近年來，卷積神經網絡（CNNs）在情感識別方面得到廣泛應用。在情緒辨識的多樣性中，包括了語音、身體語言和生理特徵等多個辨識依據。本研究特別選擇了臉部表情和身體動作作為情感辨識的指標。為此，我們採用了卡內基‧梅隆大學(CMU)提出的OpenPose技術，用於臉部和身體動作的估測。為了克服情緒辨識中常見的高參數配置和耗時的挑戰，本研究提出了一個智能嵌入式情感辨識系統。為了提高其效能，我們對OpenPose進行了修改以減少計算負擔。我們降低了Part Affinity Fields（PAF）的一個階段，從原本的4個階段減少為3個，而關鍵點的估測保留了原始的2個階段。在OpenPose生成的大量臉部與身體的關鍵點中，我們僅提取了與情感辨識相關的關鍵點進行後續辨識。在獲取所需的關鍵點後，會實時的計算這些關鍵點之間的向量作為情感特徵。接著，我們設計了一個倒傳遞類神經網絡模型（BPNN）作為分類器。為了訓練模型，我們將這些向量作為分類器的輸入，而分類器的輸出涵蓋了六個情感類別，而分類器完成了情緒的分類後會將結果輸出至雙足機器人中，使雙足機器人能夠做出與辨識出的情緒相對應的動作。最後，我們在NVIDIA Jetson Xavier NX上實現了部分提出的系統作為智能嵌入式系統，一系列的實驗結果顯示了該系統部分的有效性。
In recent years, Convolutional Neural Networks (CNNs) have been widely applied in emotion recognition. The diversity of emotional recognition encompasses various cues such as speech, body language, and physiological traits. This study specifically opted for facial expressions and body movements as indicators for emotion recognition. To accomplish this, we employed the OpenPose technology proposed by Carnegie Mellon University for estimating facial and bodily movements. In overcoming common challenges in emotion recognition, namely high parameter configurations and time-consuming processes, this research introduced an intelligent embedded emotion recognition system. To enhance its efficiency, modifications were made to OpenPose to reduce computational burdens. We reduced one stage of the Part Affinity Fields (PAF) from the original 4 stages to 3, while retaining the original 2 stages for keypoint estimation. From the numerous facial and bodily keypoints generated by OpenPose, only those relevant to emotion recognition were extracted for subsequent identification. After obtaining these required keypoints, real-time calculations of the vectors between them were performed as emotional features. Subsequently, a Back Propagation Neural Network (BPNN) was designed as the classifier. For model training, these vectors served as inputs to the classifier, which provided outputs encompassing six emotional categories. Upon the classifier's completion of emotion classification, the results were output to a bipedal robot, enabling it to perform actions corresponding to the recognized emotions. Finally, the proposed system was implemented on the NVIDIA Jetson Xavier NX as an intelligent embedded system. A series of experiments demonstrated partial effectiveness of this system.
