JAIST Repository >
School of Information Science >
Articles >
Journal Articles >

Please use this identifier to cite or link to this item: http://hdl.handle.net/10119/18103

Title: Speech Emotion and Naturalness Recognitions With Multitask and Single-Task Learnings
Authors: Atmaja, Bagus Tris
Sasou, Akira
Akagi, Masato
Keywords: Speech emotion recognition
speech naturalness recognition
multitask learning
affective computing
speech processing
Issue Date: 2022-07-07
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Magazine name: IEEE Access
Volume: 10
Start page: 72381
End page: 72387
DOI: 10.1109/ACCESS.2022.3189481
Abstract: This paper evaluates speech emotion and naturalness recognitions by utilizing deep learning models with multitask learning and single-task learning approaches. The emotion model accommodates valence, arousal, and dominance attributes known as dimensional emotion. The naturalness ratings are labeled on a five-point scale as dimensional emotion. Multitask learning predicts both dimensional emotion (as the main task) and naturalness scores (as an auxiliary task) simultaneously. The single-task learning predicts either dimensional emotion (valence, arousal, and dominance) or naturalness score independently. The results with multitask learning show improvement from previous studies on single-task learning for both dimensional emotion recognition and naturalness predictions. Within this study, single-task learning still shows superiority over multitask learning for naturalness recognition. The scatter plots of emotion and naturalness prediction scores against the true labels in multitask learning exhibit the lack of the model; it fails to predict the low and extremely high scores. The low score of naturalness prediction in this study is possibly due to a low number of samples of unnatural speech samples since the MSP-IMPROV dataset promotes the naturalness of speech. The finding that jointly predicting naturalness with emotion helps improve the performance of emotion recognition may be embodied in the emotion recognition model in future work.
Rights: Bagus Tris Atmaja, Akira Sasou, Masato Akagi, IEEE Access, 10, 2022, pp.72381-72387. DOI:10.1109/ACCESS.2022.3189481. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
URI: http://hdl.handle.net/10119/18103
Material Type: publisher
Appears in Collections:b10-1. 雑誌掲載論文 (Journal Articles)

Files in This Item:

File Description SizeFormat
IEEEAccess2022.pdf8209KbAdobe PDFView/Open

All items in DSpace are protected by copyright, with all rights reserved.


Contact : Library Information Section, Japan Advanced Institute of Science and Technology