JAIST Repository >
School of Information Science >
Articles >
Journal Articles >
Please use this identifier to cite or link to this item:
http://hdl.handle.net/10119/17067
|
Title: | Cross-Lingual Voice Conversion With Controllable Speaker Individuality Using Variational Autoencoder and Star Generative Adversarial Network |
Authors: | Ho, Tuan Vu Akagi, Masato |
Keywords: | Voice conversion cross-lingual controllable speaker individuality variational autoencoder generative adversarial network |
Issue Date: | 2021-03-02 |
Publisher: | Institute of Electrical and Electronics Engineers (IEEE) |
Magazine name: | IEEE Access |
Volume: | 9 |
Start page: | 47503 |
End page: | 47515 |
DOI: | 10.1109/ACCESS.2021.3063519 |
Abstract: | This paper proposes a non-parallel cross-lingual voice conversion (CLVC) model that can mimic voice while continuously controlling speaker individuality on the basis of the variational autoencoder (VAE) and star generative adversarial network (StarGAN). Most studies on CLVC only focused on mimicking a particular speaker voice without being able to arbitrarily modify the speaker individuality. In practice, the ability to generate speaker individuality may be more useful than just mimicking voice. Therefore, the proposed model reliably extracts the speaker embedding from different languages using a VAE. An F0 injection method is also introduced into our model to enhance the F0 modeling in the cross-lingual setting. To avoid the over-smoothing degradation problem of the conventional VAE, the adversarial training scheme of the StarGAN is adopted to improve the training-objective function of the VAE in a CLVC task. Objective and subjective measurements confirm the effectiveness of the proposed model and F0 injection method. Furthermore, speaker-similarity measurement on fictitious voices reveal a strong linear relationship between speaker individuality and interpolated speaker embedding, which indicates that speaker individuality can be controlled with our proposed model. |
Rights: | Tuan Vu Ho, Masato Akagi, IEEE Access, 9, 2021, pp.47503-47515. DOI:10.1109/ACCESS.2021.3063519. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/ |
URI: | http://hdl.handle.net/10119/17067 |
Material Type: | publisher |
Appears in Collections: | b10-1. 雑誌掲載論文 (Journal Articles)
|
Files in This Item:
File |
Description |
Size | Format |
IEEEAccess2021.pdf | | 2395Kb | Adobe PDF | View/Open |
|
All items in DSpace are protected by copyright, with all rights reserved.
|