Encompass a Broad Spectrum of Human Emotion: Mixed-emotion Speech Synthesis using Task Vector and Fine-grained Representation

0. Contents

Primary Emotion Synthesis

Angry
Happy
Neutral
Sad
Surprise

Mixed-emotion Synthesis

Surprise with Angry
Surprise with Happy
Surprise with Sad

1. Primary Emotion Synthesis

This test is mainly used to verify the ability of the fine-tuned model to synthesize primary emotions and the effectiveness of the introduced fine-grained annotations in improving the model's performance.

1.1 Angry

ID	Speaker	GT
ID	001	0013
0015	001
002	0013
002	0015
003	0013
003	0015
004	0013
004	0015
005	0013
005	0015

1.2 Happy

ID	Speaker	GT
ID	001	0013
0015	001
002	0013
002	0015
003	0013
003	0015
004	0013
004	0015
005	0013
005	0015

1.3 Neutral

ID	Speaker	GT
ID	001	0013
0015	001
002	0013
002	0015
003	0013
003	0015
004	0013
004	0015
005	0013
005	0015

1.4 Sad

ID	Speaker	GT
ID	001	0013
0015	001
002	0013
002	0015
003	0013
003	0015
004	0013
004	0015
005	0013
005	0015

1.5 Surprise

ID	Speaker	GT
ID	001	0013
0015	001
002	0013
002	0015
003	0013
003	0015
004	0013
004	0015
005	0013
005	0015

2. Mixed-emotion Synthesis

Mixed-emotion samples.

2.1 Surprise with Angry

ID	Speaker	Configure
001	0013	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
	0015	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
002	0013	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
	0015	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
003	0013	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
	0015	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
004	0013	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
	0015	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
005	0013	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
	0015	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%

2.2 Surprise with Happy

ID	Speaker	Configure
001	0013	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
	0015	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
002	0013	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
	0015	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
003	0013	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
	0015	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
004	0013	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
	0015	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
005	0013	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
	0015	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%

2.3 Surprise with Sad

ID	Speaker	Configure
001	0013	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
	0015	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
002	0013	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
	0015	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
003	0013	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
	0015	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
004	0013	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
	0015	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
005	0013	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%
	0015	20%, 80%
		40%, 60%
		60%, 40%
		80%, 20%

Code