Emotionally-Controlled Music Synthesis
-
Upload
antonio-oliveira -
Category
Engineering
-
view
266 -
download
0
Transcript of Emotionally-Controlled Music Synthesis
![Page 1: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/1.jpg)
Emotionally-Controlled Music Synthesis
António Pedro OliveiraAmílcar CardosoUniversity of Coimbra, Portugal12/12/2008
![Page 2: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/2.jpg)
2
Outline
Introduction Computational Model Features Extraction Regression Models Conclusion
![Page 3: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/3.jpg)
3
Outline
Introduction Computational Model Features Extraction Regression Models Conclusion
![Page 4: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/4.jpg)
Introduction
4
Music is accepted as a language of emotional expression
To control this expression in an automatic way, we are developing a computational model that establishes relations between emotions and musical features
Emotions are defined in 2 dimensions: Valence: degree of happiness (from very sad to very happy
music) Arousal: degree of activation (from very relaxing to very
activation music)
![Page 5: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/5.jpg)
5
Outline
Introduction Computational Model Features Extraction Regression Models Conclusion
![Page 6: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/6.jpg)
Computational Model – Features Extraction
6
Use a database of MIDI music labelled with symbolic and audio features
![Page 7: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/7.jpg)
Computational Model – Regression models
7
Use a database of MIDI music labelled with symbolic and audio features
Modelling relations between emotions and music features with regression models
![Page 8: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/8.jpg)
Computational Model
8
Use a database of MIDI music labelled with symbolic and audio features
Modelling relations between emotions and music features with regression models
Use these models to control the affective content of synthesized music
![Page 9: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/9.jpg)
Computational Model - Experiments
9
96 MIDI pieces of film music that last between 20 and 90 seconds
80 listeners Label online each affective
dimension with integer values between 0 and 10
![Page 10: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/10.jpg)
10
Outline
Introduction Computational Model Features Extraction Regression Models Conclusion
![Page 11: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/11.jpg)
Features Extraction
11
Make a music base with MIDI music labelled with symbolic and audio features
![Page 12: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/12.jpg)
Features Extraction – Correlation between audio
features and valence
12
Sharpness – ratio of high/bass frequencies Loudness – total energy Flatness – spectral distribution of energy Dissonance – perceptive interference of
sinusoids
![Page 13: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/13.jpg)
Similarity – temporal spectral correlation of energy distribution by frequency bands
Dissonance – perceptive interference of sinusoids
Sharpness – ratio of high/bass frequencies Energy – total energy
Features Extraction – Correlation between audio
features and arousal
13
![Page 14: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/14.jpg)
Bridge the gap between audio and symbolic domain:
Spectral similarity vs. note duration, interonset interval Spectral dissonance vs. prevalence of percussion
instruments
Features Extraction – Correlation between audio and symbolic
features
14
![Page 15: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/15.jpg)
15
Outline
Introduction Computational Model Features Extraction Regression Models Conclusion
![Page 16: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/16.jpg)
Regression models
16
Establish weighted relations between emotions and musical features
Use non-linear regression models Model with symbolic and audio
features
![Page 17: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/17.jpg)
Regression models – Correlation between models
and valence
17
Best hybrid (use of audio and symbolic features) non-linear regression model – 84%
Best symbolic linear regression model – 75% Best audio non-linear regression model – 61%
![Page 18: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/18.jpg)
Regression models – Best audio and symbolic features for
valence
18
![Page 19: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/19.jpg)
Regression models – Correlation between models
and arousal
19
Best hybrid (use of audio and symbolic features) non-linear regression model – 90%
Best symbolic linear regression model – 84% Best audio non-linear regression model – 75%
![Page 20: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/20.jpg)
Regression models – Best audio and symbolic features for
arousal
20
![Page 21: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/21.jpg)
21
Outline
Introduction Computational Model Features Extraction Regression Models Conclusion
![Page 22: Emotionally-Controlled Music Synthesis](https://reader035.fdocuments.in/reader035/viewer/2022062903/58ed9f8b1a28ab453a8b45d1/html5/thumbnails/22.jpg)
Conclusion
22
Hybrid non-linear regression models outperformed results of symbolic linear regression models
Non-linear models seem more appropriate than linear models
The use of features from audio and symbolic domains is more appropriate than the use of features from only one domain
Timbre/sound can be used to control/influence the emotional expression