2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx...
Transcript of 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx...
![Page 1: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/1.jpg)
Music Processing
Meinard Müller
Lecture
Introduction
International Audio Laboratories [email protected]
![Page 2: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/2.jpg)
Music
![Page 3: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/3.jpg)
Music Information Retrieval (MIR)
Sheet Music (Image) CD / MP3 (Audio) MusicXML (Text)
Music Film (Video)
Dance / Motion (Mocap) MIDI
Music Literature (Text)Singing / Voice (Audio)
Music
![Page 4: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/4.jpg)
Music Information Retrieval (MIR)
MusicMusicology
LibrarySciences
User Interfaces
Signal Processing
MachineLearning
Information Retrieval
![Page 5: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/5.jpg)
Piano Roll Representation
![Page 6: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/6.jpg)
Player Piano (1900)
![Page 7: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/7.jpg)
Time
Pitch
J.S. Bach, C-Major Fuge
(Well Tempered Piano, BWV 846)
Piano Roll Representation (MIDI)
![Page 8: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/8.jpg)
Query:
Goal: Find all occurrences of the query
Piano Roll Representation (MIDI)
![Page 9: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/9.jpg)
Matches:
Piano Roll Representation (MIDI)
Query:
Goal: Find all occurrences of the query
![Page 10: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/10.jpg)
Music Retrieval
Query
Database
Hit
Bernstein (1962) Beethoven, Symphony No. 5
Beethoven, Symphony No. 5: Bernstein (1962) Karajan (1982) Gould (1992)
Beethoven, Symphony No. 9 Beethoven, Symphony No. 3 Haydn Symphony No. 94
Audio-ID
Version-ID
Category-ID
![Page 11: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/11.jpg)
Music Synchronization: Audio-Audio
Beethoven’s Fifth
![Page 12: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/12.jpg)
Music Synchronization: Audio-Audio
Time (seconds)
Beethoven’s Fifth
Orchester(Karajan)
Piano(Scherbakov)
![Page 13: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/13.jpg)
Music Synchronization: Audio-Audio
Time (seconds)
Beethoven’s Fifth
Orchester(Karajan)
Piano(Scherbakov)
![Page 14: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/14.jpg)
Application: Interpretation Switcher
![Page 15: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/15.jpg)
Music Synchronization: Image-AudioIm
age
Audi
o
![Page 16: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/16.jpg)
Music Synchronization: Image-AudioIm
age
Audi
o
![Page 17: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/17.jpg)
How to make the data comparable?Im
age
Audi
o
![Page 18: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/18.jpg)
How to make the data comparable?Im
age
Audi
o
Image Processing: Optical Music Recognition
![Page 19: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/19.jpg)
How to make the data comparable?Im
age
Audi
o
Audio Processing: Fourier Analysis
Image Processing: Optical Music Recognition
![Page 20: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/20.jpg)
How to make the data comparable?Im
age
Audi
o
Audio Processing: Fourier Analysis
Image Processing: Optical Music Recognition
![Page 21: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/21.jpg)
Application: Score Viewer
![Page 22: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/22.jpg)
Music Structure AnalysisExample: Brahms Hungarian Dance No. 5 (Ormandy)
Time (seconds)
![Page 23: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/23.jpg)
Music Structure AnalysisExample: Brahms Hungarian Dance No. 5 (Ormandy)
Time (seconds)
![Page 24: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/24.jpg)
Music Structure AnalysisExample: Brahms Hungarian Dance No. 5 (Ormandy)
Time (seconds)A1 A2 A3B1 B2 B3 B4C
![Page 25: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/25.jpg)
Tempo Estimation and Beat Tracking
Time (seconds)
Example: Queen – Another One Bites The Dust
Basic task: “Tapping the foot when listening to music’’
![Page 26: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/26.jpg)
Time (seconds)
Tempo Estimation and Beat Tracking
Example: Queen – Another One Bites The Dust
Basic task: “Tapping the foot when listening to music’’
![Page 27: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/27.jpg)
Tempo Estimation and Beat Tracking
Light effects
Music recommendation
DJ
Audio editing
![Page 28: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/28.jpg)
Why is Music Processing Challenging?
Chopin, Mazurka Op. 63 No. 3 Example:
![Page 29: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/29.jpg)
Why is Music Processing Challenging?
Waveform
Chopin, Mazurka Op. 63 No. 3 Example:Am
plitu
de
Time (seconds)
![Page 30: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/30.jpg)
Why is Music Processing Challenging?
Waveform / Spectrogram
Chopin, Mazurka Op. 63 No. 3 Example:Fr
eque
ncy
(Hz)
Time (seconds)
![Page 31: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/31.jpg)
Why is Music Processing Challenging?
Waveform / Spectrogram
Performance– Tempo– Dynamics– Note deviations– Sustain pedal
Chopin, Mazurka Op. 63 No. 3 Example:
![Page 32: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/32.jpg)
Why is Music Processing Challenging?
Waveform / Spectrogram
Performance– Tempo– Dynamics– Note deviations– Sustain pedal
Polyphony
Chopin, Mazurka Op. 63 No. 3 Example:
Main Melody
AccompanimentAdditional melody line
![Page 33: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/33.jpg)
Audio Identification
Structure AnalysisFourier TransformAudio Features
Music ProcessingMusic Synchronization
Tempo and Beat Tracking Audio Decomposition
![Page 34: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/34.jpg)
Book: Fundamentals of Music Processing
Meinard MüllerFundamentals of Music ProcessingAudio, Analysis, Algorithms, Applications483 p., 249 illus., hardcoverISBN: 978-3-319-21944-8Springer, 2015
Accompanying website: www.music-processing.de
![Page 35: 2018 Mueller MP-Introduction - Amazon S3...Microsoft PowerPoint - 2018_Mueller_MP-Introduction.pptx Author meinard Created Date 6/25/2018 2:23:19 AM ...](https://reader034.fdocuments.in/reader034/viewer/2022042307/5ed2e9699e7a331a5e62bbb0/html5/thumbnails/35.jpg)
Book: Fundamentals of Music Processing
Meinard MüllerFundamentals of Music ProcessingAudio, Analysis, Algorithms, Applications483 p., 249 illus., hardcoverISBN: 978-3-319-21944-8Springer, 2015
Accompanying website: www.music-processing.de