MSc ProjectMusical Instrument Identification System

MIIS

Xiang LI

ee05m216

Supervisor: Mark Plumbley

Motivation of MIIS

Musical instrument identification plays an important role in

musical signal indexing and database retrieval. People can search music by the musical instruments

instead of the type or the author For instance, user is able to query ‘find piano solo parts of

a musical database’.

Introduction

Bass

Drum

Piano

Saxophone

Identification results

Musical Mixtures Musical instruments

Structure of MIIS

1s

Functional ComponentsDUET algorithm:

Separate the input musical mixture into sourcesFeature Extraction:

Extract features of each sourceClassification:

Implement classifier on testing source and find out the class it belongs to

2s3s

Input MixtureX(n)

DUET algorithmSeparation

Estimated Sources

ns

Feature Extraction

Classification

ClassificationResults

DUET algorithm Time-Frequency representation: and are representations in time-frequency domain, i.e.

Short-time Fourier Transform, Modified Cosine Discrete Transform.

Mixing parameters computation:

Time-frequency points are labeled with Mask construction: Mask equals deciding set ,which could be

achieved by grouping the time-frequency point with the same label

Source estimation

is the time-frequency representation of one source.

Time-domain conversion

Convert each to in time domain

1~x 2

~x

1

2~

~

x

xa j

jjM 1 j

ja

js~

js~

js

1~ˆ xMs jj

Feature Extraction Mel-Frequency Cepstral Coefficient (MFCC) Relationship between Mel and Hertz Spectral Rolloff It is calculated by summing up the power spectrum samples until the desired

percentage (threshold) of the total energy is reached. Bandwidth Defined as the width of the range of frequencies that the signal occupies. Root Mean Square

RMS features are used to detect boundaries among musical instruments Spectral Centroid

Correlates strongly with the subjective qualities of “brightness” or “sharpnes

s”. Zero Crossing Rate A simple measure of the frequency content of a signal

)700

1(log2595)( 10

ffMel

Classification

K-Nearest Neighbor Nonparametric classifier Large storage required

X

Class a

Class b

Class cy

x

Experiments

Musical Instruments Database Database : Downloaded from University of Iowa website.

Mixtures are composed by isolated notes.

Training set: Includes 18 classes musical instruments Testing set: Choose 3 to 5 instruments to generate mixtures

The instruments to be tested: Alto Saxophone Bassoon Double Bass Flute Viola

Experiments of three groups

Group 1 Group 2 Group 3No. of Sources 3 4 5Percentage correct

80% 60% 48%

For each group, five mixtures are tested and the result of each group is listed as follows:

Example

Source SDR Original Source

Estimated Source

Result

AltoSaxophone.C4B4 17.4453 2 2 correct

Bassoon.C3B3 10.4249 9 9 correctDouble Bass.D2B2 6.0127 4 4 correct

Estimated Sources Original Sources

Results discussion

Without MISS, the recognisation percentage of each source in 18 classes is 1/18 which is about 5.5%.

The worst case in our experiments is group 3 where each mixture consists five sources. The percentage is 48%.

The less sources mixtures have, the higher percentage system performs. More sources introduce more interferences among each other.

Conclusion

MISS is a system able to identify each musical instrument in a musical mixture.

Three functional components are introduced: DUET algorithm Feature Extraction Classification

Experiments of three groups, which is fifteen mixtures in total have been tested. Correct percentages are 80%,60%and 48% respectively.

More features could be extracted such as features of MPEG7

A more adaptive mask could help overcoming interferences among sources.