Identifying “Cover Songs” with Beat-Synchronous · PDF fileIdentifying...

Identifying Cover Songs - Ellis & Poliner 2007-04-20 - /161
1. Cover Songs2. Chroma Features3. Beat Tracking4. Matching Cover Songs
Identifying Cover Songswith Beat-Synchronous
Chroma FeaturesDan Ellis and Graham Poliner
Laboratory for Recognition and Organization of Speech and Audio Dept. Electrical Eng., Columbia Univ., NY USA
{dpwe,graham}@ee.columbia.edu http://labrosa.ee.columbia.edu/
http://labrosa.ee.columbia.eduhttp://labrosa.ee.columbia.edu

Cover Songs Cover Songs = reinterpretation of a piece
different instrumentation, characterno match with timbral features
Need a different representation!beat-synchronous chroma features
2
Let It Be - The Beatles Let It Be - Nick Cave
time / sectime / sec
freq / k
Hz
Let It Be / Beatles / verse 1
2 4 6 8 100
1
2
3
4
freq / k
Hz
chro
ma
chro
ma
0
1
2
3
4Let It Be / Nick Cave / verse 1
2 4 6 8 10
Beat-sync chroma features
5 10 15 20 25 beats
Beat-sync chroma features
5 10 15 20 25 beats
A
C
D
F
G
A
C
D
F
G

Chroma Features Chroma features map spectral energy
into one canonical octavei.e. 12 semitone bins
Can resynthesize as Shepard Tonesall octaves at once
3
Piano scale
2 4 6 8 10 100 200 300 400 500 600 700
2 4 6 8 10 time / sec
time / sec
freq
/ k
Hz
0
1
2
3
4
freq / k
Hz
0
1
2
3
4
time / frames
ch
rom
a
A
C
D
F
G
Piano chromatic scale
Shepard tone resynth
IF chroma

Calculating Chroma Features Method 1: Map every STFT bin
blurs non-tonal energy
Method 2: Map only STFT peaksstill blurry at low frequencies
Method 3: Instantaneous Frequency /tescapes frequency resolution limit
4
50 100 150 200 fft bin 2 4 6 8 10 time / sec 50 100 150 200 250 300 time / frame
freq /
kH
z
0
1
2
3
4
chro
ma
A
C
D
F
G
chro
ma
A
C
D
F
G
50 100 150 200 fft bin 2 4 6 8 10 time / sec 50 100 150 200 250 300 time / frame
freq /
kH
z
0
1
2
3
4
chro
ma
A
C
D
F
G
chro
ma
A
C
D
F
G
2 4 6 8 10 time / sec 50 100 150 200 250 300 time / frame
freq /
kH
z
0
1
2
3
4
chro
ma
A
C
D
F
G
chro
ma
A
C
D
F
G
0 2000 4000
( )

Beat Tracking (1) Goal: One feature vector per beat (tatum)
for tempo normalization, efficiency
Onset Strength Envelopesumf(max(0, difft(log |X(t, f)|)))
Autocorr. + window global tempo estimate
5
10203040
freq
/ mel
0
0
5 10 15time / sec
0 100 200 300 400 500 600 700 800 900lag / 4 ms samples
1000
0
168.5 BPM

Beat Tracking (2) Dynamic Programming finds beat times {ti}
optimizes i O(ti) + i W((ti+1 ti p)/)where O(t) is onset strength envelope (local score)W(t) is a log-Gaussian window (transition cost)p is the default beat period per measured tempoincrementally find best predecessor at every timebacktrace from largest final score to get beats
6
C*(t) = O(t) + (1)max{W(( p)/)C*()}P(t) = argmax{W(( p)/)C*()}
t
O(t)
C*(t)

Beat Tracking Results DP will bridge gaps (non-causal)
there is always a best path ...
2nd place in MIREX 2006 Beat Trackingcompared to McKinney & Moelants human data
7
10
20
30
40
0 5 10 150
20
40
freq / B
ark
band
Subje
ct #
time / s
test 2 (Bragg) - McKinney + Moelants Subject data
182 184 186 188 190 192 time / sec
10
20
30
40
freq / B
ark
band
Alanis Morissette - All I Want - gap + beats

Beat-Synchronous Chroma Features Beat + chroma features / 30ms frames average chroma within each beatcompact; sufficient?
! "# "! $# $! %# %!
$
&
'
(
"#
"$
"#
$#
%#
$
&
'
(
"#
"$
# ! "# "!)*+,-.-/,0
)*+,-.-1,2)/
34,5-.-6,7
89/,)-/)4,9:);
0;48+2-1*9/
0;48+2-1*9/
#

Matching (1): Little Fragments Cover versions may change song structure
multiple local matches at different alignments
Match query and target as many small pieces?
9
how big are the pieces?
how do we combine individual scores?
do we have all day?
100 200 300 400 500 beats
100 200 300 400 500 beats
G
EDC
A
chro
ma b
ins
G
EDC
A
chro
ma b
ins
extract
cross-correlate
Query
Candidate

Matching (2): Global Correlation Cross-correlate entire beat-chroma matrices
... at all possible transpositionsimplicit combination of match quality and duration
One good matching fragment is sufficient...?10
100 200 300 400 500 beats @281 BPM
-500 -400 -300 -200 -100 0 100 200 300 400 skew / beats
-5
0
+5
G
E
D
C
A
ch
rom
a b
ins
G
E
D
C
A
ch
rom
a b
ins
ske
w /
se
mito
ne
s
Elliott Smith - Between the Bars
Glen Phillips - Between the Bars
Cross-correlation

Filtered Cross-Correlation Raw correlation not as important as precise
local matchlooking for large contrast at 1 beat skewi.e. high-pass filter
11
-500 -400 -300 -200 -100 0 100 200 300 400
0
0.2
0.4
0.6
-500 -400 -300 -200 -100 0 100 200 300 400 skew / beats
skew / beats
-5
0
+5
ske
w /
se
mito
ne
s Cross-correlation
Cross-correlation @ skew = +2 semitones
raw
filtered

Results (1): Ellis 23 set 23 pairs of cover songs from uspop2002 +...
one correct match per query
12
Test
Qu
ery
Ab Ad Al Am Be Be Bl Ca Ce Cl Co Co Da En Fa Go Go Gr Hu I_ I_ Le Ta
Abracadabra/sugar_ray
Addicted_To_Love/tina_turner
All_Along_The_Watchtower/dave_matthews_band
America/simon_and_garfunkel
Before_You_Accuse_Me/eric_clapton
Between_The_Bars/glen_phillips
Blue_Collar_Man/styx
Caroline_No/brian_wilson
Cecilia/simon_and_garfunkel
Claudette/roy_orbison
Cocaine/nazareth
Come_Together/beatles
Day_Tripper/cheap_trick
Enjoy_The_Silence/tori_amos
Faith/limp_bizkit
God_Only_Knows/brian_wilson
Gold_Dust_Woman/sheryl_crow
Grand_Illusion/styx
Hush/milli_vanilli
I_Can_t_Get_No_Satisfaction/rolling_stones
I_Love_You/faith_hill
Let_It_Be/nick_cave
Take_Me_To_The_River/annie_lennox
Cover Songs - dpwe23 - 12/23 correct

Results (2): MIREX 06 Cover song contest
30 songs x 11 versions of each (!)(data has not been disclosed)# true covers in top 108 systems compared (4 cover song + 4 similarity)
Found 761/3300= 23% recallnext best: 11%guess: 3%
13
MIREX 06 Cover Song Results:# Covers retrieved per song per system
CS DE KL1 KL2 KWL KWT LR TP
cover song systems similarity systems
so
ng
-se
t (e
ach
ro
w is o
ne
qu
ery
so
ng
)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
2
0correct
matchesretrieved
4
6
8

Where are the matches? Look inside global cross-correlation to find
matching fragments...xcorr = t f (C1(t, f)C2(t, f)) - view along time
14
Let It Be / Beatles (beats 11-441)
50 100 150 200 250 300 350 400
0 50 100 150 200 250 300 350 400
-0.2
0
0.2
0.4
Let It Be / Nick Cave (beats 13-443)
50 100 150 200 250 300 350 400 time / beats
time / beats
time / beats
ch
rom
a
A
C
D
F
G
chro
ma
A
C
D

Identifying “Cover Songs” with Beat-Synchronous · PDF fileIdentifying...

Documents

Transcript of Identifying “Cover Songs” with Beat-Synchronous · PDF fileIdentifying...