Mining Mouse Vocalizations
Jesin ZakariaDepartment of Computer Science and Engineering
University of California Riverside
124 Time (second) 12540
kHz
100
laboratorymice
Mouse Vocalizations
Figure 1: top) A waveform of a sound sequence produced by a lab mouse, middle) A spectrogram of the sound, bottom) An idealized version of the spectrogram
AX
Q
X P A X
C
X P
The intution behind symbolizing the spectrogram
Figure 3: The two fragments of data shown in Figure 2.bottom aligned to produce the maximum overlap. (Best viewed in color)
Figure 4: The data shown in Figure 2 augmented by labeled syllables
Figure 1: top) Two 0.5 second spectrogram representations of fragments of the vocal output of a male mouse. bottom) Idealized (by human intervention) versions of the above
2
7876.3 Time (second)
kHz
120
0
30
110
original
idealized
Background
Figure 6: top) Original spectrogram, bottom) Idealized spectrogram (after thresholding and binarization)
31 1 1
4
4 8
87
Figure 7: left) A real spectrogram of a mouse vocalization can be approximated by samples of handwritten Farsi digits (right). Some Farsi digits were rotated or transposed to enhance the similarity
90.1 91.1Time (sec)
Figure 5: A snippet spectrogram that has seven syllables
I LSP
connected components
Figure 8: from left to right)snippet spectrogram, matrix corresponding to an idealized spectrogram I, matrix corresponding to the set of connected components L, mbrs of the candidate syllables
Extracting syllables from spectrogram
I J K L M N O PA B C D E F G H
a b c d e f
g h i j k
NewClass
Editing Ground truthI J K L M N O P
A B C D E F G H
Figure 9: Sixteen syllables provided by domain experts
Figure 11: Ambiguity reduction of the original set of syllable classes. Representative examples from the reduced set of eleven classes are labeled as small letters
Editing Ground truth
Figure 10: Thick/red curve represents the accuracy of classifying syllables of edited ground truth. Thin/blue curve represents the accuracy of classifying 692 labeled syllables using edited ground truth
0 100 200 300 400 500 600 700
0
0.2
0.4
0.6
0.8
1
Adding more instances
Cla
ssifi
catio
n A
ccur
acy
for edited ground truth
for all the labeled syllables
Data mining Mouse Vocalizations
cccc
ccgc
eccccc
cc eccc
ccc
ciacia
ci
ciac
iaci
dcibfcd
ddci
bfcd
ccccccgc
Figure 12: A clustering of eight snippets of mouse vocalization spectrograms using the string edit distance on the extracted syllables (spectrograms are rotated 90 degrees for visual clarity)
Figure 13: A clustering of the same eight snippets of mouse vocalization shown in Figure 12 using the correlation method. The result appears near random
Clustering mouse vocalizations
cccc
query image
Data mining Mouse Vocalizations
Similarity search / Query by content
Figure 15: top) The query image from [2] was transcribed to cccc. Similar patterns are found in CT (first row) and KO (second row) mouse vocalizations in our collection
Figure 14: top) A query image from [1], The syllable labels have been added by our algorithm to produce the query ciabqciacia, bottom) the two best matches found in our dataset; corresponding symbolic strings are ciafqcicia and ciqbqcaacja, with edit distance 2 and 3, respectively
[1] J. M. S. Grimsley, et al., Development of Social Vocalizations in Mice. PLoS ONE 6(3): e17460 (2011).
[2] T. E. Holy, Z. Guo, Ultrasonic songs of male mice, PLoS Biol 3(12): e386, (2005).
query imagec c c
aa
ib q
a
ii
ciafqciciaEdit dist 2
ciqbqcaacjaEdit dist 3
944.7 – 945.2 sec194.8 – 195.2 sec
motif
Data mining Mouse Vocalizations
Figure 1: A motif that occurred in two different time intervals of a vocalization. The left and right one correspond to the symbolic strings ciaciacia and ciacjacia
b c c c c q g c c
c c c c c c c cg
0 0.5 1 1.5 2 2.5 3 3.50
10
20
30
40 3983
4416
18
118
11
# of
subs
tring
s (lo
g sc
ale)
Z-score
motif 1
motif 2
c c c
ccc i i
i i i
ja a a
a a a
Figure 1: top) Distribution of z-scores, bottom) two sets of motifs from spectral space with a z-score of approximately two and three, respectively
Assessing Motif Significance using z-score
16
17
Overrepresented in Control
Overrepresented in Knock-out
Figure 18: Examples of contrast set phrases. top) Three examples of a phrase ciacia that is overrepresented in KO, appearing 24 times in KO but never in CT. bottom) Two examples of a phrase dccccc that appears 39 times in CT and just twice in KO
Contrast set mining
using information gain
Top Related