MediaEval 2016 - Tag Propagation in Talking Face Graphs
-
Upload
multimediaeval -
Category
Science
-
view
26 -
download
1
Transcript of MediaEval 2016 - Tag Propagation in Talking Face Graphs
TheMOTIFteam
Tagpropaga(onintalkingfacegraphs
GabrielSargent,GabrielFonseca,IzabelaLyonFreire,SilvioJ.Guimarães,GuillaumeGravier
GraphconstrucBonnode:speakingface——>facetrackwithoverlappingspeechsegment
+co-occurringtextoverlays—>withnameenBtydetecBon
edgeweight:visual/audiovisualsimilaritybetweentwospeakingfaces• Visualsimilarity:-eachfacetrack=centralface
-eachcentralface=CNN-basedfeaturevector-cosinesimilarity
• Audiosimilarity1:binarysimilarityfromspeakerdiarizaBonannotaBons
• Audiosimilarity2:-eachspeechsegmentdescribedbyGMMsusingcepstralfeatures-similarity:approximaBonofKullback-Leiblerdivergence
• Fusion:weightedaverageofaudioandvisualsimilariBes
TagpropagaBonRandomWalk
• TagpropagaBoniscomputedbyapplyingarandomwalkwithabsorbingstatesonaprobabilitygraph.
0.3
0.20.9
0.8
0.5
0.9
1
1
0.8
TagpropagaBonHierarchical
• BasedonKruskal’salgorithmforcalculaBngminimumspanningtrees.
• LabelpropagaBonhappenswhentwosetsaremergedintoone.
merge
TaggedSet UntaggedsetNewTaggedSet