COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP...
Transcript of COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP...
![Page 1: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/1.jpg)
COMP9517ComputerVision
Tracking
5/09/16 1COMP9517S2,2016
![Page 2: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/2.jpg)
Mo@onTracking• Trackingistheproblemofgenera@nganinferenceaboutthe
mo@onofanobjectgivenasequenceofimages– Whatdoweinferaboutanobject’sfutureposi@onfromasequenceof
measurements?
• Applica@ons– Mo@oncapture
• Controlacartoon• Modifythemo@onrecordtoobtainslightlydifferentmo@ons
– Recogni@onfrommo@on• Determinetheiden@tyoftheobject• Tellwhatitisdoing
– Surveillance• Monitorac@vi@esandgiveawarningifitdetectsaproblemcase
– Targe@ng• DecidewhattoshootandhiRngit
5/09/16 2COMP9517S2,2016
![Page 3: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/3.jpg)
Mo@onTracking• Whenmovingpointsarenottaggedwithuniquetextureorcolourinforma@on,thecharacteris@csofthemo@onitselfmustbeusedtocollectpointsintotrajectories
• Assump@on:– Theloca@onofanobjectchangessmoothlyover@me– Thevelocity(speedanddirec@on)ofanobjectchangessmoothlyover@me
– Anobjectcanbeatonlyoneloca@oninspaceatagive@me
– Twoobjectscannotoccupythesameloca@onatthesame@me
5/09/16 3COMP9517S2,2016
![Page 4: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/4.jpg)
TrackingwithDynamicModels• Trackingcanbeconsideredastheproblemofgenera@ngan(probabilis@c)inferenceaboutthemo@onofanobjectgivenasequenceofimages
• Trackingisproperlythoughtofasanprobabilis@cinferenceproblem– Themovingobjecthasinternalstate,whichismeasuredateachframe
– Measurementsarecombinedtoes@matethestateoftheobject
5/09/16 4COMP9517S2,2016
![Page 5: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/5.jpg)
TrackingwithDynamicModels• State:therepresenta@onofanobjectat@me(frame)t– Posi@on– Transforma@onparameters– Class– Etc
• Measurement:theobserva@on– Colour– Texture– Etc
5/09/16 5COMP9517S2,2016
![Page 6: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/6.jpg)
TrackingwithDynamicModels• Given– Xi:thestateoftheobjectattheithframe– Yi:themeasurementobtainedintheithframe
• Therearethreemainproblems:– Predic@on:predictthestatefortheithframebyhavingseenasetofmeasurementsy0,y1,…,yi-1
– Dataassocia@on:selectthemeasurementsthatarerelatedtotheobject’sstate
– Correc@on:correctthepredictedstatebyobtainedrelevantmeasurementsyi
5/09/16 6COMP9517S2,2016
),...,,|( 111100 −− === iii yYyYyYXP
),...,,|( 1100 iii yYyYyYXP ===
![Page 7: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/7.jpg)
TrackingwithDynamicModels• IndependenceAssump@ons(MarkovAssump@on)– Onlytheimmediatepastma`ers
– Measurementsdependonlyonthecurrentstate
– Theseassump@onsmeanthatatrackingproblemhasthestructureofinferenceonahiddenMarkovmodel
5/09/16 7COMP9517S2,2016
)|,...,()|()|,...,,( ikjiiikji XYYPXYPXYYYP =
)|(),...,,|( 1121 −− = iiii XXPXXXXP
Xi-1
Yi-1
Xi
Yi
Xi+1
Yi+1
![Page 8: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/8.jpg)
LinearDynamicModels• LinearDynamicModels– Arandomprobabilitydistribu@onwithmeanandcovariance
– Thestateisadvancedbymul@plyingitbysomeknownmatrixandthenaddinganormalrandomvariable
– Themeasurementisobtainedbymul@plyingthestatebysomematrixandthenaddinganormalrandomvariableofzeromeanandknowncovariance
5/09/16 8COMP9517S2,2016
).,(~ ∑µNx
).,(~ 1 idiii xDNx ∑−
).,(~imiii xMNy ∑
![Page 9: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/9.jpg)
ProbabilityDensityPropaga@on
5/09/16 9COMP9517S2,2016
![Page 10: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/10.jpg)
ProbabilityDensityPropaga@on
5/09/16 10COMP9517S2,2016
![Page 11: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/11.jpg)
TrackingwithDynamicModels• TrackingasInference– Predic@on
– Correc@on
5/09/16 11COMP9517S2,2016
111011
111011101
11101110
),...,,|()|(
),...,,|(),...,,,|(
),...,,|,(),...,,|(
−−−−
−−−−−
−−−−
∫∫
∫
=
=
=
iiiii
iiiiii
iiiiii
dXyyyXPXXP
dXyyyXPyyyXXP
dXyyyXXPyyyXP
iiiii
iiii
i
iiiii
i
iiiiii
i
iiii
dXyyyXPXyPyyyXPXyP
yyyPyyyPyyyXPXyP
yyyPyyyPyyyXPyyyXyP
yyyPyyyXPyyyXP
∫ −
−
−−
−−−
=
=
=
=
),...,,|()|(),...,,|()|(
),...,,(),...,,(),...,,|()|(
),...,,(),...,,(),...,,|(),...,,,|(
),...,,(),...,,,(),...,,|(
110
110
10
110110
10
110110110
10
1010
![Page 12: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/12.jpg)
TrackingwithDynamicModels• Someconvenienttransforma@ons
5/09/16 12COMP9517S2,2016
),0;(),;( vxgvxg µµ −=
),;(),;( vmngvnmg =
)/,/;(),;( 2avaxgvaxg µµ =
),;(),0;(),;( baba vvxgduvugvuxg +=−∫∞
∞−µµ
),,,(),;(),;(),;( dcbafdb
bddbcdadxgdcxgbaxg
++
+=
![Page 13: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/13.jpg)
TrackingwithDynamicModels• KalmanFiltering– Dynamicmodel:
– Define:
– Predic@on:
5/09/16 13COMP9517S2,2016
);,(~)|( 211 idiiii xdNXXP σ−−
).,(~)|( 2imiiii xmNXYP σ
21
21
21
21
12
1112
1
111011110
)(,
))(,;(
))(,;(),;(
),...,,|()|(),...,,|(
+−
−+−
−
+−
+−
−+−
+−−−
−−−−−
+==⇒
+∝
∝
=
∫
∫
iidiiii
iidiii
iiiidiii
iiiiiii
dXdX
dXdXg
dXXXgXdXg
dXyyyXPXXPyyyXP
i
i
i
σσσ
σσ
σσ
).,...,,|(__var),,...,,|(__ 1010 iiiiii yyyXPofianceyyyXPofmeanX −−++
σ
).,...,,|(__var),,...,,|(__ 110110 −
−
−
−−− iiiiii yyyXPofianceyyyXPofmeanX σ
![Page 14: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/14.jpg)
TrackingwithDynamicModels• KalmanFiltering– Correc@on
5/09/16 14COMP9517S2,2016
⎟⎟⎠
⎞⎜⎜⎝
⎛
+=⎟
⎟
⎠
⎞
⎜⎜
⎝
⎛
+
+=⇒
∝
∝
=
−
−+
−
−−
+
−−
−
−
−
∫
222
22
222
22
22
110
110
11010
)()(
,)()(
))(,;(),;(
),...,,|()|(
),...,,|()|(),...,,|()|(),...,,|(
iim
imi
iim
iiimi
i
iiimiii
iiii
iiiii
iiiiii
mmymX
X
XXgXmyg
yyyXPXyP
dXyyyXPXyPyyyXPXyPyyyXP
i
i
i
i
i
σσ
σσσ
σσ
σσ
σσ
![Page 15: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/15.jpg)
TrackingwithDynamicModels• KalmanFiltering–thealgorithm(1-D)– StartAssump@on: areknown– UpdateEqua@on:Predic@on
– UpdateEqua@on:Correc@on
5/09/16 15COMP9517S2,2016
−−
00 ,σx
21
2
1
)( +−
−
+−
−
+=
=
iidi
iii
d
xdx
iσσσ
⎟⎟⎠
⎞⎜⎜⎝
⎛
+=
⎟⎟
⎠
⎞
⎜⎜
⎝
⎛
+
+=
−
−+
−
−−
+
222
22
222
22
)()(
)()(
iim
imi
iim
iiimii
m
mymx
x
i
i
i
i
σσ
σσσ
σσ
σσ
![Page 16: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/16.jpg)
TrackingwithDynamicModels• KalmanFiltering–thealgorithm(2-D)– StartAssump@on: areknown– UpdateEqua@on:Predic@on
– UpdateEqua@on:Correc@on
5/09/16 16COMP9517S2,2016
−−Σ00 ,x
iiidi
iii
DDxDx
i
+−
−
+−
−
+Σ=
=
1
1
σσ
[ ][ ]
[ ] −+
−−+
−−−
Σ−=Σ
−+=
Σ+ΣΣ=
iiii
iiiiii
mTiii
Tii
MKIdxMyKxx
MMMKi
1
![Page 17: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/17.jpg)
TrackingwithDynamicModels• Par@cleFiltering
– Anon-lineardynamicmodel– Alsoknownvariouslyas:
• Sequen@alMonteCarlomethod,• bootstrapfiltering,• thecondensa@onalgorithm,• interac@ngpar@cleapproxima@onsand• survivalofthefi`est
5/09/16 17COMP9517S2,2016
![Page 18: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/18.jpg)
TrackingwithDynamicModels• Par@cleFiltering
– Thecondi@onalstatedensityat@metisrepresentedbyasetofsamplingpar@cleswithweight(samplingprobability).
– Theweightdefinetheimportanceofasample(observa@onfrequency)
5/09/16 18COMP9517S2,2016
![Page 19: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/19.jpg)
TrackingwithDynamicModels• Par@cleFiltering– Thecommonsamplingschemeisimportancesampling
• Selec@on:selectNrandomsamples• Predic@on:generatenewsampleswithzeromeanGaussianErrorandnon-nega@vefunc@on
• Correc@on:computeweightscorrespondingtothenewsamplesusingthemeasurementequa@onwhichcanbemodelledasaGaussiandensity
5/09/16 19COMP9517S2,2016
![Page 20: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/20.jpg)
TrackingwithDynamicModels• Par@cleFiltering(A.VlakeandM.Isard,Ac@veContour)
5/09/16 20COMP9517S2,2016
![Page 21: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/21.jpg)
TrackingwithDynamicModels• Par@cleFiltering–thealgorithm(SIS),A.VlakeandM.Isard
5/09/16 21COMP9517S2,2016
![Page 22: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/22.jpg)
TrackingwithDynamicModels• Par@cleFiltering(A.VlakeandM.Isard,Ac@veContour)
5/09/16 22COMP9517S2,2016
![Page 23: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/23.jpg)
Trackingcanbecomplex• Lossofinforma+oncausedbyprojec@onofthe3Dworldona2Dimage,
• Noiseinimages,• Complexobjectmo+on,• Non-rigidorar@culatednatureofobjects,• Par@alandfullobjectocclusions,• Complexobjectshapes,• Sceneillumina+onchanges,and• Real-+meprocessingrequirements.
5/09/16 23COMP9517S2,2016
![Page 24: COMP 9517 Computer Visionwebcms3.cse.unsw.edu.au/static/uploads/course/COMP9517/16s2/... · COMP 9517 Computer Vision Tracking 5/09/16 COMP 9517 S2, 2016 1 Mo@on Tracking • Tracking](https://reader034.fdocuments.in/reader034/viewer/2022050503/5f956c95b8dea11bdb77f5ea/html5/thumbnails/24.jpg)
ReferencesandAcknowledgements• ShapiroandStockman2001• Chapter19ForsythandPonce2003• Chapter5Szeliski2010• Imagesdrawnfromtheabovereferencesunlessotherwisemen@oned
5/09/16 COMP9517S2,2016 24