Final Project 776: “Computer Vision”, Fall...
3
Final Project 776: “Computer Vision”, Fall 2015 Due date: Nov 30, 2015, 12:00 pm (by e-mail to [email protected]) Data: http://cvlabwww.epfl.ch/data/multiview/denseMVS.html What should you turn in: A single pdf file with all solutions and a discussion of your solution. Along with the pdf file you should provide a link for your results and code. In addition to the above you should prepare a short (~5 min) presentation describing your work. 1) Image Data Association The goal of the assignment is to apply an unsupervised clustering algorithm to segregate a set of images observing different scenes. a) Download the datasets fountain-P11, Herz-Jesu-P8, entry-P10 from the Strecha MVS evaluation website and form an aggregate dataset. b) Extract a global image descriptor for each image. You may choose any available descriptor and are not required to develop descriptor code. Example descriptors include color histograms, GIST, bag of visual words, etc. Alternatively, you may choose to engineer your own image descriptor if desired. c) Implement a mean-shift unsupervised clustering algorithm to partition the aggregated dataset into disjoint image sets. d) Extra Credit. Download the Castle-P19 image dataset and aggregate with the entry-P10 image dataset. Perform the same clustering procedure using the same parameters. Please note, there is scene overlap among these datasets, making correct segregation more ambiguous. Include in your report the attained clusters; discuss your choice of image descriptor as well as the mechanisms used for parameter tuning of your clustering algorithm.
Transcript of Final Project 776: “Computer Vision”, Fall...
FinalProject776:“ComputerVision”,Fall2015Duedate:Nov30,2015,12:00pm([email protected])Data:http://cvlabwww.epfl.ch/data/multiview/denseMVS.htmlWhatshouldyouturnin:Asinglepdffilewithallsolutionsandadiscussionofyoursolution.Alongwiththepdffileyoushouldprovidealinkforyourresultsandcode.In addition to the above you should prepare a short (~5 min) presentationdescribingyourwork.1)ImageDataAssociationThe goal of the assignment is to apply an unsupervised clustering algorithm tosegregateasetofimagesobservingdifferentscenes.
a) Download the datasets fountain-P11, Herz-Jesu-P8, entry-P10 from theStrechaMVSevaluationwebsiteandformanaggregatedataset.
b) Extract a global image descriptor for each image. You may choose any
available descriptor and are not required to develop descriptor code.Exampledescriptorsincludecolorhistograms,GIST,bagofvisualwords,etc.Alternatively, you may choose to engineer your own image descriptor ifdesired.
c) Implement amean-shift unsupervised clustering algorithm to partition theaggregateddatasetintodisjointimagesets.
d) ExtraCredit.DownloadtheCastle-P19imagedatasetandaggregatewiththeentry-P10 image dataset. Perform the same clustering procedure using thesameparameters.Pleasenote,thereissceneoverlapamongthesedatasets,makingcorrectsegregationmoreambiguous.
Includeinyourreporttheattainedclusters;discussyourchoiceofimagedescriptoraswellasthemechanismsusedforparametertuningofyourclusteringalgorithm.
2)StructureEstimationThegoaloftheassignmentistoimplementthemultiviewplanesweepalgorithmfordepthmap generation. To this end you shall use the provided camera pose andcalibrationinformationprovidedforeachimageintheaboveStrechadatasets.
a) Manually select fromeachof theclustergenerated for task1d)a referenceimagetogenerateadepthmap.
b) Use the remaining images in the cluster as matching images for themultiviewplanesweepalgorithm
Present in your report the attained depthmaps, explain the plane inducedhomography used for planesweeping and discuss your choices for photo-consistencymeasure,depthsamplingstrategyandmultiviewcostaggregation
3)SegmentationThegoalofthisassignmentistodevelopaframeworktoperformsegmentationofaspecificforegroundobject.TheobjectofinterestisaforegroundtractorfoundintheimagesoftheCastle-19dataset.
Thedesiredoutputisapixelwisesegmentationforeachoftheimagesinthedataset.
This is an open-ended problem for which we seek a solution that balancessegmentation qualitywith the level of supervision required to carry out the task.Moreover, in designing your solution you may leverage the shared informationamongdatasets(e.g. Castle-P19hascommonsceneregionswithentry-P10,whiletheavailableCastle-P30datasetalsodepictstheforegroundobjectandcanbeseenas a superset of Castle-P19), the available 3D ground truth data for each dataset(providedasadense3Dmesh),orevenelicituserinputaspartofasemi-automatedsegmentationframework.Includeinyourreporttheattainedsegmentationandprovideadetaileddiscussionofyoursolution.
