Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for...
Transcript of Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for...
![Page 1: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/1.jpg)
SiameseNeuralNetworkbasedGaitRecognitionforHumanIdentification
ChengZhang,WuLiu,Huadong Ma,Huiyuan FuBeijingUniversityofPostsandTelecommunications
ICASSP2016
![Page 2: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/2.jpg)
Outline
• Introduction• Proposedmethod– ConventionalCNNbasedGaitRecognition– SiameseNetworkbasedGaitRecognition
• Experiments• Conclusions
![Page 3: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/3.jpg)
Definition• Gait analysis is the systematic study of animal locomotion,
more specifically the study of human motion, using the eyeand the brain of observers, augmented by instrumentation formeasuring body movements, body mechanics, and theactivity of the muscles.
![Page 4: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/4.jpg)
Background
• Socialsecurity– Videobigdataandcameranetwork– Remotesurveillance– Identificationandattributeclassification
• Biometricauthenticationtechniques– Facialrecognition– Irisrecognition– Fingerprinttechnologies– Voiceverification– Handgeometry
![Page 5: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/5.jpg)
Characteristics
• Remoteaccessed– Itcanidentifysubjectsfromadistancewithoutinterruptingthesubject
• Robust– Eveninlowresolutionvideos,thegaitstillworkswell
• Security– Itisdifficulttoimitateorcamouflagehumangait
iris fingerprintface voice gait
![Page 6: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/6.jpg)
Whygaitworks?
• A plethora of technique and data continue to showthat a person’s walking is indeed unique
MurrayMP,DroughtAB,KoryRC.Walkingpatternsofnormalmen[J].TheJournalofBone&JointSurgery,1964,46(2):335-360.JohanssonG.Visualperceptionofbiologicalmotionandamodelforitsanalysis[J].Attention,Perception,&Psychophysics,1973,14(2):201-211.
![Page 7: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/7.jpg)
Challenges• Inconspicuousinter-classdifferencefromthedifferentpeople
• Thelargeintra-classvariationsfromthesameperson– Walkingspeeds– Viewpoints– Clothing– Belongings– Occasion normal clothes backpack
Gaitsilhouettesofdifferentsubject
![Page 8: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/8.jpg)
RecentEffortsandMajorDrawback• Model-basedmethods
– Extractinghumanbodystructurefromtheimages– Requiringahighresolutionaswellashighercomputationalcostand
arenotyetsuitableforoutdoorsurveillance
• Model-freemethods– Usingthewholemotionpattern/featuresofthehumanbody,and
performingrecognitionatlowerresolutions– Human-craftedgaitfeaturescanextremelyhardtobreakthrough
featurerepresentationbottleneckwhenfacingwiththegaitandappearancechanges
![Page 9: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/9.jpg)
GeneralStepsofOurSystem
![Page 10: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/10.jpg)
GaitEnergyImage
• Averagingofsilhouetteoveronegaitcycle– Representahumanmotionsequenceinasingleimagewhilepreservingtemporalinformation
– Robusttoincidentalsilhouetteerrorsinindividualimage
![Page 11: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/11.jpg)
ConventionalCNNbasedGR
• RetraintheCNNsonthegaitdataset– CNNsareabletolearndiscriminativefeatures– Fine-tuningfromapre-trainedmodel(e.g.,AlexNet)isagoodsolutiontosolvethedatalimitationproblemandspeeduptheconvergenceofnewmodel
– EmploytheAlexNet andonlychangethe1,000labeloutputtothenumberofsubjectsingaitdataset
![Page 12: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/12.jpg)
Problems
• Datalimitation– Tolearnsufficientfeatures,theCNNrequiresamassoftrainingdataforallcategories
– Forgaitrecognition,thenumberofsubjectscanbelarge,whilewithonlyafewexamplespersubjectinpublicdatabase
• Domaingap– Gaitrecognitionforhumanidentificationisessentiallyasearchproblembutnotclassification
![Page 13: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/13.jpg)
MetricLearning
= ≠
![Page 14: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/14.jpg)
ProposedFramework
• SiameseNeuralNetworkbasedgaitrecognition
![Page 15: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/15.jpg)
Sampling
• Trainingdataishighlyunbalanced– Usingasamplertogenerateequalnumberofpositiveandnegativeineachmini-batch,avoidoverlybiased towardstonegativedecisions
– Usingasamplertoenforcevarietytopreventoverfitting toalimitednegativeset
• Specially,thetrainingsetisselectedfromOULP-C1V1-A-Gallery dataset,with20,000similarGEIpairsandrandomlyselected20,000dissimilarpairs
![Page 16: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/16.jpg)
LossFunction
• ThedistancebetweenapairofGEIscanbemeasuredby:
• Wecandefinethecontrastivefunctionasfollows:
![Page 17: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/17.jpg)
TrainingandFeatureExtraction
• Supervisedsetting• MinimizedthecontrastivelossfunctionoveratrainingsetofNpatchpairsusingstochasticgradientdescent
• Experimentedwithdifferentparametersandgavethebestperformanceoffeaturerepresentation
![Page 18: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/18.jpg)
Experiments• Database:OU-ISIRLargePopulation
• Evaluation:Rank-1andRank-5identificationrates
• Baselines:STOAgaitrecognitionmethods,i.e.,GEI,FDF,HWLD,VTM,andRankSVM
• Pipeline:Backgroundsegmentation->Periodicidentification->GEIsgeneration->DNNtraining->DNNfeatureextraction->K-Nearest-Neighborsearching
![Page 19: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/19.jpg)
Database• OU-ISIRLargePopulationGaitDatabase
– Containstheworld’slargestnumberofsubjects– Recordstwosequencesforeachsubject:probe(query)andgallery
(source)sequence,offersfaircomparisontestbed
![Page 20: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/20.jpg)
Intra-viewrecognition
[9]H.Iwama,M.Okumura,Y.Makihara,andY.Yagi,“Theou-isirgaitdatabasecomprisingthelargepopulationdatasetandperformanceevaluationofgaitrecognition,”IEEETIFS.[7]Sivapalan,D.Chen,S.Denman,S.Sridharan,andC.Fookes,“Histogramofweightedlocaldirectionsforgaitrecognition,”inCVPRW,2013.
![Page 21: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/21.jpg)
SomeResults
![Page 22: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/22.jpg)
Inter-viewrecognition
D.Muramatsu,A.Shiraishi,Y.Makihara,M.Uddin,andY.Yagi,“Gait-basedpersonrecognitionusingarbitraryviewtransformationmodel,”IEEETIP,2015.R.Mart´ın-F´elez andT.Xiang,“Gaitrecognitionbyranking,”inECCV,2012.
![Page 23: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/23.jpg)
Conclusions• Wepresentoneofthefirstattemptstostudythedeepneural
networkbasedgaitrecognitionforhumanidentificationwithdistancemetriclearning
• Intheend-to-endframework,weleveragethecompetitiveGEIpresentationastheinputofnetworkwhileholisticallyexploittheSiameseneuralnetworktolearneffectivefeaturerepresentationsforhumanidentification
• Thecomprehensiveevaluationsshowthatweimpressivelyoutperformthestate-of-the-artsontheworld’slargestchallengegaitbenchmarkdataset
![Page 24: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/24.jpg)
FutureWorks
• 3-DimensionalSiameseneuralnetwork• Quasi-periodicorsub-framegaitrecognition• Unconstrainedenvironment,likeilluminationchanges,darkillumination,clutteredbackground,motionblur,andimagecompressionnoise
• …
“High’st Queenofstate,GreatJunocomes;Iknowherbyhergait”—— TheTempest[Act4Scene1],Shakespeare
![Page 25: Siamese Neural Network based Gait Recognition for Human ... · network based gait recognition for human identification with distance metric learning • In the end-to-end framework,](https://reader034.fdocuments.in/reader034/viewer/2022042802/5f3a828b89829d24097893f1/html5/thumbnails/25.jpg)
Anyquestions?