Recognition of Traffic Lights in Live Video Streams on Mobile Devices
description
Transcript of Recognition of Traffic Lights in Live Video Streams on Mobile Devices
![Page 1: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/1.jpg)
Recognition of Traffic Lights in Live Video Streams on Mobile
Devices
Jan RotersXiaoyi JiangKai Rothaus
2011 IEEE Transactions on CSVT
![Page 2: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/2.jpg)
OutlineIntroductionProblemsSystem Architecture
IdentificationClassificationVideo AnalysisTime-Based Verification
Experiment ResultsEvaluationsConclusion
![Page 3: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/3.jpg)
IntroductionPeople with visual disabilities are limited in
mobility.Orientate pedestrians with zebra crossings at
intersectionsPortable PC with a digital camera and a pair of
auricular stereoPresent a system for mobile devices to help
sightless people cross roads.
![Page 4: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/4.jpg)
ProblemsProgram usage
Real world conditionsCamera resolutionDifferent appearances
![Page 5: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/5.jpg)
ProblemsThe scale of traffic lightsMany traffic lightsOccludedIlluminationRotation
![Page 6: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/6.jpg)
Pedestrian Lights in Germany
1) Installation2) Shape3) Color arrangement4) Circuitry5) Background
![Page 7: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/7.jpg)
Mobile Device & DatabasesNokia N95330MHZ ARM processor18Mb RAM320240
2 publicly available databaseGround truth segmentation was made manually
![Page 8: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/8.jpg)
System Architecture
1.
2.
3.
4.
![Page 9: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/9.jpg)
1. Localization
![Page 10: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/10.jpg)
Red and Green Color Filter(1/3)
1. Analyze the data
![Page 11: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/11.jpg)
Red and Green Color Filter(2/3)
2. Design the filter rules (ex : red traffic light)
The Gaussian distribution of the red cluster is defined by its mean color = (0.48,0.06,0.07) and has three eigenvectors
A color c = (r, g, b) is a red traffic light color when
![Page 12: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/12.jpg)
Red and Green Color Filter(3/3)
3. Optimize parameters different parameter settings for each color Use 300 images to train Measure the quality of each setting by TP, FP, FN
Recall = , Precision =
![Page 13: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/13.jpg)
Size/Circuitry FilterAssume the traffic light is 4 to 24 meters awayFixed camera focal length and possible aspect
ratios
1. Filter out regions that are too small or too large2. Vertical neighbor should not have different color
![Page 14: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/14.jpg)
Background Color FilterInspect the region under a red light candidate or
above a green light candidateIf there are no dark pixels within search region,
refuse this candidate
Search region
Search region
![Page 15: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/15.jpg)
Validation of LocalizationValidate the localization results with 201 images
Optimal Validationrecall precision recall precision
Red 76% 89.5% 71.8% 87%
green 85% 98.5% 83.3% 92.6%
Error = 33.7%
![Page 16: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/16.jpg)
2. ClassificationTLC is the broadestTLC has the smallest distance to the top of imageNo other traffic light has similar height with TLC
![Page 17: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/17.jpg)
Performance of Classification
Red GreenRecall 86.3% 86.3%
Precision 97.4% 98.1%
![Page 18: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/18.jpg)
![Page 19: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/19.jpg)
3. Video Analysis(1/2)Temporary OcclusionFalsified ColorsContradictory SceneRepeating Results
![Page 20: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/20.jpg)
3. Video Analysis(2/2)Find the motion vector between two frames
Use KLT tracker to track feature pointsOnly search in a small area around crucial traffic light
candidate (30 pixels in each direction)Correlate the features by using SAD
Search region
Crucial traffic light
Candidate region
Feature point
𝑡𝑖 −1 𝑡𝑖
![Page 21: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/21.jpg)
4. Time-Based VerificationReduce the false positive detections by comparing
2 kinds of resultsUse state queue with 4 scenarios
1) Identification and video analysis are both successful and the locations match with each other.
2) Identification and video analysis are successful but the locations are different.
3) Video analysis succeeds but identification fails.4) Video analysis fails but identification succeeds.
![Page 22: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/22.jpg)
Experiment Results and Compute at least 5 frames per secondAt least consecutive correct detection with the
same color
![Page 23: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/23.jpg)
Experiment Results
![Page 24: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/24.jpg)
EvaluationsReliability
Prevent false positive green light detection
![Page 25: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/25.jpg)
EvaluationsInteractivity
Temporal analysis reduce the interactivityThe feedback is normally given within 2 seconds
![Page 26: Recognition of Traffic Lights in Live Video Streams on Mobile Devices](https://reader036.fdocuments.in/reader036/viewer/2022062410/568163a5550346895dd4ab73/html5/thumbnails/26.jpg)
ConclusionThe system can be helpful on driver assistance
systemsLimited computational power on mobile devicesThe verification ideas can be improved