MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA
description
Transcript of MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA
![Page 1: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/1.jpg)
MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING
AN ONBOARD INFRARED CAMERA
Dr. Gerard Medioni
Cheng Hua Pai
Yu Ping Lin
![Page 2: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/2.jpg)
Introduction
Input: Infrared runway sequence
Goal: Detect moving objects on runway
![Page 3: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/3.jpg)
Approach
We do it in two steps:1. Stabilize the sequence
2. Detect motion on the stabilized sequence
![Page 4: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/4.jpg)
Flow chart of the system
Runway Identification
Image Stabilization
Motion Detection
irefH ,
Blobs in motionUpdate Referenceframe?
Update Referenceframe
Yes
ReferenceFrame
![Page 5: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/5.jpg)
Stabilization
Issues: Planar region containing the runway Feature choice and matching Transformation between consecutive frames
![Page 6: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/6.jpg)
Stabilization
Approach Manually Label
planar region SIFT provides
sufficient and descriptive features
RANSAC to estimate best transformation
![Page 7: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/7.jpg)
Stabilization
Result:
Stabilized runway sequence
![Page 8: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/8.jpg)
Adaptive Reference Frame
Issues: For longer sequence
Small errors accumulate Big scale difference
Beginning of a Sequence End of a Sequence
![Page 9: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/9.jpg)
Adaptive Reference Frame
When to change reference frame? Check the lower edge length ratio
![Page 10: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/10.jpg)
Stabilization algorithm
Landing UAV image sequence
Manually labeled planar region
3. Match features to previous frame to establish correspondence
1. Extract SIFT features
2. Region of Interest
4. Use RANSAC to remove outliers and estimate homography
6. Warp to the reference frame
5. Update reference frame if necessary
input
output
Locally stabilized image sequence and for all sirefiH ,
![Page 11: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/11.jpg)
Adaptive Reference Frame
Result:
Original Sequence Locally Stabilized Sequence
![Page 12: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/12.jpg)
Detection module
Issues: Detection method Global intensity variation Noise
Moire in the sequence Poor stabilization Local intensity variation Random noise
![Page 13: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/13.jpg)
Detection
Approach: Use simple Gaussian background model
t = (1-) * (t-1) + * (It)t2 = (1-) * (t-1) 2 + * (It- t)
Foreground: More than 4t2 from mean
t
4t2 4t
2
Foreground Background
Intensity distribution of an image
![Page 14: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/14.jpg)
Global intensity variation
Approach: Compensate gain with affine transformation
[Yalcin 05]
Before compensation After compensation
ttttt bImI 1
![Page 15: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/15.jpg)
Noise reduction
Approach: Moire in the sequence
Compare 8 neighbouring background pixels Poor stabilization
Restabilize with gradient map (also SIFT)
To Gradient
![Page 16: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/16.jpg)
Noise reduction
Approach: Local intensity variation
Intensity normalization on the foreground pixels
Random noise Compare consecutive foreground masks
iifg
iifg
i bu= mI
With random noise Without random noise
![Page 17: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/17.jpg)
Detailed flow chart of Motion Detection Module
Locally Stablized Runway
Sequence
Reference Frame update?
Filter Runway
Intensity Normalization
Intensity compensated runway image
Image Subtraction
Foreground mask & Quality
Indicator
QI. Score
Noise reduction
Update Runway Filter
Runway Filter
Update Reference Frame
Reference Frame
Update Background Model
Background model
Foreground mask
Yes
No
Good
Bad
Motion Detection Module
Homographies Hi,ref
![Page 18: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/18.jpg)
Detection Result
Result:
Locally Stabilized Sequence Foreground mask
![Page 19: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/19.jpg)
Evaluation
Tested on 150 synthesized and 18 real-world sequences
Results (synthetic data):
Obj. size
![Page 20: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/20.jpg)
Conclusion
Detection affected by: Object speed and size Threshold parameters
Program limitation: Moving objects fade in and out Bad result near the end of the sequence
Future work: More test on larger dataset Speed improvement
![Page 21: MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA](https://reader035.fdocuments.in/reader035/viewer/2022062517/56813c49550346895da5c82d/html5/thumbnails/21.jpg)
Reference W. G. Chris Stauffer. Adaptive background mixture models for real-time tracking. 1999 IEEE
Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'99), 2:2246, 1999.
M. Fischler and R. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381-395, 1981.
C. Harris and M. Stephens. A combined corner and edge detector. Proceedings of The Fourth Alvey Vision Conference, (1):147-152, 1988.
T. G. R. Kasturi, O. Camps and S. Devadiga. Detection of obstacles on runway using ego-motion compensation and tracking of signicant features. Proceedings 3rd IEEE Workshop on Applications of Computer Vision, 1996 (WACV'96), pages 168-173, 1996.
D. G. Lowe. Distinctive image features from scale invariant keypoints. International Journal of Computer Vision, 60(2):91-110, 2004.
R. S. B. Sridhar and B. Hussien. Passive range estimation for rotor-craft low-altitude flight. Machine Vision and Applications, 6(1):10-24, 1993.
S. Sull and B. Sridhar. Runway obstacle detection by controlled spatiotemporal image Low disparity. IEEE Transactions on Robotics and Automation, 15(3):537-547, 1999.
R. C. H. Yalcin and M. Hebert. Background estimation under rapid gain change in thermal imagery. Second IEEE Workshop on Object Tracking and Classification in and Beyond the Visible Spectrum (OTCBVS'05), 2005.
Q. Zheng and R. Chellappa. Motion detection in image sequences acquired from a moving platform. Proc. Int. Conf. Acoustics, Speech, and Signal Processing, Minneapolis, 5:201-204, 1993.