1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman...
-
date post
22-Dec-2015 -
Category
Documents
-
view
218 -
download
1
Transcript of 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman...
![Page 1: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/1.jpg)
1
Automated Rivet Inspection System for Aging Aircrafts
Unsang Park, Lalita Udpa, George C. Stockman
Computer Science and Engineering
Michigan State University
![Page 2: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/2.jpg)
2
Contents
Nondestructive Inspection (NDI)Magneto-optic Imager in NDIMotion-based Filtering (MBF)Real-time implementation of MBFAutomated rivet inspection systemRivet detectionRivet classificationResults and conclusionsFuture work
![Page 3: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/3.jpg)
3
Nondestructive Inspection for Aircrafts
Detect subsurface defects
Crack
Rivet
Seam
![Page 4: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/4.jpg)
4
Nondestructive Inspection for Aircrafts
Increase service life of airplane
Prevent disasters
Aloha Airlines B-737-200 lost part of its front fuselage during a flight in Hawaii, 1985
![Page 5: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/5.jpg)
5
Magneto-optic Imager (MOI)
CCD Camera
Induction Sheet
MO Sensor
Sample
Bias Coil
Light SourcePolarizer Analyzer
Eddy current excitationMagneto-optic sensingImaging
![Page 6: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/6.jpg)
6
Magneto-optic Imager (cont.)
Produce real-time analog images of inspected part
Images both surface breaking and subsurface
cracks
Easy to interpret with minimal training
Applicable both on conducting samples as well as
composites by tagging with ferromagnetic
particles
![Page 7: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/7.jpg)
7
Sample MOI Images
Crack along seam Crack between two rivets,
Radial crack on a rivet
Corrosion dome
![Page 8: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/8.jpg)
8
Drawback of MOI images
MOI image contains serpentine pattern
noises due to the magnetic domain walls
in magneto-optic sensor
Rivet
Signals due to domain walls
![Page 9: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/9.jpg)
9
Motion-based Filtering (MBF)
Additive Frame Subtraction
Moving direction of objects
Moving direction of MOI
In-5 In-4 In-3 In-2 In-1 In
D1D2D3D4D5
Sn
),(),(),(
)},({),(1
yxIyxIyxD
yxDMAXyxS
nini
iwi
n
![Page 10: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/10.jpg)
10
Motion-based Filtering (cont.)
Preprocessing RGB to Gray
Additive Frame Subtraction
Threshold
Median Filter
Stretch
Post processing
Input Image
Output Image
![Page 11: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/11.jpg)
11
MB filtered images
FilteredOriginal
![Page 12: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/12.jpg)
12
MB filtered images (cont.)
FilteredOriginal
![Page 13: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/13.jpg)
13
Real-time implementation of MBF
Experimental setup for proof of concept
Record to VHS
Collects MOI image Data
Record to movie file
Play on a Video player
Play on a PC monitor
Frame grabber
Web cameranoise
![Page 14: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/14.jpg)
14
Real-time implementation of MBF (cont.)
Data transfer rate4.6 Mbytes /sec ( 320240 pixels 16 bit 30 fps )
Data are down sampled as the input images are dropped while an image is processed
Diagram of real-time Motion-based Filtering
Sensing
Displaying
Imaging
RGB to Gray Subtract
ThresholdMedian FilteringStretch
Max
MOI Real-time MBF
Max
![Page 15: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/15.jpg)
15
Real-time implementation of MBF (cont.)
Optimizing MBF algorithm in C++
RGB to Gray: 20 ~ 23 ms
Additive Frame Subtraction: 1~2 ms/image
Threshold: 1 ~ 2 ms
Median Filter: 200 ~ 250 ms
Stretch: 1 ~ 2 ms
Image capture: 20 ms
Output Image
![Page 16: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/16.jpg)
16
Real-time implementation of MBF (cont.)
RGB to Gray conversion
Additive frame subtractionMAX(I1-I3,I2-I3) MAX(I1,I2) - I3
Median filter
Normal Table Lookup Table build time
320*240
16 bit image
20~23 ms 1~2 ms 5~6 ms (1M bytes)
5 by 5 7 by 7
MATLAB 50 ms 88 ms
Modified Quick Sort (C++) 200 ~ 250 ms 400 ~ 450 ms
Moving Median with Sorting (C++) 100 ~ 150 ms 150 ~ 200 ms
Moving Median with Histogram (C++)
20 ~ 25 ms 20 ~ 40 ms
![Page 17: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/17.jpg)
17
Real-time implementation of MBF (cont.)
Execution time of MBF algorithm in C++
Before optimizationIntel 2GHz, C++
After optimizationIntel 2GHz, C++
Capture 20 ms 20 ms
RGB to gray 20 ~ 23 ms 1 ~ 2 ms
Subtraction (x10)Max (x10)Threshold
1 ~ 2 ms (x10)1 ~ 2 ms (x10)
1~2 ms
1 ~ 2 ms (x10)1 ~ 2 ms
Median Filter (3x3) 200 ~ 250 ms 20 ~ 30 ms
Stretch 1 ~ 2 ms 1 ~ 2 ms
Total 262 ~ 337 ms 53 ~ 76 ms
![Page 18: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/18.jpg)
18
Drawbacks of current MOI inspection
No measure for quantitative interpretation Data interpretation is subjective
Manual inspection by human operator (more than 10 hours per airplane)
Expensive labor cost Error due to fatigue
![Page 19: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/19.jpg)
19
Automated MOI inspection system
PRI Research and Development Corporation (PRI) Developing and improving magneto-optic imager (MOI)
Michigan State University, ECE department Image processing algorithm for filtering and classification
Boeing Phantom Works Self-guided, suction cup robot – crawls over airplane skin
![Page 20: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/20.jpg)
20
Automated MOI inspection (cont.)
Currently focusing on radial cracks on rivets
Quantification of defects in MOI imagesImplementing real-time rivet inspection algorithm
Motion-based Filtering
Rivet detection
Rivet classification
![Page 21: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/21.jpg)
21
Rivet detection
Hough transformation-based method Circular Hough transformation
Morphological operation-based method
Obtain center and radius, c1, r1
Erode rivet with a circle of radius r1
Segment out each rivet
Area(rivet) = 0
Obtain center and radius, c2, r2
yes, r1 r1-1
no
![Page 22: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/22.jpg)
22
Rivet detection (cont.)
Original
Hough transformation
Morphological operation
![Page 23: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/23.jpg)
23
Rivet classification
Two-pass Hough transformation1st pass – Rivet detection
2nd pass – Blob detection
Original image Filtered image - After 1st pass
After 2nd Pass
good bad
![Page 24: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/24.jpg)
24
Rivet classification (cont.)
Bayesian classifier Feature selection
df
rcycxd yx
max
)()(
1
22
Hough transformation Morphological op.Original image
![Page 25: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/25.jpg)
25
Off-line test
Training 10 normal, 10 defective rivet images
Obtain mean and variance of feature f1
Testing 222 rivet images including 66 defective rivet images
Two-pass Hough Morph. - BayesHough - Bayes
![Page 26: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/26.jpg)
26
Experimental Results
Accuracies of three algorithms
Inspection
algorithm
Rivet detection
Rivet classificationFalse negative False positive
Two-pass Hough 1/242 0/242 90% (200/222)
Hough-Bayes 1/242 0/242 96% (214/222)
Morph.-Bayes 1/242 1/242 99% (220/222)
![Page 27: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/27.jpg)
27
Conclusions
MB filtered image is optimal in image processing for automated rivet inspectionMorphological operation-based rivet detection is superior to Hough-based rivet detection both for execution time and accuracyBayesian classifier is superior to Hough-based classifierRadial crack detection on rivets showed 99% accuracy in off-line test
![Page 28: 1 Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University.](https://reader033.fdocuments.in/reader033/viewer/2022051415/56649d7a5503460f94a5dfc2/html5/thumbnails/28.jpg)
28
Future work
Implement MBF and rivet inspection algorithms on the Digital Signal Processing (DSP) board
Improve robustness of the algorithms with the feedback from field test
Develop MOI inspection algorithms for other types of defects in aircrafts