Introduction to Computer Vision Sebastian van Delden USC Upstate [email protected].
-
Upload
vivien-boone -
Category
Documents
-
view
213 -
download
0
Transcript of Introduction to Computer Vision Sebastian van Delden USC Upstate [email protected].
![Page 2: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/2.jpg)
What is Computer Vision?
The goal of Computer Vision (or Machine Vision) is:
to make useful decisions about real physical objects and scenes based on sensed images.
to construct scene descriptions from images.
![Page 3: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/3.jpg)
Issues
Sensing: How do sensors obtain images from the world?
Encoded Information: How do images yield information for understanding the 3D world, including the geometry, texture, motion, and identity of objects in it?
Representations: How to represent objects, their parts, properties, and relationships, in a computer?
Algorithms: How to process image information and construct descriptions of the objects and the world?
![Page 4: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/4.jpg)
Digital Images
A 2D image is a projection of a scene from a specific viewpoint; many 3D features are captured, some not.
A 2D arrangement of pixels (picture elements) with a fixed number rows and columns.
In grey scale images, each pixel is a single value of grey usually in the range [0…255] where 0 is black and 255 is white.
In color images, each pixel has color information associated with it RGB Color scheme – quantities of Red, Green and Blue True color uses 1 byte for Red, 1 for Green, and 1 for blue For many computer vision applications color is not needed.
![Page 5: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/5.jpg)
Digital Images cont…
Example digital image
with 8 x 8 block of pixels
from left eye.
![Page 6: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/6.jpg)
Numerous Applications
Image Database Querying Aerial Images and GIS Medical Imaging Processing Scanned Text Understanding a Scene of Parts Inspection applications Automated navigation Etc, etc, etc…
![Page 7: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/7.jpg)
Understanding a Scene of Parts A robot needs to classify (or inspect) parts
and act accordingly.
![Page 8: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/8.jpg)
Combining Multiple Images
Images with a constant background are subtracted to detect change over time.
Pixel differences at boundary reveals moving object and its shape
![Page 9: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/9.jpg)
Combining Multiple Images
Images can also be added to blend them together.
![Page 10: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/10.jpg)
Reality Check
Success is hard won Some potential issues:
Lighting Fluctuations or inadequacies Object positioning and occlusion Background noise or other un-important image
features. Good news
Industrial robotics usually provides a very controlled environment.
![Page 12: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/12.jpg)
Imaging Devices CCD Camera
Charge-coupled Device (CCD) Instead of chemical film that reacts to light (like 35mm
film), tiny solid cells convert light energy into electrical charge.
![Page 13: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/13.jpg)
Problem with Digital Images
Blooming
Difficult to insulate adjacent sensing elements.
Charge often leaks from hot cells to neighbors, making bright regions larger.
![Page 14: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/14.jpg)
Problem with Digital Images
Clipping or Wraparound Dark grid
intersections at left were actually brightest of scene.
In A/D conversion the bright values were clipped to lower values.
![Page 15: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/15.jpg)
Problem with Digital Images
Lens distortion distorts image
“Barrel distortion” of rectangular grid is common for cheap lenses ($50)
Precision lenses can cost $1000 or more.
Zoom lenses often show severe distortion.
![Page 16: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/16.jpg)
Problem with Digital Images
CCD Variations CCD sensors imperfections can cause different
reading for the same light intensity. Chromatic Distortion
Different light wavelength bent differently. Quantization Effects
Mixing and Rounding problems.
![Page 17: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/17.jpg)
Spatial Quantization Problems Both pixel size relative position make a difference.
Mixed pixel represents a mixture of intensity values in a real scene.
Small features can be lost or blended.
![Page 18: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/18.jpg)
Portable Bit Map Image (PGM) P2 means
ASCII gray Comments W=16; H=8 192 is max
intensity Can be made
with editor
![Page 19: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/19.jpg)
Compression
Lossless – if a decompression method exists to precisely recover the original image.
Lossy – information is lost during compression and cannot to recovered during decompression (JPG, GIF) GIF – (Graphics Interchange Format) only 8 bits used for
color; can contain transparency and animation. JPG – (Joint Photographic Experts Group) for high quality
images; considers human vision systems and uses discrete cosine transform and Huffman coding.
![Page 21: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/21.jpg)
Pixels and Neighborhoods
A binary image consists of only two intensities – 0 and 1 (or 0 and 255).
A binary image B can be obtained from a grayscale image I through an operation that selects a subset of image pixels as foreground pixels, the pixels of interest in an image. Everything else would be considered as background pixels.
000100100010000001111000100000010010001000
![Page 22: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/22.jpg)
Thresholding and Segmentation
Gray level thresholding is the simplest segmentation process.
Many objects or image regions are characterized by constant reflectivity or light absorption of their surface.
Thresholding is computationally inexpensive and fast.
Thresholding can easily be done in real time using specialized hardware
![Page 23: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/23.jpg)
Thresholding Background is black Healthy cherry is bright Bruise is medium dark
This Histogram shows two cherry regions (black background has been removed)
gray-tone values
pixelcounts
0 256
![Page 24: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/24.jpg)
Thresholding - Example
Original Image Thresholded Image (95)
![Page 25: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/25.jpg)
Thresholding Example
Over-segmentation (225) Under-segmentation (25)
![Page 26: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/26.jpg)
Algorithm
Thresholding algorithm Search all the pixels f(i,j) of the image f. An image element g(i,j) of the segmented image is an
object pixel if f(i,j) >= T, and is a background pixel otherwise
Correct threshold selection is crucial for successful threshold segmentation
Threshold selection can be interactive or can be the result of some threshold detection method
![Page 27: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/27.jpg)
Region Properties
Once a binary image has been processed we could obtain properties about the regions in the processed image.
Some of those properties are Area, centroid Measure of circularity and elongation
![Page 28: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/28.jpg)
Area and Centroid
![Page 29: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/29.jpg)
Connected Components
Components are objects that share at least one common neighbor (in 4- or 8- neighborhood).
Definition: A connected component labeling of binary image B is a labeled image LB in which the value of each pixel is the label of its connected component.
![Page 30: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/30.jpg)
![Page 31: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/31.jpg)
Recursive Labeling Algorithm Given a binary image B
Negate the image (make all 1-pixels to –1) Search and find a –1 pixel, label it and find its (4-
or 8-) neighboring pixels with –1 and assign the same label.
Recursively apply to resolve (merge or split) components. Increment label each time…
![Page 33: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/33.jpg)
Vision Input Sensor
A camera can be used to provide visual input data to the robot Objects in the input images must be represented
in the robot world coordinate system so that the robot can manipulate them
Two Approaches: Visual Servoing Calibration
![Page 34: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/34.jpg)
Visual Servoing
A target image is compared against the current image
An error vector is generated which indicates how the robot should be moved in order to minimized the error between the current and target images.
The machine is incrementally moved. PRO: No need for a mm/pixel ratio to be calculated.
Very nice!
![Page 35: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/35.jpg)
Visual Servoing Example: Camera
mounted to end-effector and end-effector must move so that circular piece in the center of the image.
![Page 36: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/36.jpg)
Camera Calibration
Common in Industry CON: Usually must be manually done and
can become un-calibrated over time. Steps:
Must calculate mm/pixel ratio Must train a reference frame that can be seen in
the input image
![Page 37: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/37.jpg)
Example Setup: Frame {f} has been training in the robot work area. Parts can coordinates in this area. Consider Z to be fixed. Note: cannot recover depth this way…
![Page 38: Introduction to Computer Vision Sebastian van Delden USC Upstate svandelden@uscupstate.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062804/56649ef25503460f94c04108/html5/thumbnails/38.jpg)
Same example as before, but this is what the camera is seeing: