An Introduction to An Introduction to Computer VisionComputer Vision

George J. Grevera, Ph.D.George J. Grevera, Ph.D.

The science of analyzing images and videos in The science of analyzing images and videos in order to recognize or just model 3D objects, order to recognize or just model 3D objects, persons, and environments.persons, and environments.

Computer VisionComputer Vision

How does the Sony AIBO dog find its way How does the Sony AIBO dog find its way “home” (to its charging stations)?“home” (to its charging stations)?

Computer VisionComputer Vision

How does the yellow, virtual first-down line How does the yellow, virtual first-down line work?work?

Computer VisionComputer Vision

How do cameras perform (digital) image How do cameras perform (digital) image stabilization?stabilization?

In this class, we study the underlying principles In this class, we study the underlying principles and produce working examples.and produce working examples.

Computer VisionComputer Vision

Visualization includes…Visualization includes…

Computer graphicsComputer graphics

Computer / Machine Computer / Machine visionvision

Image understandingImage understanding

Database and Database and communicationscommunications

Computer gamesComputer games

Medical imagingMedical imaging

Image processingImage processing

Pattern recognitionPattern recognition

Computer graphics/games vs. Computer graphics/games vs. computer visioncomputer vision

Computer graphics/games creates a 2D Computer graphics/games creates a 2D image from a 3D world/model.image from a 3D world/model.

3D to 2D3D to 2D

Computer graphics/games vs. Computer graphics/games vs. computer visioncomputer vision

Computer vision estimates a 3D Computer vision estimates a 3D world/model from a 2D image.world/model from a 2D image.

2D to 3D2D to 3D

ExamplesExamplesgray (b&w) and colorgray (b&w) and color

Ansel Adams: El CapitanAnsel Adams: El Capitan

Bill Brandt: Lambeth WalkBill Brandt: Lambeth Walk

Lewis HineLewis Hine

Lewis HineLewis Hine

George Grevera: Horse FishingGeorge Grevera: Horse Fishing

#1 Major problems in Computer #1 Major problems in Computer Vision: SegmentationVision: Segmentation

SegmentationSegmentation

Recognition: Is a t-shirt present?Recognition: Is a t-shirt present? Delineation: Can you accurately outline the t-Delineation: Can you accurately outline the t-

shirt (what size is it)?shirt (what size is it)?

Segmentation tasks:Segmentation tasks:

1.1. RecognitionRecognition Human is typically better.Human is typically better. more qualitativemore qualitative

2.2. DelineationDelineation Computer is typically better.Computer is typically better. more quantitativemore quantitative

Model buildingModel building

Models from CT (Computed Models from CT (Computed Tomography) head dataTomography) head data

3D visualization of CT head data3D visualization of CT head data

MRI Diffusion Tensor ImagingMRI Diffusion Tensor Imaging

#2 Major problem in Computer #2 Major problem in Computer Vision: RegistrationVision: Registration

RegistrationRegistration

A.K.A.:A.K.A.: alignmentalignment warpingwarping mosaicingmosaicing morphingmorphing fusionfusion

Simple MRI Example (rigid)Simple MRI Example (rigid)

DeformableDeformable

Thirion’s “Demons” algorithm applied to Thirion’s “Demons” algorithm applied to pre- and post-contrast MRI of the breast.pre- and post-contrast MRI of the breast. excellent resultsexcellent results

Deformable registration Deformable registration exampleexample

pre

post (no reg)

post(after Thirion’s

Demons registration)

diff

diff (after reg)

Thirion’s “Demons” algorithm applied to Thirion’s “Demons” algorithm applied to PET chest emission and transmission PET chest emission and transmission images.images. poor resultspoor results

PET transmission image

PET emission image

PET emission warped to match transmission

Deformable registration Deformable registration exampleexample

NON-MEDICAL NON-MEDICAL VISUALIZATIONVISUALIZATION

Red eye reductionRed eye reduction

What is a distance transform?What is a distance transform?

Input:Input: a a binarybinary image image

Output:Output: a grey imagea grey image for all points . . .for all points . . .

assign the minimum distance from that particular point assign the minimum distance from that particular point to the nearest point on the border of an objectto the nearest point on the border of an object

Applications of distance Applications of distance transforms:transforms:

skeletonization/medial axis transformskeletonization/medial axis transform interpolationinterpolation registrationregistration efficient ray tracingefficient ray tracing classification of plant cellsclassification of plant cells measuring cell wallsmeasuring cell walls characterize spinal cord atrophycharacterize spinal cord atrophy

Experimental ResultsExperimental Results

binary input image distance transform result

Application areas:

• Object recognition

• Tracking

• Registration

• Fusion

Intelligence, industrial and medical projects

• FBI Automatic Fingerprint Identification System

• FOCUS: Monitor change in satellite images

• FBI Facial Reconstruction

Software: Target Junior

Image UnderstandingImage Understanding

Image UnderstandingImage Understanding

Image UnderstandingImage Understanding

Image UnderstandingImage Understanding

Image UnderstandingImage Understanding

TextbookTextbook

L.G. Shapiro, G.C. Stockman, Computer L.G. Shapiro, G.C. Stockman, Computer Vision, Prentice-Hall, 2001.Vision, Prentice-Hall, 2001.

TopicsTopics

Imaging and image representationImaging and image representation SensorsSensors Problems (including noise)Problems (including noise) Image file formatsImage file formats

Color representation and shadingColor representation and shading Binary image analysisBinary image analysis

Connected componentsConnected components MorphologyMorphology Region propertiesRegion properties

TopicsTopics

Pattern recognition conceptsPattern recognition concepts Classifiers and classificationClassifiers and classification

Filtering (enhancing) imagesFiltering (enhancing) images

SegmentationSegmentation

Registration (matching)Registration (matching)

TopicsTopics

RegistrationRegistration

Texture representation and segmentationTexture representation and segmentation

Motion from sequences of 2D imagesMotion from sequences of 2D images

Homework #1Homework #1

Read chapter 1.Read chapter 1.

Hand in 1.1, 1.2, and 1.3.Hand in 1.1, 1.2, and 1.3.

Survey questions…Survey questions…

1. Do you have access to a digital camera?1. Do you have access to a digital camera?

2. Write a function that, given a 2D array, returns a 2. Write a function that, given a 2D array, returns a 1D array where each entry is the sum of the 1D array where each entry is the sum of the corresponding row in the 2D array.corresponding row in the 2D array. (So result[0] (So result[0] contains the sum of values in m for row 0, result[1] contains the contains the sum of values in m for row 0, result[1] contains the sum of values in m for row 1, etc.)sum of values in m for row 1, etc.)

Java:Java:

int[] sumOfRows ( int m[][], int rows, int cols ) {int[] sumOfRows ( int m[][], int rows, int cols ) {

……

}}

109 4 86 19178 7 88 83174 95 48 31170 33 66 71116 47 4 65174 20 87 6793 39 24 30136 33 72 31191 95 78 1819 6 10 3120 30 26 64167 64 23 8099 29 2 68179 34 89 56100 8 3 89