Image Processing Pattern Recognition Computer...
Transcript of Image Processing Pattern Recognition Computer...
1
Image ProcessingPattern Recognition
Computer VisionXiaojun Qi
Utah State University
Image Processing (IP)• Manipulate and analyze digital images
(pictorial information) by computer.
• Applications: The applications applied to almost every area of human activities– Biological Research, Defense/Intelligence, Document
Processing, Factory Automation, Law Enforcement, Medical Diagnostic Imaging, Photography, Astronomy, Image Database Retrieval and etc.
1. Biological Research• Automatic analysis of a biological example
(specimen analysis)• Bone, tissue, and cell analysis (counting and
classification)• Analysis, classification, and matching of DNA
material
2. Defense/Intelligence• Automatic interpretation of
earth satellite imagery• Recognize and track targets in
real time• Security and surveillance
3. Document Processing• Scanning, archiving, and
transmission of documents• Automatic detection and
recognition of printed characters
4. Factory Automation• Visual inspection and assembly• Industrial Inspection
5. Law Enforcement• Fingerprint feature extraction, classification,
and identification• DNA Matching
6. Medical Diagnostic Imaging• Digital Angiography• Skin Cancer Detection• Computed Tomography • Brain Tumor• Mammography (Breast
Cancer)
7. Photography• Add/Subtract objects
to and from a scene• Special effects
(Morphing, Warping)
2
8. Astronomy• Separating stars
from galaxies• Galaxy classification
9. Image Database Retrieval• Shape Retrieval• Color Retrieval• Texture Retrieval• Content-based Image
Retrieval Image query by example: Query Image (left), and two most similar images produced by an image database system
Pattern Recognition• Classify what inside of the image
• Applications:– Speech Recognition/Speaker Identification– Fingerprint/Face Identification– Signature Verification– Character Recognition– Biomedical: DNA Sequence Identification– Remote Sensing– Meteorology– Industrial Inspection– Robot Vision
Linear Classifier
Computer Vision
• Focus on view analysis using techniques from IP, PR and artificial intelligence (AI). It is the area of AI concerned with modeling and replicating human vision using computer software and hardware.
• Applications:– Robotics– Traffic Monitoring– Face Identification– 3D Modeling in Medical Imaging
Current Research-- Content-based Image Retrieval
and Annotation System• The driving forces
– Internet– Storage devices– Computing power
• Two approaches– Text-based approach– Content-based approach
3
• Input keywords descriptionsText-Based Approach Text-Based Approach
Index images using keywords• Advantages: (Google, Lycos, etc.)
– Easy to implement– Fast retrieval– Web image search (surrounding text)
• Disadvantages:– Manual annotation is not always available– Manual annotation is impossible for a large DB– Manual annotation is not accurate– A picture is worth a thousand words– Surrounding text may not describe the image
How to describe this image? Content-Based ApproachIndex images using low-level features
Content-Based Approach Index images using images
• Advantages– Visual features, such as color, texture, and
shape information, of images are extracted automatically
– Similarities of images are based on the distances between features
Query Formation
Visual Content
Description
Feature Vectors
Similarity Comparison
Image Database
Visual Content
Description
Feature Database
Relevance Feedback
Indexing & Retrieval
Retrieval results
user
output
Diagram for content-based retrieval system
4
A Data Flow Diagram CBIR is a highly interdisciplinary research area
CBIR Applications
• Commerce (fashion, catalogue, … …)• Biomedicine (X-ray, CT, ……)• Crime prevention (security filtering, … …)• Cultural (art galleries, museums, … …)• Military (radar, aerial, … …)• Entertainment (personal album, … …)
Open Problems• Nature of digital images: arrays of numbers• Descriptions of images: high-level concepts.
– Sunset, mountain, dogs, … …• Semantic gap
– Discrepancy between low-level features and high-level concepts
– High feature similarity may not always correspond to semantic similarity
– Different users at different time may give different interpretations for the same image.
Image Categorization-- High-Level Concepts
• What is image categorization– To label images into one or several predefined
categories (e.g., Dinosaur, Elephant, Horse, Bus, Building, etc.)
– To map low-level visual features to high level semantics.
• Challenges faced by automatic image categorization– Various imaging condition.– Complex and hard-to-describe objects.– Highly textured background.– Occlusions.
Common Techniques for Categorization
• General used techniques– Statistics– Support Vector Machines (SVMs)– Neural Networks– Multiple-Instance Learning (MIL)
Our Approach: Expand SVMs to multi-Category SVMs
5
Our Categorization Approach-- Feature Extraction
• Only global features are used to avoid the problems of inaccurate image segmentation
• Features include global color histogram and edge histogram
• HSV color space is used for the color histogram, which is one of the MPEG-7 color descriptors.
• MPEG-7 also defines the edge histogram descriptor (EHD), which captures the edge distribution in 16 non-overlapping sub-images.
• Based on the original EHD, we construct global EHD (gEHD) and semi-global EHD (sEHD).
• gEHD represent the edge distribution of the whole image.
• sEHD can be constructed as follows:
R3
C4C3C1 C2
R4
R2
1 2
3
4
5
R1
Our Categorization Approach-- Feature Extraction
Our Categorization Approach-- Multi-Category SVMs
• Radial Basis Function kernel is used• 3-fold cross-validation and grid-search
algorithm are used to decide the parameters C and .
• Pairwise coupling approach is used to handle the multiple category case.
• The output of the SVMs is also mapped to the probability so we can assign confidence to each labeled keywords.
γ
Our Categorization Results
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
0 1 2 3 4 5 6 7 8 9 10Category ID
Ave
rage
Acc
urac
yProp.1Prop.2ALIPHistSVM
Our Categorization Results
Horse: 96%Food: 1%
Building: 92%Beach: 3%
Beach: 47%Mountain: 40%
Vehicle: 31%Building: 25%
• Our system can classify images by a set of confidence values for each automatically labeled keywords.
Our Retrieval Results-- using both global and regional
features
(7) 0.8982 0.889922 0.8869 0.8856
0.8855 0.8844 0.8832 0.8826 0.8805 0.8786
(A) 10 matches out of 11, 18 matches out of 20
6
Our Retrieval Results
(B) 10 matches out of 11, 17 matches out of 20
(5) 0.8844 0.882922 0.8821 0.8810
0.8805 0.8783 0.878322 0.8750 0.8726 0.8702
Our Retrieval Results
(C) 9 matches out of 11, 14 matches out of 20
(3) 0.9531 0.916322 0.9088 0.9079
0.9069 0.9065 0.902922 0.9009 0.8957 0.8954
Our Retrieval Results
(D) 10 matches out of 11, 19 matches out of 20
(6) 0.9453 0.912622 0.9100 0.9093
0.9019 0.8970 0.895222 0.8944 0.8918 0.8904
Our Retrieval Results
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5 6 7 8 9 10 11
Category ID
Aver
age
Prec
isio
nProp.NFECRHisC
Average retrieval precision for 20 returned images
Our Retrieval Results
0.250.300.350.400.450.500.550.600.650.700.75
20 30 40 50 60 70 80 90 100
Number of Returned Images
Ave
rage
Pre
cisi
on
Prop.UFM
NFECRHisC
Average retrieval precision for different number of returned images
Image Semantics
• Image semantics may be related to objects in the image
• Semantically similar images may contain semantically similar objects
• Can a computer program learn semantic concepts about images based on objects?
7
Our Image Segmentation Approach
Sample Segmentation Results
Original Image 2 Regions 3 Regions 4 Regions
5 Regions 6 Regions 7 Regions
Original Image 2 Regions 3 Regions 4 Regions
Learning
• Semantically similar images may contain semantically similar objects.– Find similar objects (feature vectors) among
positive images– At the same time, they should be as distinct
from all objects in “negative” images as possible
• Conceptual feature vector:– Multiple-instance Learning (MIL) using diverse
density Learn which region represent the semantic meaning!
Example – Data Mining
• Three conceptual feature vectors– Water, Rock, Trees.
• Rule description of a semantic concept– If one of the regions is similar to water AND
one of the regions is similar to rock, then it is a waterfall image, OR
– If one of the regions is similar to water AND one of the regions is similar to trees, then it is a waterfall image.
What is What?
Current Research-- Shape Representation and Matching
8
Shape Representation
• Shape representation methods:– Region based– Boundary based
• Shape descriptors:– Fourier descriptors– Moments– Chain codes– Etc.
Our Shape Representation and Matching Approach
• Shape indexing: – global signature construction– local signature construction
• Shape retrieval:– calculate similarity score using global
signature– calculate similarity score using local signature– Use a fuzzy method to combine the scores.– Retrieval results are those with higher scores
Our Shape Retrieval Results Current Research-- Face Detection
• Face detection: To determine whether or not there are any faces in the arbitrary still images with cluttered background and to return the image location and extent of each face if present
• Significance:– The most important first step of face identification
series. – It is the preprocessing of face recognition, face
tracking, etc.
Our Approach1. Apply color quantization and segmentation to
preprocess the original image.2. Apply a skin model to find possible skin regions.3. Apply morphological processing to remove noise.4. Merge skin regions if needed.5. Apply some constraints to eliminate non-faces.6. Apply wavelet packet to extract features.7. Apply the neural networks to classify face and
non-face.8. Solve overlapping areas.
Our Results
9
Current Research-- Digital Watermarking and steganography
• Watermarks: Secret messages used for protecting copyrights of digital multimedia data (images, audio, and video)– Content and/or authentication– For detecting unauthorized copies of images
• Characteristics: Imperceptible, security, robustness, and blindness.
• Common Techniques Used: Spatial Domain Approach, Frequency Domain Approach, and Hybrid Approach.
Watermark
Our Watermarking Results-- Wavelet-based Approach
Our Watermarking Results-- Content-based Approach
Stegnography• Steganography: A way of hiding a classified
message.
• Cover image + classified message = StegoObject Transmit over an insecure communication channel (Internet)
• The designated recipient will retrieve the classified message from the stego object, while others do not know the existence of the classified message in the “innocent” looking stego object.
10
Our Stegnography Approach-- Preliminary Test
• Study the OutGuess approach
• Study the JPEG images
• Study the characteristics of the DCT coefficients
• Study several attacks– Histogram analysis– statistics– OutGuess attacks
2χ
Current Research-- Vision-based Navigation (Road Detection)
Our Approach-- Preliminary Test
• Apply Principal Component Analysis (PCA) to find the remote scene.
• Apply Bayes statistics to learn the road features.
• Apply deformable templates to get rid of the shadows.
• Apply the curve functions to approximate the road.
Research Interests• Speech Recognition• Intrusion Detection (One Student)• Gene Sequence Analysis (One Student)• Multi-media Data Mining• Time/Spatial Data Mining• Visualization• Microarray Analysis• Network simulation (One Student)