1. Introduction to Pattern Recognition and

Machine Learning.Prof. A.L. Yuille.

Dept. Statistics. UCLA.

Stat 231. Fall 2004.

Structure.

• Examples of Patterns.

• Discriminate/Decisions about Patterns.

• Schools of Pattern Recognition.

• Learning Theory.

What are Patterns?

• Laws of Physics & Chemistry generate patterns.

Patterns in Astronomy.

• Humans tend to see patterns everywhere.

Patterns in Biology.

• Applications: Biometrics, Computational Anatomy, Brain Mapping.

Patterns of Brain Activity.

• Relations between brain activity, emotion, cognition, and behaviour.

Variations of Patterns.

• Patterns vary with expression, lighting, occlusions.

Speech Patterns.

• Acoustic signals.

Goal of Pattern Recognition.

• Recognize Patterns. Make decisions about patterns.

• Visual Example – is this person happy or sad?

• Speech Example – did the speaker say “Yes” or “No”?

• Physics Example – is this an atom or a molecule?

Applications of Pattern Recognition.Handwritten digit/letter recognitionBiometrics: voice, iris, fingerprint, face, and gait recognitionSpeech recognitionSmell recognition (e-nose, sensor networks)Defect detection in chip manufacturingInterpreting DNA sequencesFruit/vegetable recognitionMedical diagnosisTerrorist DetectionCredit Fraud DetectionCredit Applications.… …

Two Extreme Approaches

• Generative Methods: Determine models of how patterns are formed. Use these models to perform discrimination. Pattern Theory. Grenander.

• Discriminative Methods: Don’t model pattern formation. Instead extract features from patterns and make

decision using these features.

Example: Salmon versus Sea Bass.

• Generative methods attempt to model the full appearance of Salmon and Sea Bass.

• Discriminative methods extract features sufficient to make the decision (e.g. length and brightness).

Fish Features. Length.

• Salmon are usually shorter than Sea Bass.

Fish Features. Lightness.

• Sea Bass are usually brighter than Salmon.

Decision Boundaries.

• Classify fish as Salmon or Sea Bass based on a decision boundary in feature space.

Generative Models for Speech.

• Stochastic Grammars for Speech & Natural Language. (Manning & Schutze).

Bayes Decision Theory

• Bayes Decision Theory gives a framework for Generative and Discriminative approaches.

• Current Wisdom: (i) Discriminative methods are simpler,

computationally faster, and easier to apply. (ii) Generative methods are needed for most

complex problems.• Hybrid methods are increasingly popular.• Stat 231 concentrates on Discriminative Methods

and simple Generative Models.• Other courses by Prof.s Zhu & Yuille deal with

complex Generative Models.

Learning Theory.

• Both Generative and Discriminative methods require training data to learn the models/features/decision rules.

• Machine Learning concentrates on learning discrimination rules.

• Key Issue: do we have enough training data to learn?

Course Elements.

• Bayes Decision Theory as theoretical basis.

• Simple discriminative and generative methods.

• Machine Learning.

• Advanced Discriminative Methods.