Computer vision: models, learning and inference

Computer vision: models, learning and inference

Chapter 6

Learning and Inference in Vision

Structure

22Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

• Computer vision models

– Two types of model

• Worked example 1: Regression

• Worked example 2: Classification

• Which type should we choose?

• Applications

Computer vision models

• Observe measured data, x• Draw inferences from it about state of world, w

Examples:

– Observe adjacent frames in video sequence– Infer camera motion

– Observe image of face– Infer identity

– Observe images from two displaced cameras– Infer 3d structure of scene


Regression vs. Classification

• Observe measured data, x

• Draw inferences from it about world, w

When the world state w is continuous we’ll call this regression

When the world state w is discrete we call this classification


Ambiguity of visual world

• Unfortunately visual measurements may be compatible with more than one world state w

– Measurement process is noisy

– Inherent ambiguity in visual data

• Conclusion: the best we can do is compute a probability distribution Pr(w|x) over possible states of world


Refined goal of computer vision

• Take observations x

• Return probability distribution Pr(w|x) over possible worlds compatible with data

(not always tractable – might have to settle for an approximation to this distribution, samples from it, or the best (MAP) solution for w)


Components of solution

We need

• A model that mathematically relates the visual data xto the world state w. Model specifies family of relationships, particular relationship depends on parameters q

• A learning algorithm: fits parameters q from paired training examples xi,wi

• An inference algorithm: uses model to return Pr(w|x) given new observed data x.


Types of Model

The model mathematically relates the visual data x to the world state w. Two main categories of model

1. Model contingency of the world on the data Pr(w|x)

2. Model contingency of data on world Pr(x|w)


Generative vs. Discriminative

1. Model contingency of the world on the data Pr(w|x)

(DISCRIMINATIVE MODEL)

2. Model contingency of data on world Pr(x|w)

(GENERATIVE MODELS)

Generative as probability model over data and so when we draw samples from model, we GENERATE new data


Type 1: Model Pr(w|x) -Discriminative

How to model Pr(w|x)?

1. Choose an appropriate form for Pr(w)

2. Make parameters a function of x

3. Function takes parameters q that define its shape

Learning algorithm: learn parameters q from training data x,w

Inference algorithm: just evaluate Pr(w|x)


Type 2: Pr(x|w) - Generative

How to model Pr(x|w)?

1. Choose an appropriate form for Pr(x)

2. Make parameters a function of w



Inference algorithm: Define prior Pr(w) and then compute Pr(w|x) using Bayes’ rule


Summary

Two different types of model depend on the quantity of interest:1. Pr(w|x) Discriminative

2. Pr(w|x) Generative

Inference in discriminative models easy as we directly model posterior Pr(w|x). Generative models require more complex inference process using Bayes’ rule


Structure



– Two types of model




• Applications

Worked example 1: Regression

Consider simple case where

• we make a univariate continuous measurement x• use this to predict a univariate continuous state w

(regression as world state is continuous)


Regression application 1:Pose from Silhouette


Regression application 2:Head pose estimation


Worked example 1: Regression


• we make a univariate continuous measurement x• use this to predict a univariate continuous state w

(regression as world state is continuous)








1. Choose normal distribution over w2. Make mean m linear function of x

(variance constant)

3. Parameters are f0, f1, s2.

This model is called linear regression.

Parameters are y-offset, slope and variance


Learning algorithm: learn q from training data x,y. E.g. MAP


Inference algorithm: just evaluate Pr(w|x) for new data x







1. Choose normal distribution over x2. Make mean m linear function of w

(variance constant)

3. Parameter are f0, f1, s2.


Learning algorithm: learn q from training data x,w. e.g. MAP


Pr(x|w) x Pr(w) = Pr(x,w)

Can get back to joint probability Pr(x,y)


Inference algorithm: compute Pr(w|x) using Bayes rule


Structure



– Three types of model




• Applications

Worked example 2: Classification


• we make a univariate continuous measurement x • use this to predict a discrete binary world

w

(classification as world state is discrete)


Classification Example 1:Face Detection


Classification Example 2:Pedestrian Detection


Classification Example 3:Face Recognition


Classification Example 4:Semantic Segmentation


Worked example 2: Classification


• we make a univariate continuous measurement x • use this to predict a discrete binary world

w

(classification as world state is discrete)




– Choose an appropriate form for Pr(w)

– Make parameters a function of x

– Function takes parameters q that define its shape










1. Choose Bernoulli dist. for Pr(w)2. Make parameters a function of x

3. Function takes parameters f0 and f1

This model is called logistic regression.

Two parameters

Learning by standard methods (ML,MAP, Bayesian)Inference: Just evaluate Pr(w|x)







1. Choose a Gaussian distribution for Pr(x)

2. Make parameters a function of discrete binary w

3. Function takes parameters m0,

m1, s2

0, s2

1 that define its shape



Learn parameters m0, m1, s2

0, s2

1 that define its shape

Structure







• Applications

Which type of model to use?

1. Generative methods model data – costly and many aspects of data may have no influence on world state


Which type of model to use?

2. Inference simple in discriminative models

3. Data really is generated from world –generative matches this

4. If missing data, then generative preferred

5. Generative allows imposition of prior knowledge specified by user


Structure







• Applications

Application: Skin Detection


Application: Background subtraction


Application: Background subtraction

But consider this scene in which the foliage is blowing in the wind. A normal distribution is not good enough! Need a way to make more complex distributions


Future Plan

• Seen two types of model

– Probability density function

– Linear regression

– Logistic regression

• Next three chapters concern these models


Conclusion


• To do computer vision we build a model relating the image data x to the world state that we wish to estimate w

• Three types of model• Model Pr(w|x) -- discriminative• Model Pr(w|x) – generative

Computer vision: models, learning and inference

Documents

Transcript of Computer vision: models, learning and inference