Image Segmentation Using Gm Ms

ALLAN CAINEADCAINE@CS.UWATERLOO.CA

HTTP: / /WWW.CS.UWATERLOO.CA/~ADCAINE/

MAY 28 , 2009

Chroma Keying using Gaussian Mixture Models (GMMs)

Allan Caine, http://www.cs.uwaterloo.ca/~adcaine/

Expectations

You should take notes in case this material appears on an exam or assignment, possibly in a different form

It is expected that you will use the Matlab software found at http://www.cs.uwaterloo.ca/~adcaine/ to generate a toy problem and solve it

These lecture notes are on-line at the above address

Inspiration

Have you ever wondered how is this done? Can we emulate this process on a computer?

Co-ordination

By viewing an off-camera monitor, theweather announcer can co-ordinate herhand movements with the background.

The result is a seemingly realistic televisionbroadcast, in spite of the fact that theannouncer is not actually in front of anyweather map.

Teaching Objectives

1. Describe how chroma keying (blue screening) works

2. Theorize how classification might solve the chroma keying problem on a computer

3. Enumerate the axioms of probability 4. Explain the concept behind digital images5. Express the intuition behind the Expectation-

Maximization algorithm in terms of a Gaussian Mixture Model (GMM), using the definition of a Gaussian.

6. Generate a toy GMM problem, and solve the toy problem, recovering the hidden variables

Outline

Chroma Keying and our AssumptionsRBG Color ModelAxioms of ProbabilityGaussian DistributionsGaussian Mixture Model (GMM)Bayes’ Rule and the Expectation StepUpdate Rules and the Maximization StepSolving a Toy ProblemSegmentation using a GMM and MasksFurther Work

Chroma Keying and our Assumptions

Basic Idea behind Chroma Keying

If the background is of one color, saygreen, and the foreground person is notof that color, then the person can besegmented from the background.

The person can then be superimposed on abackground of our choosing.

Normally, the chroma key is assumed to beknown. To make this problem moreinteresting, we will assume that the chromakey is unknown, and must be learned. So,it’s an AI problem.

Assumptions for the Segmentation Problem

1. We want to segment the foreground from the background

2. There is a background of a single color3. However, the background color is unknown;

the background color must be learned4. No part of the person or object in the

foreground has the background color5. The foreground person or object can be in

any position in the image; we cannot deduce the background color simply by examining a pixel in a certain fixed position

RBG Color Model

Explanation of RGB Color Model

Every color is expressed as an ordered triple

Each value can be between 0 and 255; i.e. 256 possibilities 8 bits can store 256 possibilities

The numbers represent the amounts of red, green, and blue in that order (R,G,B)

If R = G = B, then the pixel is a shade of gray

200 222 32 12

88 22 45 23

30 31 56 22

11 98 99 100

89 120 56 23

89 32 90 111

45 67 23 29

49 111 121 122

RGB as a Data Structure in Matlab

45 188 64 90

111 255 0 11

88 44 21 211

45 32 45 88

)12,23,90()32,56,64(

)222,120,188()200,89,45(

Scatter Diagrams

If only we could determine which class a pixel belonged to: foreground orbackground; the purple sphere or the green background, respectively.

Make your own Scatter Plot

Actual Scatter Plot

Axioms of Probability

(Some) Axioms of Probability

falseP

Bayesian Philosophy of Choosing

Suppose two events, A and B, occur with probability P(A) and P(B) such that P(A) + P(B) = 1.

If you had to place a bet on which event would occur and if P(A) ≠ P(B), what event would you bet on?

Suppose P(A) = P(B). What event would you bet on?

Gaussian Distributions

μxμxx

One-D Gaussian

Variations in μ

Variations in σ

2D Gaussian

Comparison of two 2D Gaussians

2D Gaussian with σx ≠ σy

3D Gaussian

Gaussian Mixture Model (GMM)

GMM Model

The number of classes is knownEach class has a particular mean and

varianceEach class has a mixing (or membership)

weight. All mixing weights add up to one

Toy Problem Parameters

ClassMixingWeights μ Σ

1 .2 (0.0 -0.1) 1.0 0.20.2 0.8

2 .4 (10.1 10.1)

2.6 1.81.8 1.9

3 .4 (9.9 -2.1) 3.3 1.6 1.6 2.1

Generating the Toy Problem’s DataSet

8.02.0

2.00.1)1.0,0.0(

9.18.1

8.16.2)1.0,0.10(

1.26.1

6.13.3)1.2,9.9(

The GMM Problem

??????)(??,

Bayes’ Rule and the Expectation Step

Bayes’ Rule

)()|()|(

BPBAPABP

The likelihood of A given B

A normalization constantThe likelihood of B given A

A prior probability on B

The Main of Idea of GMMs: E-Step

)()|()|(

iCPiCPiCP

The probability that position j is generatedby class 1 or 2: foreground or background

The probability of position j havinga pixel with values x given class i. Basically, evaluate a Gaussian at x using μi and Σi as parameters

The prior probability ofclass i; i.e. the mixingweights

Essentially, a normalization constantto ensure that P(C=1|xj) +P(C=2|xj) = 1for j fixed.

)()|()|(

BPBAPABP

Update Rules and the Maximization Step

Update Rules: M-Step

records ofnumber

Tijjij

Update

Define

μxμxΣ

iWhat do you think is the intuitionbehind these three update rules?

In words, define what each ofthese mathematical statementsmean.

Solving a Toy Problem

Toy Problem Visually

1 .2 (0.0 -0.1) 1.0 0.20.2 0.8

2 .4 (10.1 10.1)

2.6 1.81.8 1.9

3 .4 (9.9 -2.1) 3.3 1.6 1.6 2.1

Typical Output

Class 1Class 3

Class 2

Solution to the Toy Problem

Comparison of Statistics

ClassToy Problem GMM Solver

Mix μ Σ Mix μ Σ

1 .2 (0.0 -0.1) 1.0 0.20.2 0.8 .2 (0.0 -0.1) 1.0 0.2

0.2 0.8

2 .4 (10.1 10.1)

2.6 1.81.8 1.9 .4 (10.1

10.1)2.6 1.81.8 1.9

3 .4 (9.9 -2.1) 3.3 1.6 1.6 2.1 .4 (9.9 -2.1) 3.0 1.6

1.6 2.1

Recovery of the Model

3D GMM Problem

Segmentation using a GMM and Masks

Segmentation Example

Segmentation

The upshot of all of this is that the foreground has a parameterization of:

Background 0.9695 (62 195

137 42 159

42 149 49159 49

Foreground 0.0305 (122 35

167 26 191

26 134 30191 30

The novelty of this result is that we have a complete and numericalcharacterization of the image without knowing beforehand thecolor of the background. These parameters have been learned.

Producing the Mask

)2()2|()1()1|(

)1()1|()|1(

CPCPCPCP

CPCPCP

The mixing weight 0.9695

The mixing weight 0.0305

A Gaussian computed using themean and variance of the foregroundclass.

A Gaussian computed using themean and variance of the backgroundclass.

How would we use a Bayesian philosophy to decide ifa pixel represents the foreground or background?

This mask permits the sphereto show through while excludingthe background. It’s used tosegment the sphere; that is,segment out the foreground.

This mask is the logical complementof the first mask. It indicates wherethe background pixels can be found.This mask is used to prepare the newbackground so that it may becombined with the foreground.

Using the Masks for the Foreground and Background

Adding the Images Together

Before and After

Before After

Further Work

Complex foregrounds and backgroundsTracking objectsSpeeding up processingSamplingSmoothing the transition from background to

foreground to avoid ragged edgesUsing alternative color spaces such as Hue-

Saturation-Value (HSV) for better classification

Image Segmentation Using Gm Ms

Documents

Transcript of Image Segmentation Using Gm Ms

GM Parts Workbench Users Guide. GM Security.jarviscomputer.com/jcs/gmpwbug.pdf · GM Parts Workbench Users Guide. GM Security. GM Parts workbench is a system set up by GM SPO to allow

· PSA Analysis Classification from Photography Classification from Vibrocorer Greater )ublin 30 (g)M gM (g)sM msG msG (g)ms ms SAND Mud Sandy mud o MUD (g)M (g)sM gM (g)ms gmS mG

Spatial Decision Forests for MS Lesion Segmentation in Multi-Channel Magnetic ... · 2011-09-23 · Spatial Decision Forests for MS Lesion Segmentation in Multi-Channel Magnetic Resonance

Life Insurance Customers segmentation using fuzzy … Insurance Customers segmentation using fuzzy ... Market segmentation; customer segmentation; ... Life Insurance Customers segmentation

NOTICE INVITING Re e-TENDER...49 Metalaxyl 35% WS 50 gm 100 gm 250 gm 500 gm 50 Tricyclazole 75% WP 50 gm 100 gm 120 gm 250 gm 500 gm 51 Tricyclazole 34.2% + Propiconazole 10.7% SE

inTernATiOnAL mArKeT fOr sOYBeAns cOeXisTence Of gm ...fdir.idei.fr/wp-content/uploads/2012/newsletter7.pdfthe economic impacts resulting from the regulation: i) market segmentation

Star Shape Prior for Graph-Cut Image Segmentationimagine.enpc.fr/~de-la-gm/cours/UPEM/projects/Star... · Star Shape Prior for Graph-Cut Image Segmentation Olga Veksler University

SLMP Communication (/E4) User’s Manual · GM GM Data Acquisition System ... SLMP Communication (/E4) User’s Manual ... Read cycle 100 ms/200ms/500ms/1s/2s/5s/10s/20s/30s/1min

chess24 Press Kit · PDF filechess24 | Press Kit | 06/2017 ... GM Artur Yusupov GM Jan Gustafsson GM Niclas Huschenbeth GM Timur Gareyev GM Ivan Saric GM Roeland Pruijssers IM Mark

9 de julio ed - · PDF file9 de Julio José Padula A Gm D7 D7 Gm G7 Cm D7 Gm D7 Gm (D7) Gm D7 D7 Gm 1. 2. G7 Cm D7 para el trío Gm D7 Gm Gm D7 B G Am

CAUTION GM-D8704 GM-D9704 · 2 Fuse 80A (GM-D8704) / 100A (GM-D9704) (sold separately) Each amplifier must be separately fused at 80A (GM-D8704) / 100A (GM-D9704). 3 Positive + terminal

Market Segmentation and Segmentation Strategies

Chapter 5 Market Segmentation - Taft University · 5-2 . Key Terms . Market Segmentation, Post Hoc Segmentation, Priori Segmentation, Benefit Segmentation, Psychographic Segmentation,

Traitement des Images et Vision Artificielle - IMAGINEimagine.enpc.fr/~de-la-gm/cours/UPEM/cours_image_3.pdf · Traitement des Images et Vision Artificielle Segmentation Martin de

Video Object Segmentation based on Pixel-level Annotated ... · Introduction The objective of Video Object Segmentation is ... or MS-COCO for image captioning and object instance

Blowers, Compressors & Turbos – Made by AERZEN...2004/02/05 · GM 60 sm, GM 90 sm, GM 150Sm, GM 240 sm, GM 400Lm, GM 570 Lm, GM 1080Lm, s.r.o. (Hi GLa/GMa/b 12.5 GLa/GMa/b 12.6

Phoneme Segmentation Practice - Ms. Moshkowski's ...moshkowskisclass.weebly.com/uploads/5/4/5/9/54599537/...Phoneme Segmentation Practice Created by Kinder Karla Thank you for downloading

Marketing 7 Segmentation & Targeting. 7.1 Segmentation and Targeting - 7 Bases for segmentation Requirements for effective segmentation Segmentation and.

precote 80.pdf · 2020. 12. 17. · GM-6175M GM-6194M GM-6193M WSS-M11P45-A1 ES20007-S100 DIN 267 part 27 DIN 267 part 27 DIN 267 part 27 Bosch N38A SR9.3 N38A SR9.5 MS-CC76 (Type

Segmentation of MS lesions using entropy-based EM ... › pdf › JBiSE20110800005_50615799.pdf · radiological criteria for MS include the number of lesion on the MRI, their locations