CS6825: Probability CS6825: Probability DistributionsDistributions

An IntroductionAn Introduction

Recall Probability PropertiesRecall Probability Properties

The probability of an event, say The probability of an event, say event A, is denoted P(A).event A, is denoted P(A).

All probabilities are between 0 and 1.All probabilities are between 0 and 1.

(i.e. 0 < P(A) < 1)(i.e. 0 < P(A) < 1) The sum of the probabilities of all The sum of the probabilities of all

possible outcomes must be 1.possible outcomes must be 1.

Remember our Shoe Examples Remember our Shoe Examples and the Probabilitiesand the Probabilities

46%

24.50%

18.50%6.50%

4.50% Nike

Adidas

ReebokAsics

Other

P(Nike) = 46/100 = .46P(Adidas) = 24.5/100 = .245P(Reebok) = 18.5/100 = .185P(Asics) = 6.5/100 = .065P(Other) = 4.5/100 = .045

Frequency distributionFrequency distribution To better understand the main features To better understand the main features

of the data, we portray the of the data, we portray the frequency frequency distributiondistribution which quickly reveals the which quickly reveals the shape of the data. shape of the data.

Frequency Distribution Frequency Distribution is a grouping is a grouping of data into mutually exclusive classes of data into mutually exclusive classes (categories for nominal data, ranks for (categories for nominal data, ranks for ranking, numerical ranges for interval ranking, numerical ranges for interval and ratio data) showing the number of and ratio data) showing the number of observations in eachobservations in each

Probability DistributionProbability Distribution

Probability Distribution is a Frequency Probability Distribution is a Frequency Distribution where the frequency is Distribution where the frequency is represented by the probability (scale 0 to represented by the probability (scale 0 to 1.0)1.0)

So, a probability distribution gives the So, a probability distribution gives the chance of every event (if you have a chance of every event (if you have a discrete variable) or the probability across discrete variable) or the probability across all ranges of events (if you have a all ranges of events (if you have a continuous variable).continuous variable).

Probability Distribution of our Shoe Probability Distribution of our Shoe ExamplesExamples

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Nike Adidas Reebok Asics Other

Steps to build frequency distributionSteps to build frequency distribution(Suggestions but not the rules)(Suggestions but not the rules)

1.1. Decide on the number of classesDecide on the number of classes Sometimes you have natural knowledge of your Sometimes you have natural knowledge of your

problem and can define the classes you care about. problem and can define the classes you care about. Like in our shoe example “Nike”, “Reebok”, etc.Like in our shoe example “Nike”, “Reebok”, etc.

Sometimes you do not know the number of classes Sometimes you do not know the number of classes you need: Network Router Working, Network Router you need: Network Router Working, Network Router Failed. But, what about Router working at X% Failed. But, what about Router working at X% capacity? What about Y% and Z%?capacity? What about Y% and Z%?

When you cant decide on your number of classes: When you cant decide on your number of classes: Good rule of thumb. To estimate the number of Good rule of thumb. To estimate the number of classes for “n” observations (test samples you have), classes for “n” observations (test samples you have), find the smallest number k, such that 2find the smallest number k, such that 2kk>n>n

Steps to build frequency distributionSteps to build frequency distribution2. Define your class: Determine the class interval or width.2. Define your class: Determine the class interval or width.

Not all classes are represented by distinct values like “Nike” in our Not all classes are represented by distinct values like “Nike” in our shoe example. Instead they may have a continuous range of values. shoe example. Instead they may have a continuous range of values. For example, think about a set of classes representing colors. You For example, think about a set of classes representing colors. You might have classes like “light blue” , “medium blue” and “dark blue”. might have classes like “light blue” , “medium blue” and “dark blue”. How are these blue classes separated? This is referred to as the class How are these blue classes separated? This is referred to as the class interval or width in this case with respect to the color wheel (or color interval or width in this case with respect to the color wheel (or color space).space).

Rule of Thumb: the class interval should be greater or equal than the Rule of Thumb: the class interval should be greater or equal than the difference between the highest and the lowest value divided by the difference between the highest and the lowest value divided by the number of classes. With no prior information can split a range of 0 to number of classes. With no prior information can split a range of 0 to X values between k classes by spans of X/k. Hence:X values between k classes by spans of X/k. Hence:

Class 1 = range 0 to X/kClass 1 = range 0 to X/k Class 2 = range X/k to 2X/kClass 2 = range X/k to 2X/k Class 3 = range 2X/k to 3X/kClass 3 = range 2X/k to 3X/k Class 4 = range 3X/k to 4X/kClass 4 = range 3X/k to 4X/k Class k = range (k-1)X/k to XClass k = range (k-1)X/k to X

Steps to build frequency distributionSteps to build frequency distribution(Suggestions but not the rules)(Suggestions but not the rules)

3. Tally the observations (test data) into the appropriate 3. Tally the observations (test data) into the appropriate classes.classes.

4. Count the number of tallies (items) in each class. 4. Count the number of tallies (items) in each class.

If doing Probability distribution (rather than just If doing Probability distribution (rather than just frequency), you simply divide the number of tallies of each frequency), you simply divide the number of tallies of each class by the total number of observations. This gives us an class by the total number of observations. This gives us an estimated probability for each classestimated probability for each class

EXAMPLEEXAMPLE

Professor X wishes prepare to a report Professor X wishes prepare to a report showing the number of hours per week showing the number of hours per week students spend studying. She selects a students spend studying. She selects a random sample of 30 students and determines random sample of 30 students and determines the number of hours each student studied last the number of hours each student studied last week.week.

Organize the data into a frequency distributionOrganize the data into a frequency distribution 15.0, 23.7, 19.7, 15.4, 18.3, 23.0, 14.2, 20.8, 15.0, 23.7, 19.7, 15.4, 18.3, 23.0, 14.2, 20.8,

13.5, 20.7, 17.4, 18.6, 12.9, 20.3, 13.7, 21.4, 13.5, 20.7, 17.4, 18.6, 12.9, 20.3, 13.7, 21.4, 18.3, 29.8, 17.1, 18.9, 10.3, 26.1, 15.7, 14.0, 18.3, 29.8, 17.1, 18.9, 10.3, 26.1, 15.7, 14.0, 17.8, 33.8, 23.2, 12.9, 27.1, 16.6.17.8, 33.8, 23.2, 12.9, 27.1, 16.6.

ExampleExample Step 1 Determine the number of classesStep 1 Determine the number of classes

• There are 30 observations so n=30.There are 30 observations so n=30.• 2255=32>30=32>30• Lets start with the assumption of at least 5 Lets start with the assumption of at least 5

classes, i.e., k=5 (p.s. this may not be valid classes, i.e., k=5 (p.s. this may not be valid but, is a place to start….there are much better but, is a place to start….there are much better ways than guessing like this and are referred ways than guessing like this and are referred to a the estimation of k or the number of to a the estimation of k or the number of modes in a multi-modal distribution…..this is modes in a multi-modal distribution…..this is an area of continued research)an area of continued research)

Example continuedExample continued

7.45

3.108.33

i

Step 2 Define ClassesStep 2 Define Classes• First determine the range of our possible class First determine the range of our possible class

values.values.• Min = 10.3Min = 10.3• Maximum = 33.8Maximum = 33.8• Interval range i:Interval range i:

round up interval to 5round up interval to 5• Set the lower limit of the first class at 10 hours Set the lower limit of the first class at 10 hours

(could make it 10.3)(could make it 10.3)• Class 1 = range 10 to 15Class 1 = range 10 to 15• Class 2 = range 15 to 20Class 2 = range 15 to 20• Class 3 = range 20 to 25Class 3 = range 20 to 25• Class 4 = range 25 to 30Class 4 = range 25 to 30• Class 5 = range 30 to 35Class 5 = range 30 to 35

AGAIN this like choosing k=5 Is a GUESS and may (willoften) be wrong.

There are much more informedtechniques than this…but, thisis simple for you as a beginnerto understand.

Example continuedExample continuedStep 3 &4 place and count Step 3 &4 place and count observationsobservations

Class 1 = 8 / 30Class 1 = 8 / 30 Class 2 = 11/ 30Class 2 = 11/ 30 Class 3 = 7 / 30Class 3 = 7 / 30 Class 4 = 3 / 30Class 4 = 3 / 30 Class 5 = 1 / 30Class 5 = 1 / 30

P(Class 1) = .26667P(Class 1) = .26667P(Class 2) = .36667P(Class 2) = .36667P(Class 3) = .23333P(Class 3) = .23333P(Class 4) = .1P(Class 4) = .1P(Class 5) = .03333P(Class 5) = .03333

TEST DATA:TEST DATA:15.0, 23.7, 19.7, 15.0, 23.7, 19.7, 15.4, 18.3, 23.0, 15.4, 18.3, 23.0, 14.2, 20.8, 13.5, 14.2, 20.8, 13.5, 20.7, 17.4, 18.6, 20.7, 17.4, 18.6, 12.9, 20.3, 13.7, 12.9, 20.3, 13.7, 21.4, 18.3, 29.8, 21.4, 18.3, 29.8, 17.1, 18.9, 10.3, 17.1, 18.9, 10.3, 26.1, 15.7, 14.0, 26.1, 15.7, 14.0, 17.8, 33.8, 23.2, 17.8, 33.8, 23.2, 12.9, 27.1, 16.6.12.9, 27.1, 16.6.

Normal DistributionNormal Distribution

If you have an outcome that is If you have an outcome that is continuous and is caused by lots of continuous and is caused by lots of independent factors it can probably independent factors it can probably be modeled with a Normal (aka be modeled with a Normal (aka Gaussian) distribution. The Normal Gaussian) distribution. The Normal distribution looks bell shaped and it distribution looks bell shaped and it is described by two values, a mean is described by two values, a mean and variance.and variance.

Using the Normal function is commonly a technique used. Sometimes aCombination of normal functions (called a mixture of normals) is used.However, there are many techniques to automatically figure this out aswell as to use other functions to model your distribution.

Note –even a single point can be a normal function with 0 variance...even If this a possibly foolish model.

Normal approximationNormal approximation The actual distribution of baby weights at The actual distribution of baby weights at

a hospital and approximated by Normal a hospital and approximated by Normal Distribution with mean=3400 grams Distribution with mean=3400 grams variance=variance=360000 (standard deviation = 600 360000 (standard deviation = 600 grams)grams)