1

Topic 3:Introduction to

Probability

2

Contents1. Introduction2. Simple Definitions3. Types of Probability4. Theorems of Probability5. Probabilities under conditions of statistically

independent events6. Probabilities under conditions of statistically

dependent events7. Counting Rule8. permutation9. combinations

3

Introduction If an experiment is repeated under essentially

homogeneous & similar conditions we generally come across 2 types of situations:

Deterministic/ Predictable: - The result of what is usually known as the ‘outcome’ is unique or certain.

Example:- The velocity ‘v’ of a particle after time ‘t’ is given by

v = u + atEquation uniquely determines v if the right-hand quantities are known.

4

Unpredictable/ Probabilistic: - The result is not unique but may be one of the several possible outcomes.

Examples: -

(i) In tossing of a coin one is not sure if a head or a tail will be obtained.

(ii) If a light tube has lasted for t hours, nothing can be said about its further life. It may fail to function any moment.

5

Simple Definitions

Trial & Event

Example: - Consider an experiment which, though repeated under essentially identical conditions, does not give unique results but may result in any one of the several possible outcomes.

Experiment is known as a Trial & the outcomes are known as Events or Cases. Throwing a die is a Trial & getting (1,2,3,…,6) is an

event. Tossing a coin is a Trial & getting Head (H) or Tail (T)

is an event.

6

Exhaustive Events: - The total number of possible outcomes in any trial. In tossing a coin there are 2 exhaustive cases, head &

tail. In throwing a die, there are 6 exhaustive cases since

any one of the 6 faces 1,2,…,6 may come uppermost.

Experiment Collectively Exhaustive EventsIn a tossing of an unbiased coin Possible solutions – Head/ Tail

Exhaustive no. of cases – 2In a throw of an unbiased cubic die

Possible solutions – 1,2,3,4,5,6Exhaustive no. of cases – 6

In drawing a card from a well shuffled standard pack of playing cards

Possible solutions – Ace to KingExhaustive no. of cases – 52

7

Favorable Events/ Cases: - It is the number of outcomes which entail the happening of an event. In throwing of 2 dice, the number of cases favorable to

getting the sum 5 is:(1,4), (4,1), (2,3), (3,2).

In drawing a card from a pack of cards the number of cases favorable to drawing an ace is 4, for drawing a spade is 13 & for drawing a red card is 26.

Independent Events: - If the happening (or non-happening) of an event is not affected by the supplementary knowledge concerning the occurrence of any number of the remaining events. In tossing an unbiased coin the event of getting a head in

the first toss is independent of getting a head in the second, third & subsequent throws.

8

Mutually exclusive Events: - If the happening of any one of the event precludes the happening of all the others. In tossing a coin the events head & tail are mutually

exclusive. In throwing a die all the 6 faces numbered 1 to 6 are

mutually exclusive since if any one of these faces comes, the possibility of others, in the same trial, is ruled out.

Experiment Mutually Exclusive EventsIn a tossing of an unbiased coin

Head/ Tail

In a throw of an unbiased cubic die

Occurrence of 1 or 2 or 3 or 4 or 5 or 6

In drawing a card from a well shuffled standard pack of playing cards

Card is a spade or heartCard is a diamond or clubCard is a king or a queen

9

Equally likely Events: - Outcomes of a trial are said to be equally likely if taken into consideration all the relevant evidences, there is no reason to expect one in preference to the others. In tossing an unbiased coin or uniform coin, head or tail are

equally likely events. In throwing an unbiased die, all the 6 faces are equally likely

to come.

Experiment Collectively Exhaustive EventsIn a tossing of an unbiased coin

Head is likely to come up as a Tail

In a throw of an unbiased cubic die

Any number out of 1,2,3,4,5,6 is likely to come up

In drawing a card from a well shuffled standard pack of playing cards

Any card out of 52 is likely to come up

10

Probability: Probability of a given event is an expression of likelihood of occurrence of an event. Probability is a number which ranges from 0 to 1. Zero (0) for an event which cannot occur and 1 for an

event which is certain to occur.

Importance of the concept of Probability

Probability models can be used for making predictions. Probability theory facilitates the construction of

econometric model. It facilitates the managerial decisions on planning and

control.

11

Types of Probability

There are 3 approaches to probability, namely:

1. The Classical or ‘a priori’ probability2. The Statistical or Empirical probability3. The Axiomatic probability

12

Mathematical/ Classical/ ‘a priori’ Probability Basic assumption of classical approach is that the

outcomes of a random experiment are “equally likely”.

According to Laplace, a French Mathematician:“Probability, is the ratio of the number of ‘favorable’ cases to the total number of equally likely cases”.

If the probability of occurrence of A is denoted by p(A), then by this definition, we have:

Number of favorable cases mp = P(E) = ------------------------------ = ----

Total number of equally likely cases n

13

Probability ‘p’ of the happening of an event is also known as probability of success & ‘q’ the non-happening of the event as the probability of failure.

If P(E) = 1, E is called a certain event &if P(E) = 0, E is called an impossible event

The probability of an event E is a number such that 0 ≤ P(E) ≤ 1, & the sum of the probability that an event will occur & an event will not occur is equal to 1.

i.e., p + q = 1

14

Classical probability is often called a priori probabilitybecause if one keeps using orderly examples ofunbiased dice, fair coin, etc. one can state theanswer in advance (a priori) without rolling a dice,tossing a coin etc.

Classical definition of probability is not very satisfactory because of the following reasons: It fails when the number of possible outcomes of the

experiment is infinite. It is based on the cases which are “equally likely” and

as such cannot be applied to experiments where the outcomes are not equally likely.

Limitations of Classical definition

15

It may not be possible practically to enumerate all the possible outcomes of certain experiments and in such cases the method fails.

Example it is inadequate for answering questions such as: What is the probability that a man aged 45 will die within the next year?

Here there are only 2 possible outcomes, the individual will die in the ensuing year or he will live. The chances that he will die is of course much smaller than he will live.

How much smaller?

16

Relative/ Statistical/ Empirical Probability

Probability of an event is determined objectively by repetitive empirical observations/ Experiments. Probabilities are assigned a posterior.

According to Von Mises “If an experiment is performedrepeatedly under essentially homogeneous conditions andidentical conditions, then the limiting value of the ratio of thenumber of times the event occurs to the number of trials, as thenumber of trials becomes indefinitely large, is called theprobability of happening of the event, it being assumed that thelimit is finite and unique”. Example: - When a coin is tossed, what is the probability that the

coin will turn heads? Suppose coin is tossed for 50 times & it falls head 20 times, then

the ratio 20/50 is used as an estimate of the probability of heads of this coin.

17

Symbolically, if in n trials an event E happens m times, then the probability ‘ p’ of the happening of E is given by

In this case, as the number of trails increase probabilities of outcomes move closer to the real probabilities and tend to be real probabilities as the number of trails tends to infinity (a large number).

The empirical probability approaches the classical probability as the number of trails becomes indefinitely large.

mp = P(E) = Lim ----

N -> N

18

The Empirical probability P(A) defined earlier can never be obtained in practice and we can only attempt at a close estimate of P(A) by making N sufficiently large.

The experimental conditions may not remain essentially homogeneous and identical in a large number of repetitions of the experiment.

The relative frequency of m/N, may not attain a unique value, no matter however large N may be.

Limitations of Statistical/ Empirical method

19

The Axiomatic Approach

Modern theory of probability is based on the axiomatic approach introduced by the Russian Mathematician A. N. Kolmogorov in 1930’s.

Classical approach restricts the calculation of probability to essentially equally likely & mutually exclusively events.

Empirical approach requires that every question be examined experimentally under identical conditions, over a long period of time considering repeated observations.

Axiomatic approach is largely free from the inadequacies of both the classical & empirical approaches.

20

Given a sample space of a random experiment, the probability of the occurrence of any event A is defined as a set function P(A) satisfying the following axioms.

1. Axiom 1: - P(A) is defined, is real and non-negative i.e.,P(A) ≥ 0 (Axiom of non-negativity)

2. Axiom 2: - P(S) = 1 (Axiom of certainty)3. Axiom 3: - If A1, A2, …., An is any finite or infinite sequence of

disjoint events of S, then

n nP ( U Ai) = ∑ P( Ai )

i=1 i=1

21

Theorems of Probability

There are 2 important theorems of probability which are as follows:

The Addition Theorem and The Multiplication Theorem

22

Addition theorem when events are Mutually Exclusive

Definition: - It states that if 2 events A and B are mutually exclusive then the probability of the occurrence of either A or B is the sum of the individual probability of A and B.

Symbolically

The theorem can be extended to three or more mutually exclusive events. Thus,

P(A or B) or P(A U B) = P(A) + P(B)

P(A or B or C) = P(A) + P(B) + P(C)

23

Addition theorem when events are not Mutually Exclusive (Overlapping or Intersection Events)

Definition: - It states that if 2 events A and B are not mutually exclusive then the probability of the occurrence of either A or B is the sum of the individual probability of A and B minus the probability of occurrence of both A and B.

Symbolically

P(A or B) or P(A U B) = P(A) + P(B) – P(A ∩ B)

24

Multiplication theorem

Definition: States that if 2 events A and B are independent, then the probability of the occurrence of both of them (A & B) is the product of the individual probability of A and B.

Symbolically,

Probability of happening of both the events:

Theorem can be extended to 3 or more independent events. Thus,

P(A and B) or P(A ∩ B) = P(A) x P(B)

P(A, B and C) or P(A ∩ B ∩ C) = P(A) x P(B) x P(C)

25

How to calculate probability in case of Dependent Events

Case Formula

1. Probability of occurrence of at least A or B1. When events are mutually

2. When events are not mutually exclusive

2. Probability of occurrence of both A & B

3. Probability of occurrence of A & not B

4. Probability of occurrence of B & not A

5. Probability of non-occurrence of both A & B

6. Probability of non-occurrence of atleast A or B

P(A U B) = P(A) + P(B)

P(A U B) = P(A) + P(B) – P(A ∩ B)

P(A ∩ B) = P(A) + P(B) – P(A U B)

P(A ∩ B) = P(A) - P(A ∩ B)

P(A ∩ B) = P(B) - P(A ∩ B)

P(A ∩ B) = 1 - P(A U B)

P(A U B) = 1 - P(A ∩ B)

26

How to calculate probability in case of Independent Events

Case Formula

1. Probability of occurrence of both A & B

2. Probability of non-occurrence of both A & B

3. Probability of occurrence of A & not B

4. Probability of occurrence of B & not A

5. Probability of occurrence of atleast one event

6. Probability of non-occurrence of atleast one event

7. Probability of occurrence of only one event

P(A ∩ B) = P(A) x P(B)

P(A ∩ B) = P(A) x P(B)

P(A ∩ B) = P(A) x P(B)

P(A ∩ B) = P(A) x P(B)

P(A U B) = 1 - P(A ∩ B) = 1 – [P(A) x P(B)]

P(A U B) = 1 - P(A ∩ B) = 1 – [P(A) x P(B)]

P(A ∩ B) + P(A ∩ B) = [P(A) x P(B)] +

[P(A) x P(B)]

27

Problem

An inspector of the Alaska Pipeline has the task of comparing the reliability of 2 pumping stations. Each station is susceptible to 2 kinds of failure: Pump failure & leakage. When either (or both) occur, the station must be shut down. The data at hand indicate that the following probabilities prevail:

Station P(Pump failure) P(Leakage) P(Both)1 0.07 0.10 02 0.09 0.12 0.06

Which station has the higher probability of being shut down.

28

Solution

P(Pump failure or Leakage)= P(Pump Failure) + P(Leakage Failure)

– P(Pump Failure ∩ Leakage Failure)

Station 1: 0.07 + 0.10 – 0 = 0.17

Station 2: 0.09 + 0.12 – 0.06 = 0.15

Thus, station 1 has the higher probability of being shut down.

29

Probability Rules

30

Probabilities under conditions of Statistical Independence

Statistically Independent Events: - The occurrence of one event has no effect on the probability of the occurrence of any other event.

Most managers who use probabilities are concerned with 2 conditions.1. The case where one event or another will occur.2. The situation where 2 or more events will both occur.

31

There are 3 types of probabilities under statistical independence.

Marginal Joint Conditional

Marginal/ Unconditional Probability: - A single probability where only one event can take place.

Joint probability: - Probability of 2 or more events occurring together or in succession.

Conditional probability: - Probability that a second event (B) will occur if a first event (A) has already happened.

32

Example: Marginal Probability - Statistical Independence

A single probability where only one event can take place.

Example 1: - On each individual toss of an biased or unfaircoin, P(H) = 0.90 & P(T) = 0.10. The outcomes of severaltosses of this coin are statistically independent events too,even though the coin is biased.

Example 2: - 50 students of FST-MU drew lottery to see whichstudent would get a free trip to the Carnival at Dar es salaam.Any one of the students can calculate his/ her chances ofwinning as:

P(Winning) = 1/50 = 0.02

Marginal Probability of an EventP(A) = P(A)

33

The probability of 2 or more independent events occurring together or in succession is the product of their marginal probabilities.

Example: - What is the probability of heads on 2 successive tosses?

P(H1H2) = P(H1) * P(H2)= 0.5 * 0.5 = 0.25

The probability of heads on 2 successive tosses is 0.25, sincethe probability of any outcome is not affected by anypreceding outcome.

Joint Probability of 2 Independent EventsP(A then B) = P(AB)= P(A) * P(B)

Example: Joint Probability - Statistical Independence

34

We can make the probabilities of events even more explicit using a Probabilistic Tree.

1 Toss 2 Toss 3 Toss

H1 0.5 H1H2 0.25 H1H2H3 0.125T1 0.5 H1T2 0.25 H1H2T3 0.125

T1H2 0.25 H1T2H3 0.125T1T2 0.25 H1T2T3 0.125

T1H2H3 0.125T1H2T3 0.125T1T2H3 0.125T1T2T3 0.125

35

For statistically independent events, conditional probability of event B given that event A has occurred is simply the probability of event B.

Example: - What is the probability that the second toss of a fair coin will result in heads, given that heads resulted on the first toss?

P(H2|H1) = 0.5For 2 independent events, the result of the first toss have absolutely no effect on the results of the second toss.

Example: Conditional Probability - Statistical Independence

Conditional Probability for 2 Independent EventsP(B|A) = P(B)

36

Probabilities under conditions of Statistical Dependence

Statistical Dependence exists when the probability of some event is dependent on or affected by the occurrence of some other event.

The types of probabilities under statistical dependence are:• Marginal• Joint• Conditional

37

Consider the following example

Assume that a box contains 10 balls distributed as follows: -

3 are colored & dotted 1 is colored & striped 2 are gray & dotted 4 are gray & striped

Event Probability of Event

1 0.1Colored & Dotted2 0.1

3 0.1

4 0.1 Colored & Striped

5 0.1Gray & Dotted6 0.1

7 0.1

Gray & Striped8 0.1

9 0.1

10 0.1

38

It can be computed by summing up all the joint events in which the simple event occurs.

From example compute the marginal probability of the event colored.

It can be computed by summing up the probabilities of the two joint events in which colored occurred:

P(C) = P(C and D) + P(C and S)= 0.3 + 0.1=0.4

Marginal Probability - Statistically Dependent

39

Joint probabilities under conditions of statistical dependence is given by

• What is the probability that this ball is dotted and colored?Probability of colored & dotted balls =

P(D and C) = P(D|C) * P(D) = (0.75) * 0.5= 0.3 (Approximately)

Joint probability for Statistically Dependent EventsP(A and B) = P(A|B) * P(B)

Example: Joint Probability - Statistically Dependent

40

Given A & B to be the 2 events then,

What is the probability that this ball is dotted, given that it is colored?

The probability of drawing any one of the ball from this box is 0.1 (1/10) [Total no. of balls in the box = 10].

Conditional probability for Statistically Dependent EventsP(A ∩ B)

P(A|B) = ----------P(B)

Example: Conditional Probability - Statistically Dependent

41

We know that there are 4 colored balls, 3 of which are dotted & one of it striped.

P(D ∩ C) 0.3

P(D|C) = ------------- = ------

P(C) 0.4

= 0.75

P(D ∩ C) = Probability of colored & dotted balls (3 out of 10 --- 3/10)

P(C) = 4 out of 10 --- 4/10

The Multiplication Rule for Counting

Multiplication Rule for counting :In a sequence of n events in which the firstone has k1 possibilities of occuring and thesecond event has k2 and the third has k3,and so forth, the total possibilities of thesequence will be k1k2k3kn.

The Multiplication Rule for Counting – Example 1

A nurse has three patients to visit. Howmany different ways can she make herrounds if she visits each patient onlyonce?

The Multiplication Rule for Counting – solution 1

She can choose from three patients for the firstvisit and choose from two patients for thesecond visit, since there are two left. On thethird visit, she will see the one patient who isleft. Hence, the total number of differentpossible outcomes is 3 2 1= 6.

The Multiplication Rule for Counting – Example 2

Employees of a large corporation are to be issued special coded identification cards. The card consists of 4 letters of the alphabet. Each letter can be used up to 4 times in the code. How many different ID cards can be issued?

The Multiplication Rules for Counting – Solution 2

Since 4 letters are to be used, there are 4 spaces to fill ( _ _ _ _ ). Since there are 26 different letters to select from and each letter can be used up to 4 times, then the total number of identification cards that can be made is 26 2626 26= 456,976.

The Multiplication Rule for Counting – Example 3

The digits 0, 1, 2, 3, and 4 are to be used in a 4-digit ID card. How many different cards are possible if repetitions are permitted?

Solution 3: Since there are four spaces to fill and five

choices for each space, the solution is 5 5 5 5 = 54 = 625.

The Multiplication Rule for Counting – Example 4

What if the repetitions were not permitted in the previous example?

Solution 4:The first digit can be chosen in five ways. Butthe second digit can be chosen in only fourways, since there are only four digits left; etc.Thus the solution is 5 4 3 2 = 120.

Permutations Consider the possible arrangements of the letters

a, b, and c. The possible arrangements are: abc, acb, bac, bca,

cab, cba. If the order of the arrangement is important then we

say that each arrangement is a permutation of thethree letters. Thus there are six permutations ofthe three letters.

Permutations An arrangement of n distinct objects in a

specific order is called a permutation of the objects.

Note:To determine the number of possibilities mathematically, one can use the multiplication rule to get: 3 2 1 = 6 permutations.

Permutations Permutation Rule :The arrangement of n objects in a specificorder using r objects at a time is called apermutation of n objects taken r objects at atime. It is written asType equation here. nPrand the formula is given by

nPr = !!

Permutations - Example

How many different ways can a chairperson and an assistant chairperson be selected for a research project if there are seven scientists available?

Solution: Number of ways = 7P2 = 7! / (7 – 2)! = 7!/5! = 42.

Permutations - Example

How many different ways can four books be arranged on a shelf if they can be selected from nine books?

Solution: Number of ways=9P4 = 9! / (9 – 4)! = 9!/5! = 3024.

Combinations

Consider the possible arrangements of the lettersa, b, and c.

The possible arrangements are: abc, acb, bac, bca, cab, cba.

If the order of the arrangement is not importantthen we say that each arrangement is the same. We say there is one combination of the three letters.

Combinations Combination Rule :The number of combinations of of r objects from n objects is denoted by nCr and the formula is given by

nCr = !! !

Combinations – Example 1

How many combinations of four objects are there taken two at a time?

Solution 1Number of combinations is 4C2

4C2 = 4! / [(4 – 2)! 2!] = 4!/[2!2!] = 6.

Combinations – Example 2

In order to survey the opinions of customers atlocal malls, a researcher decides to select 5 mallsfrom a total of 12 malls in a specific geographicarea. How many different ways can the selectionbe made?Solution 2:Number of combinations is 12C5

12C5 = 12! / [(12 – 5)! 5!] = 12!/[7!5!] = 792.

Combinations – Example 3 In a club there are 7 women and 5 men. A

committee of 3 women and 2 men is to be chosen.How many different possibilities are there?

Solution 3Number of possibilities are as follows: (number of ways of selecting 3 women from 7) (number of ways of selecting 2 men from 5) =7C3 5C2 = (35)(10) = 350.

Combinations – Example 4 A committee of 5 people must be

selected from 5 men and 8 women. Howmany ways can the selection be made ifthere are at least 3 women on thecommittee?

Combinations – Solution 4 The committee can consist of 3 women and 2 men, or 4 women and 1 man, or 5 women.

To find the different possibilities, find each separately and then add them:

8C3 5C2 + 8C4 5C1 + 8C5 5C0

= (56)(10) + (70)(5) + (56)(1) = 966.

Application of Combinations in probability

A committee of 5 people must be selectedfrom 5 men and 8 women. What is theprobability of forming a committee whichconsist of at least 3 women?

MORE EXAMPLES

1. Suppose you want to buy two lines for your double lines mobilephone. How many choices do you have from vodacom, Tigo, Airtel and Zantel mobile companies?

2. If a box contains 2 black balls and 3 red balls and twoballs are selected at randomly. Find the probability that:

(a)both are red (b)both are of the same color (c)one is red and one is black

62