Medical ScreeningThe government has a screening program for a potentially fatal medical condition which is thought to affect 1 in 1000 of the population. 100 000 people are tested. The test gives the correct result in 98 out of 100 patients.

People will naturally worry if they have a result which suggests they have the condition, but how much of this concern is misplaced?

Where do we start?• How worried should someone be if they have a

positive result?

– Extremely worried, – Very worried, – A little worried, – Not at all worried.

• Why?

• Do you think many will get a FALSE positive result? Explain why.

Some calculations that might help(i) How many of the 100 000 people tested would you

expect to have the condition?

(ii) How many of the 100 000 people tested would you expect to NOT have the condition?

(iii) How many people would be expected to get a positive test result?

False Positives• A ‘false positive’ is a result which suggests a person has

a condition when they actually don’t have it at all• Fill in the ‘tree diagram’ on the following slide, with

numbers, to see how many people will receive genuine positive results and how many will receive false positive results

• Remember that the test result is correct 98% of the time: – 98% of the time it says you do have it when you do– 98% of the time it says you don’t have it when you

don’t

Number tested100000Have the medical condition

???

Test Negative???

Don't have the medical condition???

Test Positive???

Test Negative???

1 in 1000

999 in 1000

98%

98%

2%

2%

False Positives

Number tested100000Have the medical condition

100

Test Negative2

Don't have the medical condition99900Test Positive1998

Test Negative97902

1 in 1000

999 in 1000

98%

98%

2%

2%

False Positives

• What percentage of the people who have received positive results actually have the condition?

• Does this surprise you?

• Should doctors use this test?• Is it better to worry people who don’t really have the

condition, or to miss people who do have it?

False Positives

Hit and RunThe police receive a report of a hit and run accident involving a taxi. Although it is dark, an eye witness is 90% sure it was a green cab.

There are 1000 taxis in the city, 10 are green and the rest are red.

Is it more likely the accident involved

a red or green taxi?

??%

??%

Taxis ???

Green ??

Involved ??

Not involved ??

Red ???

Involved ??

Not involved ??

Fill in the blanks:• Eye witness is 90% sure• There are 1000 taxi’s in the city• 10 are green and the rest are red

??%

??%

Hit and Run

90%

90%

Taxis 1000

Green 10

Involved 9

Not involved 1

Red 990

Involved 99

Not involved 891

Is it more likely the accident involved a red or green taxi?

10%

10%

Hit and Run

Drug TestingIn the USA, 20 000 air traffic controllers

undergo random drug testing.

The test is good but not perfect:• 96% of those who use drugs test positive• 93% of those that do not use drugs test negative

Based on previous figures the Federal Airline Authority believe that 99% of air traffic controllers are drug free.

Do you think it’s likely that people identified as positive by the test are guilty of taking drugs?

How is this problem different to the previous ones?

Assuming that the Federal Aviation Authority are correct – that 99% of pilots are drug free – what percentage of those testing positive are actually drug free?

Complete the tree diagram on the next slide and discuss your results.

Drug Testing

Number tested20000

Number expected to test Negative???

Number expected to be guilty of taking drugs

???Number expected to test Positive???

Number expected to test Negative???

??%

??%

??%

??%

??%

??%

Drug Testing

Number tested20000

Number expected to test Negative

18414Number expected to be guilty of

taking drugs200

Number expected to test Positive

192Number expected to test

Negative8

99%

4%

96%

93%

7%

1%

Drug Testing

Lie Detector TestA TV show uses a lie detector test to try to establish which one of two people is telling the truth.

Assume that just one of the two people in the dispute is not being honest, and it is equally likely to be either guest.

The American Polygraph Association claim the tests are 89% accurate for a single issue response.

Out of 1000 shows, how many people will be falsely accused of lying?

How many guests will receive the wrong result?

• Draw out a diagram to help you ascertain how many inaccurate results there are likely to be.

• What will happen on the shows if one or other or both of the results are inaccurate?

Lie Detector Test

Number tested2000Truthful1000Pass Lie Detector890

Fail Lie Detector110

Not Truthful1000

Pass Lie Detector110

Fail Lie Detector890

Lie Detector Test

Down’s Syndrome OccurrenceIn an article in 2009, a “top doctor” called for changes to the pre-natal screening for Down’s Syndrome.

To consider the concerns expressed by the doctor here are the most recent figures from the Office for National Statistics which show that there were approximately 730,000 babies born in the UK in 2012 and gives the approximate age of their mothers.

As stated in the article, screening for Down’s Syndrome is currently offered to all prospective mothers.

2012 Live Births In England and Wales

All ages Under 20 20-24 25-29 30-34 35-39 40-44 45 and over729,674 33,815 132,456 202,370 216,242 114,797 28,019 1,975

The current test has a false positive rate of about 3% according to information from the NHS.

To complicate matters further the chances of having a baby with Down’s Syndrome increases with the age of the mother. (Figures from the NHS) • 25 years of age has a risk of 1 in 1,250 • 30 years of age has a risk of 1 in 1,000 • 35 years of age has a risk of 1 in 400 • 40 years of age has a risk of 1 in 100 • 45 years of age has a risk of 1 in 30

How many false positives are there likely to be for the different age groups? Use this evidence to decide whether you think pre-test counselling is a good idea, explaining your response.

Down’s Syndrome Occurrence

Births:

The current test has a false positive rate of about 3%

Chances of having a baby with Down’s Syndrome increases with the age of the mother. • 25 years of age has a risk of 1 in 1,250 • 30 years of age has a risk of 1 in 1,000 • 35 years of age has a risk of 1 in 400 • 40 years of age has a risk of 1 in 100 • 45 years of age has a risk of 1 in 30

Down’s Syndrome Occurrence

Teacher notes: It’s a Risk

Pupils will be familiar with situations in which ‘risk’ is used, such as the use of ‘Lie Detector tests’ on day-time TV shows and athletes testing positive for the use of banned substances, but may be less familiar with the impact of test accuracy in medical tests.

Through these activities, pupils will gain a better understanding of false positives and how to interpret ‘risk'.

Teacher notes: It’s a Risk

There are 5 activities with a decreasing amount of support and structure.

It is important to give students time to think, discuss and absorb. Many of the results will challenge what they believe or suspect to be true.

Attaining an understanding of why there are so many false negatives is helpful.

It is suggested that students work in pairs or small groups in order to discuss their initial thoughts and then to make sense of the outcomes.

Activities could all be tackled within a lesson or each could be used as a starter activity in a series of lessons.

Teacher notes: Medical ScreeningWhere do we startMost students in the trials said very or extremely worried. The cause of this worry was generally stated as the 98% accuracy of the test.

Some calculations that might help• Leave a little time between each question for the students to think and

respond. • Students should attempt to produce a figure by calculation… the answers

will be confirmed in the next section ‘False Positives’

False Positives• Of the 2096 positive results, only 98 are genuine – that’s less than 5%• Why, with such an accurate test, are there so many false positives?• Because 98% of small amount (in this case 100) < 2% of much larger

amount (in this case 99900).• This is a key point so take some time to ensure that the students fully

appreciate it.

Teacher notes: Hit and Run

Eye witness testimony is notoriously inaccurate. In this case, the eye-witness is 90% sure it was a green taxi, which means that there’s a 90% chance of it being one of the green taxis and a 10% chance of it being one of the red taxis.

This is similar in structure to the ‘Medical Screening’ question, where there are a lot more items in the ‘non-target’ group. In this case there’s a lot more red taxis than green ones.

Out of the 108 taxis that could have been involved, 9 are green and 99 are red, so it’s far more likely that the taxi involved was actually red.

Teacher notes: Drug Testing

What is different in this problem? The probabilities are conditional this time. 96% for correctly identifying a drug user, but only 93% chance of correctly identifying a non-user.

Ask the students to fill in the blanks in the diagram and discuss the outcomes.

Pose the question: “If an air traffic controller has a positive result, how likely are they to actually have taken drugs?”

Of the 1578 likely to test positive, only 192 are likely to have done. That means approximately 88% of those testing positive in this test are actually innocent.

Teacher notes: Lie Detector TestThis is a relatively straight forward problem.

If students have tried the previous problems with some structure, then this would be a good one to let them try without teacher input and without being given a blank tree diagram.

In the 1000 shows, there are likely to be 110 false negatives and 110 false positives.

This would mean that on some shows the following could happen:• If one result is accurate and one is erroneous, then either both

people are found to be telling the truth or both people are found to be lying

• Occasionally, the results are completely the wrong way round – the liar is found to be telling the truth and the honest person is found to be lying

Teacher notes: Down’s SyndromeSensitivity must be shown with this content; this activity is simply about identification of a condition and allowing prospective mothers to be prepared.

Students should be encouraged to explore the information for themselves and justify their responses to the questions. It might be helpful to print out copies of the information slide for students to refer to more easily. Working with a partner or in a small group would be helpful for students to share the workload and to interpret their answers.

If students struggle to get started, encourage them to think about the following:• Number of Down’s Syndrome babies expected for each age group• Number of non-Down’s Syndrome babies expected for each age groupThen consider how many babies would fall into each of the following categories for each age group:

Correct test result Incorrect test result

Down’s baby

Non-Downs baby

Teacher notes: Down’s SyndromeAll Under 20-24 25-29 30-34 35-39 40-44 45+

ages 20Numbers 729,674 33,815 132,456 202,370 216,242 114,797 28,019 1,975Risk of Down's (1 in ……..) 1250 1250 1250 1000 400 100 30Expected Number of Children born with Down's 1144 27 106 162 216 287 280 66Expected Number of children Without Down's 33,788 132,350 202,208 216,026 114,510 27,739 1,909

Test False indication rate 3%False Positives 21856 1014 3971 6066 6481 3435 832 57False Negatives 34 1 3 5 6 9 8 2Correct positives 1110 26 103 157 210 278 272 64Correct negatives 706,674 32,774 128,380 196,142 209,545 111,075 26,907 1,852

Probability of a positive result being false 97.5% 97.5% 97.5% 96.9% 92.5% 75.4% 47.3%

Teacher notes: Down’s Syndrome

• Looking at the results it’s possible to conclude that mothers below 35 years of age are much more likely to get a false positive result.

• However, about half the children with Down’s Syndrome are born to mothers aged under 35; this is because they make up the largest proportion of mothers.

• Perhaps mothers should be made aware of the probability of a false positive, for their age group, and they can then make an informed choice of whether they want the test.

ReferencesAmerican Polygraph Association accessed 3-2-2014http://www.polygraph.org/section/resources/polygraph-validity-research

Lie Detector Test accessed 3-2-2014http://www.dailymail.co.uk/femail/article-1203070/Jeremy-Kyle-nearly-killed-The-horrifying-story-womens-decision-daytime-TVs-notorious-show.html

Down’s Syndrome article accessed 3-2-2014http://www.dailymail.co.uk/health/article-1223406/Doctor-calls-mothers-opt-Downs-Syndrome-test.html

Data from the ONS accessed 3-2-2014http://www.ons.gov.uk/ons/taxonomy/index.html?nscl=Births+by+Mother%27s+Area+of+Residence#tab-data-tables

NHS information accessed 3-2-2014http://www.nhs.uk/news/2013/06June/Pages/New-Downs-syndrome-blood-test-more-reliable.aspxhttp://www.nhs.uk/Conditions/Downs-syndrome/Pages/Causes.aspx