Biology seminar

“One of the most striking characteristics of modern biology is the pervasiveness of statistical thinking. However, discussions with colleagues suggest that some teachers resist the idea that statistical thinking is a necessary part of learning science. As a result, many students, both majors and nonmajors, remain ignorant of this fundamental aspect of biological thought. Deprived of this knowledge, they continue to view the living world from a simplistic, deterministic perspective. They also fail to master crucial intellectual skills that should be used in everyday life.”

Kugler, Charles, Hagen, Joel, and Singer, Fred, “Teaching Statistical Thinking,” Journal of College Science Teaching, Vol. 32 No. 7, p. 434-439, May 2003

“We argue that basic statistical concepts are needed by educated people, but statistical calculations should be kept simple to maintain focus on the reasoning. Most importantly, statistical thinking should be explicitly taught in all parts of biology courses. Only by routinely using statistical reasoning in lecture, laboratory, and homework will students understand the importance of biological variation and the basis of many scientific conclusions.”

• Study involved 277 medical residents in 11 residency programs

• instrument developed to reflect the statistical methods and results most commonly represented in contemporary research studies

• Authors reviewed all 239 original articles published from January to March of 2005 in each issue of 6 general medical journals American Journal of Medicine, Annals of Internal Medicine, BMJ, JAMA, Lancet, and New England Journal of Medicine) and summarized the frequency of statistical methods described

• Questions developed based on this research

Biostatistics test of medical residents

To determine if fasting is associated with dengue fever, data from 40 patients with dengue fever were collected. These patients were matched for age, sex, and race to 40 patients without dengue fever. The hospital charts of these patients were then reviewed to determine whether they also fasted prior to their illness. This study type is known as:

A. cross-sectional studyB. concurrent cohort studyC. case-control studyD. retrospective cohort studyE. randomized clinical trial

Percentage with correct answer

39.4 (33.6-45.1)

Biostatistics test of medical residents

Any systematic error in the design, conduct, or analysis of a study that results in a mistaken estimate of an exposure’s effect on the risk of disease is called:

A.ConfoundingB.BiasC.InteractionD.Stratification

Percentage with correct answer

• 46.6 (40.7-52.4)

Biostatistics test of medical residents

Researchers designed a study looking at cardiovascular deaths comparing a new drug to placebo. They determined they would need 200 patients in each group to detect a 15% difference in cardiovascular end points given 90% power and a significance level of .01.

Which of the following changes would require the members to increase their sample size?

A. Aim to detect a difference of 20%B. Specify a power of 80%C. Use a significance level of .05.D. Aim to detect a difference of 10%.

Percentage with correct answer

• 30.3 (24.9-35.7)

Biostatistics test of medical residents

In a placebo-controlled study of the use of aspirin and dipyridamole to prevent arterial restenosis after coronary angioplasty, 38% of patients receiving the treatment had restenosis and 39% of patients receiving the placebo had restenosis. In reporting this finding, the authors stated that P>.05. This means

A. The chances are greater than 1 in 20 that a difference would be found again if the study were repeated.

B. The probability is less than 1 in 20 that a difference this large could occur by chance alone.

C. The probability is greater than 1 in 20 that a difference this large could occur by chance alone

D. The chance is 95% that the study is correct.

Percentage with correct answer

• 58.8 (53.0-64.6)

• Residents scored well on three questions• 87% correctly understood the purpose of

double-blinded studies• 82% correctly understood relative risk• Residents with prior training in biostatistics,

but still averaged below 50% correct

http://www3.interscience.wiley.com/journal/121432447/abstract

“In a 2007 campaign advertisement, former New York City mayor Rudy Giuliani said, ‘‘I had prostate cancer, 5, 6 years ago. My chance of surviving prostate cancer—and thank God, I was cured of it—in the United States? Eighty-two percent. My chance of surviving prostate cancer in England? Only 44 percent under socialized medicine’’ (Dobbs, 2007). For Giuliani, these health statistics meant that he was lucky to be living in New York and not in York, since his chances of surviving prostate cancer appeared to be twice as high. This was big news. As we will explain, it was also a big mistake.”

Survival rate:

Imagine a group of people all diagnosed with cancer at the same time.

5-yr survival rate =Number of patients diagnosed with cancer still

alive 5 years after diagnosis / number of people diagnosed with cancer

Giuliani’s data

• British study found that the diagnosis rate was 49 men per 100,000, and of these 28 died within 5 years. (Thus about 43% were still alive after 5 years.)

Mortality rate:

Imagine a specific group of people (British men, for example):

Annual mortality rate =number of people who die from cancer over 1

year /number of people in the group

Gigerenzer, et. al. p. 57

Absolute vs. relative riskThe contraceptive pill scare

“In October 1995, the U.K. Committee on Safety of Medicines issued a warning that third-generation oral contraceptive pills increased the risk of potentially life-threatening blood clots in the legs or lungs twofold—that is, by 100%. This information was passed on in ‘‘Dear Doctor’’ letters to 190,000 general practitioners, pharmacists, and directors of public health and was presented in an emergency announcement to the media.”

Gigerenzer, et.al., p. 54

The data

• Second generation pill: 1 in 7000 instances of thrombosis

• Third generation pill: 2 in 7000 instances of thrombosis• Relative risk increased 100%• Absolute risk of 3G pill: .00029• Impact: for five years before pill scare, number of

abortions had steadily declined about 5000 per year. Year after pill scare: abortions increased by 13000.

• Births by girls under 16 increased 800 that same year

Example of absolute risk reporting as required by the FDA

http://www1.astrazeneca-us.com/pi/Seroquel.pdf

MammographyTest conducted by one of the authors on 160 gynecologists at a

continuing education seminar:

Assume you conduct breast cancer screening using mammography in a certain region. You know the following information about the women in this region:

• The probability that a woman has breast cancer is 1% (prevalence)• If a woman has breast cancer, the probability that she tests positive

is 90% (sensitivity)• If a woman does not have breast cancer, the probability that she

nevertheless tests positive is 9% (false-positive rate = 1-specificity)

A woman tests positive. She wants to know from you whetherthat means that she has breast cancer for sure, or what thechances are. What is the best answer?

A. The probability that she has breast cancer is about 81%.B. Out of 10 women with a positive mammogram, about 9 have breast cancer.C. Out of 10 women with a positive mammogram, about 1 has breast cancer.D. The probability that she has breast cancer is about 1%.

Not sure? Need help from the audience?

Here are the responses from the doctors.

A. The probability that she has breast cancer is about 81%. (47% of the doctors selected A.)

B. Out of 10 women with a positive mammogram, about 9 have breast cancer. (13% selected B.)

C. Out of 10 women with a positive mammogram, about 1 has breast cancer. (21% selected C.)

D. The probability that she has breast cancer is about 1%. (19% selected D.)

Before you choose, suppose the data are presented like this:

Assume you conduct breast cancer screening using mammography in a certain region. You know the following information about the women in this region:

• Ten out of every 1,000 women have breast cancer• Of these 10 women with breast cancer, 9 test

positive• Of the 990 women without cancer, about 89

nevertheless test positive.

• What fraction of women who test positive actually have breast cancer?

• Out of 1000 women, 9+89 = 98 test positive.• Only 9 of these women actually have cancer.• The fraction is 9/98 = .092

Answer C is best. Answers A, B, and D are off by approximately an order of

magnitudeA. The probability that she has breast cancer is

about 81%.B. Out of 10 women with a positive

mammogram, about 9 have breast cancer.C. Out of 10 women with a positive

mammogram, about 1 has breast cancer.D. The probability that she has breast cancer is

about 1%.

Gigenrenzer, et.al., p. 56

After training using natural frequencies instead of conditional

probability, 87% of the doctors were able to correctly answer the question.

• Gigerenzer, et.al., don’t just point out the problem and measure it, but also provide ideas for solutions as well

Basic points of statistical literacy in health

• Learning to Live With Uncertainty– Understand that there is no certainty and no zero-

risk, but only risks that are more or less acceptable.

• Questions to Ask About All Risks– Risk of what?– Time frame?– How big?– Does it apply to me?

Basic points of statistical literacy in health

• Screening Tests– Understand that screening tests may have benefits

and harms. – Understand that screening tests can make two errors:

false positives and false negatives.– Understand how to translate specificities,

sensitivities, and other conditional probabilities into natural frequencies.

– Understand that the goal of screening is not simply the early detection of disease; it is mortality reduction or improvement of quality of life.

Basic points of statistical literacy in health

Treatment• Understand that treatments typically have

benefits and harms.• Understand the size of the benefit and harm.– What are the absolute risks with and without

treatment

Basic points of statistical literacy in health

Questions About the Science Behind the Numbers

• Quality of evidence?• What conflicts of interest exist?

Guidelines for Assessment and Instruction in Statistical Education (GAISE)

• Endorsed by the American Statistical Association• Guidelines for K-12 and separate guidelines for

college• https://www.amstat.org/education/gaise/index.cfm

Primary recommendations• Emphasize statistical literacy and develop

statistical thinking;• Use real data;• Stress conceptual understanding rather than

mere knowledge of procedures;• Foster active learning in the classroom;• Use technology for developing conceptual

understanding and analyzing data;• Use assessments to improve and evaluate

student learning

Students should believe and understand why:

• Data beat anecdotes.• Variability is natural and is also predictable and quantifiable.• Random sampling allows results of surveys and experiments to be

extended to the population from which the sample was taken.• Random assignment in comparative experiments allows cause and

effect conclusions to be drawn. • Association is not causation.• Statistical significance does not necessarily imply practical

importance, especially for studies with large sample sizes.• Finding no statistically significant difference or relationship does

not necessarily mean there is no difference or no relationship in the population, especially for studies with small sample sizes.

Students should recognize:

• Common sources of bias in surveys and experiments.• How to determine the population to which the results

of statistical inference can be extended, if any, based on how the data were collected.

• How to determine when a cause and effect inference can be drawn from an association, based on how the data were collected (e.g., the design of the study)

• That words such as “normal”, “random” and “correlation” have specific meanings in statistics that may differ from common usage.

Students should understand the parts of the process through which statistics works to answer questions,

namely:• How to obtain or generate data.• How to graph the data as a first step in analyzing data,

and how to know when that’s enough to answer the question of interest.

• How to interpret numerical summaries and graphical displays of data - both to answer questions and to check conditions (in order to use statistical procedures correctly).

• How to make appropriate use of statistical inference.• How to communicate the results of a statistical

analysis.

WHAT MAKES REAL DATA REAL? MEASURING QUALITY AND STRUCTURE OF DATA SETS

1. Length (# observations). Real data sets should be large enough to make the need for computers, rather than calculators, obvious. They should be large enough that graphical and numerical summary techniques reveal structure that is otherwise not apparent.

2. Width (# variables). Real data sets should have enough variables that students have room for exploration, for testing alternative hypotheses, and for performing residual diagnostics.

3. Form. Real data include missing values, un-coded values, and sometimes misspelled values; real data can include both numerical and character-valued variables.

4. Structure. Real data typically have complex structure. For example, linear relations are rare and even then high correlations are unusual. Distributions can be highly skewed, or multi-modal. Students should also see that some forms of data, for example longitudinal, can be stored in different “shapes,” depending on whether rows represent an observation made on a subject at a particular time, or contain all observations for a particular subject

http://www.ime.usp.br/~abe/ICOTS7/Proceedings/PDFs/InvitedPapers/7A2_GOUL.pdf

Improving statistical skills of research scientists in pharmaceutical discovery research

1) Time spent thinking on the conceptualization and design of an experiment is time wisely spent;

2) The design of an experiment reflects the contributions from different sources of variability;

3) The design of an experiment balances between its internal validity (proper control of noise) and external validity (the experiment’s generalizability);

4) Good experimental practice provides the clue to bias minimization; 5) Good experimental design is the clue to the control of variability; 6) Experimental design integrates various disciplines; 7) A priori consideration of statistical power is an indispensable pillar of an

effective experiment.

Vandenbroeck, Philippe, Wouters, Luc, Molenberghs, Geert, Van Gestel, Jef and Bijnens, Luc(2006)'Teaching Statistical Thinking to Life Scientists a Case-Based Approach',Journal of Biopharmaceutical Statistics,16:1,61 — 75To link to this Article: DOI: 10.1080/10543400500406520URL: http://dx.doi.org/10.1080/10543400500406520

“Habits of mind” for students • consideration of how to best obtain meaningful and relevant

data to answer the question at hand• constant reflection on the variables involved and curiosity for

other ways of examining and thinking about the data and problem at hand

• seeing the complete process with constant revision of each component

• omnipresent skepticism about the data obtained• constant relation of the data to the context of the problem

and interpretation of the conclusions in non-statistical terms• thinking beyond the textbook

Chance, Beth L., Components of Statistical Thinking and Implications for Instruction and Assessment, Journal of Statistics Education Volume 10, Number 3 (2002)www.amstat.org/publications/jse/v10n3/chance.html

Handy summaries of types of studies

• http://www.lib.uconn.edu/research/bysubject/nursingtutorial/studies.htm

• http://www.hsrmethods.org/glossary.aspx

Hierarchy of Evidence for Intervention Studies

Adapted from: Guyatt et al. for the Evidence-Based Medicine Working Group. JAMA. 2000;284:1290-1296.

Unsystematic clinical observations

Systematic review of randomized trials

Single randomized trial

Systematic review of observational studies addressing patient-important outcomes

Single observational study addressing patient-important outcomes

Physiologic studies

From: Freedman, Pisani, Purves and Adhikari, Statistics, Norton, 1991

Thank you.