Lecture 7

Tests for Binary/Categorical outcomes

Binary or categorical outcomes (proportions)

Chi-square testKim et al. Effect of probiotic mix (Bifidobacterium bifidum, Bifidobacterium lactis, Lactobacillus acidophilus) in the primary prevention of eczema: a double-blind, randomized, placebo-controlled trial. Pediatric Allergy and Immunology. Published online October 2009.Table3.Cumulative incidence of eczema at 12months of age From an RCT of probiotic supplementation during pregnancy to prevent eczema in the infant:

Probiotics groupPlacebo groupp-valueAdjusted OR(95% CI)p-value

Cumulative incidence at 12months12/33 (36.4%)22/35 (62.9%)0.029*0.243(0.0750.792)0.019*Significant difference between the groups as determined by Pearson's chi-square test. p value was calculated by multivariable logistic regression analysis adjusted for the antibiotics use, total duration of breastfeeding, and delivery by cesarean section.

Chi-square testStatistical question: Does the proportion of infants with eczema differ in the treatment and control groups?What is the outcome variable? Eczema in the first year of life (yes/no)What type of variable is it? BinaryAre the observations correlated? NoAre groups being compared and, if so, how many? Yes, two groupsAre any of the counts smaller than 5? No, smallest is 12 (probiotics group with eczema) chi-square test or relative risks, or both

Chi-square test of IndependenceChi-square test allows you to compare proportions between 2 or more groups (ANOVA for means; chi-square for proportions).

Example 2Asch, S.E. (1955). Opinions and social pressure. Scientific American, 193, 31-35.

The ExperimentA Subject volunteers to participate in a visual perception study.Everyone else in the room is actually a conspirator in the study (unbeknownst to the Subject).The experimenter reveals a pair of cards

The Task CardsStandard lineComparison linesA, B, and C

The ExperimentEveryone goes around the room and says which comparison line (A, B, or C) is correct; the true Subject always answers last after hearing all the others answers.The first few times, the 7 conspirators give the correct answer.Then, they start purposely giving the (obviously) wrong answer.75% of Subjects tested went along with the groups consensus at least once.

Further ResultsIn a further experiment, group size (number of conspirators) was altered from 2-10.

Does the group size alter the proportion of subjects who conform?

The Chi-Square test

Apparently, conformity less likely when less or more group members

20 + 50 + 75 + 60 + 30 = 235 conformedout of 500 experiments.

Overall likelihood of conforming = 235/500 = .47

Expected frequencies if no association between group size and conformity

Do observed and expected differ more than expected due to chance?

Chi-Square test

Chi-Square testRule of thumb: if the chi-square statistic is much greater than its degrees of freedom, indicates statistical significance. Here 85>>4.

InterpretationGroup size and conformity are not independent, for at least some categories of group sizeThe proportion who conform differs between at least two categories of group sizeGlobal test (like ANOVA) doesnt tell you which categories of group size differ

Caveat**When the sample size is very small in any cell (

Review Question 1I divide my study population into smokers, ex-smokers, and never-smokers; I want to compare years of schooling (a normally distributed variable) between the three groups. What test should I use? Repeated-measures ANOVA.One-way ANOVA.Difference in proportions test.Paired ttest.Chi-square test.

Review Question 2I divide my study population into smokers, ex-smokers, and never-smokers; I want to compare the proportions of each group that went to graduate school. What test should I use? Repeated-measures ANOVA.One-way ANOVA.Difference in proportions test.Paired ttest.Chi-square test.

Risk ratios and odds ratiosKim et al. Effect of probiotic mix (Bifidobacterium bifidum, Bifidobacterium lactis, Lactobacillus acidophilus) in the primary prevention of eczema: a double-blind, randomized, placebo-controlled trial. Pediatric Allergy and Immunology. Published online October 2009.Table3.Cumulative incidence of eczema at 12months of age From an RCT of probiotic supplementation during pregnancy to prevent eczema in the infant:

Probiotics groupPlacebo groupp-valueAdjusted OR(95% CI)p-value

Cumulative incidence at 12months12/33 (36.4%)22/35 (62.9%)0.029*0.243(0.0750.792)0.019*Significant difference between the groups as determined by Pearson's chi-square test. p value was calculated by multivariable logistic regression analysis adjusted for the antibiotics use, total duration of breastfeeding, and delivery by cesarean section.

Corresponding 2x2 table

Risk ratios and odds ratiosStatistical question: Does the proportion of infants with eczema differ in the treatment and control groups?What is the outcome variable? Eczema in the first year of life (yes/no)What type of variable is it? BinaryAre the observations correlated? NoAre groups being compared and, if so, how many? Yes, binaryAre any of the counts smaller than 5? No, smallest is 12 (probiotics group with eczema) chi-square test or relative risks, or both

Odds vs. Risk (=probability)

Note: An odds is always higher than its corresponding probability, unless the probability is 100%.1:13:11:91:99

If the risk isThen the odds are (50%) (75%)1/10 (10%)1/100 (1%)

Risk ratios and odds ratiosAbsolute risk difference in eczema between treatment and placebo: 36.4%-62.9%=-26.5% (p=.029, chi-square test). Risk ratio:

Corresponding odds ratio:There is a 26.5% decrease in absolute risk, a 42% decrease in relative risk, and a 66% decrease in relative odds.

Why do we ever use an odds ratio??We cannot calculate a risk ratio from a case-control study (since we cannot calculate the risk of developing the disease in either exposure group).The multivariate regression model for binary outcomes (logistic regression) gives odds ratios, not risk ratios.The odds ratio is a good approximation of the risk ratio when the disease/outcome is rare (~

Interpretation of the odds ratio:The odds ratio will always be bigger than the corresponding risk ratio if RR >1 and smaller if RR

The rare disease assumption

The odds ratio vs. the risk ratio1.0 (null)Rare OutcomeCommon Outcome1.0 (null)

When is the OR is a good approximation of the RR?

RecallSplit-face trial: Researchers assigned 56 subjects to apply SPF 85 sunscreen to one side of their faces and SPF 50 to the other prior to engaging in 5 hours of outdoor sports during mid-day.Sides of the face were randomly assigned; subjects were blinded to SPF strength. Outcome: sunburn Russak JE et al. JAAD 2010; 62: 348-349.

Results:Table I -- Dermatologist grading of sunburn after an average of 5 hours of skiing/snowboarding (P = .03; Fishers exact test) The authors use Fishers exact test to compare 1/56 versus 8/56. But this counts individuals twice and ignores the correlations in the data!

Sun protection factorSunburnedNot sunburned8515550848

McNemars testStatistical question: Is SPF 85 more effective than SPF 50 at preventing sunburn?What is the outcome variable? Sunburn on half a face (yes/no)What type of variable is it? BinaryAre the observations correlated? Yes, split-face trialAre groups being compared and, if so, how many? Yes, two groups (SPF 85 and SPF 50)Are any of the counts smaller than 5? Yes, smallest is 0 McNemars test exact test (if bigger numbers, would use McNemars chi-square test)

Correct analysis of dataTable 1. Correct presentation of the data from: Russak JE et al. JAAD 2010; 62: 348-349. (P = .016; McNemars test).Only the 7 discordant pairs provide useful information for the analysis!

SPF-50 sideSPF-85 sideSunburnedNot sunburnedSunburned10Not sunburned748

McNemars exact testThere are 7 discordant pairs; under the null hypothesis of no difference between sunscreens, the chance that the sunburn appears on the SPF 85 side is 50%.In other words, we have a binomial distribution with N=7 and p=.5.Whats the probability of getting X=0 from a binomial of N=7, p=.5?

Probability =

Two-sided probability =

McNemars chi-square testBasically the same as McNemars exact test but approximates the binomial distribution with a normal distribution (works well as long as sample sizes in each cell >=5)

Recall: Political party and drinkingDrinking by political affiliation

Recall: Political party and alcoholThis association could be analyzed by a ttest or a linear regression or also by logistic regression:

Republican (yes/no) becomes the binary outcome.Alcohol (continuous) becomes the predictor.

Logistic regressionStatistical question: Does alcohol drinking predict political party?What is the outcome variable? Political partyWhat type of variable is it? BinaryAre the observations correlated? NoAre groups being compared? No, our independent variable is continuous logistic regression

The logistic modelln(p/1- p) = + 1*X

The Logit Model (multivariate)

Review question 7If X=.50, what is the logit (=log odds) of X?

.5001.02.0-.50

Example: political party and drinkingModel:Log odds of being a Republican (outcome)= Intercept+ Weekly drinks (predictor)

Fit the data in logistic regression using a computer

Fitted logistic model:Log Odds of being a Republican = -.09 -1.4* (d/wk) Interpretation: every 1 drink more per week decreases your odds of being a Republican by 75% (95% CI is 0.047 to 1.325; p=.10)

To get back to ORs

Adjusted Odds Ratio Interpretation

Adjusted odds ratio, continuous predictor

Practical InterpretationThe odds of disease increase multiplicatively by e for for every one-unit increase in the exposure, controlling for other variables in the model.

Multivariate logistic regressionLitvick JR et al. Predictors of Olfactory Dysfunction in Patients With Chronic Rhinosinusitis. The Laryngoscope Dec 2008; 118: pp 2225-2230.

Logistic regressionStatistical question: What factors are associated with anosmia (and hyposmia)?What are the outcome variables? anosmia vs. normal olfaction (and hyosmia vs. normal)What type of variable is it? BinaryAre the observations correlated? NoAre groups being compared? We want to examine multiple predictors at once, so we need multivariate regression. multivariate logistic regression

Multivariate logistic regressionLitvick JR et al. Predictors of Olfactory Dysfunction in Patients With Chronic Rhinosinusitis. The Laryngoscope Dec 2008; 118: pp 2225-2230.

Logistic regression in cross-sectional and cohort studiesMany cohort and cross-sectional studies report ORs rather than RRs even though the data necessary to calculate RRs are available. Why?If you have a binary outcome and want to adjust for confounders, you have to use logistic regression.Logistic regression gives adjusted odds ratios, not risk ratios.These odds ratios must be interpreted cautiously (as increased odds, not risk) when the outcome is common. When the outcome is common, authors should also report unadjusted risk ratios and/or use a simple formula to convert adjusted odds ratios back to adjusted risk ratios.

Example, wrinkle studyA cross-sectional study on risk factors for wrinkles found that heavy smoking significantly increases the risk of prominent wrinkles. Adjusted OR=3.92 (heavy smokers vs. nonsmokers) calculated from logistic regression.Interpretation: heavy smoking increases risk of prominent wrinkles nearly 4-fold??The prevalence of prominent wrinkles in non-smokers is roughly 45%. So, its not possible to have a 4-fold increase in risk (=180%)!

Raduan et al. J Eur Acad Dermatol Venereol. 2008 Jul 3.

Interpreting ORs when the outcome is commonIf the outcome has a 10% prevalence in the unexposed/reference group*, the maximum possible RR=10.0.For 20% prevalence, the maximum possible RR=5.0For 30% prevalence, the maximum possible RR=3.3.For 40% prevalence, maximum possible RR=2.5.For 50% prevalence, maximum possible RR=2.0.

*Authors should report the prevalence/risk of the outcome in the unexposed/reference group, but they often dont. If this number is not given, you can usually estimate it from other data in the paper (or, if its important enough, email the authors).

Interpreting ORs when the outcome is commonFormula from: Zhang J. What's the Relative Risk? A Method of Correcting the Odds Ratio in Cohort Studies of Common Outcomes JAMA.1998;280:1690-1691. Where:OR = odds ratio from logistic regression (e.g., 3.92)P0 = P(D/~E) = probability/prevalence of the outcome in the unexposed/reference group (e.g. ~45%)If data are from a cross-sectional or cohort study, then you can convert ORs (from logistic regression) back to RRs with a simple formula:

For wrinkle studyZhang J. What's the Relative Risk? A Method of Correcting the Odds Ratio in Cohort Studies of Common Outcomes JAMA.1998;280:1690-1691. So, the risk (prevalence) of wrinkles is increased by 69%, not 292%.

Sleep and hypertension studyORhypertension= 5.12 for chronic insomniacs who sleep 5 hours per night vs. the reference (good sleep) group.ORhypertension = 3.53 for chronic insomiacs who sleep 5-6 hours per night vs. the reference group.Interpretation: risk of hypertension is increased 500% and 350% in these groups?No, ~25% of reference group has hypertension. Use formula to find corresponding RRs = 2.5, 2.2Correct interpretation: Hypertension is increased 150% and 120% in these groups.

-Sainani KL, Schmajuk G, Liu V. A Caution on Interpreting Odds Ratios. SLEEP, Vol. 32, No. 8, 2009 .-Vgontzas AN, Liao D, Bixler EO, Chrousos GP, Vela-Bueno A. Insomnia with objective short sleep duration is associated with a high risk for hypertension. Sleep 2009;32:491-7.

Review problem 8In a cross-sectional study of heart disease in middle-aged men and women, 10% of men in the sample had prevalent heart disease compared with only 5% of women. After adjusting for age in multivariate logistic regression, the odds ratio for heart disease comparing males to females was 1.1 (95% confidence interval: 0.801.42). What conclusion can you draw?

Being male increases your risk of heart disease.Age is a confounder of the relationship between gender and heart disease.There is a statistically significant association between gender and heart disease.The study had insufficient power to detect an effect.

Review topic: Diagnostic Testing and Screening Tests

Characteristics of a diagnostic testSensitivity= Probability that, if you truly have the disease, the diagnostic test will catch it.

Specificity=Probability that, if you truly do not have the disease, the test will register negative.

Calculating sensitivity and specificity from a 2x2 tableAmong those with true disease, how many test positive?Among those without the disease, how many test negative?a+bc+d

Hypothetical ExampleSensitivity=9/10=.9010990Specificity= 881/990 =.891 false negatives out of 10 cases109 false positives out of 990

Positive predictive valueThe probability that if you test positive for the disease, you actually have the disease.Depends on the characteristics of the test (sensitivity, specificity) and the prevalence of disease.

Calculating PPV and NPV from a 2x2 tableAmong those who test positive, how many truly have the disease? Among those who test negative, how many truly do not have the disease? a+cb+d

Hypothetical ExamplePPV=9/118=7.6%118882Prevalence of disease = 10/1000 =1%NPV=881/882=99.9%

What if disease was twice as prevalent in the population?sensitivity=18/20=.9020980specificity=872/980=.89Sensitivity and specificity are characteristics of the test, so they dont change!

What if disease was more prevalent?PPV=18/126=14.3%126874Prevalence of disease = 20/1000 =2%NPV=872/874=99.8%

ConclusionsPositive predictive value increases with increasing prevalence of disease Or if you change the diagnostic tests to improve their accuracy.

Review question 9In a group of patients presenting to the hospital casualty department with abdominal pain, 30% of patients have acute appendicitis. 70% of patients with appendicitis have a temperature greater than 37.5C; 40% of patients without appendicitis have a temperature greater than 37.5C.

The sensitivity of temperature greater than 37.5C as a marker for appendicitis is 21/49.The specificity of temperature greater than 37.5C as a marker for appendicitis is 42/70.The positive predictive value of temperature greater than 37.5C as a marker for appendicitis is 21/30.The predictive value of the test will be the same in a different population.The specificity of the test will depend upon the prevalence of appendicitis in the population to which it is applied.

Lecture 7

Documents

Transcript of Lecture 7