University of Warwick, Department of Sociology, 2014/15SO 201: SSAASS (Surveys and Statistics) (Richard Lampard)

Week 2-3Wedding Attitude by Sex Cross-tabulation Example

First version of cross-tabulation

Wedding more celebration than commitment? by Sex

Male Female Total Agree strongly 118 167 285

9.7% 10.7% 10.3%Agree 522 710 1232

42.8% 45.7% 44.4%Neither 203 234 437

16.6% 15.0% 15.7%Disagree 265 335 600

21.7% 21.5% 21.6%Disagree strongly 68 67 135

5.6% 4.3% 4.9%Can't choose 23 19 42

1.9% 1.2% 1.5%Not answered 21 23 44

1.7% 1.5% 1.6%Total 1220 1555 2775

Commentary on first version• The attitude measure is a nominal variable (i.e. a set of

categories), but the first five categories constitute an ordinal variable (i.e. ordered categories)

• It is tempting to view the first five categories as an interval-level variable (i.e. a scale)

• (Pearson) chi-square treats the measure as nominal, and has a statistically non-significant value (7.628, 6 d.f., p=0.267 > 0.05)

• However the Linear-by-Linear Association statistic treats the variable as interval-level (N.B. the last two categories mean that it isn’t!), and is significant (4.870, 1 d.f., p=0.027 < 0.05)

Second version of cross-tabulation

Wedding more celebration than commitment? by Sex

Male Female Total Agree strongly 118 167 285

10.0% 11.0% 10.6%Agree 522 710 1232

44.4% 46.9% 45.8%Neither 203 234 437

17.3% 15.5% 16.3%Disagree 265 335 600

22.5% 22.1% 22.3%Disagree strongly 68 67 135

5.8% 4.4% 5.0%Total 1176 1513 2689

Commentary on second version

• When we render the attitude measure properly ordinal by removing the last two categories, the Linear-by-Linear Association statistic ‘unfortunately’ becomes non-significant (p=0.071 > 0.05)

• However, this statistic doesn’t treat the variable as ordinal, but as interval-level (using the codes for the categories as a scale).

Third version of cross-tabulation Wedding more celebration than commitment? by Sex

Male Female Total

Agree strongly/Agree 640 877 1517 54.4% 58.0% 56.4%

Neither/Disagree 468 569 1037 39.8% 37.6% 38.6%

Disagree strongly 68 67 135 5.8% 4.4% 5.0%

Total 1176 1513 2689

Commentary on third version

• However, if we consider the possibility that, substantively, there is not much difference between agreeing and agreeing strongly, and that there is not much different between answering ‘Neither’ and answering ‘Disagree’, then we can reduce the measure to three categories.

• If these are coded 1, 2 and 3, and then treated as interval-level, then the Linear-by-Linear Association statistic become significant (p=0.033 < 0.05)

Conclusion• As is sometimes the case, our findings are ambiguous here. • Clearly the basic chi-square statistic ignores the apparent trend in

the percentages in the table, and is not taking proper account of the variable’s ordinality.

• But if we tinker with the categorisation in a way that simply reflects the observed pattern in the cross-tabulation, we can be viewed on one hand as teasing out a ‘real’, substantively plausible underlying relationship, or on the other hand as (inappropriately) nudging the p-value in the direction of significance, and hence increasing the risk of a Type I error (i.e. a false positive).

• Either way, with p-values around the conventional 5% (0.05) significance level, we should perhaps be cautious about viewing a finding as either ‘significant’ or ‘non-significant’…