Post on 13-Jan-2016
1
On statistical significance testingCoffee talk 19/02/2015
Nora Baka
2
My
Mx
My
Mx
My
Mx
My
Mx
My
Mx
My
Mx
My
Mx
My
Mx
My
Mx
My
Mx
My
Mx
My
Mx
My
Mx
My
Mx
My
Mx
My
Mx
My
Mx
Sequence 1 Sequence 2 Sequence 3
Is there a significant difference between method X and Y?
Time fram
esMeasurements are not independent!
Clustered data
Ignore
Take cluster meansSpecial tests
3
The Wilcoxon Signed Rank Test for Paired Comparisons of Clustered Data
B. Rosner, R. J. Glynn and M. T. LeeBiometrics 2006
4
Recap: Wilcoxon Signed Rank Test
• Paired data, Notion of bigger-lesser, (Data symmetric)• Independent Random samples• H0: median difference between the pairs is zero
1. Exclude all sample size 2. Rank from small to large 3. Test statistic
, where 5. If small Compare to distribution of all possible W
(random assignment of )6. If large Gaussian approximation
5
Clustered Wilcoxon Signed Rank Test, balanced
• Paired data, Notion of bigger-lesser• Clustered samples (m clusters), g correlated
samples in each cluster• H0: median difference between the pairs is zero
1. , where i is cluster, and j is sample in cluster2. Exclude all sample size 3. Rank for pooled data from small to large 4. Test statistic
6
Clustered Wilcoxon Signed Rank Test, balanced (2)
• Test statistic
• Randomization unit is the cluster distribution of
• If m is small calculate all possible
• If m is large Gaussian approximation
,where with probability ½.
7
Clustered Wilcoxon Signed Rank Test, unbalanced
• Different number of samples per cluster
• Modified statistic
• Randomization unit is the cluster distribution of
From here on the same as before.
𝑇 𝑐𝑠𝑜𝑏𝑠=∑
𝑖=1
𝑚
𝑤𝑖𝑆𝑖=∑𝑖=1
𝑚
𝑤𝑖∑𝑗=1
𝑔𝑖
𝑅𝑖𝑗𝑉 𝑖𝑗 𝑤𝑖=1
𝑉𝑎𝑟 (𝑆𝑖𝑗)1
1+(𝑔¿¿ 𝑖−1)𝜌 ¿,where
Intra-class correlation
8
Compare with no clustering taken into account
9
Results
10
What more?
• See below paper for overview of clustered data analysis[1] S. Galbraith, J. A. Daniel, and B. Vissel, “A study of clustered data and
approaches to its analysis.,” J. Neurosci., vol. 30, no. 32, pp. 10601–8, Aug.
2010.