Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.
-
date post
22-Dec-2015 -
Category
Documents
-
view
224 -
download
8
Transcript of Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.
![Page 1: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/1.jpg)
Measurement Errors
Introduction to Study Skills & Research Methods (HL10040)
Dr James Betts
![Page 2: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/2.jpg)
Lecture Outline:•Measurement Errors Continued
•Types of Errors
•Assessment of Error
•Introduction to Inferential Statistics
•Chi-Squared tests
•Assessment Details.
![Page 3: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/3.jpg)
Measurement Errors
• Virtually all measurements have errors– i.e.
Measured Score = ‘True’ Score ErrorTherefore inherently linked to SD
• Reliability and Measurement Error are not the same, rather Reliability infers an acceptable degree of Measurement Error.
![Page 4: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/4.jpg)
Energy Intake (calories per day)
1500 2500 3500 4500 5500
Nu
mb
er
of
Pe
op
le
0
20
40
60
80
100
120
140
160 This variability between methods is
caused by both systematic and error factors
Direct Record
Retrospective Recall
SD
![Page 5: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/5.jpg)
TotalVariance
(SD2)
This total variance can then be
‘partitioned’
SystematicVariance
ErrorVariance
Caused by systematic error
Caused by random error
![Page 6: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/6.jpg)
Types of Errors• Systematic Error
– Any variable causing a consistent shift in the mean in a given direction
e.g. Retrospective diet records tend to omit the snacks between meals
• Random Error– The fluctuation of scores due to chancee.g. Innaccurate descriptions of the food consumed
![Page 7: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/7.jpg)
Systematic Error
Skin-Fold Callipers
Hydrostatic Weighing
% Body-fatSubject 1 Subject 2 Subject 3 Subject 4
10 12 8 11
17 22 14 12
![Page 8: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/8.jpg)
Random Error
Skin-Fold Callipers
Hydrostatic Weighing
% Body-fatSubject 1 Subject 2 Subject 3 Subject 4
14 18 10 9
11 15 21 17
![Page 9: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/9.jpg)
Body-Fat
0
5
10
15
20
25
Condition
%
Assessment of Error
• Systematic Error
Descriptive Statistics
4 12.00 22.00 16.2500 4.34933
4 8.00 12.00 10.2500 1.70783
4
Hydrostat
Callipers
Valid N (listwise)
N Minimum Maximum Mean Std. Deviation
Evidence of bias between means
![Page 10: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/10.jpg)
Assessment of Error• Random
Error
12.00 14.00 16.00 18.00 20.00 22.00
Hydrostat
8.00
9.00
10.00
11.00
12.00
Cal
liper
s
Correlations
1 .527
. .473
4 4
.527 1
.473 .
4 4
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Callipers
Hydrostat
Callipers Hydrostat r2 = 0.278
r = 0 infers lots of error
r = 1 infers no error In general, good agreement
requires r > 0.7
![Page 11: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/11.jpg)
Assessment of Error• Systematic &
Random ErrorCallipers HydroStat. 10.00 17.00 12.00 22.00 8.00 14.00 11.00 12.00 14.00 11.00 18.00 15.00 10.00 21.00 9.00 17.00
![Page 12: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/12.jpg)
Assessment of Error• Systematic &
Random ErrorCallipers HydroStat. Difference Mean 10.00 17.00 7.00 13.50 12.00 22.00 10.00 17.00 8.00 14.00 6.00 11.00 11.00 12.00 1.00 11.50 14.00 11.00 -3.00 12.50 18.00 15.00 -3.00 16.50 10.00 21.00 11.00 15.50 9.00 17.00 8.00 13.00
![Page 13: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/13.jpg)
Assessment of Error• Systematic &
Random Error
12.00 14.00 16.00
Mean
0.00
5.00
10.00
dif
fere
nce
s
Mean = 4.63
The “Bland-Altman” Plot 3 points of visual assessment:
-Systematic Error: are points evenly distributed about the zero line?
-Random Error: do points deviate greatly from the mean line?
-Nature of error: is the error consistent left-right?
![Page 14: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/14.jpg)
Examples of Bland-Altman Plots
12.00 13.00 14.00 15.00 16.00
Mean
0.00
5.00
10.00
Mean difference
Zero
![Page 15: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/15.jpg)
Examples of Bland-Altman Plots
12.00 13.00 14.00 15.00 16.00
Mean
0.00
5.00
10.00
Mean difference
Zero
![Page 16: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/16.jpg)
Examples of Bland-Altman Plots
12.00 13.00 14.00 15.00 16.00
Mean
0.00
5.00
10.00
Mean difference
Zero
![Page 17: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/17.jpg)
Examples of Bland-Altman Plots
12.00 13.00 14.00 15.00 16.00
Mean
0.00
5.00
10.00
Mean difference
Zero
![Page 18: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/18.jpg)
Examples of Bland-Altman Plots
12.00 13.00 14.00 15.00 16.00
Mean
0.00
5.00
10.00
Zero
![Page 19: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/19.jpg)
Why is Error Important• Measurement Error is clearly of importance when
evaluating the agreement between two measurement tools
• A consideration of error is also relevant when attempting to establish intervention effects/treatment differencesi.e. where some of the variance between trials is due to the
independent variable...
![Page 20: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/20.jpg)
Total Variance between trial 1
& trial 2
SystematicVariance
ErrorVariance
Dependent Variable
Extraneous/Confounding
(Error) Variables
Independent Variable
![Page 21: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/21.jpg)
SystematicVarianceTotal Variance
between trial 1 & trial 2
SystematicVariance
ErrorVariance
Dependent Variable
Extraneous/Confounding
(Error) Variables
Independent Variable
Primary Variance
So researchers strive to increase the proportion of variance due to IV.
![Page 22: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/22.jpg)
Total Variance between trial 1
& trial 2
SystematicVariance
Error Variance
Dependent Variable
Extraneous/Confounding
(Error) Variables
Independent Variable
Primary Variance
So researchers strive to increase the proportion of variance due to IV.
Increase control
Maximise effect
(20 pints?)
![Page 23: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/23.jpg)
Introduction to Inferential Statistics
• Before our next lecture you will be conducting some inferential statistics in your lab classes
• All you need to be aware of at this stage is that the ‘P-value’ represents the probability that total variance is not due to primary variance
i.e. P = 0.01 infers a 99 % probability variance in the DV is not due to pure chance
(i.e. 1 % likelihood of your result occurring if there is in fact no effect)
![Page 24: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/24.jpg)
Introduction to Inferential Statistics
• Before our next lecture you will be conducting some inferential statistics in your lab classes
• All you need to be aware of at this stage is that the ‘P-value’ represents the probability that total variance is not due to primary variance
i.e. P = 0.10 infers a 90 % probability variance in the DV is not due to pure chance
(i.e. 1 % likelihood of your result occurring if there is in fact no effect)
![Page 25: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/25.jpg)
Introduction to Inferential Statistics
• Before our next lecture you will be conducting some inferential statistics in your lab classes
• All you need to be aware of at this stage is that the ‘P-value’ represents the probability that total variance is not due to primary variance
In exercise science, we must be at least 95 % sure that our effect is due more than pure chance before
concluding a ‘significant’ difference.
i.e. P 0.05
n.b. this DOES NOT mean that you will find this result in 95/100 test-retests or that your false positive rate is 5 %
![Page 26: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/26.jpg)
n.b. this DOES NOT mean that you will find this result in 95/100 test-retests or that your false positive rate is 5 %
![Page 27: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/27.jpg)
Quantitative Analysis of Nominal Data
• Recall that nominal data infers that variables are
dichotomous, i.e. belong to distinct categories
e.g. Athlete/Non-Athlete, Male/Female, etc.
• We know that such qualitative data can be coded
quantitatively to allow a more objective analysis
• Nominal data does not require any consideration
of normality and is analysed used a Chi2 test.
![Page 28: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/28.jpg)
The Chi-Squared Test
• Goodness of fit χ2 test– A comparison of your observed frequency counts
against what would be expected according to the null hypothesis
i.e. null hypothesis infers equal dispersion (50:50)
• Contingency χ2 test– A comparison of two observed frequency counts
![Page 29: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/29.jpg)
Goodness of fit χ2 test• Is a leisure centre used more by males than by
females?– n =150
Observed Frequency
Expected
Frequency
Male 62 75
Female 88 75
![Page 30: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/30.jpg)
Gender
62 75.0 -13.0
88 75.0 13.0
150
Male
Female
Total
Observed N Expected N Residual
Goodness of fit χ2 testSPSS Output
Test Statistics
4.507
1
.034
Chi-Square a
df
Asymp. Sig.
Gender
0 cells (.0%) have expected frequencies less than5. The minimum expected cell frequency is 75.0.
a. P-value AKA significance level
i.e. significant difference in the proportion of users
according to gender
![Page 31: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/31.jpg)
Contingency χ2 test• Are elite athletes more likely to take nutritional
supplements than non-athletes– n =60
Do take supplements
Do not take supplements
Athletes 18 12
Non-athletes 11 19
![Page 32: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/32.jpg)
Chi-Square Tests
3.270b 1 .071
2.403 1 .121
3.301 1 .069
.120 .060
3.216 1 .073
60
Pearson Chi-Square
Continuity Correctiona
Likelihood Ratio
Fisher's Exact Test
Linear-by-LinearAssociation
N of Valid Cases
Value dfAsymp. Sig.
(2-sided)Exact Sig.(2-sided)
Exact Sig.(1-sided)
Computed only for a 2x2 tablea.
0 cells (.0%) have expected count less than 5. The minimum expected count is 14.50.
b.
Group * Response Crosstabulation
Count
18 12 30
11 19 30
29 31 60
Athletes
Non-Athletes
Group
Total
Do takesupplements
Dont takesupplements
Response
Total
Contingency χ2 test
SPSS Output
This is the test of interest
i.e. no significant difference in the proportion of users
according to group
![Page 33: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/33.jpg)
Assumptions for Chi-Squared
• Although ND not required…
• Cells in the table should all be independent
i.e. one person could have visited the leisure centre twice
• 80 % of the cells must have expected frequencies greater than 5 and all must be above 1
i.e. the more categories available, the more subjects needed
• Cannot use percentages
i.e. a 15:45 split cannot be expressed as 25%:75%
![Page 34: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/34.jpg)
Selected Reading• I know error and variance can be confusing topics, try these:
• Atkinson, G. and A. M. Nevill. Statistical methods for assessing measurement error (Reliability) in variables relevant to sports medicine. Sports Medicine. 26:217-238, 1998.
• Hopkins, W. G. et al. Design and analysis of research on sport performance enhancement. Med. Sci. Sport and Exerc. 31:472-485, 1999.
• Hopkins, W. G. et al. Reliability of power in physical performance tests. Sports Medicine. 31:211-234, 2001.
• Atkinson, G., ''What is this thing called measurement error?'' , in Kinanthropometry VIII: Proceedings of the 8th International Conference of the International Society for the Advancement of Kinanthropometry (ISAK) , Reilly, T. and Marfell-Jones, M. (Eds.), Taylor and Francis, London , 2003.
![Page 35: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/35.jpg)
Coursework (60% overall grade)• Your coursework will require you to address
2 of the following research scenarios:
– 1) Effect of Plyometric Training on Vertical Jump
– 2) Effect of Ice Baths on Recovery of Strength
– 3) Effect of Diet on the Incidence of Muscle Injury
– 4) Effect of Footwear on Sprint Acceleration
– 5) Effect of PMR on Competitive Anxiety.
![Page 36: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/36.jpg)
Coursework Outline• For each of the 2 scenarios you will need to:
– Perform a literature search in order to provide a
comprehensive introduction to the research area
– Identify the variables of interest and evaluate the
research design which was adopted
– Formulate and state appropriate hypotheses
– Summarise descriptive statistics in an appropriate
and well presented manner…
![Page 37: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/37.jpg)
Coursework Outline• Cont’d…
– Select the most appropriate statistical test with justification for your decision
– Transfer the output of your inferential statistics into your word document
– Interpret your results and discuss the validity and reliability of the study
– Draw a meaningful conclusion (state whether hypotheses are accepted or rejected).
![Page 38: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/38.jpg)
Coursework Details (see unit outline)• 2000 words maximum (i.e. 1000 for each)
• Any supporting SPSS data/outputs to be appended
• To be submitted on Thursday 11th December
Assessment Weighting
Evaluation & Analysis (30 %)
Reading & Research (20 %)
Communication & Presentation (20 %)
Knowledge (30 %)
![Page 39: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/39.jpg)
Coursework Details• All information relating to your coursework
(including the relevant data files) are accessible via the unit web page:
www.bath.ac.uk/~jb335/Y1%20Research%20Skills%20(FH10040).html
Web address also referenced on shared area
![Page 40: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/40.jpg)
Mid-Term Test (40% overall grade)• NEXT WEEK• This test will involve short answer questions
covering all the information covered so far• Mostly knowledge recall but will require
understanding and possibly some calculations• Duration = 50 min
So…
![Page 41: Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d7f5503460f94a62722/html5/thumbnails/41.jpg)
Mid-Term Test (40% overall grade)
• Surnames: A-H– Arrive promptly at 11.10 am for start of test at 11.15 am– Exit in silence afterwards
• Surnames: I-Z– Arrive promptly at 12.10 am for start of test at 12.15 am– Exit however you like!