Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.
-
Upload
gertrude-booker -
Category
Documents
-
view
223 -
download
1
Transcript of Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.
![Page 1: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/1.jpg)
Analysis of Variance (ANOVA)
Quantitative Methods in HPELS
HPELS 6210
![Page 2: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/2.jpg)
Agenda
Introduction The Analysis of Variance (ANOVA) Hypothesis Tests with ANOVA Post Hoc Analysis Instat Assumptions
![Page 3: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/3.jpg)
Introduction Recall There are two possible
scenarios when obtaining two sets of data for comparison: Independent samples: The data in the first
sample is completely INDEPENDENT from the data in the second sample.
Dependent/Related samples: The two sets of data are DEPENDENT on one another. There is a relationship between the two sets of data.
![Page 4: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/4.jpg)
Introduction Three or more data sets?
If the three or more sets of data are independent of one another Analysis of Variance (ANOVA)
If the three or more sets of data are dependent on one another Repeated Measures ANOVA
![Page 5: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/5.jpg)
Introduction: Terminology
Factor: Synonym of independent variable Level: The treatment conditions that make
up the factor or independent variable Example: What is the effect of grade (1st,
2nd, 3rd) on IQ?Dependent variable: IQFactor: GradeLevels (3): 1st, 2nd and 3rd grades
![Page 6: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/6.jpg)
Introduction: Terminology Between-Treatment Variance: Variance
between the treatments/levels As the between-treatment variance
increases: The statistic increases The p-value decreases
Greater chance of rejecting the H0
![Page 7: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/7.jpg)
Introduction: Terminology Within-Treatment Variance: Variance
within the treatments/levels As the within-treatment variance
increases: The statistic decreases The p-value increases
Lesser chance of rejecting the H0
![Page 8: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/8.jpg)
Recall the Independent-Measures t-Test
If there was a large difference between the means (between variance) t got bigger
Why? t = M1-M2 / s(M1-M2)
The t formula can be thought of as a ratio of: Between variance (M1-M2)
Within variance (s(M1-M2))
Several Scenarios can occur
![Page 9: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/9.jpg)
-Small between variance
-Large within variance
-t = BV / WV = near zero value
Accept or reject the H0
![Page 10: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/10.jpg)
-Large between variance
-Large within variance
-t = BV / WV = near value of 1.0
Accept or reject the H0
![Page 11: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/11.jpg)
-Small between variance
-Small within variance
-t = BV / WV = near value of 1.0
Accept or reject the H0
![Page 12: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/12.jpg)
-Large between variance
-Small within variance
-t = BV / WV = greater than 1.0
Accept or reject the H0
![Page 13: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/13.jpg)
Introduction The F-Ratio
ANOVA is a ratio of between variance and within variance
Distinction: Three or more groups
![Page 14: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/14.jpg)
The F Distribution Plot all possible F-ratios F distribution There is a family of F distributions As df increases, the distribution becomes more
narrow F-ratios are always positive in value
Computed with two variances Variances are always positive!
F distribution is skewed Most values cluster around 1.0 Figure 13.8 (p 413)
![Page 15: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/15.jpg)
![Page 16: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/16.jpg)
Agenda
Introduction The Analysis of Variance (ANOVA) Hypothesis Tests with ANOVA Post Hoc Analysis Instat Assumptions
![Page 17: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/17.jpg)
ANOVA Statistical Notation:
k = number of treatment conditions (levels)nx = number of samples per treatment level
N = total number of samples N = kn if sample sizes are equal
Tx = X for any given treatment level
G = TMS = mean square = variance
![Page 18: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/18.jpg)
ANOVA
Formula Considerations:SSbetween = T2/n – G2/N
SSwithin = SSinside each treatment
SStotal = SSwithin + SSbetween
SStotal = X2 – G2/N
![Page 19: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/19.jpg)
ANOVA Formula Considerations:
dftotal = N – 1
dfbetween = k – 1
dfwithin = (n – 1) dfwithin = dfin each treatment
![Page 20: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/20.jpg)
ANOVA Formula Considerations:
MSbetween = s2between = SSbetween / dfbetween
MSwithin = s2within = SSwithin / dfwithin
F = MSbetween / MSwithin
![Page 21: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/21.jpg)
Independent-Measures Designs
Static-Group Comparison Design: Administer treatment to two or more groups
and perform posttest Perform posttest to control group Compare groups
X1 O
X2 O
O
![Page 22: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/22.jpg)
Independent-Measures Designs Quasi-Experimental Pretest Posttest
Control Group Design: Perform pretest on three or more groups Administer treatments to treatment groups Perform posttests on all groups Compare delta (Δ) scores
O X1 O Δ
O X2 O Δ
O O Δ
![Page 23: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/23.jpg)
Independent-Measures Designs Randomized Pretest Posttest Control Group
Design: Randomly select subjects from three or more
populations Perform pretest on all groups Administer treatments to treatment groups Perform posttests on all groups Compare delta (Δ) scores
R O X1 O Δ
R O X2 O Δ
R O O Δ
![Page 24: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/24.jpg)
Agenda
Introduction The Analysis of Variance (ANOVA) Hypothesis Tests with ANOVA Post Hoc Analysis Instat Assumptions
![Page 25: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/25.jpg)
Hypothesis Test: ANOVA Example 13.1 (p 415) Overview:
Researchers are interested in the effectiveness different pain relievers (A, B and C) compared placebo (D)
N = 20 randomly assigned to the four treatments (n = 5)
Amount of time (s) each subject could withstand a painfully hot stimulus was measured
![Page 26: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/26.jpg)
Hypothesis Test: ANOVA
Questions:What is the experimental design?What is the independent variable/factor? How many levels are there?What is the dependent variable?
![Page 27: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/27.jpg)
Step 1: State Hypotheses
Non-Directional
H0: µA = µB = µC = µD
H1: At least one mean is different than the others
Directional?
Too many too list
Step 2: Set Criteria
Alpha () = 0.05
Critical Value:
Use F Distribution Table
Appendix B.4 (p 693)
Information Needed:
dfbetween = k – 1
dfwithin = (n – 1)
![Page 28: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/28.jpg)
Step 3: Collect Data and Calculate Statistic
Total Sum of Squares
SStotal = X2 – G2/N
SStotal = 262 – 602/20
SStotal = 262 - 180
SStotal = 82
Sum of Squares Between
SSbetween = T2/n – G2/N
SSbetween = 52/5+102/5+202/5+252/5 – 602/20
SSbetween = (5+20+80+125) - 180
SSbetween = 50
Sum of Squares Within
SSwithin = SSinside each treatment
SSwithin = 8+8+6+10
SSwithin = 32
![Page 29: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/29.jpg)
Step 3: Collect Data and Calculate Statistic
Mean Square Between
MSbetween = SSbetween / dfbetween
MSbetween = 50 / 3
MSbetween = 16.67
Mean Square Within
MSwithin = SSwithin / dfwithin
MSwithin = 32/16
MSwithin = 2
F-Ratio
F = MSbetween / MSwithin
F = 16.67 / 2
F = 8.33
Step 4: Make Decision
![Page 30: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/30.jpg)
Agenda
Introduction The Analysis of Variance (ANOVA) Hypothesis Tests with ANOVA Post Hoc Analysis Instat Assumptions
![Page 31: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/31.jpg)
Post Hoc Analysis What ANOVA tells us:
Rejection of the H0 tells you that there is a
high PROBABILITY that AT LEAST ONE difference exists SOMEWHERE
What ANOVA doesn’t tell us:Where the differences lie
Post hoc analysis is needed to determine which mean(s) is(are) different
![Page 32: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/32.jpg)
Post Hoc Analysis
Post Hoc Tests: Additional hypothesis tests performed after a significant ANOVA test to determine where the differences lie.
Post hoc analysis IS NOT PERFORMED unless the initial ANOVA H0 was rejected!
![Page 33: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/33.jpg)
Post Hoc Analysis Type I Error Type I error: Rejection of a true H0
Pairwise comparisons: Multiple post hoc tests comparing the means of all “pairwise combinations”
Problem: Each post hoc hypothesis test has chance of type I error
As multiple tests are performed, the chance of type I error accumulates
Experimentwise alpha level: Overall probability of type I error that accumulates over a series of pairwise post hoc hypothesis tests
How is this accumulation of type I error controlled?
![Page 34: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/34.jpg)
Two Methods Bonferonni or Dunn’s Method:
Perform multiple t-tests of desired comparisons or contrasts
Make decision relative to / # of testsThis reduction of alpha will control for the
inflation of type I error Specific post hoc tests:
Note: There are many different post hoc tests that can be used
Our book only covers two (Tukey and Scheffe)
![Page 35: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/35.jpg)
Tukey’s Honestly Significant Difference (HSD) Test Overview:
Computes a single value that determines the minimum difference (HSD) between any two means necessary for rejection of the H0
Compares the HSD value to all of the contrast results
If the contrast result exceeds the HSD, the H0
of that particular contrast is rejected
![Page 36: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/36.jpg)
Tukey’s HSD Calculation
Formulas: Equal sample sizes
HSD = q√MSwithin / n
Unequal sample sizesHSD = q√(MSwithin/2)(1/n1+1/n2)
![Page 37: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/37.jpg)
Tukey’s HSD Calculations
Formula Considerations: q = value found in Table B.5 (p 696)
Left column: dfwithin
Top row: k treatments Body:
Regular font: = 0.05 Bold font: = 0.01
MSwithin = value from ANOVA calculation n = number of subjects in each treatment
Example 13.5 (p 427)
![Page 38: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/38.jpg)
Step 1: State Hypotheses
Null
H0: µA = µB
H0: µA = µC
H0: µB = µC
Alternative
H1: µA µB
H1: µA µC
H1: µB µC
Step 2: Set Criteria
Alpha () = 0.05
Step 3: Calculate Statistic
Get q from Table B.5
Information needed:
dfwithin = 24
k = 3
= 0.05
q = 3.53
![Page 39: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/39.jpg)
Calculate Tukey’s HSD Value
HSD = qMSwithin / n
HSD = 3.53 4 / 9
HSD = 2.36
Step 4: Make Decision:
A significantly greater than B MA – MB = 2.44 > 2.36
A significantly greater than C MA – MC = 4.00 > 2.36
B not significantly different than C MB – MC = 1.56 < 2.36
Table 13.6
![Page 40: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/40.jpg)
Scheffe Overview:
Most conservative/cautious of all post hoc tests Uses an F-ratio (like ANOVA) on only two treatments
Controls for type I error: Uses k value from the original ANOVA thus the numerator
of the F-ratio for the Scheffe test is k – 1 Uses same critical value used for the ANOVA
Calculation of Scheffe is identical to the ANOVA however: SSbetween uses the two means of interest
Example 13.6 (p 428)
![Page 41: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/41.jpg)
Step 1: State Hypotheses
Null
H0: µA = µB
H0: µA = µC
H0: µB = µC
Alternative
H1: µA µB
H1: µA µC
H1: µB µC
Step 2: Set Criteria
Alpha () = 0.05
Step 3: Calculate Statistic
Sum of squares between:
SSbetween = T2/n – G2/N
SSbetween = (272/9 + 492/9) – 762/18
SSbetween = (81+266.78) – 320.89
SSbetween = 26.89
SSwithin from original ANOVA = 96
Critical Value 3.40
dfbetween = 2
dfwithin = 24
= 0.05
Mean square between and within
MSbetween = SSbetween/dfbetween
MSbetween = 26.89 / 2 = 13.45
MSwithin from original ANOVA = 4
F = MSbetween / MSwithin
F = 13.45 / 4
F = 3.36
![Page 42: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/42.jpg)
F = MSbetween / MSwithin
F = 13.45 / 4
F = 3.36
Step 4: Make Decision
F = 3.36 < 3.40 (critical value)
Accept or reject?
Repeat for the other two contrasts:
H0: µA = µC
H0: µB = µC
df = 2, 24
3.40
![Page 43: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/43.jpg)
Agenda
Introduction The Analysis of Variance (ANOVA) Hypothesis Tests with ANOVA Post Hoc Analysis Instat Assumptions
![Page 44: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/44.jpg)
Instat Type dependent variable data from the three or more
samples into one column: Label column appropriately
In a second column, type in the grouping variable (independent variable) next to each data point:
Label column appropriately Convert the grouping column into a “factor” column:
Highlight the grouping column. Choose “Manage” Choose “Column Properties” Choose “Factor” Select the appropriate column to be converted Indicate the number of levels in the factor Click OK
![Page 45: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/45.jpg)
Instat Choose “Statistics”
Choose “Analysis of Variance” Choose “One-Way” Y-Variate:
Choose the dependent variable Factor:
Choose the factor column or grouping/independent variable Plots:
Not necessary to choose any Click OK. Interpret the p-value!!!
Post Hoc Analysis: Perform multiple Independent-Measures t-Tests with the
Bonferonni/Dunn correction method
![Page 46: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/46.jpg)
Reporting ANOVA Results Information to include:
Value of the F statistic Degrees of freedom:
Between: k – 1 Within: (n – 1)
p-value Examples:
A significant treatment effect was observed (F(2, 24) = 8.33, p = 0.02)
![Page 47: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/47.jpg)
Reporting ANOVA Results An ANOVA summary table is often
included
Source SS df MS
Between 50 3 16.67 F = 8.33
Within 32 16 2
Total 82 19
![Page 48: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/48.jpg)
Agenda
Introduction The Analysis of Variance (ANOVA) Hypothesis Tests with ANOVA Post Hoc Analysis Instat Assumptions
![Page 49: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/49.jpg)
Assumptions of ANOVA Independent Observations Normal Distribution Scale of Measurement
Interval or ratio Equal variances (homogeneity)
![Page 50: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/50.jpg)
Violation of Assumptions Nonparametric Version Kruskall-Wallis
Test (Chapter 17) When to use the Kruskall-Wallis Test:
Independent-Measures design with three or more groups
Scale of measurement assumption violation: Ordinal data
Normality assumption violation: Regardless of scale of measurement
![Page 51: Analysis of Variance (ANOVA) Quantitative Methods in HPELS HPELS 6210.](https://reader035.fdocuments.in/reader035/viewer/2022062222/5697bfa01a28abf838c956bc/html5/thumbnails/51.jpg)
Textbook Assignment
Problems: 3, 5, 17a, 21