Statistics for Education Statistics for Education ResearchResearch

Lecture 5Lecture 5

Tests on Two Means:Tests on Two Means:Two Independent SamplesTwo Independent Samples

Independent-Sample t TestsIndependent-Sample t Tests

Instructor: Dr. Tung-hsien Instructor: Dr. Tung-hsien HeHe

the@tea.ntue.edu.twthe@tea.ntue.edu.tw

Independent-Sample t TestIndependent-Sample t Test::1. Independent-Samples t tests compare 1. Independent-Samples t tests compare

meansmeans of of two groups (two independent two groups (two independent samples)samples). .

2. The two means come from a single 2. The two means come from a single dependent variable.dependent variable.

3. Use G*Power to decide sample sizes. 3. Use G*Power to decide sample sizes. 4. Ideally, participants should be randomly 4. Ideally, participants should be randomly

selected and selected and randomlyrandomly assignedassigned to the two to the two groups, so that any difference in response groups, so that any difference in response is due to the is due to the treatmenttreatment (or lack of (or lack of treatment) and not to other factors.treatment) and not to other factors.

5. The 5. The two groupstwo groups should have should have the same number of the same number of participantsparticipants so that the so that the robustness of homogeneirobustness of homogeneityty can be ensured. can be ensured.

6. Check the 6. Check the two assumptionstwo assumptions behind this type of te behind this type of test, i.e., st, i.e., normality and homogeneitynormality and homogeneity, , before using before using itit. .

7. Check 7. Check normality by performing K-S testnormality by performing K-S test (Kolmog (Kolmogorov-Smirnov test). orov-Smirnov test).

8. Check 8. Check homogeneityhomogeneity by performing by performing Levene’sLevene’s tes test (nonsignificant is desired), if the two groups’ sit (nonsignificant is desired), if the two groups’ sizes are different. zes are different. If they have equal ns, don’t neIf they have equal ns, don’t need to run Levene’s tested to run Levene’s test. .

9. If the two assumptions are violated, 9. If the two assumptions are violated, either use nonparametric tests (e.g., either use nonparametric tests (e.g., 2-sample K-S test2-sample K-S test or or Mann-Whitney Mann-Whitney UU, but the sample size must be small , but the sample size must be small < or = 20) or use < or = 20) or use data data transformationstransformations (e.g., square root). (e.g., square root).

10. Condition: Testing Differences in 10. Condition: Testing Differences in Two Means of Two Means of Experimental Group Experimental Group and Control Group on a Single and Control Group on a Single Dependent Variable. Dependent Variable.

Bonferroni Adjustment TechniqueBonferroni Adjustment Technique::a. a. ConditionCondition: : Multiple dependent variablesMultiple dependent variables n need to analyzed by eed to analyzed by performing univariate anaperforming univariate analysis techniqueslysis techniques like paired-sample t test or i like paired-sample t test or independent-sample t tests.ndependent-sample t tests.b. Reason for Using Bonferroni Adjustment Tb. Reason for Using Bonferroni Adjustment Technique: echnique: If any univariate test is performed If any univariate test is performed repeatedlyrepeatedly to analyze different dependent va to analyze different dependent variables, the chance to make riables, the chance to make Type I error will iType I error will increasencrease (i.e., (i.e., will be inflatedwill be inflated). As a result, H). As a result, Ho become easier to be rejected when it shoulo become easier to be rejected when it should not be at the original d not be at the original level. level.

c. Example: c. Example: Independent-Sample t tests are usIndependent-Sample t tests are used twice ed twice to analyze reading achievement and sto analyze reading achievement and scores on written papers, cores on written papers, respectivelyrespectively. . d. d. Procedures of Bonferroni Adjustment TechniProcedures of Bonferroni Adjustment Techniqueque: : Dividing the original Dividing the original by the number of ti by the number of times tests are performedmes tests are performed. Then, after t tests are . Then, after t tests are performed, compare p-values to the values of tperformed, compare p-values to the values of the divided he divided to see if p-values are higher or sm to see if p-values are higher or smaller than the aller than the /n. In our case, /n. In our case, should be 0.02should be 0.0255 (i.e., (i.e., 0.05/20.05/2). ). e. Decision: If e. Decision: If p-values are smaller than 0.025p-values are smaller than 0.025, , then it is then it is significantsignificant. If . If p-values are larger than p-values are larger than 0.0250.025, then it is , then it is nonsignificantnonsignificant. .

ExampleExample::a.a. Scenario Scenario: In a study that focuses on the : In a study that focuses on the effects of Phonics instruction on reading effects of Phonics instruction on reading comprehension, the researcher comprehension, the researcher randomlyrandomly selectselect 4040 subjects from the population. subjects from the population. Then the researcher Then the researcher randomlyrandomly and and evenlyevenly assignsassigns the 40 subjects into the the 40 subjects into the experimental and control groups. Then experimental and control groups. Then phonics instruction is given to the phonics instruction is given to the experiment group, whereas the control experiment group, whereas the control group does not receive any treatment. group does not receive any treatment. After 4 weeks, the researcher tests the two After 4 weeks, the researcher tests the two groups’ reading comprehension by using a groups’ reading comprehension by using a test. test.

a.a. Scenario Scenario: : This test has 6 itemsThis test has 6 items and each and each item occupies 5 points (maximum score is item occupies 5 points (maximum score is 30). The researcher wants to know whether 30). The researcher wants to know whether the instruction will make significant effects the instruction will make significant effects on the experiment group’s reading on the experiment group’s reading comprehension.comprehension.b. b. ConditionsConditions: : Two independent samples with Two independent samples with two means on a single dependent variable. two means on a single dependent variable. c. Goal: To see c. Goal: To see whether the difference in the whether the difference in the experiment group mean and control group experiment group mean and control group meanmean is so huge ( is so huge (significantsignificant) that it is very ) that it is very UnlikelyUnlikely that sampling error will cause this that sampling error will cause this difference.difference.

d. Xd. Xcontrolcontrol = sampling error + = sampling error + reading proficiencreading proficiency y (constant: (constant: 常數常數 )) XXexperimentexperiment=sampling error + =sampling error + reading proficienreading proficiency cy (constant: (constant: 常數常數 )) + + effects of instruceffects of instructiontion

e. Between-Group Homogeneitye. Between-Group Homogeneity: Because the : Because the two groups are randomly selected, the consttwo groups are randomly selected, the constant will be assumed to be same.ant will be assumed to be same.

f. Hypothesis Testing:f. Hypothesis Testing:Ho: XHo: Xcontrolcontrol = X = Xexperiment experiment

Ha: XHa: Xcontrolcontrol ≠ X ≠ Xexperiment experiment

g. One-tailed, g. One-tailed, = .05, n1=20, n2 = 20, = .05, n1=20, n2 = 20, n = 40n = 40

h. Proper Stat Technique: h. Proper Stat Technique: Independent-Sample t Test Independent-Sample t Test

i. See “SPSS Procedures” for detailed i. See “SPSS Procedures” for detailed info.info.

j. Report j. Report effect sizeseffect sizes (post hoc) so that (post hoc) so that ““practical significancepractical significance” can be more ” can be more clarified given that “clarified given that “statistical statistical significancesignificance” is found. ” is found.

3P3PPP: Some researchers support that PP: Some researchers support that

bilingualism prompts bilingual bilingualism prompts bilingual children to reflect on and manipulate children to reflect on and manipulate structural features of a language, structural features of a language, including its phonemes, words, and including its phonemes, words, and syntactic structures (metalinguistic syntactic structures (metalinguistic awareness), whereas others do not. awareness), whereas others do not.

IP: However, (a) very few studies ever IP: However, (a) very few studies ever tackled whether foreign language tackled whether foreign language learning on word-awareness skills among learning on word-awareness skills among Mandarin-English marginal preschool Mandarin-English marginal preschool bilinguals could contribute to a better bilinguals could contribute to a better metalinguistic ability compared to that metalinguistic ability compared to that of monolinguals, and (b) previous of monolinguals, and (b) previous studies do not manipulate variables like studies do not manipulate variables like different degrees of bilingualism, socio-different degrees of bilingualism, socio-economic status (gender & age, amounts economic status (gender & age, amounts of L2 exposure), and task’s difficulties. of L2 exposure), and task’s difficulties.

SP: Marginal bilinguals may perform SP: Marginal bilinguals may perform better on certain tasks.better on certain tasks.

Criticisms:Criticisms:1. Is there a clear link between PP and IP?1. Is there a clear link between PP and IP?2. Is it reasonable to claim that young EFL 2. Is it reasonable to claim that young EFL

learners can be deemed as marginal learners can be deemed as marginal bilinguals? If it is reasonable, does the bilinguals? If it is reasonable, does the author provide the key studies and author provide the key studies and researchers that can justify this researchers that can justify this categorization?categorization?

Literature ReviewLiterature Review1. Studies that highlighted bilinguals’ 1. Studies that highlighted bilinguals’

advantages in metalinguistic awareness.advantages in metalinguistic awareness.2. Studies that failed to find these 2. Studies that failed to find these

advantagesadvantages

3. Criticisms on previous studies’ 3. Criticisms on previous studies’ being unable to find advantages (See being unable to find advantages (See p. 134)p. 134)

Criticisms: Criticisms: 1. Does the author clearly specify that 1. Does the author clearly specify that

investigating word awareness among investigating word awareness among these three kinds of metalinguistic these three kinds of metalinguistic awareness is significant? (See 1awareness is significant? (See 1stst paragraph under Subheading of paragraph under Subheading of LITERATURE REVIEW on p. 132)LITERATURE REVIEW on p. 132)

2. Is there any evidence (like previous 2. Is there any evidence (like previous studies) that can justify the author’s studies) that can justify the author’s criticisms? In other words, are these criticisms? In other words, are these criticisms strongly supported by previous criticisms strongly supported by previous research? (p. 134)research? (p. 134)

3. Is there any research that highlights EFL 3. Is there any research that highlights EFL can been seen as a type of bilinguals? Is it can been seen as a type of bilinguals? Is it important to have evidence to justify this important to have evidence to justify this category?category?

Method & ResultsMethod & Results1. Participants: 1. Participants: a. two samples that constituted 60 a. two samples that constituted 60

preschoolers;preschoolers;

b. Screening activitiesb. Screening activities2. Tasks: 2. Tasks: a. Word Segmentation Taska. Word Segmentation Taskb. Word Size Judgment Taskb. Word Size Judgment Taskc. Name Manipulation Taskc. Name Manipulation Task3. Procedures:3. Procedures:4. Analysis:4. Analysis:a. Independent T-testa. Independent T-testb. Paired-Sample T-testb. Paired-Sample T-test

Criticisms:Criticisms:1. Are participants randomly selected and 1. Are participants randomly selected and

assigned?assigned?2. Is the format of T-test tables appropriate? 2. Is the format of T-test tables appropriate?

(Table 1, p. 138: What seems missing? (Table 1, p. 138: What seems missing? Descriptive statistics of both groups)Descriptive statistics of both groups)

3. Is the result of paired-sample t-tests reported?3. Is the result of paired-sample t-tests reported?4. Are the results of t-tests properly reported? 4. Are the results of t-tests properly reported?

(e.g.: t(58)=-1.220, p>.05, p. 138)(e.g.: t(58)=-1.220, p>.05, p. 138)

Discussions & ConclusionDiscussions & Conclusion1. Discussions on non-significant 1. Discussions on non-significant

results by referring to qualitative results by referring to qualitative analysesanalyses

2. Discussions on significant results by 2. Discussions on significant results by referring to qualitative analyses referring to qualitative analyses

3. Limitations 3. Limitations 4. Implications4. Implications Criticisms:Criticisms:1. Is it appropriate to discuss non-1. Is it appropriate to discuss non-

significant results?significant results?

2. Is it appropriate to cite limitations 2. Is it appropriate to cite limitations in discussion sections?in discussion sections?

3. Is it appropriate to cite qualitative 3. Is it appropriate to cite qualitative data to augment quantitative data to augment quantitative results? results?

4. Is it appropriate to conclude that 4. Is it appropriate to conclude that metalinguistic benefits exist for EFL metalinguistic benefits exist for EFL preschoolers as the marginal preschoolers as the marginal bilingual group merely outperformed bilingual group merely outperformed the monolingual group on one (out of the monolingual group on one (out of three) task? (11three) task? (11thth line from the top, line from the top, p. 143)p. 143)

5. Is it appropriate to conclude that 5. Is it appropriate to conclude that “limited exposure to a second “limited exposure to a second language can enhance one aspect of language can enhance one aspect of word awareness” when participants word awareness” when participants are not randomly selected and are not randomly selected and English input may come from other English input may come from other sources in addition to English sources in addition to English instructions? instructions?