+

Quantitative Analysis:Supporting Concepts

EDTEC 690 – Methods of InquiryMinjuan Wang (based on previous slides)

+Agenda

Quick review of data Why analysis is necessary – beyond descriptive statistics The Culture data posted on BB

Descriptive analysis vs. inferential analysis

Review Key Concepts of Descriptive Statistics

Inferential analysis concepts Types of tests – parametric and non-parametric What test should I use when?

Next steps for your studies We will help you with inferential analysis using SPSS or

other

2

+Our Special Guests: Types of Analysis Descriptive statistics

Correlation Measuring a relationship

between studied variables

Inferential statistics Inferences from a studied

sample to a population

Parametric analyses

Nonparametric analyses

3

+What is measurement?

Measurement: process of assigning numbers, according to rules defined by the researcher. The numbers are

assigned to events or objects, such as responses to items, or to certain observed behaviors

Correspondence between event/objective/behavior and number is defined by the researcher

4

+Types of Measurement Scales

Nominal Categorization, no implied order (e.g., sex, eye color)

Ordinal Involves order of the scores/ratings on some basis (e.g.,

attitude toward the government)

Interval Unit interval is the same across the scale, doesn’t

necessarily begin at zero (e.g., time, test score)

Ratio Equal unit with a true zero point (e.g., the government

expenditures; birth weight in pounds)

5

+Inferential statistics

Making inferences from samples to populations Making inferences, then conclusions, from the statistics of a

sample – that’s inferential statistics

In practical terms, this means testing your hypothesis

6

+Inferential statistics

Inferential tests produce a level of significance

Significance level or level of significance (α- level) is a probability (for example, 0.05) used in making a decision about the hypothesis (i.e., rejecting the null hypothesis); it is called the alpha level

Significance level is set prior to commencing the study In education, typically .05

7

+

Inferential statistics - parametric

Scale: Dependent variable is measured on an interval scale

Sample: The scores (dependent variable) come from a population distribution that is normally distributed.

Distribution: When two or more populations are being studied, they have homogeneous variance.

+Inferential statistics - parametric t-test (difference between two means)

testing the statistical significance of the difference between means from two independent samples, or two sets of scores from the same sample (pre to post)

Types: T for 1 (paired samples); and T for 2 (unpaired samples)

9

+

Concepts behindInferential Statistics

Let’s work on the assumption that we’re measuring knowledge.

For EDTEC folks – think Kirkpatrick’s Level II – in other words, mastery of objectives

Let’s make our audience diesel technicians who work for dealers of a major auto manufacturer

Finally, let’s say we have two treatments:

1. Traditional classroom instruction, with limited exercises

2. Fully hands-on curriculum involving “bugged” trucks and problem solving throughout

Drawing conclusions from your data

+Diesel Technician Scores

PretestMean

PosttestMean

Gain(difference in

means)

Traditional 55.55 94.65 +39.1

Hands-on 53.45 97.76 +44.31

Course objectives-based Test of Mastery (percent mastery)

11

+Inferential statistics - parametric But what happens when you have more than two

independent variables? For example, what if there were three types of classes for the diesel technicians?

Analysis of variance (ANOVA) Tests the statistical significance when 2 or more

independent variables are present

Consider: A study on student learning with the presence of:

no music slow music fast music

12

13

Does Culture Make a Difference in learner perceptions?

The survey: Perceptions about being equal with their instructor Chinese, American, Korean students

Tests conducted Kruskal-Wallis Analysis of Variance

Non-parametric version of ANOVA

Results and Interpretation P=0.02 comparing with a=0.05 ???

14

O. Perceptions about Being Equal with Instructors (the higher the mean, the

lower the equality) n

Rank sum

Mean rank

American 31 950.0 30.65

Chinese 15 682.5 45.50

Korean 291217.

5 41.98

Kruskal-Wallis statistic 7.15

p 0.028

+More about ANOVA

But what happens when you have more than two independent variables? For example, what if there were three types of classes for the diesel technicians?

Analysis of variance (ANOVA) Tests the statistical significance when 2 or more

independent variables are present

Consider: A study on student learning with the presence of:

no music slow music fast music

15

16

Parametric versus NonparametricParametric –

Characteristic is normally distributed in the population; sample was randomly selected; data is interval or ratio

Nonparametric Use when you have a specialized population, you’ve not

randomly selected, or data is ranked or nominal

“Cooking” Analogy steamed versus fried Streamed broccoli versus baked pumpkin pie

+Assumptions of parametric analyses Scale: Dependent variable is measured on an interval

scale (or ratio) – not nominal or ordinal

Sample: random sampling & normal distribution Normal distribution is required only if sample size is less

than 30. More than 30, the sample is large enough to have a normal

distribution.

Distribution: When two or more populations are being studied, they have homogeneous variance. This means that the populations have about the same

dispersion (SD) in their distributions. Mean can differ.

When you cannot meet these assumptions (i.e., you have categorical data)…

look to non-parametric analyses…

17

+

Inferential statistics - nonparametric

Scale: Can be used with ordinal and nominal scale data

Sample/Distribution: Require few if any assumptions about the population under study

Nonparametric tests do not emphasize means; they use frequencies and other statistics to investigate significance

+

What test should I use?

Recognize there are many, many statistical tests…

And that ED 690 is not intended as a statistics course.

Still, you should be conceptually familiar with these statistical tests

+Choosing the appropriate test

Relationship between variables

Relationship between variables

About means, and parametric

assumptions are met

About means, and parametric

assumptions are met

About frequencies, etc., and parametric assumptions are met

About frequencies, etc., and parametric assumptions are met

Correlation CoefficientCorrelation Coefficient

Chi-squareChi-square

Parametric analyses

Parametric analyses Nonparametric

analysesNonparametric

analyses

Chi-squareChi-square

t-testst-tests ANOVAANOVA

Magnitude of Relationship

20

21

Inferential: Parametric tests

T-Test for means T for 1 (pre- post- tests of 1 group) T for 2 (compare the mean of 2 groups)

Analysis of Variance ANOVA Compare differences between 2 or more

groups

Analysis of Covariance

22

Inferential: Non-parametric

Nonparametric Techniques for Quantitative Data The Mann-Whitney U Test—for T(ea) for two The Kruskal-Wallis One Way Analysis of Variance

—for ANOVA 1 independent variable

The Friedman Two-Way Analysis of Variance—for ANOVA 2 or more independent variables

+Inferential statistics - nonparametricThe Chi-Square (X2) test and

distribution Unlike t-distribution, the X2 distribution

does not require symmetrical distributions

It tests hypotheses about how well a sample distribution fits some theoretical or hypothesized distribution Is there a relationship between eye and hair

color?

23

24

Tails of A Test

Two-tailed test (non-directional/both)

There is no difference in content acquisition between "discovery learning" and "direct instruction.“

One-tailed test (directional/upper/lower) difference will be in one direction only Students who use "discovery learning" exhibit greater

gains in content acquisition than students who use "direct instruction"

+Type I and Type II errors

What if we observe a difference – but none exists in the population?

What if we do not find a difference – but it does exist in the population?

These situations are called Type I and II errors

These errors cannot be eliminated; they can be minimized, but unfortunately, minimizing one type of error will increase the probability of committing the other error

25

+Type I and Type II errors

Conclusion about null hypothesis from statistical test

Accept Null Reject Null

Truth aboutnull

hypothesis in population

True Correct Type I errorObserve difference when none exists

False Type II errorFail to observe

difference when one exists

Correct

26

27

Mini-Data Activity (time to embark on it?)Salary DataCulture DataWhen you are

not heavily cognitively overloaded…..