Gifts in the treasure chest of Methodology: A personal view

Rolf SteyerFriedrich Schiller University Jena

Institute of PsychologyDepartment of Methodology and Evaluation Research

Germany

Items in the treasure chest• Latent Class Models• Mixture Distribution Models• Structural Equation Models• Generalized Linear Models • Loglinear Models• Multilevel Models• CART (Classification and Regression Trees)• ...

• LISREL and the other SEM programs• ConQuest and other IRT programs• ...

• EM-Algorithm• Newton-Raphson• ...

Outline

• Measurement• Causality• Statistics

I have deliberately choosen this order, because:

• (Causal) modeling does not make sense if we don´t have reasonable measurements

• Statistcal analysis does not make sense if we don´t have reasonable measurements and a causal model, if we are looking for causal effects (which we do most of the time)

Outline

• Measurement• Causality• Statistics

MeasurementThe fundament of every science

- Fundamental measurement theory

- IRT, uni- and multidimensional

- SEM modeling including models for ordinal variables

- Multidimensional Scaling

Measurement

Measurement

- is much more than assigning numbers to observations

- defines the concepts to which empirical research really refers to

- explicates the relationship between observations and theoretical concepts (constructs)

Measurement

Measurement

- is not reading numbers from a meter stick

- it rather means introducing the concept of length

- and spells out the rules of assigning numbers to observations representing the length of the objects considered

Measurement (cont´d)

Measurement defines our theoretical concepts by

- selecting the observables or items

- specifying a mathematical measurement model relating the observables to the theoretical concepts

Of course both these points need substantive theory and ideas. They are part of the substantive theory.

Measurement (cont´d)

The measurement model determines

- the logical nature of our theoretical variables (metric, ordinal, or nominal concepts)

- the scale level of the theoretical variables

Measurement (cont´d)

Latent variable models are flexible enough to model complex reality

SEELE1.00 Depr1

Depr2

Depr3

Depr4

MetFac

Trait 1

Trait 2

Trait 3

Trait2-1

Trait3-2

A_DEPR1 0.10

A_DEPR2 0.11

C_DEPR1 0.03

C_DEPR2 0.09

D_DEPR1 0.04

D_DEPR2 0.12

E_DEPR1 0.03

E_DEPR2 0.11

Chi-Square=35.77, df=20, P-value=0.01637, RMSEA=0.020

1.00

0.85

1.00

1.00

0.85

0.741.00

0.85

0.70

1.00

0.85

0.70

1.00

1.00

1.00

0.97

1.00

1.00

1.00

1.00

0.08

0.10

0.56

-0.26

-0.06

Measurement (cont´d)

Measurement (cont´d)

IRT-Models

- introduce metric variables on the basis of qualitative variables

- allow adaptive testing

- are recently going multivariate ...

Measurement (cont´d)

Where are we going?

- IRT-Models and Structural Equation Models will merge

- Adaquate and sophisticated latent variable modeling will increase

Causality

- we know much more than „correlation is not causality“ or „noncorrelation is not noncausality“

- what we really want are individual causal effects, what we get are expectations and their difference, which are not the average of the individual causal effects

Causality (cont´d)

- If we have a client, we have at least two alternative treatments for him

- we are able to decide which treatment to choose only when we have a hypothesis about the individual effect of the treatment compared to its alternative

- when there is no knowledge about the individual effect we need at least some knowledge about the average effect

Causality (cont´d)O

bser

vation

al u

nits

P(U

= u

)

E(Y

X =

x1, U

= u

)

E(Y

X =

x2, U

= u

)

Individual treatment assignment probabilities P(X = x1

U = u)

Example Ia Example Ib Example Ic

u1 1/2 85 91 1/4 1/2 1/3

u2 1/2 105 109 3/4 1/2 1/3

Note. According to the theorem of the total probability, the unconditional probability for treatment assignment is P(X = x1) = 2

1iP(X = x1 U = ui) P(U = ui) = 1/2 for Examples Ia and Ib and P(X = x1) = 1/3 for Example Ic.

E(Y X = x1) = uE(Y X = x1, U = u) P(U = u X = x1) = 85 1/4 + 105 3/4 = 100,

E(Y X = x2) = uE(Y X = x2, U = u) P(U = u X = x2) = 91 3/4 + 109 1/4 = 95.5.

Causality (cont´d)

we know

- sufficient conditions for causal unbiasedness

- necessary conditions for unconfoundedness

- how to analyze causal effects in nonorthogonal ANOVA

- how to test for unconfoundedness

Causality (cont´d)

Where are we going? We will learn more about

- nonexperimental design and analysis

- systems of regression equations (causality in SEMs)

- generalize to distributions

- applying it to sophisticated data

Statistics

- statistical tests important, but are we really happy with knowing the probability of the test statistic being this or more extreme under the assumption of H0 ?

- Shall we be going Bayesian and ask for the probabilities of hypotheses?

- How long will it take for the resampling procedures (bootstrapping etc.) to be an easy tool in our standard software?

Conclusions

- There is much more in our treasure chest than an individual could even learn and apply

- Hence we need to organize, distribute and teach our knowledge in a more efficient way

- Intensifying cooperation in Europe may be one way to achieve these goals

Where to find more

For example, this power point file and more useful things, such as papers, infos on workshops etc. may be found at:

http://www.uni-jena.de/svw/metheval/

Or mail to:

rolf.steyer@uni-jena.de