Understanding Variability

Unraveling the Mystery of the Data’s Message

Becoming a “Data Whisperer”

Overview of Research Issues and Goals

I. Idea – hypothesisConceptualization –

Nominal Definitions – Operational Definitions –

Measurement ProceduresII. Design (framework/procedure) –

Strategy -- Implementation –

Data Collection -- Measures

III. Statistics – Exploratory –

Descriptive – Inferential –

DecisionsIV. Conclusions in context –

Interpretation -- Meaning --

Generalization

Search for Cause – Effect connections

Information needed to establish a cause is a cause(J. S. Mill)

Cause and Effect are related – they covary

Cause comes before the Effect in time

All alternative Causes are eliminated

Cause and Effect Connections can be

Necessary and Sufficient – single clear cause

Necessary, not Sufficient – combinations of or interactions of variables needed to get effect

Sufficient, not Necessary – alternative independent causes exist

Goal of Research – to understand variability,

how do variables vary (descriptive), when do they vary (covariation),

and why do variables vary (causes)

Assume that variables cannot vary without cause

Research Strategies –

depend upon research goals/purposes/constraints

Descriptive Research what variables exist and how do variables vary

structure of a variable – qualitative or quantitativedoes the variable exist separately from othersone-dimensional or multidimensionalpossible values and range of values

variability within a sample/population

Covariation Research (when do variables vary - context)

when variables vary, with what do they covary

“response variable - response variable” covariation

assess strength and direction of relationship

“predictor variable – response variable” covariation

use one variable to ‘explain’ or ‘predict’ variability in the other variable

limitations on ‘causal’ interpretations

Search for Cause – Effect connections

Information needed to establish a cause is a cause

Cause and Effect are related – they covary

Cause comes before the Effect in timeLogic may support direction -

All alternative Causes are eliminated

Group Difference Research - focus on typical behavior

Quasi-experimental research –

covariation related to ‘existing’ differences (grouping)

“manipulation” by selection

Experimental research –

creating groups to control variation of some variables

manipulation by control

Data Collection IssuesMeasurement scale –

what type of information is providedqualitative and/or quantitative

operations that can be performed on measures

Source of measuresdirect

physiological measuresbehavioral observations

indirectself reportsinferred

secondarycollected by others

Research Setting

Controlled conditions ------- Uncontrolled conditions

reducing the noise to improve ability to hear the signal

allowing noise to see if signal still has impact

Evaluating Research Strategies

Internal Validity – can outcome be interpreted clearly

enhanced by control over all variables and precision of activities

External Validity – can outcome be generalized

enhanced by inclusion of representative variables and activities

Research involves solving a mystery (Must decide on Guilt or Innocence – and hope that decision matches the unknown ‘truth’)

understanding why things happen – search for TRUTH!

How strong a case can you make,

but NEVER 100% confident

TRUTH

DECISION Innocent Guilty

Guilt wrong right

Innocence right wrong

Research involves solving a mystery (Must decide on Guilt or Innocence – and hope that decision matches the unknown ‘truth’)

understanding why things happen – search for TRUTH! (You are like CSI and Prosecutor)

Research Strategy – collect the evidence…the data upon which your case will be based

Statistics – organize, summarize, and evaluate the strength of the evidence

Decide on your ‘interpretation’ of the evidence(Decide on your version of the “Truth” – guilt or innocence)

Persuade others of the validity of your interpretation

(Other Researchers are like Defense attorney – need only raise doubts!)

validity of your researchappropriateness of evaluation

The ‘Case”

a body is found in the library – “who” and/or “what” caused the death?

Law & Order: Psychological Research Unit

Descriptive Research

Covariation Research

Causal EvidenceMeaningful level of Explanation?

Main Characters – all the variables that might be involved

Dependent Variable(s) – the ‘response’ variablealways present, the basis for the data

used to answer the question

Independent Variable(s) – explanatory/predictor variablethe suspected cause or variable related to the cause

Extraneous Variables – all variables not cast in either of the roles above

nuisance or confound – possible impacts (evil) (Orient Express)

best dealt with in the context of the research strategy – design and procedures

(Constant, Control, Random)

alternatively, can be dealt with indirectly, using statistical adjustments

Qualities of the Main Characters – any variables which will be ‘measured’

Actually are Qualities of Operational Definitions Used

reliability – precision and consistency

validity – appropriateness and accuracy

RecapResearch involves solving a mystery – understanding why things happen

Research Strategy – collect the evidence…the data upon which your case will be based

Statistics – organize, summarize, and evaluate the strength of the evidence

Decide on your ‘interpretation’ of the evidencethe logic of statistical decision-making

Persuade others of the validity of your interpretation

validity of your researchappropriateness of evaluation

Research involves solving a mystery – understanding why things happen

Research Strategy – collect the evidence…the data upon which your case will be base

Issues:correct variables, measured well (operational

definitions)design is appropriate for questions

descriptive, covariation, causalabsence of contamination (internal validity)can generalize beyond specifics (external

validity)

Research involves solving a mystery – understanding why things happen

Research Strategy – collect the evidence…the data upon which your case will be based

Statistics – organize, summarize, and evaluate the strength of the evidence

Issues:organize/summarize data (correct descriptive

statistics)isolate signal from noise (correct inferential

statistics)evaluate the signal (understand what the

message is)

Research involves solving a mystery – understanding why things happen

Research Strategy – collect the evidence…the data upon which your case will be based

Statistics – organize, summarize, and evaluate the strength of the evidence

Decide on your ‘interpretation’ of the evidence Issues:

what do the results meanwhat decision will you make (guilt or

innocence)how confident are you (probability you are

wrong)

Research involves solving a mystery – understanding why things happen

Research Strategy – collect the evidence…the data upon which your case will be based

Statistics – organize, summarize, and evaluate the strength of the evidence

Decide on your ‘interpretation’ of the evidence

Persuade others of the validity of your interpretation

Issues:validity of your researchvalidity of your analyses of the datavalidity of your interpretation of the analyses

Mystery Solved