Data Consumerism HRU

PEOPLE ANALYT

ICS:

HOW TO BE A

SMART DATA

CONSUMER

NYC 4/22/2016

HOW TO BE A SMART DATA CONSUMER4/22/16

2


IT’S NOT THAT EASY…

3


MORE REALITY THAN FICTION…

4


THREATS TO VALIDITY OF RESULTSResource: http://horan.asu.edu/cook&

campbell.htm

From the “Bible”:

5

http://horan.asu.edu/cook&campbell.htm



http://www.amazon.com/Quasi-Experimentation-Design-Analysis-Issues-Settings/dp/0395307902


6

INTERNAL VALIDITY?


INTERNAL VALIDITYGiven that there is a relationship, is it plausible there are other explanations for the model?

7


THREATS TO INTERNAL… History (effects may be due to

unforeseen events) Maturation Testing (becoming test savvy) Instrumentation Statistical Regression Selection (self or convenient selection) Mortality

8


THREATS TO INTERNAL… Interactions With Selection Ambiguity About the Direction of Causal

Inference Diffusion or Imitation of Treatments Compensatory Equalization of

Treatments (coffee talk) Compensatory Rivalry by Respondents'

Receiving Less Desirable Treatments Resentful Demoralization of

Respondents Receiving Less Desirable Treatments9


10

EXTERNAL VALIDITY?


EXTERNAL VALIDITYGiven that there’s a causal relationship, how likely is it that the conclusion is generalizable across people, groups, companies, locations, and time?

11


THREATS TO EXTERNAL… Interaction of Selection and Treatment

(participants) Interaction of Setting and Treatment

(places) Interaction of History and Treatment

12


CONSTRUCT VALIDITY?

13


14

CONSTRUCT VALIDITY?


CONSTRUCT VALIDITYDo the relationships in the model actually reflect the meaning of variables?

15


THREATS TO CONSTRUCT… Inadequate Preoperational Explication

of Constructs Mono-Operation Bias (when the boss

asks the questions) Mono-Method Bias Hypothesis Guessing within

Experimental Conditions Evaluation Apprehension

16


THREATS TO CONSTRUCT… Experimenter Expectancies (coaching) Confounding Constructs and Levels of

Constructs Interaction of Different Treatments Interaction of Testing and Treatment

(attention time!) Restricted Generalizability Across

Constructs

17


18

STATISTICAL CONCLUSION VALIDITY?


19

STATISTICAL CONCLUSION VALIDITY?


STATISTICAL CONCLUSION VALIDITYAre we correctly analyzing the data?

20


THREATS TO STATISTICAL… Low Statistical Power Violated Assumptions of Statistical

Tests Fishing and the Error Rate Problem (in

Kansas City…) The Reliability of Measures The Reliability of Treatment

Implementation Random Irrelevancies in the

Experimental Setting Random Heterogeneity of Respondents21


22


STEVE LEVYwww.linkedin.com/in/stevenmlevywww.twitter.com/[email protected]

23

http://www.linkedin.com/in/stevenmlevy

http://www.twitter.com/levyrecruits

http://www.recruitinginferno.com/

mailto:[email protected]

Data Consumerism HRU

Documents

Transcript of Data Consumerism HRU