Présentation PowerPointA Focus on Evidence-Based Management
Leading scholars have promoted EBM as an integrated and
comprehensive approach to help bridge the rigor-relevance gap and
improve the management of organizations
Denise M. Rousseau’s 2005 Presidential Address to the
Academy of Management
Jeffrey Pfeffer’s 2007 Keynote Address to the Society for
Industrial and Organizational Psychology
Resources: A Primer and Summary of Current Literature.
Human Resources, Law and Management Department - September
2009
The Origins: Evidence-Based Medicine
“the conscientious, explicit and judicious use of current best
medical/clinical research evidence in making decisions about the
care of individual patients.”
Dr. David Sackett (McMaster University)
Human Resources, Law and Management Department - September
2009
Evidence-Based Management
Ignorance is arguably higher in management than in medicine…
“If doctors practiced medicine like many companies practice
management, there would be more unnecessarily sick or dead patients
and many more doctors in jail or suffering other penalties for
malpractice”
(Pfeffer & Sutton, 2006, p1)
Pfeffer, J. & Sutton, R. I. (2006). Evidence-based management.
Harvard Business Review, January, pp1-12.
Confer also Rousseau, D. M. (2006). Is there such a thing as
"evidence-based management"? Academy of Management Review. 31, pp
256-269
Human Resources, Law and Management Department - September
2009
What is Evidence-Based Management?
A “way of thinking” about organizational decisions in a systematic
manner
Being committed to doing things based critically on the “best”
available scientific evidence
Not solely based upon unsystematic experience, instincts,
intuition, tradition, untested beliefs, trends and fashions
etc.
Translating the best scientific evidence into organizational
practices (“Big E Evidence”)
Systematically gathering local data to inform decision-making
(“little e evidence”)
Learning about human behavior within organizations and using that
knowledge to enhance managerial practice.
Human Resources, Law and Management Department - September
2009
What is Evidence-Based Management?
practitioner expertise and judgment
a critical evaluation of the best available research evidence
the perspectives of those people who might be affected by the
decision
Briner, R. B., Denyer, D., & Rousseau, D. M. (2009).
Evidence-Based Management: Concept Cleanup Time? Academy of
Management Perspectives. 23, pp19-32.
“Using evidence does not mean slavishly following it, acting only
when there is good evidence, or doing nothing if there is
none.”
Briner, R. B. and Rousseau, D. M. (2011). Evidence-Based I–O
Psychology: Not There Yet. Industrial and Organizational
Psychology.4, pp3–22.
Human Resources, Law and Management Department - September
2009
What EBM is !
Something practitioners already do to some extent
About the practice of management
A family of related approaches to decision making
A way of thinking about how to make decisions
About using a wide range of different kinds of research evidence
depending on the problem
Practitioners using research as only one source of information to
inform decisions
A means of getting existing management research out to
practitioners
Likely to help both the process and outcome of practitioner
decision-making
About questioning ideas such as “best practice”
Human Resources, Law and Management Department - September
2009
What EBM is NOT!
NOT A brand-new way of making decisions
NOT About conducting particular types of research
NOT A single decision-making method
NOT A rigid, one-size-fits all decision-making formula
NOT About using only certain types of research evidence
irrespective of the problem
NOT Scholars or researchers telling practitioners what to do
NOT About conducting research only about management practice
NOT The solution to all management problems
NOT About identifying and promoting “best practice”
Human Resources, Law and Management Department - September
2009
EBM QUIZ
Rynes, S. L., Colbert, A. E., & Brown, K. G. (2002). HR
professionals‘ beliefs about effective human resource practices:
Correspondence between research and practice. Human Resources
Management, 41(2), pp149-174.
Rynes, S. L., Giluk, T. L., & Brown, K. G. (2007). The very
separate worlds of academic and practitioner periodicals in human
resource management: Implications for evidence-based management.
Academy of Management Journal, 50(5), pp987-1008.
TRUE or FALSE?
General mental ability is the strongest, or one of the strongest,
predictors of job performance
TRUE
Setting goals and providing feedback is a highly effective
motivational practice
TRUE
FALSE
TRUE
TRUE
EBM QUIZ
On average, encouraging employees to participate in decision making
is more effective for improving organizational performance than is
setting performance goals
FALSE
Most people under-evaluate how well they perform on the job
FALSE
Training for simple skills will be more effective if it is
presented in one concentrated session than if it is presented in
several sessions over time
FALSE
“Integrity tests” designed to predict whether someone will steal,
be absent, etc. can predict those types of counterproductive
behaviors
TRUE
Graphology (handwriting analysis) is able to predict job
performance effectively because it is difficult to “fake”
FALSE
Theory?
An organized framework of concepts aimed at illustrating,
interpreting and/or predicting reality
“It is the theory that decides (determines) what we can
observe”
→ A Frame or an Abstraction of Reality Albert Einstein (via
Gregory, 1990)
“There is nothing more practical than a good theory” Kurt Lewin
(1951,p169)
*
Independent Variable (Predictor) & Dependent Variable
(Criterion)
a theoretical model establishes a causal relationship between 2
variables which is then tested in research studies
the independent variable (IV) can be either qualitative or
quantitative and is typically the variable being manipulated or
changed in the study→ What is being changed
the dependent variable (DP) can be either qualitative or
quantitative and is the observed result of the independent variable
being manipulated or changed → What is being observed?
Be aware that finding a relation between the IV & the DP
doesn't establish the direction of causality
Causality or which is the cause and which is the effect can only be
established by the type of the research design
Human Resources, Law and Management Department - September
2009
2 Types of Research Design…
Concurrent or Cross-Sectional Research Design: both the independent
and the dependent variable are measured in the same point in time
(roughly simultaneously)→ A Cross-Section in the
Time-Continuum
Predictive and/or Longitudinal Research Design: the independent
variable is measured at an earlier point in time than the dependent
variable and/or repeatedly for a long time period
Only Predictive and/or Longitudinal Research Designs can establish
Causality and consequently they are considered scientifically
superior
However in the position of IV you ought to find that variable which
is at least theoretically assumed to be the cause of an effect in
the DP, with the former being treated as the predictor and the
later as the criterion
Human Resources, Law and Management Department - September
2009
Reliability
reliability: the estimated consistency of the measurement for a
given instrument/method (e.g. test) employed with itself (internal
validity), across time (test-retest) and within/across raters
(intra/inter-rater)→ the repeatability of the measurement
the level of reliability puts an upper limit to the possible level
of validity
high reliability is necessary but not sufficient for high
validity
internal reliability: coefficient alpha 0.80(at least 0.70) for a
measure to have an acceptable degree of reliability
Inter-rater agreement: minimum level 75% or higher
test-retest reliability: concerned with time stability of
measurement; level of r should range between r = .50 to .90
Intra-rater agreement: for short time intervals between measures, a
fairly high relationship is expected; r = .80 to .90
Human Resources, Law and Management Department - September
2009
Validity
conclusion or criterion related validity: the degree to which the
independent variable measures or is correlated to the dependent
variable (correlation coefficient: r)
concurrent validity under a concurrent or cross-sectional research
design
predictive validity under a predictive research design
the square of the correlation coefficient states the amount of
variance in the DV explained by the IV
internal validity: an overall inductive estimation of the degree of
established causality between IV & DV
external validity: an estimation of the degree of generalizability
of our conclusions at the population level
Human Resources, Law and Management Department - September
2009
Intervening Variables: Moderator
intervening variables: variables which interact with the IV
affecting its relationship with the DV and which when incorporated
in our models upgrade theoretical rigor
moderator: a variable that interacts with the IV affecting the
direction (positive or negative) and/or strength of its relation
with the DV
a type of multiplicative statistical interaction
the moderator can be either a qualitative (e.g. gender, race…) or a
quantitative (e.g. amount of pay…) variable
be aware that the moderator stands at the same point of causal
precedence or supremacy with the IV
Human Resources, Law and Management Department - September
2009
Intervening Variables: Mediator
mediator: a variable through which the IV relates with the DV
either partially or fully
a mediation is not a statistical interaction
a mediator can be either a qualitative or a quantitative
variable
when the independent variable is related to the DV both indirectly
through the mediator and directly we have a partial mediation
when the IV is related to the DV only through the mediator then we
have a full mediation
the mediator is causally inferior to the IV but causally superior
to the DV
Human Resources, Law and Management Department - September
2009
Control Variables
control variables: variables known from research to be having an
effect on the DV and for that reason are kept constant to prevent
their influence on the effect of the IV on the DV→ alternative
causes (predictors) of the dependent variable (criterion)
→ What other variables have to be held constant in order to isolate
the net effect of the IV to the DV?