Applying Science Towards Understanding Behavior in Organizations

Chapters 2 & 3

Research Issues in Organizations

Approaches to collecting data Experimental Observational/correlational

Data collection issues Sampling

How should we select participants? What impact does it have on the results?

Experimental design Controlling potential confounds Assigning participants to experimental conditions

Measurement issues Describing and interpreting the results

Experiments: A Review

Experiments - Do changes in one variable (X) “cause” changes in another variable (Y)? Independent Variable (X)

condition or event that is manipulated by experimenter

Dependent Variable (Y) variable that is affected (hopefully) by manipulating

independent variable Extraneous Variable(s)

any variable other than independent variable that may influence dependent variable

Experiments: Pros and Cons

Advantage: Allows conclusions about direct effects of one

variable on another

Disadvantages: Experimental conditions are artificial

results may not “generalize” to the real world Some questions can’t be tested in an

experiment Require control that is not always available in the

“real” world

Experimental Design

Controlling potential confounds Goal of experiment is to “rule out” alternate

explanations of what affected dependent variable

Confounds are threats to internal validity Can be controlled through appropriate

experimental design and procedures

Internal ValidityHistoryMaturationTestingInstrumentationStatistical RegressionSelectionMortalitySelection-MaturationDiffusion of Treatment

External ValiditySampleSetting (e.g., culture)Time (e.g., 60s vs. 90s)Replication (lack of)

Do the results of this experiment generalize (apply) to settings other than the experimentIs there another reason (other than the independent variable) that could explain the results of the experiment.

Validity

How participants are selected for a study influences the extent to which the results can be applied to a larger group (external validity). A wide variety of techniques are available

Two Main types of sampling Probability

predetermined chance of any individual in the population being selected for the study

Nonprobability Typically nonrandom sampling

Sampling

Sampling Techniques

Probability Sampling1. Simple random sampling2. Systematic sampling3. Stratified random sampling4. Cluster sampling5. Multistage sampling

Nonprobability Sampling1. Convenience sampling2. Quota sampling3. Snowball sampling

Post with no Control Group

Training Posttest

Pre – Post with no Control Group

Pretest Training Posttest

Control Group with no PretestExperimental

Group Training Posttest

Control Group Placebo Posttest

GroupDifferences

Pre – Post with Control Group

Pretest ExperimentalTraining Posttest

Pretest Control Posttest

GroupDifferences

GroupDifferences

Measurement

Measurement – the process of assigning numbers to objects or events according to rules (Linn & Gronlund, 1995).

Psychological Measurement – concerned with evaluating individual differences in psychological traits. Trait – descriptive label applied to a group of

behaviors (e.g., friendly; intelligent)

Utilizing Individual Differences Psychologists assume that most traits are normally

distributed in the population. e.g., height, intelligence, KSAs

Psychologists study: measuring these differences using these differences to predict performance

I/O Psychologists typically primarily rely on these as predictors of job performance: Cognitive abilities Personality

Types of Assessment

Biographical Information Interviews Work Samples Letters of Recommendation Psychological Tests

Biographical Data

Good questions are about events that are: historical external discrete controllable (by the individual) verifiable equal access job relevant non-invasive(Mael, 1991)

Rationale vs. empirical method

Biographical Data

Strong criterion validity drug use, criminal history predicts dysfunctional police

behavior (Sarchione et al., 1998) not redundant with personality (McManus & Kelly, 1999)

Measurement issues Generalizability Faking Fairness Privacy concerns

Interviews Structured vs. Unstructured Info. gathering vs. interpersonal behavior sample Situational interview

“How would you handle a circumstance in which you needed the help of a person you did not like?”

Measurement issues structured has more criterion related validity value of unstructured? Illusion of validity

Guidelines for structured interviews

Work Samples

perform a task under standardized conditions historically were for blue collar jobs

e.g. use of tools, demonstrate driving skills white collar examples

speech interview for foreign worker, test of basic chemistry knowledge,

Measurement issues high criterion validity if skills are similar to job costly to administer work best with mechanical, rather than people-oriented

tasks

Assessment Centers Realistic tasks done in groups Assessed by multiple of raters rating multiple domains Multiple methods

in basket group exercise leaderless group exercise

Strong criterion validity (e.g., teachers, police) overall scores predict job performance

Measurement issues costly to administer different ratings on a task too highly correlated dimension ratings not correlated strongly across tasks fix? focus on behavior checklists and rater training

Drug Testing opinion? People are more accepting of it if job involves risks

to others (Paronto, et al., 2002) Measurement issues

reliability is very high, but not perfect Validity?

Normands, Salyards, & Mahoney (1990) over 5000 postal service applicants those who tested positive had 59% higher absenteeism,

47% more likely to be fired no differences in injury or accidents

Letters of Recommendation ever written a letter of recommendation for

someone? worst criterion validity of all commonly used

assessment tools some use for screening extremely bad candidates

Measurement issues restriction of range writer bias/investment

Psychological Test Characteristics

Group vs. individual Objective vs. open-ended Paper & pencil vs. performance Power vs. speed

Psychological Test Types

Ability Tests Cognitive ability Psychomotor ability

Knowledge and skill or achievement Integrity Personality Emotional Intelligence Vocational interest

Integrity Tests

Designed to predict whether employee will engage in counterproductive work behavior (CWB) overt vs. personality (covert)

Better at predicting general CWB and performance than theft (r = .30 -.40)

Measurement issues difficult to measure criteria! proprietary issues legal and privacy issues faking

Personality Tests

measures predispositions toward particular feelings and behaviors

not all tests are based on past research many have shown incremental validity

e.g., predict when controlling for IQ Measurement issues

job relevance not easily/often faked or a problem if faked (e.g.,

job faking too)

The Big Five Inventory Openness

Highs: imaginative, creative, and to seek out cultural and educational experiences.

Lows: more down-to-earth, less interest in art & more practical. Conscientiousness

Highs: methodical, well organized and dutiful. Lows: less careful, less focused & more likely to be distracted

Extraversion Highs: energetic and seek out the company of others. Lows (introverts): tend to be more quiet and reserved.

Agreeableness Highs: tend to be trusting, friendly and cooperative. Lows: tend to be more aggressive and less cooperative

Neuroticism Highs: prone to insecurity and emotional distress. Lows: more relaxed, less emotional and less prone to distress.

Cognitive Tests Have greatest validity Often very easy and inexpensive to use

Wonderlic Personnel Test 50 items 12 minute time limit Sample questions Interpreting scores?

Scores vary as a function of race and ethnicity Ethical issues? Face validity?

Psychological Test Characteristics

Group vs. individual Objective vs. open-ended Paper & pencil vs. performance Power vs. speed

Reliability and Validity Reliability

Test-retest Parallel (Alternate) forms Internal Consistency

Validity Face Content Criterion-related Construct-related