Doug Reynolds, Development Dimensions International

Presenter:

Data-Driven Talent Management:Using assessment and technology to run better organizations

Presented by:

Doug Reynolds, Ph.D. Senior Vice President & CTODevelopment Dimensions International (DDI)

Big Data:Old Wine or New Opportunity?

Presented by:

Doug Reynolds, Ph.D. Senior Vice President & CTODevelopment Dimensions International (DDI)

“Big Data” about people is hot!

Anatomy of a trend

Reference to

Moneyball

Uninformed

over-

generalization

Pivot point

Anatomy of a trend

Review of

common

problems

Anatomy of a trend

Discovery of

known

knowledge

Nuggets of

insight from

new

technologies

What is Big Data for HR?

• A visit to the dust bowl

• A sea of spurious correlation

• High school science

• New insights from new tools

A few definitions

• Big data: Large quantity and variety of data

generated through internet-based systems.

• Digital exhaust: trace information remaining after

use of an online tool, often irrelevant to the

purpose of the tool.

• HR analytics: organizationally relevant statistics

regarding people associated with the

organization

Overview

• Why are we talking about this now?

• The promise of big data in HR

• A few examples

• The answer

A few important trends

Labor Market Management Technology

• Talent is a differentiator

• Aging workforce

• Skill gaps

• Automation

• Globalization

• Outsourcing

• Interoperability

• Services architecture

• Cloud computing

A (simplified) view

of the evolution

of business software

Start with an inefficient business process…

Input

Step 1

Step 2

Step 3

Step 4

Output

… design software to improve it

Input

Step 1

Step 2

Step 3

Step 4

Output

Simplify, automate,

increase availability,

globalize, etc.

It doesn’t take much to get started…

Competitors may capture different parts of the process

Company 1

Company 2

The next challenge: add value beyond your step

Interconnect with other

software tools to automate

more of the business

process

Acquire, merge, or build to own the whole process

Attempt to support the

whole process:

• Build more pieces

• Buy your neighbor

• Sell out

• Or, go out of

business

Once you own one process, you get hungry for others…

Process: A B C D

Once you own one process, you get hungry for others…

Process: A B C D

Interconnections allow for more insight and strategic value

Within

Process

A B C D

Across

Processes

In the HR context:

Within

Process

Across

Processes

Big data about people

Recruit

Hire

TrainManage

Promote

The promise of big data for

talent management

Strategic Impact

Process Automation

Insight

Predictive Hiring Analytics

Manager

Satisfaction

with Quality

of CandidatesRatio Offers to

Acceptance and

Diversity of

New Hires Candidate

feedback

on the hiring

process

Confidence

of Hiring Manager

in the New Hire

and Confidence

of the New Hire

that they are in

the Right Job

Job Performance

and

Engagement

SourceFinal

Interview Job Offer First Day

on the Job

6 Months

on the Job

1 year

on the jobAssessment

Candidate

Source

Assessment

Data for

Individual

and Group

A pervasive issue: assessment rigor

Talent quality?

1 2 3 4 5

Examples of

Assessment-driven

Analytics

About DDI

• 40+ Years in 3 markets:

– Personnel Selection

– Executive Assessment

– Leadership Development

• 42 offices in 26 countries

• 4M+ tests delivered/yr

• 1,500 exec assessments/yr

• Multiple software delivery

systems

Example 1: selection testing

• Selection test for graduate hiring

• Relevant and effective across cultures

• Strong security (difficult to cheat on)

• Strong predictor of performance

• Available anytime, anywhere

• Brief

Test features

• Figural reasoning: measure of reasoning ability,

critical thinking, and problem-solving

• Non-verbal/graphical items

– No translations

– Applicable regardless of candidate reading level

– Culture-free/fair for all candidate groups

– Allows for comparisons across cultures/countries

Internet-based computer adaptive testing (CAT)

CAT addresses several common issues:

Test SecurityItems drawn from extensive bank;

low item exposure rates

Cheating

Different combination of items for

each candidate; no single key

available to be used for cheating

Length of

Candidate

Experience

Compared to traditional tests,

shorter test time but superior

precision

CAT: development process

Calibration Research

• 200,000+ candidates for entry-

level professional jobs, globally

• Items researched via internet

delivered test forms

• Test timing and question

functioning developed from

response data

• Ensured the test is inclusive to all

candidates globally

Then, criterion-related validation study conducted

Results from CAT validation

Criterion Validity Coefficient

Composite Performance 0.36 (0.29)**

Gathering Information 0.18 (0.14)**

Reviewing and Analyzing Information 0.34 (0.27)**

Decision Making 0.33 (0.26)**

Strategic and Operational Agility 0.20 (0.16)**

Innovation 0.20 (0.16)**

Potential 0.27 (0.21)**

Adaptability 0.25 (0.20)**

Note. N=596 ** p < 0.01. Validity coefficients have been corrected only for unreliability in the criterion using a reliability estimate of 0.63. Values in parentheses represent uncorrected validity coefficients.

Interview scores by score group

0

500

1000

1500

2000

2500

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

RT scores with passing RS score RT scores with or without RS scoreNext step scores with passing CAT Next Step scores with or without passing CAT

Assessment-driven metrics

• Employer

brand index

• Recruiting

effectiveness

metrics

Test scores + HRIS data:

Turnover survival analysis

Key characteristics:

• Day-in-the-life format

• Technology driven

• Live interactions

• Deployed globally

Example 2: Tech-facilitated assessment center

72

87

25

100

98

88

95

Adjustment

Ambition

Sociability

Interpersonal Sensitivity

Prudence

Inquisitiveness

Learning Orientation

Leadership Effectiveness Inventory

33

10

55

24

15

13

11

53

69

24

72

Volatile

Argumentative

Risk Averse

Imperceptive

Avoidant

Arrogant

Impulsive

Attention Seeking

Eccentric

Perfectionistic

Approval Dependent

Leadership Challenge Inventory

Sample Output

PE

RS

ON

AL

ITY

PA

TT

ER

NS

CO

MP

ET

EN

CIE

SKaren GatesVice President , Operations

Started: 02.04.2000

Previous Position: Director, Operations

Education: MBA, Wharton Business School

Known Aspirations: VP of Eastern Region

Interpersonal Skills

Compelling Communication

Cultivating Networks

Navigating Politics

Influence

P

S

D

P

Business Management

D

S

P

P

P

P

Building Organizational Talent

Driving Execution

Financial Acumen

Operational Decision Making

Entrepreneurship

Establishing Strategic Direction

Leadership

S

P

S

P

Personal Competencies

Leading Change

Coaching and Developing Others

Selling the Vision

Empowerment/Delegation

P

S

Executive Disposition

Passion for Results

Multi-organization analyses

8000+ Executives

Strengths

Driving for Results

Executive Disposition

Communicating with Impact

Decision Making

Customer Focus

Strengths

Coaching

Establishing Strategic Direction

Financial Acumen

Entrepreneurship

Building Talent

Strong Managers

Strategic Leaders

Vary by:

Level?

Industry?

Experiences?

Personality profile?

Company Performance

Prior Assessments

HRIS

Company Profiles

Low

High

Med

Coaching

Customer

Focus

Business

Savvy

Empowerment

2nd

Level

4th

Level

3rd

Level

Level differences in leadership

Vacancy rate + competency analysis

Expected

vacancies:

10 Directors

66 Managers

184 Supervisors

“Cultivating Innovation” scores by position

Strength

Not Ready at this Time

Ready with Development

Ready

What if priorities change?

Competency analysis:

Driving Efficiency

Strength

Not Ready at this Time

Ready with Development

Ready

Big Data: Old wine or new opportunity

The answer:

Big data:

Old wine

and

New opportunity

(with some significant challenges ahead)

“Big Data” about people

Challenges in practice:

• Software is often an expensive hollow shell

• People data can be of poor quality

• Interpretations can be terribly flawed

• Managers defer to gut instinct

Big data: big skills required

• Complex data analysis and modeling

• Knowledge of people & systems

in organizations

• Theory building and testing

• Communication and

action planning

“Big Data” about people

Opportunities:

• Interconnections across steps add new

insights and strategic value

• Strong theory, modeling, hypothesis testing

are essential to extract meaning and order

from huge complexity

• Insight about people can be packaged to

better inform organizational strategy

Concluding thoughts:

• Multi-disciplinary approach is important

• New opportunities are emerging for

the role of assessment

• Potential for better people strategy

if we can overcome common barriers

• We have an obligation to respond to the popular

trend

Thank you!Thank you.

Big Data:

Old Wine or New Opportunity?

Doug Reynolds

July 23, 2013

Post-SIOP survey:

“What are you most excited about?”

Vacancy rate estimation +

competency analysis

Years Projected Preliminary Talent Gap Projected Talent GapProjected Talent Gap after

internal sourcing

Additional talent needed

through internal talent

identification, development

and promotion

Current

Year

Forecast

Year

Current

Headcount

Planned

Headcount

Current

Vacancies

% promoted

out of role

per year

Retire % per

year

Resign,

Terminate %

per year

% positions

historically filled

successfully

through Internal

Promotions

Target % filled

through

external hires

2013 2016 3

Director 30 45 3 18 5% 4% 1% 22 15% 18 40% 10

Manager 150 300 12 162 7% 6% 2% 188 35% 122 30% 66

Supervisor 300 550 30 280 12% 8% 4% 461 40% 277 20% 184

Total 480 895 45 460 671 417 260

Summary

Year

Talent

Supply

Talent

Demand Talent Gap

2013 435 480 45

2016 635 895 260Projected Talent Gap, given current rate of

expansion, promotions, retirments, resignations

-- and targeted % of external hires