Post on 18-Jul-2020
Doug Reynolds, Development Dimensions International
Presenter:
Data-Driven Talent Management:Using assessment and technology to run better organizations
Presented by:
Doug Reynolds, Ph.D. Senior Vice President & CTODevelopment Dimensions International (DDI)
Big Data:Old Wine or New Opportunity?
Presented by:
Doug Reynolds, Ph.D. Senior Vice President & CTODevelopment Dimensions International (DDI)
“Big Data” about people is hot!
Anatomy of a trend
Reference to
Moneyball
Uninformed
over-
generalization
Pivot point
Anatomy of a trend
Review of
common
problems
Anatomy of a trend
Discovery of
known
knowledge
Nuggets of
insight from
new
technologies
What is Big Data for HR?
• A visit to the dust bowl
• A sea of spurious correlation
• High school science
• New insights from new tools
A few definitions
• Big data: Large quantity and variety of data
generated through internet-based systems.
• Digital exhaust: trace information remaining after
use of an online tool, often irrelevant to the
purpose of the tool.
• HR analytics: organizationally relevant statistics
regarding people associated with the
organization
Overview
• Why are we talking about this now?
• The promise of big data in HR
• A few examples
• The answer
A few important trends
Labor Market Management Technology
• Talent is a differentiator
• Aging workforce
• Skill gaps
• Automation
• Globalization
• Outsourcing
• Interoperability
• Services architecture
• Cloud computing
A (simplified) view
of the evolution
of business software
Start with an inefficient business process…
Input
Step 1
Step 2
Step 3
Step 4
Output
… design software to improve it
Input
Step 1
Step 2
Step 3
Step 4
Output
Simplify, automate,
increase availability,
globalize, etc.
It doesn’t take much to get started…
Competitors may capture different parts of the process
Company 1
Company 2
The next challenge: add value beyond your step
Interconnect with other
software tools to automate
more of the business
process
Acquire, merge, or build to own the whole process
Attempt to support the
whole process:
• Build more pieces
• Buy your neighbor
• Sell out
• Or, go out of
business
Once you own one process, you get hungry for others…
Process: A B C D
Once you own one process, you get hungry for others…
Process: A B C D
Interconnections allow for more insight and strategic value
Within
Process
A B C D
Across
Processes
In the HR context:
Within
Process
Across
Processes
Big data about people
Recruit
Hire
TrainManage
Promote
The promise of big data for
talent management
Strategic Impact
Process Automation
Insight
Predictive Hiring Analytics
Manager
Satisfaction
with Quality
of CandidatesRatio Offers to
Acceptance and
Diversity of
New Hires Candidate
feedback
on the hiring
process
Confidence
of Hiring Manager
in the New Hire
and Confidence
of the New Hire
that they are in
the Right Job
Job Performance
and
Engagement
SourceFinal
Interview Job Offer First Day
on the Job
6 Months
on the Job
1 year
on the jobAssessment
Candidate
Source
Assessment
Data for
Individual
and Group
A pervasive issue: assessment rigor
Talent quality?
1 2 3 4 5
Examples of
Assessment-driven
Analytics
About DDI
• 40+ Years in 3 markets:
– Personnel Selection
– Executive Assessment
– Leadership Development
• 42 offices in 26 countries
• 4M+ tests delivered/yr
• 1,500 exec assessments/yr
• Multiple software delivery
systems
Example 1: selection testing
• Selection test for graduate hiring
• Relevant and effective across cultures
• Strong security (difficult to cheat on)
• Strong predictor of performance
• Available anytime, anywhere
• Brief
Test features
• Figural reasoning: measure of reasoning ability,
critical thinking, and problem-solving
• Non-verbal/graphical items
– No translations
– Applicable regardless of candidate reading level
– Culture-free/fair for all candidate groups
– Allows for comparisons across cultures/countries
Internet-based computer adaptive testing (CAT)
CAT addresses several common issues:
Test SecurityItems drawn from extensive bank;
low item exposure rates
Cheating
Different combination of items for
each candidate; no single key
available to be used for cheating
Length of
Candidate
Experience
Compared to traditional tests,
shorter test time but superior
precision
CAT: development process
Calibration Research
• 200,000+ candidates for entry-
level professional jobs, globally
• Items researched via internet
delivered test forms
• Test timing and question
functioning developed from
response data
• Ensured the test is inclusive to all
candidates globally
Then, criterion-related validation study conducted
Results from CAT validation
Criterion Validity Coefficient
Composite Performance 0.36 (0.29)**
Gathering Information 0.18 (0.14)**
Reviewing and Analyzing Information 0.34 (0.27)**
Decision Making 0.33 (0.26)**
Strategic and Operational Agility 0.20 (0.16)**
Innovation 0.20 (0.16)**
Potential 0.27 (0.21)**
Adaptability 0.25 (0.20)**
Note. N=596 ** p < 0.01. Validity coefficients have been corrected only for unreliability in the criterion using a reliability estimate of 0.63. Values in parentheses represent uncorrected validity coefficients.
Interview scores by score group
0
500
1000
1500
2000
2500
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
RT scores with passing RS score RT scores with or without RS scoreNext step scores with passing CAT Next Step scores with or without passing CAT
Assessment-driven metrics
• Employer
brand index
• Recruiting
effectiveness
metrics
Test scores + HRIS data:
Turnover survival analysis
Key characteristics:
• Day-in-the-life format
• Technology driven
• Live interactions
• Deployed globally
Example 2: Tech-facilitated assessment center
72
87
25
100
98
88
95
Adjustment
Ambition
Sociability
Interpersonal Sensitivity
Prudence
Inquisitiveness
Learning Orientation
Leadership Effectiveness Inventory
33
10
55
24
15
13
11
53
69
24
72
Volatile
Argumentative
Risk Averse
Imperceptive
Avoidant
Arrogant
Impulsive
Attention Seeking
Eccentric
Perfectionistic
Approval Dependent
Leadership Challenge Inventory
Sample Output
PE
RS
ON
AL
ITY
PA
TT
ER
NS
CO
MP
ET
EN
CIE
SKaren GatesVice President , Operations
Started: 02.04.2000
Previous Position: Director, Operations
Education: MBA, Wharton Business School
Known Aspirations: VP of Eastern Region
Interpersonal Skills
Compelling Communication
Cultivating Networks
Navigating Politics
Influence
P
S
D
P
Business Management
D
S
P
P
P
P
Building Organizational Talent
Driving Execution
Financial Acumen
Operational Decision Making
Entrepreneurship
Establishing Strategic Direction
Leadership
S
P
S
P
Personal Competencies
Leading Change
Coaching and Developing Others
Selling the Vision
Empowerment/Delegation
P
S
Executive Disposition
Passion for Results
Multi-organization analyses
8000+ Executives
Strengths
Driving for Results
Executive Disposition
Communicating with Impact
Decision Making
Customer Focus
Strengths
Coaching
Establishing Strategic Direction
Financial Acumen
Entrepreneurship
Building Talent
Strong Managers
Strategic Leaders
Vary by:
Level?
Industry?
Experiences?
Personality profile?
Company Performance
Prior Assessments
HRIS
Company Profiles
Low
High
Med
Coaching
Customer
Focus
Business
Savvy
Empowerment
2nd
Level
4th
Level
3rd
Level
Level differences in leadership
Vacancy rate + competency analysis
Expected
vacancies:
10 Directors
66 Managers
184 Supervisors
“Cultivating Innovation” scores by position
Strength
Not Ready at this Time
Ready with Development
Ready
What if priorities change?
Competency analysis:
Driving Efficiency
Strength
Not Ready at this Time
Ready with Development
Ready
Big Data: Old wine or new opportunity
The answer:
Big data:
Old wine
and
New opportunity
(with some significant challenges ahead)
“Big Data” about people
Challenges in practice:
• Software is often an expensive hollow shell
• People data can be of poor quality
• Interpretations can be terribly flawed
• Managers defer to gut instinct
Big data: big skills required
• Complex data analysis and modeling
• Knowledge of people & systems
in organizations
• Theory building and testing
• Communication and
action planning
“Big Data” about people
Opportunities:
• Interconnections across steps add new
insights and strategic value
• Strong theory, modeling, hypothesis testing
are essential to extract meaning and order
from huge complexity
• Insight about people can be packaged to
better inform organizational strategy
Concluding thoughts:
• Multi-disciplinary approach is important
• New opportunities are emerging for
the role of assessment
• Potential for better people strategy
if we can overcome common barriers
• We have an obligation to respond to the popular
trend
Thank you!Thank you.
Big Data:
Old Wine or New Opportunity?
Doug Reynolds
July 23, 2013
Post-SIOP survey:
“What are you most excited about?”
Vacancy rate estimation +
competency analysis
Years Projected Preliminary Talent Gap Projected Talent GapProjected Talent Gap after
internal sourcing
Additional talent needed
through internal talent
identification, development
and promotion
Current
Year
Forecast
Year
Current
Headcount
Planned
Headcount
Current
Vacancies
% promoted
out of role
per year
Retire % per
year
Resign,
Terminate %
per year
% positions
historically filled
successfully
through Internal
Promotions
Target % filled
through
external hires
2013 2016 3
Director 30 45 3 18 5% 4% 1% 22 15% 18 40% 10
Manager 150 300 12 162 7% 6% 2% 188 35% 122 30% 66
Supervisor 300 550 30 280 12% 8% 4% 461 40% 277 20% 184
Total 480 895 45 460 671 417 260
Summary
Year
Talent
Supply
Talent
Demand Talent Gap
2013 435 480 45
2016 635 895 260Projected Talent Gap, given current rate of
expansion, promotions, retirments, resignations
-- and targeted % of external hires