A GAME CHANGER IN EMPLOYEE...

Innovations in Gamified Assessment

A GAME CHANGER IN

EMPLOYEE SELECTION

TALEGENT I n n o v a t i o n s i n G a m i f i e d A s s e s s m e n t

“I did my online test yesterday, thought it was quite fun actually. Got way better for the

second part and I think I got 2 or 3 'nailed it!' where I got them all right, but then some I got maybe half only, they kept buzzing out even though I swear they belonged in that spot lol”

From Whirlpool – Graduate Testing Online Forum


- Overview of Gamified Assessments

- Value of Gamified Assessments

- What can we measure?

- Case Study Development of Gamified Assessment

Implementation of a Gamified Assessment

Outcomes of using a Gamified Assessment

- Reliability and Validity of Gamified Assessments

Reliability

Validity

- Next Generation of Gamified Assessments

CONTENTS

Engaging Assessments for the Digital Age

OVERVIEW OF GAMIFIED ASSESSMENTS


Gamified assessments blend game elements with robust psychometric methodologies.

Engaging

Reliable

Valid

Making data gathering more enjoyable

VALUE OF GAMIFIED ASSESSMENTS


Gamification has consistently toped the list of

upcoming tech trends in HR for the last 5 years.

http://www.forbes.com/sites/jeannemeister/2015/03/30/future-of-work-using-gamification-for-human-resources/#56b5ef4032ba


Drive to provide a positive

recruitment experience for

candidates.

Leveraging game elements

and narrative to create a

two way interaction.

Why gamify?

WHAT CAN WE MEASURE?


Leads to increased

intrinsic motivation

Can measure similar

constructs to standard

assessments with

correlations ranging from

rs = 0.45 - 0.60

Performance may be

multidimensional

Lumsden et al (2016) Gamification of Cognitive Assessment and Cognitive Training: A Systematic Review of Applications and Efficacy. JMIR Serious Games, 4 (2)

Previous Research on Gamified Assessments

High volume Graduate Recruitment

CASE STUDY


- A large professional services firm

- Assesses large volumes (10K+) of

graduates for recruitment purposes

- Create a recruitment solution which

measured cognitive ability

- Positive candidate experience

- Mobile first

- Promote the companies employee

value proposition


Interactive Gamified Assessments

RECRUITMENT CAMPAIGN INFO

GAMIFIED ASSESSMENT

Interesting Stats So Far VIDEO DEMO


1798

4843

3113

338 215

815

1351

Feb Mar Apr May Jun Jul Aug

Completions so far… G

am

ifie

d A

sse

ssm

en

t C

om

ple

tion

s b

y M

onth

12,558

Completions

45 calls to the Helpdesk

(0.35% of total

candidates)

38 Sessions completed

on a mobile device

and 26 on a tablet


- Chrome was the most

popular browser with

69% share

- Safari made up 17%

- Firefox 9% and IE 5%

Internet

Explorer

5%

Firefox

9%

Safari

17%

Chrome

69%

Browser Usage


Example Output


REPORT


Positive Candidate Experience

Whirlpool - Graduate Testing Online Forum


CANDIDATE COMMENTS

“The online testing…very cool and different which i

enjoyed”

“The test was pretty enjoyable”

“Had a lot of fun in the testing, prefer this to any other type

of testing.”

“Yep loved this type of test.”

“The test was pretty enjoyable imo. All about logic no

comprehension or math bull lol….”


“Talegent has really brought this idea to life. They identified the job-relevant competencies that would translate well to be tested via a game interface and designed the questions and game logic”

Tanyth Lloyd | Talent Acquisition Manager

- Engaging Candidates

with innovative gamified

experience

- Strengthen employment

brand

- Psychometrically sound

Benefits Innovation


We matched the gamified campaign

with a traditional recruitment campaign

Graduates

Australian Organisation

Well regarded brand

Found a campaign which shared

31.17% of its candidates with the

gamified recruitment campaign

Gamified

(N =12,763) Traditional (N = 1,745)

Comparing it to traditional assessments


The gamified

assessment had lower

drop out rates

compared to the

traditional assessment

campaign.

Drop out Rates vs Traditional Assessments

Traditional Gamified

13.92%

1.18%

Do they measure anything?

RELIABILITY AND VALIDITY


0

100

200

300

400

500

600

700

800

900

1000

-555.7 -469.1 -382.5 -295.9 -209.3 -122.7 -36.1 50.5 137.1 223.7 310.3 396.9 483.5 570.1 656.7

Nu

mb

er

of

Ap

plican

ts

Raw Ability Score

Distribution of Gamified Assessment Raw Score

Distribution Of Raw Score

Item Response Theory Model

a

Measuring Ability

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1 2 3 4 5 6 7 8 9 10

Pro

bab

ilit

y o

f p

lacin

g o

pe

rato

r

co

rre

ctl

y

Learning Experience With Operator

• Previous assessments have been static.

• Not enabling applicants to experiment or learn.

• The gamified assessment provides applicants with this opportunity enabling

the measurement of learning agility.

Measuring Learning Agility

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1 2 3 4 5 6 7 8 9 10

Pro

bab

ility

of pla

cing

opera

tor

corr

ect

ly

Operator Usage

Probability of placing operator correctly over operator usage by agility level

High

Average

Low

Looking at the response data we can see that applicants differ in the speed with which they learn the operator function.

Some applicants show high levels of learning agility and learn the dynamic systems quickly, while other applicants take more

experimentation attempts to learn.

Measuring Learning Agility

Dynamic IRT Linked to Learning Agility

The reliability of the Gamified Assessment

“… So gamified assessment with Dynamic IRT are a reliable predictor

of candidate ability.”

And how valid is gamified assessment…?

Gamified Verbal Numeric Logical

Gamified 1.00 0.11* 0.42** 0.41**

Verbal 0.11* 1.00 0.40** -0.02

Numeric 0.42** 0.40** 1.00 0.20*

Logical 0.41** -0.02 0.20* 1.00

** < 0.01, * = 0.05

As expected is weak correlation with Verbal

Reasoning

Verbal Reasoning with r (248) = 0.11, p < 0.05

Moderate correlations with

Numeric Reasoning with r (248) = 0.42, p < 0.01

Logical Reasoning with r (248) = 0.41, p < 0.01

“… and can measure similar abilities to traditional assessments.”

Where to next? NEXT GENERATION OF GAMIFIED ASSESSMENTS


New technology is opening up a virtual world to

assessment developers full of amazing new

opportunities to measure abilities and traits in

new and engaging ways. In near future

developers will create high fidelity immersive

simulations of work environments.

These new experiences will provide

opportunities to measure new constructs and

new challenges.

Virtual Reality, Blended

Reality, High Fidelity

Simulations

Gamified Assessments SUMMARY


Gamified assessments

can measure similar

abilities to traditional

cognitive ability

assessments

Has lower drop

out rates than

traditional

assessments

Sound reliability

and validity

© 2016 Talegent. All Rights Reserved.

Applying new psychometric assessment

technologies to meet challenges of online

recruitment.

Auckland, New Zealand

Level 18, 48 Emily Place, Auckland City 1010

PO Box 105429 Auckland City 1143

0508 ASSESS (277 377) or (09) 282 4739

A GAME CHANGER IN EMPLOYEE...

Documents

Transcript of A GAME CHANGER IN EMPLOYEE...