LIST

honey candy dinner present sword belief shore kitchen cradle snake

TARGETS

honey candy dinner present sword belief shore kitchen cradle snake

Hit Rate= .80

FOILS

drama folly thorn message drink ground doctor woods journal sister

FA Rate= .30

Hit(Correct ID)

TrueState

Present

Absent

Present Absent

DiagnosticDecision

Miss

CorrectRejection

False Alarm(False ID)

.80

.30

.20

.70

Eyewitness Identification Procedures

Simultaneous Lineup

Suspect:Innocent or

Guilty?

Fillers:All are known to be

innocent

Eyewitness Identification Procedures

Sequential LineupSimultaneous Lineup

Suspect:Innocent or

Guilty?

Simultaneous LineupSimultaneous Lineup

TARGET-PRESENT LINEUP(N=100)

TARGET-ABSENT LINEUP(N=100)

Mock-Crime Laboratory StudiesEach participant (n = 200) first watches a simulated crime

1 2 3 4 5 6 7 8 9 10

1 S S S S S S S S S S

2 S S S S S S S S S S

3 S S S S S S S S S S

4 S S S S S S S S S S

5 S S S S S S S S S S

6 S S S S S S S S F F

7 F F F F F F F F F F

8 N N N N N N N N N N

9 N N N N N N N N N N

10 N N N N N N N N N N

Suspect ID Rate: 0.58

Filler ID Rate: 0.12

No ID rate: 0.30

Target-Present Lineup

(Guilty)

1 2 3 4 5 6 7 8 9 10

1 S S S S S S S S S S

2 S S S S S S S S S S

3 S S S S S S S S S S

4 S S S S S S S S S S

5 S S S F F F F F F F

6 F F F F F F F F N N

7 N N N N N N N N N N

8 N N N N N N N N N N

9 N N N N N N N N N N

10 N N N N N N N N N N

Suspect ID Rate: 0.43

Filler ID Rate: 0.15

No ID rate: 0.42

Target-Absent Lineup

(Innocent)

Lindsay & Wells (1985)

Lindsay & Wells (1985)

Lindsay & Wells (1985)

Simultaneous lineup Correct Suspect ID rate = 0.58 False Suspect ID rate = 0.43

Sequential lineup Correct Suspect ID rate = 0.50 False Suspect ID rate = 0.17

.58—— .43

Diagnosticity Ratio

= 2.94

= 1.35

.50—— .17

Lindsay & Wells (1985)

Simultaneous lineup Correct Suspect ID rate = 0.58 False Suspect ID rate = 0.43

Sequential lineup Correct Suspect ID rate = 0.50 False Suspect ID rate = 0.17

SimultaneousSequential

False ID Rate

0.0 0.1 0.2 0.3 0.4 0.5

Cor

rect

ID

Rat

e

0.0

0.2

0.4

0.6

0.8

Lindsay & Wells (1985)

Simultaneous lineup Correct Suspect ID rate = 0.58 False Suspect ID rate = 0.43

Sequential lineup Correct Suspect ID rate = 0.50 False Suspect ID rate = 0.17

SimultaneousSequential

False ID Rate

0.0 0.1 0.2 0.3 0.4 0.5

Cor

rect

ID

Rat

e

0.0

0.2

0.4

0.6

0.8

1.352.94

Target-Present Lineup

1 2 3 4 5 6 7 8 9 10

1 1 1 2 2 2 2 2 3 3 3

2 3 3 3 3 3 3 4 4 4 4

3 4 4 5 5 5 5 5 5 5 6 Suspect ID Rate: 0.58 (Guilty)

4 6 6 6 6 6 7 7 7 7 7

5 7 7 7 7 7 7 7 7 7 7 Filler ID Rate: 0.12

6 7 7 7 7 7 7 7 7 F F

7 F F F F F F F F F F No ID rate: 0.30

8 N N N N N N N N N N

9 N N N N N N N N N N

10 N N N N N N N N N N

Target-Present Lineup

1 2 3 4 5 6 7 8 9 10

1 N N 2 2 2 2 2 3 3 3

2 3 3 3 3 3 3 4 4 4 4

3 4 4 5 5 5 5 5 5 5 6 Suspect ID Rate: 0.56 (Guilty)

4 6 6 6 6 6 7 7 7 7 7

5 7 7 7 7 7 7 7 7 7 7 Filler ID Rate: 0.12

6 7 7 7 7 7 7 7 7 F F

7 F F F F F F F F F F No ID rate: 0.32

8 N N N N N N N N N N

9 N N N N N N N N N N

10 N N N N N N N N N N

Target-Absent Lineup

1 2 3 4 5 6 7 8 9 10

1 1 1 1 1 1 1 1 1 1 1

2 2 2 2 2 2 2 2 3 3 3

3 3 3 3 3 3 4 4 4 4 4 Suspect ID Rate: 0.43 (Innocent)

4 4 5 5 5 5 5 6 6 6 6

5 7 7 7 F F F F F F F Filler ID Rate: 0.15

6 F F F F F F F F N N

7 N N N N N N N N N N No ID rate: 0.42

8 N N N N N N N N N N

9 N N N N N N N N N N

10 N N N N N N N N N N

Target-Absent Lineup

1 2 3 4 5 6 7 8 9 10

1 N N N N N N N N N N

2 2 2 2 2 2 2 2 3 3 3

3 3 3 3 3 3 4 4 4 4 4 Suspect ID Rate: 0.33 (Innocent)

4 4 5 5 5 5 5 6 6 6 6

5 7 7 7 F F F F F F F Filler ID Rate: 0.15

6 F F F F F F F F N N

7 N N N N N N N N N N No ID rate: 0.52

8 N N N N N N N N N N

9 N N N N N N N N N N

10 N N N N N N N N N N

False ID Rate

0.0 0.1 0.2 0.3 0.4 0.5C

orre

ct I

D R

ate

0.0

0.2

0.4

0.6

0.8

Varying Response Bias from liberal to conservative

SimultaneousSequential

≥ 1≥ 2

False ID Rate

0.0 0.1 0.2 0.3 0.4 0.5C

orre

ct I

D R

ate

0.0

0.2

0.4

0.6

0.8

Varying Response Bias from liberal to conservative

SimultaneousSequential

≥ 1≥ 2≥ 3≥ 4

≥ 5

≥ 6

≥ 7

False ID Rate

0.0 0.1 0.2 0.3 0.4 0.5C

orre

ct I

D R

ate

0.0

0.2

0.4

0.6

0.8

Varying Response Bias from liberal to conservative

SimultaneousSequential

3.90

5.14

7.2

2.941.351.812.28

False ID Rate

0.0 0.1 0.2 0.3 0.4 0.5C

orre

ct I

D R

ate

0.0

0.2

0.4

0.6

0.8

Varying Response Bias from liberal to conservative

SimultaneousSequential

Diabetes

1.253.31

16.3

False ID Rate

0.0 0.1 0.2 0.3 0.4 0.5C

orre

ct I

D R

ate

0.0

0.2

0.4

0.6

0.8

The Concept of Discriminability

SimultaneousSequential

False ID Rate

0.0 0.1 0.2 0.3 0.4 0.5C

orre

ct I

D R

ate

0.0

0.2

0.4

0.6

0.8

The Concept of Discriminability

SimultaneousSequential

Lindsay & Wells (1985)

Simultaneous lineup Correct Suspect ID rate = 0.58 False Suspect ID rate = 0.43

Sequential lineup Correct Suspect ID rate = 0.50 False Suspect ID rate = 0.17

SimultaneousSequential

False ID Rate

0.0 0.1 0.2 0.3 0.4 0.5

Cor

rect

ID

Rat

e

0.0

0.2

0.4

0.6

0.8

Results from ROC Analysis (#1)Simultaneous vs. Sequential

False ID Rate

0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08

Co

rre

ct I

D R

ate

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Simultaneous

Sequential

Mickes, L., Flowe, H. D., & Wixted, J. T. (2012). Journal of Experimental Psychology: Applied, 18, 361–376.

Results from ROC Analysis (#2)

Gronlund et al. (2012). Journal of Applied Research in Memory and Cognition, 1, 221–228.

Simultaneous vs. Sequential

SIMSEQ

Results from ROC Analysis (#3)

Dobolyi, D. G., & Dodson, C. S. (2013). Journal of Experimental Psychology: Applied, 19, 345-357.

Simultaneous vs. Sequential

SIM x 4SEQ x 4

SIM x 2SEQ x 2

Results from ROC Analysis (#4)

Carlson, C. A. & Carlson, M. A. (2014). Journal of Applied Research in Memory and Cognition.

False ID Rate

Corr

ect I

D R

ate

SimultaneousSequential

Results from ROC Analysis (#5)

Andersen, S. M., Carlson, C. A., Carlson, M. A. & Gronlund (2014). Personality and Individual Differences, 60, 36-40.

False ID Rate

Corr

ect I

D R

ate

SimultaneousSequential

“Despite its merits, a single diagnosticity ratio thus conflates the influences of discriminability and response bias on binary classification, which muddies the determination of which procedure, if any, yields objectively better discriminability in eyewitness performance.”

“Perhaps the greatest practical benefit of recent debate over the utility of different lineup procedures is that it has opened the door to a broader consideration of methods for evaluating and enhancing eyewitness identification performance. ROC analysis is a positive and promising step, with numerous advantages.”

“The committee concludes that there should be no debate about the value of greater discriminability – to promote a lineup procedure that brings less discriminability would be akin to advocating that the lineup be performed in dim instead of bright light.”

Diagnosticity Ratio or ROC Analysis?

What About Recent ROC Analyses?

• “…a small set of recent studies using ROC analysis has reported that discriminability (area under the ROC curve) for simultaneous lineups is as high, or higher, than that for sequential lineups.”

• “Amendola and Wixted re-analyzed a subset of the data for which proxy measures of ground truth were available…Their analyses suggested that identification of innocent suspects is less likely and identification of guilty suspects is more likely when using the simultaneous procedures. While future studies are needed, these latter findings raise the possibility that diagnosticity is higher for the simultaneous procedures.”

LIST

honey candy dinner present sword belief shore kitchen cradle snake

TARGETS

honey candy dinner present sword belief shore kitchen cradle snake

Hit Rate= .80

FOILS

drama folly thorn message drink ground doctor woods journal sister

FA Rate= .30

Hit(Correct ID)

TrueState

Present

Absent

Present Absent

DiagnosticDecision

Miss

CorrectRejection

False Alarm(False ID)

.80

.30

.20

.70

Confidence Ratings

"New" "Old"

LuresTargets

Memory StrengthH

ighM

ediu

m

Med

ium

Low

Low

Hig

h

1 2 3 4 5 6

"New" "Old"

LuresTargets

High

Medium

Low

"New" "Old"

LuresTargets

High

Medium

Low

FA = .01

"New" "Old"

LuresTargets

High

Medium

Low

FAR = .01

HR = .31

"New" "Old"

LuresTargets

High

Medium

Low

FAR = .07

"New" "Old"

LuresTargets

High

Medium

Low

FAR = .07

HR = .69

"New" "Old"

LuresTargets

High

Medium

Low

FAR = .16

"New" "Old"

LuresTargets

High

Medium

Low

FAR = .16

HR = .84

1 2 3 4 5 6

1 2 3 4 5 6

"New" "Old"

LuresTargets

High

Medium

Low

FAR = .31

"New" "Old"

LuresTargets

High

Medium

Low

FAR = .31

HR = .93

"New" "Old"

LuresTargets

High

Medium

Low

FAR = .69

"New" "Old"

LuresTargets

High

Medium

Low

FAR = .69

HR = .99

1 2 3 4 5 6

1 2 3 4 5 6

ROC Analysis

FA Rate

0.0 0.2 0.4 0.6 0.8 1.0

Hit

Rat

e

0.0

0.2

0.4

0.6

0.8

1.0

z(FA)

-3.0 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0

z(H

it)

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Slope = Lure / Target

0.0 0.2 0.4 0.6 0.8 1.0

Hit

Rat

e

0.0

0.2

0.4

0.6

0.8

1.0

False Alarm Rate

-3 -2 -1 0 1 2 3

z-H

it

-3

-2

-1

0

1

2

3

z-FA

slope = Target

Lure

Slope ≈ .80.31, .01

.69, .01

.84, .16

.93, .31.99, .67

Asymmetrical ROC

Memory Strength

"Old""New"

The Unequal-Variance Signal-Detection Model

Likelihood Ratio

Familiarity

10 20 30 40 50 60 70 80 90

"Old""New"

7.38/1

Target

p

1-p

Hit

Hit

Miss

g

1-g

Foil

g

1-gCorrect Rejection

False Alarm

Target

p

1-p

Hit

Hit

Miss

g

1-g

Foil

g

1-gCorrect Rejection

False Alarm

pr(Hit) = p + (1-p)g

pr(FA) = g

pr(Hit) = p + (1-p)FA

pr(Hit) = p + (1-p)g

pr(FA) = g

pr(Hit) = p + (1-p)FA

Just solve for p (because p is the measure of interest)

p = [pr(Hit) – pr(FA)] / [1 – pr(FA)]

p = (Hit – FA) / (1 – FA) Standard “correction for guessing”

p 0.5

FA (g) Hit0 0.5

0.1 0.550.2 0.6 00.3 0.65 10.4 0.70.5 0.750.6 0.80.7 0.850.8 0.9 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.2 0.4 0.6 0.8 1

Hit

Rate

FA Rate