The impact of discredited evidence David Lagnado Nigel Harvey Evidence project, UCL.
Thinking about Evidence David Lagnado University College London.
-
Upload
kaylynn-hannah -
Category
Documents
-
view
220 -
download
1
Transcript of Thinking about Evidence David Lagnado University College London.
Thinking about Evidence
David LagnadoUniversity College London
Leonard Vole accused of murdering a rich elderly lady Miss French
Vole had befriended French and visited her regularly including night of murder
Vole needed money
French changed her will to include him; shortly after he enquired about luxury cruises
Maid testified Vole was with French at time of death
Blood on Vole’s jacket same type as French
Romaine, Vole’s wife, was to testify that he was with her at time of murder
But instead Romaine appears as witness for prosecution
Testifies that Vole was not with her, returned later with blood on his jacket, and said “I’ve killed her”
Letters written by Romaine to lover – reveals her plan to lie and incriminate Vole
Vole is acquitted!
Evidential reasoning
• How do people reason with uncertain evidence?
• How do they assess and combine different items of evidence?– What representations do they use?– What inference processes?
• How do these compare with normative theories?
Reasoning with legal evidence
• Legal domain– E.g. juror, judge, investigator, media
• Complex bodies of interrelated evidence– Forensic evidence; witness testimony; alibis;
confessions etc
• Need to integrate wide variety of evidence to reach singular conclusion (e.g. guilt of suspect)
Descriptive models of juror reasoning
• Belief adjustment model (Hogarth & Einhorn, 1992)
– Sequential weighted additive model – Over-weights later items– Ignores relations between items of evidence
• Story model (Pennington & Hastie, 1992)
– Evidence evaluated through story construction – Holistic judgments based on causal models– No formal, computational or process model
Descriptive models of juror reasoning
• Coherence-based models (Simon & Holyoak, 2002)
– Mind strives for coherent representations– Evidential elements cohere or compete– Judgments emerge through interactive process that
maximizes coherence– Bidirectional reasoning (evidence can be re-evaluated
to fit emerging conclusions)
How should people do it?• Bayesian networks?• Nodes represent evidence statements or hypotheses• Directed links between nodes represent causal or
evidential relations• Permits inference from evidence to hypotheses (and
vice-versa)
Guilt
Maid
Vole is guilty
Maid testifies that Vole was with Miss French
Blood
Blood on Vole’s cuffs
CutVole cut wrist slicing ham
Partial BN of ‘Witness for Prosecution’
Partial Bayesian net for Sacco and Vanzetti trial
Applicable to human reasoning?
• Vast number of variables
• Numerous probability estimates required
• Complex computations
Applicable to human reasoning?
• Fully-fledged BNs unsuitable as model of limited-capacity human reasoning
• BUT –
a key aspect is the qualitative relations between variables (what depends on what)
• Judgments of relevance & causal dependency critical in legal analyses
• And people seem quite good at this!– Blood match raises probability of guilt – Alibi lowers it (not much!) Guilt
Blood Alibi
+ -
Qualitative causal networks (under construction!)
• People reason using small-scale qualitative networks
• Require comparative rather than precise probabilities
• Guided by causal knowledge
• More formalized & testable version of story model?
Empirical studies
• Discrediting Evidence
• Alibi Evidence
Discredited evidence• How do people revise their beliefs once an item
of evidence is discredited? – When testimony of one witness is shown to be
fabricated, how does this affect beliefs about testimony of other witnesses, or even other forensic evidence?
– E.g., Romaine’s discredited testimony
• Involves a distinctive pattern of inference
Explaining away
Vole cut himself
P(G|B&C) < P(G|B)
Finding out C too lowers the probability of G
Despite its simplicity and ubiquity, this pattern of inference is hard to capture on most psychological models of inference (e.g., associative models, mental models, mental logic)
P(G|B) > P(B)
Finding out B raises probability of G
Blood on Vole’s cuffs
Vole is guilty of murder
Guilt
Blood
Cut
Discrediting vs. direct evidence
Guilt
Blood
Cut
Bayesian network model
Causal model
CUT only becomes relevant to guilt given BLOOD
Important to distinguish ‘explaining away’ from simply adding (negative) evidence
Weighted additive model
Standard regression model
Guilt
Blood Cut
Experimental questions• Do people use causal models to reason with
evidence in online tasks?• Do they model discrediting evidence by
‘explaining away’?• How does the discredit of one item of evidence
affect other items?
YES when same source
EVIDENCE 1
Neighbour says that suspect has stolen previously
NO when different source
EVIDENCE 1
Footprints outside house match suspect’s
HYPOTHESIS: Local man did it
Scenario: House burglary, local man arrested
EVIDENCE 2
Neighbour says he saw suspect outside house on night of crime
Neighbour is lying because he dislikes suspect
?Does the discredit of item 2 affect item 1?
Extension of discredit• When do people extend the discredit of one item to other
items?• SAME
– E.g. two statements from same neighbour• SIMILAR
– E.g. two statements from two different neighbours• DIFFERENT
– E.g., one statement and one blood test
• Causal model approach would expect people to distinguish SAME from DIFFERENT cases
BN models
GUILT
Witness A
Witness B
Discredit
Same/Similar
GUILT
Blood test
Witness
Discredit
Different
Experiment 1• Mock jurors given simplified criminal cases• Four probability judgments (of guilt)
– Baseline– Stage 1 (Evidence 1) Footprint match– Stage 2 (Evidence 2) Neighbour sees suspect– Final (Discredit 2) Neighbour is lying
Compare judgments at Final stage and Stage 1 Does discredit return judgments to Stage 1?
Vary relations between items of evidence– SAME, SIMILAR, DIFFERENT source
Witness1 Witness2 Discredit2 Both items undermined
Forensic1 Witness2 Discredit2
When discredit presented LAST, it is extended regardless of relations between items
Results Final judgments significantly
lower than at Stage 1 for all conditions
Discredit does not simply remove item 2; also affects belief in item 1
Summary• Discrediting information extended regardless of
relation to other evidence• This pattern is consistent with Belief Adjustment
model– Recency effect leads to over-weighting of discrediting
information– Neglect relations between items
• Further test of BAM: manipulate order of evidence presentation
Experiment 2• Vary order of presentation of evidence
– LATE……E1 E2 D2– EARLY….E2 D2 E1
– Both orders ‘ought’ to lead to same conclusions
• Relatedness – SAME, DIFFERENT
Witness1 Witness2 Discredit2 Both items undermined
Forensic1 Witness2 Discredit2
When discredit presented LAST, it is extended regardless of relations between items
Results: Late condition Final judgments lower
than at Stage 1 for both conditions
Discredit does not simply remove item 2
Replicates EXP 1
When discredit presented EARLY, only extended to related items
Results: Early condition Pattern of judgments differ
for SAME and DIFF SAME
Final = Stage 2 DIFF
Final > Stage 2 Appropriate sensitivity to
relation between items
Witness1 Discredit1 Both items underminedWitness2
Witness1 Forensic1Discredit1 Only 1st item undermined
Problematic for current models
• Why are people ‘rational’ in early but not late condition?
• Belief Adjustment model – Cannot explain early condition because does not
consider relations between evidence
• Story model – Cannot explain bias in late condition (and needs to be
adapted to online processing)
Coherence-based/grouping account
• Mind strives for most coherent representation• Evidence grouped as +ve or -ve relative to guilt• +ve and -ve groups compete, but within-group
items mutually cohere (irrespective of exact causal relations)
• When an item of one group is discredited, this affects other items that cohere with it
LATE condition• Incriminating evidence grouped together
(regardless of source)• Discredit affects the group (not just individual
item)
GUILT
A
+
B
+
D
+ +
EARLY condition• First item of evidence discredited• Second item only discredited if from related
source• No grouping effect
GUILT
B
+
D
A
+++
Study 3• Grouping hypothesis predicts that coherent
groupings only emerge with elements that share the same direction (cf. Heider, 1946)
• Therefore discredit extended when evidence items both +ve or both -ve, but not with mixed items
Design• Four evidence conditions
1. A+, B+, discredit B+
2. A-, B-, discredit B-
3. A+, B-, discredit B-
4. A-, B+, discredit B+
• Two levels of relatedness: similar and different• Predictions
– 1&2 non-mixed -> discredit affects both items– 3&4 mixed -> discredit affects only second item
Examples: Condition 2 - - different
Neighbour says she was with suspect at time of crime
Neighbour lying because in love with suspect
Lab tests reveal no footprint match
Evidence 1 Evidence 2 Discredit
Examples: Condition 3 + - different
Lab tests reveal footprint match
Evidence 1 Evidence 2 Discredit
Neighbour says she was with suspect at time of crime
Neighbour lying because in love with suspect
Results
Summary• Grouping hypothesis
supported• Discredit extended when
items share common direction, not when mixed
• Mutually coherent elements stand or fall together (even when no clear causal relation between them)
• Romaine & Agatha Christie knew this!
Alibi evidence
• Often crucial evidence (if true, absolves suspect)• Treated with suspicion• Hard to generate (even if innocent)• Very little formal or empirical work• Ongoing psychological studies – what makes a good
alibi? (e.g., how much detail is best)• Also interesting from normative viewpoint
Witness vs. Alibi models
H
E
E*
Suspect committed crime
Witness report of suspect at crime scene
Suspect at crime scene
H
E
A Suspect claims he was not at crime scene
DSuspect motivated to lie
In alibi case – if suspect says he wasn’t there, but he was, this raises likelihood of guilt (beyond that if you just find out he was there)
To understand alibi evidence – need to represent potential deception
With impartial witness – knowing that suspect was at crime scene ‘screens off’ witness report from guilt judgment
+
++
+
+-
P(H|E&E*)=P(H|E) P(H|E&A)>P(H|E)Even though P(H|A)<P(H)
Pilot study
• Compare discredit of witness vs. alibi evidence • Manipulate reason for discredit
– Deception (X was lying in his statement)– Error (X was mistaken in his statement)
• Mock jurors given crime scenarios• 3 judgments of guilt
– Baseline– After statement (alibi/witness)– After discredit of statement
0
10
20
30
40
50
60
70
80
90
100
1 2 3
Judgment stage
Pro
bab
ility
of g
uilt
Witness/Error
Witness/Deception
Alibi/Error
Alibi/Deception
Alibi – discredit returns belief to baseline in error condition, but greatly enhances guilt in deception condition
Fits with causal network predictions
Witness – discredit returns belief to baseline (j1 = j3) irrespective of reason
Results
General alibi model
H
E
A Suspect claims he was not at crime scene
DSuspect motivated to lie
Case 1: Suspect provides alibi
Higher motivation to lie if guilty than if innocent
(hence link from H to D)
Given alibi, discovery of E incriminates via two routes
E raises likelihood of H directly
E raises likelihood of H indirectly
(via its effect on D)
++
+-
No screening-off ie P(H|E&A) > P(H|E)
General alibi model
H
E
A Friend claims suspect was not at crime scene
DFriend motivated to lie
Case 2: Close relative/friend provides alibi
AND they know whether or not suspect is guilty
Higher motivation to lie if guilty than if innocent
(hence link from H to D)
Given alibi, discovery of E incriminates via two routes
++
+-
No screening-off ie P(H|E&A) > P(H|E)
General alibi model
H
E
A Friend claims suspect was not at crime scene
DFriend motivated to lie
Case 3: Close relative/friend provides alibi
BUT they do NOT know whether suspect is guilty
Motivation to lie irrespective of actual guilt or innocence of suspect
(effectively no link from H to D)
Given alibi, discovery of E incriminates only via direct route
+
+-
Screening-off ie P(H|E&A) = P(H|E)
General alibi model
H
E
A Stranger claims that suspect was not at crime scene
DStranger motivated to lie
Case 4: Impartial stranger provides alibi
AND they do NOT know whether suspect is guilty
Low Motivation to lie AND this is unrelated to actual guilt or innocence of suspect
(effectively no link from H to D)
Given alibi, discovery of E incriminates only via direct route
+
+-
Screening-off ie P(H|E&A) = P(H|E)
Experimental study
• Do people conform to these models?• Background info:
– eg Victim is attacked on her way home … suspect is arrested
• Alibi: ‘suspect was elsewhere at time of crime’• Manipulate who provides the alibi• Discredit Alibi
– e.g., suspect seen on CCTV near crime scene at time of crime
Alibi provider
Motivated to lie?
Knows H? Prediction
Suspect YES YES P(H|E&A) >
P(H|E)
Close friend YES YES P(H|E&A) >
P(H|E)
Work colleague
MAYBE NO P(H|E&A) =
P(H|E)
Stranger NO NO P(H|E&A) =
P(H|E)
Results so far
>
=
=
=
• Scenarios don’t clarify that close friend knows H (as shown by subjects’ judgments about this)
• Strong order effects ---ALIBI, CCTV >> CCTV, ALIBI
Conclusions so far
• People construct and use causal models• ‘Explaining-away’ inferences• Grouping of variables can lead to biases• Sensitive to Alibi model• Puzzling order effect with Alibis
• Judgment involves both causality and coherence?
Thank you!• Leverhulme/ESRC Evidence project
– Nigel Harvey– Phil Dawid– Amanda Hepler– Gianluca Baio
• Students– Miral Patel– Nusrat Uddin– Charlotte Forrest