CHALLENGING YOUR LOCAL FORENSICS LABS

Bruce Livingston & Kathleen Elliott

Strengthening Forensic Science in the United States: A Path Forward (2009)

◦ NAS Report

◦ NRC Report

◦ 2009 Report

Start Here….

Looks at forensic science differently

Easy to read

Begin with the Summary (p 1- 33)

E-format (on disk) and hardbound (order)

Don’t be Scared of the Report

Asked for by forensic analysts Authorized by Republican Congress/President Researched by NRC of National Academies

Created by an Act signed by President Lincoln in 1863

200 members have won Nobel Prize

Other National Academies of Engineering and Medicine Authored by Committee

◦ Co-chaired by US Court of Appeals Judge & Statistician

Embrace the Report

Forensic Sciences Reviewed in the

Report

Fingerprints Firearms Toolmarks Bite Marks Impressions

◦ Tires◦ Footwear

Bloodstain Pattern Handwriting Hair

Pattern/Experience Evidence

DNA Coatings (i.e.., paint) Chemicals including Drugs Materials including Fibers Fluids Serology Fire & Explosives

Digital Evidence

Analytical Evidence

Collection/Flow from crime scene to courtroom

Manner in which forensic practitioners testify

Misinterpretation

Adversarial system in criminal/civil litigation

Lawyers’ Use/Misuse

Judge’s Handling

The NAS Report: Use of Forensic Evidence

What is Science?“The Hallmark of Science is

Transparency”

◦ Hypothesis

◦ Testing

◦ Falsifiability

◦ Replication

◦ Peer Review

Scientific Method for Forensic

Practice

To Determine the Limitations of Forensic Science . . .

Collection and Analysis of Forensic Data

Accuracy and Error Rates of Forensic Analysis

Potential Bias & Human Error in Interpretation

Proficiency Testing of Forensics Experts

Assess the Method & Technology

Validation Studies: ◦Must be performed to confirm the validity

of a method or process for a particular purpose

◦Appropriate error rates have been defined and estimated

How to Assess

Proficiency Testing: ◦Routine and Random◦Mandatory ◦Realistic ◦Representative cross-section of analyst’s casework areas

How to Assess …

CSI ?

The Reality of Forensic Science

San Francisco(2010)

North Carolina(2010)

Drug TheftDestruction of Records

◦ DNA Mix-upMisleading Forensic Report

Misrepresented ResultsHid Evidence from DefenseManual: Be aware of “defense whores”230 Flawed Cases

3 Ended in Execution

Errors and Fraud

New York Dep’t of Forensic Investigation

LAPD

Connecticut

Oklahoma

Misconduct and Falsifications “could be the basis for a criminal prosecution”

File Missing after Errors found

“Major Deficiencies” in Dr. Henry Lee’s Lab

Joyce Gilchrist, Lab Supervisor, Dismissed for Falsifying Evidence

Errors and Fraud…

West Virginia

Houston

Washington

FBI

Lab employee lied in 100+ casesCredentials questioned

Grossly Incompetent Work“Misleading manner designed to unfairly

help prosecutors obtain convictions”

“Troubling Flaws” in Drug Cases◦ Failure to Disclose Brady

DNA Contamination and Errors

Brandon Mayfield Case◦ Bias ◦ Circular Reasoning

Quantico Lab Analyst DNA Error and Falsified Report

Errors and Fraud…

Idaho Possession of Untraceable C/S

Possession of Unauthorized Amounts of C/S

Hiding C/S during Audits

Errors and Fraud….

“With the exception of nuclear DNA analysis,… no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source.” – NAS Report

Is it Science?

Hair

Bite Marks

Fingerprints

Absent DNA testing, No individualization possible

“Could” ≠ Accurate

Continuing dispute over the value and scientific validity of comparing and identifying bite marks

Studies demonstrated that identification decisions on same fingerprint can change solely by presenting the print in a different context

Is it Science?

Arson

DNA

Despite the paucity of research, some arson investigators continue to make unsupported determinations about whether or not a particular fire was set

Mislabeling & Losing SamplesMisinterpreting dataUnderestimating probability of

family matchLCN

Is it Science…?

What is Reported?

Possible sources of error and statistical data

Evidence of analysis and the chemist’s notes

Some laboratories might mention the tests that were conducted

What is Not Reported…?

“From a scientific perspective, this style of reporting is often inadequate, because it may not provide enough detail to enable a peer or other courtroom participant to understand and, if needed, question the sampling scheme, process(es) of analysis, or interpretation”

- NAS Report

What is Reported…?

Get the GoodsWhat Exists?

Who has it?

How do I get it?

Why do I want it?

How do I use it?

Challenging Forensic Science

What Exists? Audit Reports◦ Internal ◦ External

Manuals and SOPsValidation StudiesQuality Control

◦ Unexpected Results Log

◦ Error Log◦ Corrective Action Log

Proficiency TestingMaintenance Records

The Goods…

What Exists…?

ASCLAD Accreditation

◦Legacy v. ISO 17025 Standards

◦Subjective Judgment as to compliance or corrective action request

◦Not Transparent

◦Look for units outside lab

The Goods…

What Exists…?

Analysts

◦ Case File including Bench

Notes

◦ CV

◦ Proficiency Testing

◦ Personnel Files

◦ Certificates

◦ Number of Analyses

◦ Courts/Cases

Qualified

The Goods…

Who has it? Law Enforcement Agencies

Crime Lab

ASCLAD

Prosecutor

DOJ

The NAS Report

Government Oversight Agencies

The Goods

How do I get it? Informal Request

Discovery

Subpoena Duces Tecum

FOIA/Public Records

Online Sources

◦ SWG s

◦ Libraries

The Goods…

Why do I want it? Provide Trier of Fact with Accurate Information

Allow Trier of Fact to Assess Proper Weight

Rebut Zero Error Myth

Public Entities Paid for by Public

The Goods…

The Result is Not Inculpatory

Alternative Interpretations

General Acceptance

Lab Compliance

Analyst Competence

Bias

The Goods…How do I use it?

“Based on my experience…”

Scientific Method◦ Hypothesis

◦ Testing

◦ Falsifiability

◦ Replication

◦ Peer Review

Scientific Assessment ◦ Collection and Analysis

◦ Accuracy and Error Rates

◦ Potential Bias & Human Error

◦ Proficiency

How do I really use it in Trial?

“Match…”

“You can’t see what I saw…”

No Bias

DefineNot Recognized By…

Scientific MethodBring Whatever it Takes for

Us to See

Contextual BiasPeer Review

How do I really use it in Trial?

Case Law

Melendez-Diaz v. Mass,129 S.Ct. 2527 (2009) (Confrontation re Lab Analysis)

“Nor is it evident that what [the State] calls ‘neutral scientific testing’ is as neutral or as reliable as [it] suggests. Forensic evidence is not uniquely immune from the risk of manipulation”

Case Law….

“The majority of [laboratories producing forensic evidence] are administered by law enforcement agencies, such as police departments, where the laboratory administrator reports to the head of the agency”

Justice Scalia on the Report

“Because forensic scientists often are driven in their work by a need to answer a particular question related to the issues of a particular case, they sometimes face pressure to sacrifice appropriate methodology for the sake of expediency”

“A forensic analyst responding to a request from a law enforcement official may feel pressure-or have an incentive-to alter the evidence in a manner favorable to the prosecution”

“Confrontation is designed to weed out not only the fraudulent analyst, but the incompetent one as well”

Citing the NAS Report concerning “problems of subjectivity, bias, and unreliability of common forensic tests such as latent fingerprint analysis, pattern/impression analysis, and toolmark and firearms analysis”

Scalia on the Value of Confrontation

There is “wide variability across forensic science disciplines with regard to techniques, methodologies, reliability, types and numbers of potential errors, research, general acceptability, and published material” – Justice Scalia quoting NAS Report

“Contrary to respondent’s and the dissent’s suggestion, there is little reason to believe that confrontation will be useless in testing analysts’ honesty, proficiency, and methodology-the features that are commonly the focus in the cross-examination of experts” – Justice Scalia

Scalia’s Roadmap to Cross

Brady

Deficient Performance

Anything the prosecution didn't disclose that Scalia mentioned as a possible subject of cross–examination is Brady

Everything that trial counsel didn’t request in discovery or use as a basis for cross-examination is deficient performance

How do I use it in Post-Conviction?

US v. Oliveira ◦(D. Mass 2010)

“The NAS Report called for sweeping changes in the presentation and production of evidence ....

The Standing Order …

“In the past, the admissibility of this kind of evidence was effectively presumed, largely because of pedigree-the fact that it has been admitted for decades.

The Standing Order

“As such, counsel rarely challenged it, and if it were challenged, it was rarely excluded or limited ….

The Standing Order…

“The NAS report suggests a different calculus-that admissibility of such evidence ought not to be presumed; that it has to be carefully examined in each case….”

The Standing Order….