MINORITY REPORT: PREDICTING SOLVABILITY WITHOUT PRECOGSTOM OLPHIN

OLPHIN, T.W.L. (2015) ‘SOLVING VIOLENT CRIME: TARGETING FACTORS THAT PREDICT CLEARANCE OF NON-DOMESTIC VIOLENT OFFENCES’, (UNPUBLISHED M.ST THESIS, UNIVERSITY OF CAMBRIDGE)

SETTING THE SCENE

Vast amount of harm caused by violence Shrinking police budget Need to maintain or improve level of service to

public in relation to violent crime Necessity to work towards being more effective

and efficient Given the information available to police, it is

not possible to solve all crimes Between March 2012 and December 2013,

43.4% positive clearance rate for non-domestic violent crime in WMP

As the precogs do for murder in Minority Report, wouldn’t it be great to be able to predict which crimes it is possible to solve at the start?

SETTING THE SCENE

West Midlands Police use an officer-derived experiential case-screening model Sherman (2013) found that by 2012 it was “hard to find a police agency that employs a

statistical model of solvability factors to allocate the very scarce resource of investigators’ time.”

Kahneman (2011, p.224) asserts that statistical judgements are frequently more accurate than those of even the most experienced practitioners

KAHNEMAN, D. (2011) THINKING, FAST AND SLOW, NY: PENGUINSHERMAN, L.W. (2013) THE RISE OF EVIDENCE-BASED POLICING: TARGETING, TESTING, AND TRACKING. CHAPTER DUE TO APPEAR IN CRIME AND JUSTICE IN AMERICA, 1975-2025. TONRY, M. (ED.) CRIME

AND JUSTICE, 43, 1-59. CHICAGO: UNIVERSITY OF CHICAGO PRESS

MINORITY REPORT FOUR STEP METHOD

1. Identify three mutants (pre-cogs) and put them in a pool

2. Design a computer system to turn their visions into recognisable predictions of the future

3. Base all policing on these predictions4. Pretend there are no errors (minority

reports)

METHODOLOGY

Large Scale Analysis (29,105 cases) – Non-domestic violent crimes with offences involving child abuse, vulnerable adults, public officials, driving offences and murder/manslaughter excluded

Utilised the wealth of electronically recorded data available in Crimes, Crimescan, Docutrak and Socrates as well as information gathered from officers by the Crimes Service Team

Date range 1st March 2012 to 31st December 2013 was chosen to maximise data quality and sample size

All cases were randomly allocated into two groups (one for analysis and one for testing of the predictive model)

Each individual factor was analysed separately using clearance as the outcome measure using the first data set

Effect size of these individual analyses was then examined Prevalence was examined to identify factors with room for development

METHODOLOGY

Analysis was repeated for all factors using only cases with no suspect information Logistic Regression was used with significant useful variables to examine the predictive

strength of each variable The Logistic Regression was then reverse-engineered to provide a predictive formula Precog errors would lead to incorrect imprisonment or unpredicted murders; errors in

solvability prediction formulae would lead to either wasted resources or filing of solvable investigations

The cut-off point was optimised in order to reduce false negatives (0.39 instead of 0.5) – Not all errors are equal

A series of automatic opt-in criteria were designed The predictive formula was tested on the second data set

ALLOCATION OF INVESTIGATIVE RESOURCES IN WEST MIDLANDS

Current allocation model was designed during the Continuous Improvement programme

Factors were identified by officers, as dictated by experience and perception, then combined with public interest factors and mandatory offences

Each factor received the same weighting

The presence of any one factor dictates that the case is allocated

Chi-squared tests were used to examine individual factors

Factor Significance NotesSuspect Information P < 0.001 * Strong PositiveWitness Outstanding P = 0.310 -Mandatory P = 0.667 -Critical P = 0.888 -Media Interest P = 0.654 -Community Tension P = 0.673 -Financial Impact P = 0.770 -Profile High P = 0.919 -Forensic Evidence P = 0.003 * PositiveSeries P = 0.001 * PositiveProperty Flagged P = 0.009 * NegativeVRM Details P = 0.001 * Positive when controlling for SusInfoCCTV Flag P < 0.001 * NegativeAdditional Factors P = 0.610 Positive when controlling for SusInfoResolved X Automatically Updates – Inaccurate

SOLVABILITY FACTORS

WMP Suspect Information WMP Forensic Evidence WMP Series GBH with Intent Attempted Murder Links between victim and

offender Private Indoor Location Private Outdoor Location Public Indoor Location Offence at Dwelling

Two or more Statements Scanned

At least one PROP Scanned

At least one SOCO Scanned

At least one MISC Scanned

At least one SUSP Scanned

Offence occurred at time with lower risk of violent crime

* WMP VRM Details controlling for SusInfo

* WMP Additional Factors controlling for all other factors

Reported to Patrol Reported within 5

minutes of occurrence Offence duration 15

minutes or under Offence commenced in

daylight Drugs Involved Alcohol Involved FSI Dispatched Items recovered by FSI No Statements Scanned

CASE LIMITING FACTORS

WMP Property Flagged WMP CCTV Flag Actual Bodily Harm Wounding GBH without Intent Use of Opportunistic Weapon Use of Weapon causing Injury/Damage Public Outdoor Location Offence on Licensed Premises Reported to Front Office

Reported to Contact Centre/Help Desk One Statement Scanned * WMP VRM Details

NON-SIGNIFICANT FACTORS

WMP Witness Outstanding WMP Mandatory WMP Critical WMP Media Interest WMP Community Tension WMP Financial Impact WMP Profile High Common Assault Presence of visible injury Hate Offence

All Weapon Types other than Opportunistic All Weapon Uses other than causing

injury/damage At least one DRAW Scanned

EARLY TACTICAL INDICATIONS

Items being recovered by the forensic scene examiner are more prevalent in cleared cases

WMP only dispatched an FSI in 4.43% of cases

Cases where an officer identifies an outstanding witness are not associated with higher solvability

Cases with more than one statement scanned to Crimescan are more prevalent in cleared casesPrevalence of forensic variables in cleared

and uncleared cases

A NEW PREDICTIVE MODEL

Predictive Model Logit(p) = -2.06012231166749 + (Private Indoor x 0.21896234166784) +(Private Outdoor x 0.482221853049798) + (Public Indoor x 0.614168153975186) +(Reported to Patrol x 0.135507823875585) + (Something Recovered by FSI x0.922678539797806) + (Suspect Information x 1.65451761371856) - (Witness Outstandingx 0.00509247368812278) - (Forensic Evidence x 0.124114369542983) + (Series x0.179382232899131) + (VRM Details x 0.0552156712817925) - (CCTV Flag x0.118196466193996) + (Additional Factors x 0.13909164461101) - (Two or More MG11s x0.140966342491925) + (At least one PROP x 0.254753361095196) + (Duration 15 minutesor under x 0.225334514108994) + (Drugs Involved Marker x 0.23158147033591) + (AlcoholInvolved Marker x 0.139861549604373) + (Reported within 5 minutes x0.324619578460559) + (Offender Link to Victim x 0.338284590056064) + (Daylight Offencex 0.196372030404582) + (Public Outdoor x 0.261198299233381) - (Licensed Premises x0.234972861437699) - (Reported in Front Office x 0.164408083792076) - (Reported to HelpDesk or Contact Centre x 0.0734407903273885) - (Opportunistic Weapon Used in Offence x0.409479586358932) - (One MG11 Exactly x 0.561505039063743)

Screen-In Category Description of Specific Screen-In Factor RationaleHate CrimeMedia InterestCommunity TensionCritical IncidentHigh Profile Offence

Case is ResolvedCase is already resolved and has been cleared

Case is already solved

Allocated by Logit ModelCase is allocated based on its score on the solvability equation

Case is deemed by the equation to be solvable

GBH without Intent (or Attempt)GBH with Intent (or Attempt)Attempt Murder

Overwhelming Evidence

It has not been possible to create these yet but examples would be; CCTV which is clear, shows the offender and the offence taking place or Offence witnessed by a Police Offi cer

These are pieces of evidence where offi cers can identify that there is suffi cient, even if in weak evidential types to permit a prosection

Offence SeverityThese are offences where serious injury has been caused, or intended

Incident TypeThese are offence types that may cause severe reputational damage if investigation does not occur

Model for application with automatic screen-in criteria

Predictive Formula

STATISTICAL VS EXPERIENTIAL PREDICTION

Accuracy of WMP Predictive Model

Case Cleared Case Not Cleared

Case Cleared 100% (6334) 84% (6922)

Case Not Cleared

0% (0) 16% (1296)

Real Case Outcome

Mod

el

Pred

icte

d Ca

se

Out

com

e

For the weighted statistical model, 9321 out of 14552 cases are correctly predicted. Therefore the overall accuracy of the weighted statistical model is 64.05%

Accuracy of Olphin (2015) Statistical Predictive Model

For the WMP current model, 7630 out of 14552 cases are correctly predicted. Therefore the overall accuracy of the WMP current model is 52.43%

Case Cleared Case Not Cleared

Case Cleared 93% (5864) 58% (4761)

Case Not Cleared

7% (470) 42% (3457)

Real Case Outcome

Mod

el

Pred

icte

d Ca

se

Out

com

e

STATISTICAL VS EXPERIENTIAL PREDICTION

22.16% increase in accuracy of prediction through use of the weighted statistical model over the WMP current model

This equates to an additional 1736 cases being accurately predicted in the second data set when using the Olphin (2015) statistical predictive model

This equates to 3006 cases per annum that would be filed following initial investigation rather than being allocated

Based on analysis of Docutrak report age and quantity, this equates to approximately 27 FTE worth of investigations just for violent crime

Difference in Model Accuracy

RESEARCH LIMITATIONS

Large data set allowed for an extremely powerful analysis but did not facilitate in-depth examination of reports

Only electronically recorded variables are included Quality of factors is not available, only whether present Causality can not be argued due to this being a descriptive study

Whilst it is limited by the factors detailed above, this research has conducted the largest scale examination of solvability factors to date and has combined traditional techniques with novel examinations of information to produce a predictive model which is capable of increasing the accuracy of allocation for WMP by 22.16%

POLICY IMPLICATIONS

Whilst probable that this model can be improved further, this research provides a tool which could dramatically increase WMP investigative efficiency if implemented

Policing agencies may be able to improve detection rate through review of forensic attendance strategy Policing agencies may be able to improve detection rate through additional statements being obtained

during initial investigation, rather than identifying additional potential witnesses Other experiential predictive tools should be reviewed for accuracy against statistical models There may be additional factors which can further improve accuracy of screening Additional examination is required for CCTV and VRM variables, along with information that is not stored

electronically at present Any implementation of this, or a similar, model should be tested rigorously, ideally using an RCT WMP data is not stored in a manner that facilitates research, this could be dramatically improved Cost-benefit analysis would be beneficial This methodology is likely to be applicable to other crime types, and work is ongoing This demonstrates that we should look at algorithmic prediction modelling for other assessments such as

risk

TOM OLPHINEmail: tolphin@gmail.com Mobile: 07921 808988

OLPHIN, T.W.L. (2015) ‘SOLVING VIOLENT CRIME: TARGETING FACTORS THAT PREDICT CLEARANCE OF NON-DOMESTIC VIOLENT OFFENCES’, (UNPUBLISHED M.ST THESIS, UNIVERSITY OF CAMBRIDGE)