UNWOMEN/IPEN Transformative Mixed Methods Evaluation: Day 3 Mixed Methods
Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety:...
Transcript of Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety:...
![Page 1: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/1.jpg)
Jail Safety: Using Mixed Methods to Audit a Changing Environment
• Darrell Finke, CGAP, Senior Auditor• Robert Persichitte, CPA, CFE, Senior Data Analytics Auditor• Samuel Gallaher, PhD, Audit Analytics Supervisor
Team members not present:• LaKeshia Allen-Horner, MPA, Audit Supervisor• Shaun Wysong, MA, Senior Auditor• Daniel Summers, Staff Auditor
![Page 2: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/2.jpg)
Presentation Objectives
• Identify the challenges of auditing a sensationalized, dynamic, and culturally-closed environment.• Discuss qualitative and quantitative methods used to address
those challenges.• Review the utility of mixed-methods approach to auditing
challenging auditees.
denverauditor.org 2
![Page 3: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/3.jpg)
Denver Jail Safety Audit Objective
To assess the effectiveness of Denver Sheriff Department efforts to ensure the safe and secure operation of the City and County of Denver’s jails – including an examination of operational trends and management practices that affect the safety and security of the jails for the public, staff, and inmates.
denverauditor.org 3
![Page 4: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/4.jpg)
A Range of Methods AppliedResearch and analysis• Reform efforts• Leading best practices• Department strategic planning and risk
management documentation• Department policies and procedures• Department training course content
Interviews• External stakeholders• Department command staff• Department supervisors and deputies• Department training staff• Department special unit supervisors
Walkthrough and observation• Facility familiarization• Targeted areas of facility• Pre-service Training Academy• Field Training Officer• Specialty training
Data review and analysis• Classification/Housing• Incident reports• Inmate grievance• Inmate disciplinary• Internal Affairs
denverauditor.org 4
![Page 5: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/5.jpg)
Challenges to Meeting Our Objective
Jail External Pressure
Jail Management Jail Culture Jail
Environment
denverauditor.org 5
![Page 6: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/6.jpg)
Jail External Pressure
- 2 deaths resulted in payments over $10 million- Allegations of excessive/inappropriate use of force- Inmate on deputy assaults causing serious injury
High-profile media
coverage
denverauditor.org 6
![Page 7: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/7.jpg)
Jail External PressureResearch and analysis• Reform efforts• Leading best practices• Department strategic planning and risk
management• Department policies and procedures• Department training course content
Interviews• External stakeholders• Department command staff• Department supervisors and deputies• Department training staff• Department special unit supervisors
Walkthrough and observation• Facility familiarization• Targeted areas of facility• Pre-service Training Academy• Field Training Officer• Specialty training
Data review and analysis• Classification/Housing• Incident reports• Inmate grievance• Inmate disciplinary• Internal Affairs
denverauditor.org 7
![Page 8: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/8.jpg)
Jail Management
8
- August 2014 to June 2018 - multiple task forces- Senior leadership changes- Reviews of jail operations, including Denver Auditor
Lengthy reform efforts
- Risk management and integration of new units- Convergence of multiple tactics
Undefined strategies &
tactics
- 300+ documents, inconsistent or duplicative content- Revisions during audit
Changing policies and procedures
denverauditor.org
![Page 9: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/9.jpg)
Jail ManagementResearch and analysis• Reform efforts• Leading best practices• Department strategic planning and risk
management• Department policies and procedures• Department training course content
Interviews• External stakeholders• Department command staff• Department supervisors and deputies• Department training staff• Department special unit supervisors
Walkthrough and observation• Facility familiarization• Targeted areas of facility• Pre-service Training Academy• Field Training Officer• Specialty training
Data review and analysis• Classification/Housing• Incident reports• Inmate grievance• Inmate disciplinary• Internal Affairs
denverauditor.org 9
![Page 10: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/10.jpg)
Jail Culture
- Evasiveness or not cooperative- Not receptive to recommendations
Fatigue to scrutiny
- Weak relationships with department- Questioning of audit team subject matter expertise
Lack of trust and
credibility
denverauditor.org 10
![Page 11: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/11.jpg)
Jail CultureResearch and analysis• Reform efforts• Leading best practices• Department strategic planning and risk
management• Department policies and procedures• Department training course content
Interviews• External stakeholders• Department command staff• Department supervisors and deputies• Department training staff• Department special unit supervisors
Walkthrough and observation• Facility familiarization• Targeted areas of facility• Pre-service Training Academy• Field Training Officer• Specialty training
Data review and analysis• Classification/Housing• Incident reports• Inmate grievance• Inmate disciplinary• Internal Affairs
denverauditor.org 11
![Page 12: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/12.jpg)
Jail Environment
- 2 serious inmate on deputy assaults- Several serious inmate fights
Assaults and fights
- Disruptive inmates- Use-of-force incident- Staff incident response
Distractions and dangers
to audit team
denverauditor.org 12
![Page 13: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/13.jpg)
Jail EnvironmentResearch and analysis• Reform efforts• Leading best practices• Department strategic planning and risk
management• Department policies and procedures• Department training course content
Interviews• External stakeholders• Department command staff• Department supervisors and deputies• Department training staff• Department special unit supervisors
Walkthrough and observation• Facility familiarization• Targeted areas of facility• Pre-service Training Academy• Field Training Officer• Specialty training
Data review and analysis• Classification• Incident reports• Inmate grievance• Inmate disciplinary• Internal Affairs
denverauditor.org 13
![Page 14: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/14.jpg)
Data Solutions
denverauditor.org 14
![Page 15: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/15.jpg)
Jail External Pressure and Culture
Respond to attempts to downplay issues
Use data to overcome objections
denverauditor.org 15
![Page 16: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/16.jpg)
Number of Bookings for Intake Records by Calendar Day
denverauditor.org 16
![Page 17: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/17.jpg)
High-Risk Assaults, by Facility
17
![Page 18: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/18.jpg)
Workers’ Compensation Claims for Denver Sheriff Department
18
![Page 19: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/19.jpg)
Jail Management
Consistency in a changing environment
Understanding impact of changes
denverauditor.org 19
![Page 20: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/20.jpg)
Daily Jail Population and Capacity for the Downtown Detention Center
denverauditor.org 20
![Page 21: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/21.jpg)
Frequency of Incident Reports by Day
denverauditor.org 21
![Page 22: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/22.jpg)
Jail Environment
Identification of risk areas
Auditing remotely
denverauditor.org 22
![Page 23: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/23.jpg)
Identify Risks
denverauditor.org 23
![Page 24: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/24.jpg)
Mixed-Methods to Address Uncertainty
Barriers to gathering sufficient
evidence
Solutions found through multiple
sources
This Photo by Unknown Author is licensed under CC BY
This Photo by Unknown Author is licensed under CC BY-NC-ND
denverauditor.org 24
![Page 25: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/25.jpg)
Law, Policy, Rule, etc.
ObservationsInterviews Hand-coded
Incident DatabaseJail
Management System
Workman’s Compensation
Database
Sources of Information and Evidence
denverauditor.org 25
![Page 26: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/26.jpg)
ObservationsInterviews
Findings
QuantitativeQualitative
Analysis and interpretation
Analysis and interpretation
Compare, interpret, and
integrate results
Hand-coded Incident Database
Jail Management
System
Workman’s Compensation
Database
26
Law, Policy, Rule, etc.
denverauditor.org
![Page 27: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/27.jpg)
Risk Assessment
27
Law, Policy, Rule, etc.
Interviews Hand-coded Incident Database
![Page 28: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/28.jpg)
Developing Fieldwork Methods
28
Law, Policy, Rule, etc.
ObservationsInterviews Hand-coded
Incident Database
Progression of steps and methods• Planned to observe• Analyzed incidents by location• Stratified on severity• Planned risk-based observations• Observed a use-of-force incident• Contrasted observation to policy• Resulted in sub-finding and recommendation
![Page 29: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/29.jpg)
Audit Finding
29
Management Processes Prevent Understanding of Safety and Security Risks.
denverauditor.org
![Page 30: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/30.jpg)
A Long-Standing Condition: Crowding
Law, Policy, Rule, etc.
Jail Management
System 30denverauditor.org
![Page 31: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/31.jpg)
Hand-coded Incident Database
Potential Effects: Increased High-Risk Assault
31
![Page 32: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/32.jpg)
Potential Effects: Claims Related to Assault
Worker’s Compensation
Database32denverauditor.org
![Page 33: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/33.jpg)
Audit Finding
33
Management Processes Prevent Understanding of Safety and Security Risks.
Law, Policy, Rule, etc.
Hand-coded Incident Database
Jail Management
System
Workman’s Compensation
Database
Interviews
denverauditor.org
![Page 34: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/34.jpg)
Where Types of Evidence Often Fit
Organizational performance
metrics
Process and incident dataPolicies and
procedures
Financial and transactional
data
ObservationsInterviews
Criteria Condition
Cause Effect
Qualitative Quantitative
Survey
denverauditor.org
![Page 35: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/35.jpg)
Mixed-Methods to Address Uncertainty
What we have shown is known, good audit practice.
These fundamentals are difficult to maintain in certain situations.
Solutions found through:• Creative with reformatting data to analyze risks;• Diligent synthesis of policies;• Targeted and repeated observations;• Continued communication between the analysts and the auditors.
35denverauditor.org
![Page 36: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/36.jpg)
Thank You!
denverauditor.org 36
Darrell Finke, CGAP
Robert Persichitte, CPA, CFE
Sam Gallaher, PhD
![Page 37: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/37.jpg)
Attributes
• Qualitative/Quantitative Image adapted from: Stojanov, Zeljko. (2016). Inductive Approaches in Software Process Assessment. I-XV. 10.20544/AIIT2016.I01. and Creswell, J. W., Plano Clark, V. L., Gutmann, M. L., & Hanson, W. E. (2003). Advanced mixed methods research designs. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 209–240). Thousand Oaks, CA: Sage.
• .
• .
• .
denverauditor.org 37
This Photo by Unknown Author is licensed under CC BY-NC
This Photo by Unknown Author is licensed under CC BY-SA-NC
This Photo by Unknown Author is licensed under CC BY
![Page 38: Jail Safety: Using Mixed Methods to Audit a Changing Environment · 2019-11-25 · Jail Safety: Using Mixed Methods ... - August 2014 to June 2018 - multiple task forces - Senior](https://reader033.fdocuments.in/reader033/viewer/2022050519/5fa2e13391e76873600a8972/html5/thumbnails/38.jpg)
Attributes (cont’d)
• .
• .
• .
• .
denverauditor.org 38
This Photo by Unknown Author is licensed under CC BY
This Photo by Unknown Author is licensed under CC BY-SA-NC
This Photo by Unknown Author is licensed under CC BY-SA
This Photo by Unknown Author is licensed under CC BY