Virginia Department of Health Professions Core Competency: Board Performance Measured
description
Transcript of Virginia Department of Health Professions Core Competency: Board Performance Measured
1Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Virginia Department of Health Professions
Core Competency:
Board Performance Measured
October 22, 2008Neal Kauder, President, VisualResearch, Inc.
2Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Two Major Projects Underway
1. Agency-wide Performance Measurement
2. Sanctioning Reference Points (SRPs)
3Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
4Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Note: Vertical scales on line charts change, both across boards and measures, in order to accommodate varying degrees of data fluctuation.
CNA - In Q1 2009, the clearance rate was 116%, the Pending Caseload older than 250 business days was 28% and the percent closed within 250 business days was 80%.Q1 2009 Caseloads:Received=138, Closed=160Pending over 250 days=96Closed within 250 days=110
Nurses - In Q1 2009, the clearance rate was 102%, the Pending Caseload older than 250 business days was 18% and the percent closed within 250 business days was 80%.Q1 2009 Caseloads:Received=313, Closed=319Pending over 250 days=118Closed within 250 days=224
Clearance Rate
Nursing - In Q1 2009, the clearance rate was 106%, the Pending Caseload older than 250 business days was 22% and the percent closed within 250 business days was 80%.Q1 2009 Caseloads:Received=451, Closed=479Pending over 250 days=219Closed within 250 days=334
Age of Pending Caseload Percent Closed in 250 Business Days(percent of cases pending over one year)
0%
50%
100%
150%
200%
250%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
10%
20%
30%
40%
50%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
10%
20%
30%
40%
50%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
50%
100%
150%
200%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
50%
100%
150%
200%
250%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 090%
10%
20%
30%
40%
50%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
90% Goal
90% Goal
90% Goal
25% Goal
25% Goal
25% Goal
100% Goal
100% Goal
100% Goal
Agency-wide Performance Measurement
5Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Note: Vertical scales on line charts change, both across boards and measures, in order to accommodate varying degrees of data fluctuation.
Dentistry - In Q1 2009, the clearance rate was 202%, the Pending Caseload older than 250 business days was 45% and the percent closed within 250 business days was 81%.Q1 2009 Caseloads:Received=58, Closed=117Pending over 250 days=87Closed within 250 days=75
Clearance Rate
Medicine - In Q1 2009, the clearance rate was 186%, the Pending Caseload older than 250 business days was 26% and the percent closed within 250 business days was 92%.Q1 2009 Caseloads:Received=267, Closed=497Pending over 250 days=96Closed within 250 days=422
Pharmacy - In Q1 2009, the clearance rate was 125%, the Pending Caseload older than 250 business days was 9% and the percent closed within 250 business days was 77%.Q1 2009 Caseloads:Received=59, Closed=74Pending over 250 days=10Closed within 250 days=51
Age of Pending Caseload Percent Closed in 250 Business Days(percent of cases pending over one year)
0%
100%
200%
300%
400%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
15%
30%
45%
60%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
10%
20%
30%
40%
50%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
50%
100%
150%
200%
250%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
50%
100%
150%
200%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 090%
10%
20%
30%
40%
50%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
90% Goal
90% Goal
90% Goal
25% Goal
25% Goal
25% Goal
100% Goal
100% Goal
100% Goal
Agency-wide Performance Measurement
6Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Note: Vertical scales on line charts change, both across boards and measures, in order to accommodate varying degrees of data fluctuation.
Counseling - In Q1 2009, the clearance rate was 71%, the Pending Caseload older than 250 business days was 0% and the percent closed within 250 business days was 100%.Q1 2009 Caseloads:Received=7, Closed=5Pending over 250 days=0
Closed within 250 days=4
Clearance Rate
Veterinary Medicine - In Q1 2009, the clearance rate was 144%, the Pending Caseload older 250 business days was 3% and the percent closed within 250 business days was 93%.Q1 2009 Caseloads:Received=34, Closed=49Pending over 250 days=2Closed within 250 days=39
Social Work - In Q1 2009, the clearance rate was 180%, the Pending Caseload older than 250 business days was 17% and the percent closed within 250 business days was 78%.Q1 2009 Caseloads:Received=5, Closed=9Pending over 250 days=2Closed within 250 days=7
Age of Pending Caseload Percent Closed in 250 Business Days(percent of cases pending over one year)
0%
100%
200%
300%
400%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
15%
30%
45%
60%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
10%
20%
30%
40%
50%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
50%
100%
150%
200%
250%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
50%
100%
150%
200%
250%
300%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 090%
15%
30%
45%
60%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
90% Goal
90% Goal
90% Goal
25% Goal
25% Goal
25% Goal
100% Goal
100% Goal
100% Goal
Agency-wide Performance Measurement
7Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Note: Vertical scales on line charts change, both across boards and measures, in order to accommodate varying degrees of data fluctuation.
Age of Pending Caseload Percent Closed in 250 Business Days(percent of cases pending over one year)
Optometry - In Q1 2009, the clearance rate was 200%, the Pending Caseload older than 250 business days was 0% and the percent closed within 250 business days was 86%.Q1 2009 Caseloads:Received=4, Closed=8Pending over 250 days=0Closed within 250 days=6
Long-Term Care Administrators - In Q1 2009, the board recieved no new cases, and closed 4 cases. No cases were pending over 250 business days, 100 percent of cases cosed within 250 business days.Q1 2009 Caseloads:Received=0, Closed=4Pending over 250 days=0Closed within 250 days=4
Clearance Rate
Psychology - In Q1 2009, the clearance rate was 160%, the Pending Caseload older than 250 business days was 18% and the percent closed within 250 business days was 86%.Q1 2009 Caseloads:Received=5, Closed=8Pending over 250 days=2Closed within 250 days=6
0%
100%
200%
300%
400%
500%
600%
700%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
10%
20%
30%
40%
50%
60%
70%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
10%
20%
30%
40%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
100%
200%
300%
400%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
100%
200%
300%
400%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 090%
10%
20%
30%
40%
50%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
90% Goal
90% Goal
90% Goal
25% Goal
25% Goal
25% Goal
100% Goal
100% Goal
100% Goal
Agency-wide Performance Measurement
8Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Note: Vertical scales on line charts change, both across boards and measures, in order to accommodate varying degrees of data fluctuation.
Age of Pending Caseload Percent Closed in 250 Business Days(percent of cases pending over one year)
Audiology - In Q1 2009, the board closed 2 cases and received none. 33% of the Pending Caseload was older than 250 business days; no cases were closed within 250 business days.Q1 2009 Caseloads:Received=0, Closed=2Pending over 250 days=2
Closed within 250 days=0
Funeral - In Q1 2009, the clearance rate was 75%, the Pending Caseload older than 250 business days was 0% and the percent closed within 250 business days was 100%.Q1 2009 Caseloads:Received=4, Closed=3Pending over 250 days=0Closed within 250 days=3
Clearance Rate
Physical Therapy - In Q1 2009, the clearance rate was 100%, the Pending Caseload older than 250 business days was 50% and the percent closed within 250 business days was 100%.Q1 2009 Caseloads:Received=2, Closed=2Pending over 250 days=3Closed within 250 days=2
0%
100%
200%
300%
400%
500%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
20%
40%
60%
80%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
100%
200%
300%
400%
500%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
50%
100%
150%
200%
250%
300%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 090%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
90% Goal
90% Goal
90% Goal
25% Goal
25% Goal
25% Goal
100% Goal
100% Goal
100% Goal
Agency-wide Performance Measurement
9Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Note: Vertical scales on line charts change, both across boards and measures, in order to accommodate varying degrees of data fluctuation.
Clearance Rate
Medicine - In Q1 2009, the clearance rate was 186%, the Pending Caseload older than 250 business days was 26% and the percent closed within 250 business days was 92%.Q1 2009 Caseloads:Received=267, Closed=497Pending over 250 days=96Closed within 250 days=422
Age of Pending Caseload Percent Closed in 250 Business Days(percent of cases pending over one year)
0%
10%
20%
30%
40%
50%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
0%
50%
100%
150%
200%
250%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 090%
25%
50%
75%
100%
Q1 05 Q1 06 Q1 07 Q1 08 Q1 09
90% Goal
90% Goal
90% Goal
25% Goal
25% Goal
25% Goal
100% Goal
100% Goal
100% Goal
Data will be interactively visual and accessible (downloadable in Excel)
Q1 05 295 160 54%
Q2 05 303 190 63%
Q3 05 308 290 94%
Q4 05 338 204 60%
Q1 06 329 322 98%
Q2 06 334 220 66%
Q3 06 337 260 77%
Q4 06 299 268 90%
Q1 07 316 202 64%
Q2 07 278 237 85%
Q3 07 350 238 68%
Q4 07 294 463 157%
Q1 08 235 329 140%
Q2 08 247 439 178%
Q3 08 285 569 200%
Q4 08 258 558 216%
Q1 09 267 497 186%
Received ClosedClearance
Rate
Q1 05 906 27%
Q2 05 1019 27%
Q3 05 1037 29%
Q4 05 1171 29%
Q1 06 1178 30%
Q2 06 1292 32%
Q3 06 1369 33%
Q4 06 1400 35%
Q1 07 1514 37%
Q2 07 1555 38%
Q3 07 1662 40%
Q4 07 1499 42%
Q1 08 1409 45%
Q2 08 1210 45%
Q3 08 934 32%
Q4 08 626 26%
Q1 09 363 26%
% of Pending
# Pending 250 days or more
Q1 05 147 85%
Q2 05 181 83%
Q3 05 272 81%
Q4 05 178 78%
Q1 06 287 81%
Q2 06 191 77%
Q3 06 231 79%
Q4 06 227 69%
Q1 07 163 74%
Q2 07 207 54%
Q3 07 208 69%
Q4 07 391 67%
Q1 08 259 73%
Q2 08 356 69%
Q3 08 410 50%
Q4 08 479 83%
Q1 09 459 92%
# Closed w/in 250 days
% Closed w/in 250 days
Board of Medicine
Agency-wide Performance Measurement
10Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Data will be interactively visual and accessible (downloadable in Excel)
• PDF file linked to Excel files that automatically open – we can implement in the
short-term, quickly (L2K reporting to come....)
• All files on a shared drive – accessible by many or restricted access
• A “dashboard” approach
• Management can gauge performance in a paperless, easy to understand
visual format.
• Can also print on PC printers if desired.
Agency-wide Performance Measurement
11Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Other Performance measure work:
• Narrowing case category codes for more valid and reliable recording of
patient care vs. non-patient case
• 2 KPM measures by stage – enforcement vs. board
• Recommendations related to old case standards
• Examining impact of new agency policies on performance
• Recidivism concept to be explored as an internal management tool
Agency-wide Performance Measurement
12Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Recidivism as possible internal management measure...
• Multiple measures of recidivism possible
• Measure which respondents return after a violation and sanction
are handed down
• How long to recidivate, and for what type of “case” – similar to
original case?
Agency-wide Performance Measurement
13Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Purpose
“…to provide an empirical, systematic analysis of board sanctions for
offenses and, based upon this systematic analysis, to derive
reference points for board members and an educational tool for
respondents and the public”
Virginia Board of Health Professions, Work plan, Spring 2001
Sanctioning Reference Points
14Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
SRPs are:
• Data driven, historically based worksheets that assess factors that predict
sanction outcomes
• Factors include those board members and staff indicate are important
• Designed so similarly situated respondents receive similar sanctions
• Designed to be modified as history changes, not static
• Voluntary; reasons for departing are recorded
Sanctioning Reference Points
15Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Medicine sample worksheetAdopted August 2004
Sparrow 010199999
60
30
80
60
50
Sanctioning Reference Points
16Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Board Start DateTotal Number
of Cases
Overall Agreement
Rates
BHP Overall 664 80%
Medicine Aug-04 92 70%
Dentistry Jul-06 73 85%
Nursing (Nurses and CNAs) Jun-06 466 82%
Funeral May-07 9 89%
Veterinary Medicine May-07 31 52%
Pharmacy Nov-07 17 82%
Optometry Dec-08 2 50%
*
* November 6th, 2008 – Veterinary Board will be considering adjusting SRP worksheet to include CE as possible sanction at the lowest thresholds. This is consistent with other boards more recent practice as well.
SRP Agreement Rates(through October 1, 2008)
Sanctioning Reference Points
17Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
SRP Example of Departure Reasons Cited
Mitigating (less harsh than SRP result):
• Isolated incident
• Closing practice/ No longer practicing
• No patient involvement
• Sincerity/ Remorse
• Corrective action take
• Actions were not the cause of injury
Aggravating (more harsh than SRP result):
• Drug Abuse/ Diversion
• No call/ No show
• Repeat SOC issues
• Sexual abuse with patient harm
• Severity of allegation
• Denial of addiction problem
• Multiple allegations
• Failed drug screens
Sanctioning Reference Points
18Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Optometry was the first Board to incorporate CCAs into the SRP system...
Sanctioning Reference Points
19Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Sanctioning Reference Points
20Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Remaining Boards:
Behavioral sciences
Psychology and Social Work have approved draft WS
Meeting with Counseling Nov. 14
Move forward to draft SRP manual
Train Board members and staff on use
Physical Therapy, Long-Term Care, Audiology & Speech PathologyReviewed limited number of cases
Interviewed board members that hear cases
Preparing draft data collection instrument
Collect data, draft worksheet
Sanctioning Reference Points
21Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
Evaluating the Effectiveness of Sanctioning Reference Points
Primary Outcomes
1. Consistency
2. Proportionality
3. Neutrality
4. Board and staff satisfaction
Secondary Outcomes:
1. Transferability of SRP system
2. Case processing time
3. Violation, CCA, or Consent Order rates change
Possible outcomes to measure:
Sanctioning Reference Points
22Prepared by VisualResearch, Inc. 10/17/08, 804.794.3144
That’s all!
Questions or comments?