Post on 16-Aug-2019
Continuous Improvement Platforms
Boosting Operating Performance
KPI & Dashboard
in Quality Operation
www.cresultsconsulting.com
cResults / ERD confidential 1
Resource Planning, Scheduling,
Cost of Quality and Efficiency
Management for QC Laboratories
www.smart-qc.com
Batch Record Release Management
and Overall QA Scheduling &
Efficiency Management
www.cmanageefficiency.com
By cResults an IPS Affiliate
WEBINAR AGENDA
Introduction - Today's Drivers, and the need for improved
visibility
Key Performance Indicators Goals and Objectives
KPI Overview and Examples / What Data Need to Be Collected /
How to calculate an more
cResults / ERD confidential 2
How to calculate an more
Demo of cME / Smart-QC as a platform to capture all related QA
& QC data requirements & Reports / KPI / Dashboard
Case Study
Summary and Q&A
Today’s Business Drivers
Release Time Reduction
On-Time Delivery &Delivery Consistency(customer service)
Overall Cycle Time
cResults / ERD confidential 3
Efficiency
Variability
Overall Cycle Time
& Reduce Cost
Compliance
XX … these challenges have been
recognized by corporate leaders
as imperative issues that need
to be addressed.
Definition of KPI
In today’s Webinar we will focus on several KPIs, and how to define,
cResults / ERD confidential 4
In today’s Webinar we will focus on several KPIs, and how to define,
measure, and report them:
Batch Record Management
QA Professionals Efficiency (QA Operation)
Service level – Focused on cycle time and on-time delivery
RFT Documentation - QC
Non Traditional Compliance KPI
�
����
����
�
�
# Focus Area Key Performance Indicators (KPI) QA Operation
KPI Examples QC Operation
1 Compliance • Investigation Open above 30 days; New vs.
Closed; Investigation TAT; CAPA Effectiveness
• Regulatory Commitments
• Complaints
• Activities based KPI (e.g. daily audits, clearances, swabs) & Failures / observations while performing these activities.
• OOS Rate / Investigations
• OOS Closure Turn-around Time (TAT)
• Stability Test On Time %• Right First Time Documentation
• Error per analyst
• Errors Type Distribution
2 Right First Time
• Right First Time (by area / product)• RFT by QA tech / by Owner• Breakdown of documentation errors
• Re-test Rate• Pending Information
3 Financial and Efficiency
• Overtime Rate
• Budget Variance
• Reviewer’s efficiency
• Overtime Rate
• Budget Variance
• Analyst Earned Hr. vs. Actual Hrs.
cResults / ERD confidential 5
• Reviewer’s efficiency• QA Techs Earned Hr. vs. Actual Hrs.• QA Tech Workload vs. QA Tech Availability %
• % of Time Spent On QA VA vs. Non-QA Activities
• Analyst Earned Hr. vs. Actual Hrs.• Analyst weekly efficiency %• Cost per sample / product / VS• Efficiency losses Pareto• Instrument utilization (Actual/STD)
4 Customer Service
• Release cycle time• Schedule Adherence• Reviewer cycle time / Mfg./Pkg. response time
for documentation errors• Investigations / CAPA closure Turn Around Time
• Sample turn around time / Cycle Time
• On Time Delivery / Schedule Adherence
• Backlog
5 People / Training
• Cross Training
• Turn-over / Hiring
• Safety
• Cross Training• Turn-over / Hiring
• Safety
6 New Product Introduction
• Quality of the Batch Record • Re-test Rate / Turn-around time
QC Dashboard Example
Throughput
cResults / ERD confidential 6
On-Time Delivery
Cycle TimeEfficiency
cME – QA Dashboard Example
Efficiency
cResults / ERD confidential 7
Errors Recorded RFT / Errors by Area
Throughput Cycle Time
cME – QA Dashboard Example
cResults / ERD confidential 8
KPI – measure progress toward Operation Excellence QC and QA
Meaningful KPI –measuring the progress
An effective labs’ structurethat support the business goals
Efficient Layout / Material Flow and Work Method / Make or Buy
Meaningful KPI –measuring the
progress
Reviewers performance,guidelines and harmonization
Clear roles and responsibilities(Mfg./Pkg./QA/QC)
QA Release Optimization QC Efficiency Improvement
cResults / ERD confidential 9
and Work Method / Make or Buy
Effective coordination & alignmentwith Mfg. / Planning / purchasing
Automation and adequate resources level(Instruments and Analysts)
Systems: LIMS / Data Acquisition / Planning / Daily Scheduling / Test Allocation / Training
Detailed Labs’ standards as a foundation for effective planning and Scheduling of the Labs’ resources
(Mfg./Pkg./QA/QC)
Value added vs. Non Value added reviews (Stage / Full Review)
Right First Time documentation –awareness, standardization, training, and
accountability
Investigation Closure improvement, early detection and determination, effective root
cause and CAPA
Batch Record design Streamlining (Layout, Clarity, Simplicity, and Standardization)
Batch Record KPI needs
What is the Right First Time (by area / product)?
What is the QA review cycle time?
What is the Mfg./Pkg. response time to correct documentation errors?
What are the main delays / pending issues causing BR delays?
What is the impact of investigations / CAPA closure on the BR
����
cResults / ERD confidential 10
What is the impact of investigations / CAPA closure on the BR release time?
What is my reviewer’s efficiency?
What is the RFT by QA tech, and what leads to the variability between QA techs?
What are the causes for time losses for my reviewers?
What are the value added reviews / stages throughout the release process?
What is the breakdown of the documentation errors?
Print error logPrint error log
1st Review 2nd Review
1st Mfg./Pkg.
Response time
2nd Mfg./Pkg.
Response time
Improve Batch Record release process
?
����
cResults / ERD confidential 11
Insert Time
Assigned Date
Review Start
Review End
Mfg./Pkg.
Insert Time
Assigned Date
Review Start
Review End
Pass 2nd
(1) Cycle Time
(2) RFT
(3) Efficiency
Pending
?
Pass 1st
Meaningful KPI����
cResults / ERD confidential 12
Earned Hours = Σ All Activities Performed * Standard time to perform these activities
Available Hours = Available direct staffing hrs – Σ All the planned
Efficiency in QA����
cResults / ERD confidential 13
hrs – Σ All the planned unavailable activities (i.e., meetings, training)
During A Given shift a QA Tech performed the following:
� Release 2 BR – Type A (2 * 1.5 Hr.).
� Fail BR – Type B after review (45 min. for Review and 15 min. if
errors found).
� Performed an Audit (1 Hr.).
� Participant in 1 Hr. EHS training.
Efficiency Calc. Example (QA)����
cResults / ERD confidential 14
Based on the system standards the overall Earned Hrs. were:
� Earned Hours = Σ 3 hrs. + (45min+15min) + 1 Hr. = 5 Hrs.
� The shift duration is 8 Hrs.
� Available Hours = 8 hrs. – 1 hr. = 7 hrs.
QA Tech Efficiency = (5 / 7) * 100% = 71.4%
Tests / Audits
Tests & Audit [Earned Hrs] / (Weekly Schedule – Unavail – Non Tests) * 100% = Weekly Efficiency
Efficiency Calc. Example (QC)����
cResults / ERD confidential15
Unavail Events
Weekly Schedule
Non Tests
���� Unavailable time
cResults / ERD confidential
Available Hours = Available direct staffing hrs – Σ All the planned unavailable activities (i.e.,
meetings, training)
Overall Mfg. / Packaging Execution
Cycle Time Measured from Batch Record Issuance to BR Release
Main Components:
Batch Record Review Time
QA tech workload ���� Efficiency ���� Affect Cycle Time
Cycle Time / Turn Around Time (QA)����
cResults / ERD confidential 17
QA tech workload ���� Efficiency ���� Affect Cycle Time
BR Delays:
Investigation / CAPA – Turn Around Time
QC Sample Release
Mfg. Turn Around Time (BR Errors)
Other …
S0
E4
S1
*E1
*S2
E2
S3
E3
S4
Arrival of the Sample
Start of Batch 1 out of N
Complete the Run of the instrument
Start Post Run Calc.
End Post Run
Start Audit
End Audit
Start Approval
End Approval
Q1
Q2
Q3
Q4
Qu
eu
e T
ime
P1
P2
P3
P4
P5
Cycle
Tim
e / T
urn
Aro
un
d T
ime (Q
C)
��� �
cR
es
ults
/ ER
D c
on
fide
ntia
l18
Arrival of the Sample
Start of Batch 1 out of N
Complete the Run of the instrument
Start Post Run Calc.
End Post Run
Start Audit
End Audit
Start Approval
End Approval
Pro
cess / T
est
Sample ID
Product Id
Product Desc
Batch ID
Test ID
Sub Test ID
Test Desc
Lims Unique ID
Insert Date
Start CT
Batch Start
Cycle Time Report from Smart-QC
����
cResults / ERD confidential 19
Batch Finish
Testing CT
Audit Start
Audit End
Audit CT
Review Start
Review End
Review CT
Comments
• Audit Report – Documentation Review in QC
– RFT per Week
– RFT per Week Chart
– RFT BY SUPERVISOR
– RFT BY Analyst
RFT Documentation in QC (From Smart-QC)����
cResults / ERD confidential
– RFT BY Analyst
– Errors Type Distribution
– Errors BY Owner
20
����
Errors Type Distribution
RFT Documentation in QC (From Smart-QC)
cResults / ERD confidential 21
• More Common: – Investigation Open above 30 days; New vs. Closed
– Regulatory Commitments
– Complaints
– More …
Non Traditional Compliance KPI�
cResults / ERD confidential 22
DEMO – Dashboard and KPI Examples
cResults / ERD confidential 23
Case Study – Par PharmaceuticalA Full Article could be found on cResults Website –
www.cresultsconsulting.com
cResults / ERD confidential 24
StepStepStepStep----1111:::: cME provides an
overview on error reasonsbreakdown for all areas. Thisgives us the first indication ofwhere the issues could befound
StepStepStepStep----2222:::: Takes us a step further
to find out by month which ofthe error code are the dominantones
StepStepStepStep----3333:::: Identify if the errors are
contributed by a specific area.In this example, Pharmacy was
Step-1
Step-2
StepStepStepStep----5555:::: Provide the right first time metric by each
individual QA reviewer. Once a substantialdifference is detected, it is pointing the team toimprove the standardization process among thereviewers.
StepStepStepStep----6666:::: In this step we will look closer on the type
of errors made. While in previous steps, the mainfocus was on the type of errors, in this step, we arefocusing on the details level which can lead us to aspecific page that may cause issues, a specificinformation that is often missed etc. This can leadto changes of the actual batch record design alongwith some targeted training for the production
Step-5
Step-6
RFT by QA Reviewers
Detailed Comments by BR
Leveraging KPI to Reduce Batch Record Documentation Errors
In this example, Pharmacy wasthe lead area in terms of errorsgeneration, which greatly helpsthe team to focus on this area toget a quick win.
StepStepStepStep----4444:::: Focus on a pareto – find
out which of the errors are themost dominant in a specificmonth. As in step-1 we werefocusing on the overall errors,in step-4 we are looking if atrend is detected for a monthspecific details. This is anotherstep in identifying if the errorsare related to a specific areaor/and a few specific errorcodes.
Step-3
Step-4
team.
StepStepStepStep----7777:::: cME provides the view of the same data
previously analyzed from the productionownership. In other words, once an error isrecorded, in addition to the area, the error code,the QA check that are being recorded, theproduction owner that could be a group leader, ora supervisor is recorded as well. This allows theteam to identify some specific issues related to anindividual that may need additional training orsome clarifications.
StepStepStepStep----8888:::: Finally, cME provides a robust cycle time
and efficiency data that could show us not just theerrors but also the impact of QC samples,investigation, pending for errors response, etc. sothe team could once again focus on the mostcritical area / individual / events that lead to cycletime losses and overall deficiencies.
Step-7
Step-8
3 Executive Drive 2nd Floor, Somerset, NJ 08873 P: 732-748-1990; F: 732-748-1993www.cmanageefficiency.com
Case Study – KPI To Measure Progress
� Errors in batch records were reduced in the month of August
by 29.7%
� Response time of Manufacturing to correct errors was
The case study outlines the results observed after 3 months of cME implementation at Par Pharmaceutical a
leading generic companies:
cResults / ERD confidential 26
� Response time of Manufacturing to correct errors was
reduced by 18.7%
� Reviewer efficiency is up over 10%
� With cME information, the number of reviews are expected to
be reduced by 50%
� Many traditional QA activities are being challenged via
cME’s visibility of value-added / non-value-added activities
Case Study
Errors in batch records were reduced
29.7%
Errors in batch records were reduced
29.7%
cResults / ERD confidential
29.7%29.7%
Efficiency Metrics – For the first time in QA
cResults / ERD confidential 28
Case Study-2.1: Efficiency Gain: Earned
Hrs. Since Smart-QC Kick off
223.2
258.1267
279.2 282.6
240
260
280
300
Eff
icie
ncy [
%]
Earned Hrs. Per WeekSince Smart-QC Go-Live
Earned Hrs. increases by
cResults / ERD confidential 29
192.5
212.5
223.2
180
200
220Eff
icie
ncy [
%]
Earned Hrs
Earned Hrs = Sum of ALL Test related activities times the STD
Earned Hrs. increases by
almost 50%
42.4%
36.4%41.3%
59.7%63.6%
55.4%
40.8%
72.6%
93.8%
83.1%
70.1%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
300
400
500
600
700
800
900
1,000
Ax
is T
itle
Efficinecy & Earned Hrs for past 12 wks
Case Study-2.1: Efficiency Gain: Earned
Hrs. Since Smart-QC Kick off
cResults / ERD confidential
27.1%
0.0%
10.0%
20.0%
30.0%
0
100
200
300
Axis Title
Earned Hrs Efficiency
30
Earned Hrs = Sum of ALL Test related activities times the STD
The average efficiency for the last 4 weeks is 79.9% while the first 4 weeks of
that 12 weeks period was 36.8%
Case Study-2.1: Efficiency Gain: Earned
Hrs. Since Smart-QC Kick off
Week Date Avail Hrs Unavail HrsNon Test
HrsRemaining Hrs Earned Hrs Unaccounted Hrs
% Earn Hrs Of
OverallEfficiency
35 29-Aug-10 1,718 323 461 934 655 279 38.1% 70.1%
34 22-Aug-10 1,712 270 582 861 715 145 41.8% 83.1%
33 15-Aug-10 1,552 220 397 934 876 58 56.4% 93.8%
32 8-Aug-10 1,535 217 411 906 658 248 42.9% 72.6%
31 1-Aug-10 1,522 237 402 883 360 523 23.7% 40.8%
30 25-Jul-10 1,521 174 433 914 507 407 33.3% 55.4%
cResults / ERD confidential 31
30 25-Jul-10 1,521 174 433 914 507 407 33.3% 55.4%
29 18-Jul-10 1,557 120 612 825 525 301 33.7% 63.6%
28 11-Jul-10 1,529 189 665 675 403 272 26.3% 59.7%
27 4-Jul-10 1,535 321 479 736 303 432 19.8% 41.3%
26 27-Jun-10 1,914 250 586 1,077 292 786 15.2% 27.1%
25 20-Jun-10 1,906 315 617 974 354 620 18.6% 36.4%
24 13-Jun-10 1,902 319 603 980 415 565 21.8% 42.4%
Standard Smart-QC Report: The average efficiency for the last 4 weeks is 79.9% while the first 4
weeks of that 12 weeks period was 36.8%
THANK YOUTHANK YOU
cResults / ERD confidential 32
Web Site: www.cManageEfficiency.comEmail: info@cresultsconsulting.comcResults Web Site: www.cresultsconsulting.comContact Info: Rafi Maslaton; 1-732-748 1990 ext. 250