Evaluation of the Quality System Appraisal of Conformity

Evaluation of the Quality System

Appraisal of Conformity

Internal Audit

Prof. Dr. F. Vanstapel, MD PhD

Laboratory Medicine

UZ-KULeuven

Internal Audit Slide 1version 061017

F. Vanstapel © - Laboratory Medicine - University Hospitals - K.U.Leuven

mailto:[email protected]?subject=Laboratory%20Medicine%20Teaching%20Modules%20:%20Internal%20Audit


Teaching Goals



Teaching Goals

- understand auditing as an “ investment in conformance ” - understand “ added value auditing ” - learn audit techniques - learn to code and report effectively



Operational Definition


Operational DefinitionInternal Audit

Systematic / independent / documented appraisal of conformity / validity of procedures

Systematic evaluation of conformity

with the norm-requirements to which we subscribe ?

Is the audit trail credible ?

Are the procedures valid = effective ?

Internal Audit

Within and by the organization itself : “ value-added ” auditing Are the procedures effective / efficient :

Variance with prescribed procedures ?

Sound / lean / failure-proof design of procedures ?

Training adequate ?

System of prevention adequate ?

Managerial supervision adequate ?

Proper starting point& end point of this exercise




Requirements of the Standards


Applicable standardsBelgian regulator: KB (3 dec 1999) based on EN 45001 Praktijkrichtlijn

updates based on ISO 17025ISO 15189



Understanding the standard All standards: hierarchical systems

defining responsibilitiesallowing implementation adjusted to the organization

allowing a hierarchical document system EN 45001 : main focus on audit-trail & traceability ISO 17025 : + focus on “Shewart-Deming”-cycle ISO 15189 : + focus on “Clinical Practice” & “Health Economy”

http://www.iph.fgov.be/ClinBiol/bckb33/activities/licence/_down/03_12_1999.pdf

http://www.iph.fgov.be/ClinBiol/bckb33/commission/document_nl/praktijkrichtlijn.pdf

ISO 15189:2003



Understanding the norm Internal Audit is an exercise in Quality Assurance

Requirements of Standards

4.14.1 In order to verify that operations continue to comply with the requirements of the quality management system, internal audits of all elementsof the system, both managerial and technical, shall be conducted at intervals defined by the system itself. The internal audit shall progressively address these elements and emphasize areas critically important to patient care.4.14.2 Audits shall be formally planned, organized and carried out by the quality manager or designated qualified personnel. Personnel shall not audit their own activities. The procedures for internal audits shall be defined and documented and include the types of audit, frequencies, methodologies and required documentation. When deficiencies or opportunities for improvement are noted, the laboratory shall undertake appropriate corrective or preventiveactions, which shall be documented and carried out within an agreed upon time.The main elements of the quality system should normally be subject to internal audit once every twelve months.4.14.3 The results of internal audits shall be submitted to laboratory management for review.


Understanding the standardThe standards are “open” ended: you determine yourself how to implement it (creative interpretation)

The economy of your quality system: - quality = fit for purpose = value for one of the stakeholders- quality = value for your investment = create added value- minimize costs of a lack of quality - maximize return of your investment

- weed out all waste steps

- lean / failure-proof processes

FOCUS ON OPTIMALIZATION OF PROCESSES




QUALITY INVESTMENT

Costs of

QUALITY FAILURE

Investment in

CONFORMANCE

PREVENTION APPRAISAL INTERNAL EXTERNAL

Training

Maintenance

Lean

DESIGN

for QUALITY

End-of-Line

Inspection

(internal & external)

AUDITS

Scrap

Rework

Adverse Effects

in Patients

Complaints

Customer support

A.V. Feigenbaum

Action plans

ROOT-CAUSE APPROACH

Audits address A/B : conformity with norm / procedures* : process excellence

RepairDo it Right theFirst Time

Maintenance

Quality at no Cost

Norm.ppt

Norm.ppt

Norm.ppt

Norm.ppt

http://w1.uzleuven.be/labo/Leermodule/QUALITY_SYSTEM/3_PROCESS_CONTROL/Outlines/3_RISK_ANALYSIS/Risk.pdf


Internal vs External Audits




Internal Auditcan our creative interpretation

of the standard * stand scrutiny?

External Auditcredible

adherence to standard ?

vs

Compliance with standard ?

Hierarchical system ?

Audit trail & Traceability ?

Procedures ?

Training ?

Corrective and Preventive Interventions ?

Correct ?

Acceptable quality ?

State of the art ?

Correct ?

Desired quality ?

Desired effectiveness ?

* Auditing refers to compliance with applicable standardsStandards and integral quality thinking are covered in a separate module

http://w1.uzleuven.be/labo/Leermodule/QUALITY_SYSTEM/1_BUILDING_THE_QUALITY_SYSTEM/Outlines/1_1_Norm/Norm.pdf



Internal versus external audit

External auditorhas to stick to the standard:

will almost exclusively report on non-conformities examines a limited test scope

technical auditor can judge state of the art, & comes with expectation patterns

does not know the quality system, thoroughly

Internal auditorhas to detect non-conformities,

but can also report on other issuescovers the complete scope is unrelated to the audited section of the lab,

and thus lacks expertise & prejudice knows the inner workings of the organization


Nuts and Bolts




How to generate added value ?

Audit is an “ unannounced sample of the reality ”, suited for analysis of the workings of the “ integral quality system ”.

(Analyze system with respect to an established standard,* pay attention to system of prevention,management has to act)

In an audit the organization can show that it believesin the ability of its personnel to detect shortcomings and in its willingness and ability to remedy these. (Audit can be a motivator)

A correctly performed audit is an ideal didactic means to educate the personnel in the workings of the system. (Audit can be a motivator)

* Standards and integral quality system thinking are treated in a separate outline


BS EN ISO 19011: 2002 Guideline for quality management systems auditing

Save time and effort by:

• Avoid confusion over the objectives of the audit programme

• Secure agreement of the goals for individual audits

• Reduce duplication of effort

• Ensure audit reports contain all the relevant information

• Ensure audit reports are suited for actions by management

• Evaluate competence of members of an audit teamagainst objective criteria



Quality of scoring- thoroughness / yield- referencing norm-elements- evaluating the importance of non-conformities

Murphy’s Law

Better be prepared !

Risk

Failures

Adversity

System of Prevention

PrimaryPrevention

Prevention

DamageMitigation

Universal Lean ProtocolsMaintenance Schemes

Managing Critical SuppliesStart-up Inspections

If it can break,

it will break one day.

POKA

In-line InspectionsIntervention Cascades

Corrective Actions

Focus of “value added” Action Plan

Focus of “value added” Audit

Fail-safeDesign for Quality

Secundary



BS EN ISO 19011: 2002 Guideline for quality management systems auditing

Save time and effort by:

• Avoid confusion over the objectives of the audit programme

• Ensure audit reports are suited for actions by management

Management

Workfloor

Role of the hierarchical system

For maximal effectivenessthe quality system should never step in for the management



author & actor

integrateinformation &communicate business goals

stay on track

dissipateknowledge

facilitatecommunication

QualitySystem



How to audit ?

Returning ritual : INTRO - The ritual helps to restrain from becoming “personal”

- We are equals, examining together, how to improve our organization

- I come as an outsider, with less expertise, please teach me,

please take advantage of my lack of prejudice - There are no wrong answers, but there are

procedures which have to be improved, & explanations, training that has to be given

- Please use the exercise to think now and in the future about how to improve our & “your” procedures

(treat the interviewee as an author)



How to audit ?

Returning ritual : CHECKLISTS - norm based lists - scenario-based lists (available on request)

Returning ritual : QUESTIONING - “total” vs “sampling” - “horizontal” vs “vertical”- “closed” vs “open”

R) a mixture of all techniques

mailto:[email protected];[email protected]?subject=Laboratory%20Medicine%20Teaching%20Modules%20:%20Request%20for%20LAG%20internal%20audit%20form%20

http://economie.fgov.be/organization_market/belac/documents/docs_pdf/6-102_EN.doc

//uz/data/LaboratoriumGeneeskunde/Web_Server/HB/PRC/PRC_300_KS/PRC_380_Evaluatie_KS/PRC_383_Interne_Audit/document/bijlagen/2_CHK/Start.xlsm.lnk



Interview techniques

Open questions: allow the interviewee to become an author of ideas

Stop supervisors intervening in the discourse of the technicians

Keep interview going by grouping items

Do not go for a total audit, but move vertically, when you find a non-conformity

Note taking should not interfere with the interview

Produce a report that invites for a “constructive” response



While you listen, continuously ask yourself

Traceability – Audit trail: Can results be traced to personnel, reagents, calibrators, … ?

Training: Does technician operate on “common sense” or on “knowledge” ?

Quality of supervision: Do supervisors leave traces of their interventions ?

Communication: Are complaints, evaluation of suppliers, etc.

processed by the organization ?

Quality of interventions: Are corrective actions expanded to collateral damage ?Are preventive actions undertaken ? Are they evaluated ?

Validity of procedures: Norm-compliance ?

Argued/documented in the test validation file ?



How to audit ?

Returning ritual : PLANNING - on short notice

- no shuffling- better many short then a few long audits

Returning ritual : REPORTING - be fast

while you can still reconstruct your short-hand noteswhile interviewee still remembers the interview

- be factual and precise(procedures, reagents, tests)

- refer to elements of the “standard” & “grade” non-conformities - recognize the interviewee as author of ideas,

and identify yourself as author of opinions: this is important to sustain mutual trust



Frequency ?

+ +*

Sporadic

B A

CODINGFLOWCHART

IDENTIFY ITEM

RECORD FINDINGS

ImmediatelyDeploy Procedure

Adjust Processes

Analyse± actie

NO

Remark ?

Systematic

NO

YES

YES

NO

YES

ALL IT

EM

S R

EQ

UIR

ED

Patiënt SafetyRepeating B

Standard/ProceduresPatient-safety ?

Greatest Added Value



How to audit ?

Returning ritual : ACTION PLANSregistered by a single person, but authored by many- be fast

while you can still remember the interview - be factual and precise

(refer to the non-conformities of the report) - start with analysis by the person at the origin of the problem - let that person become the author of the solution - always indicate action, deadline, and responsibilities

Returning ritual : FOLLOW UP - transfer follow-up of action to a higher organ

and simultaneously feed the management review process- use repeat audits as a means of follow up


Summary




Take home message

The content and topics of the audit relate to the daily supervision & the efficacy of management

The ritual allows to perform an audit without becoming threatening

Systematic questioning allows to recognize blind spots of personnel, supervisors & management

Overall the audit creates value by assisting in improved managementby stressing that quality is realized on workfloor / responsibility of coworkers



Aids

Know the standard *

Know your implementation of the standard: = your quality systemthe document system, the internal quality assurance procedures, the validation procedures, …

Checklists Introduction to the audit Evaluation form

(hard-copy versions can be obtained upon request)

* The standard is treated in a separate module

C:/Documents and Settings/fvstap0/1_BUILDING_THE_QUALITY_SYSTEM/Outlines/1_1_Norm/Norm.ppt

mailto:[email protected];[email protected]?subject=Laboratory%20Medicine%20Teaching%20Modules%20:%20Request%20for%20LAG%20internal%20audit%20form%20

//uz/data/LaboratoriumGeneeskunde/Web_Server/HB/PRC/PRC_300_KS/PRC_380_Evaluatie_KS/PRC_383_Interne_Audit/document/bijlagen/1_Intro/document.doc

//uz/data/LaboratoriumGeneeskunde/Web_Server/HB/PRC/PRC_300_KS/PRC_380_Evaluatie_KS/PRC_383_Interne_Audit/document/bijlagen/2_CHK/Start.xlsm.lnk




Recommended literature

Standard ISO 15189:2003 (CEN – Comité Européen de Normalisation)

BELAC 6-102 Checklist for evaluation of conformity towards requirements of the standard NBN EN ISO 15189

BELAC 2-102Toepassing van de accreditatiecriteria: bijkomende interpretatie voor de sector klinische biologie

CPA 0204 An approach to audit in the medical laboratory

http://economie.fgov.be/organization_market/belac/documents/docs_pdf/6-102_EN.doc

http://economie.fgov.be/organization_market/belac/documents/docs_pdf/2-102_NL.pdf

http://www.cpa-uk.co.uk/files/aataud.pdf

Evaluation of the Quality System Appraisal of Conformity

Documents

Transcript of Evaluation of the Quality System Appraisal of Conformity