[ 1 ] CONFIDENTIAL

Risk Management – Performing FAT and SAT

Jorge A. Cordero-Monroig; BSChE, MBA-GM

Validation Site Leader

B+L Greenville Solutions Plant

[ 2 ]

Risk Management – Performing FAT and SAT

Agenda:

Concept and Definitions

Why risk management

How to implement risk management to FAT and SAT

Interactive Exercise: Develop a risk management model to be implemented during FAT and SAT

Bonus Material

Q&A

[ 3 ]

Risk Management – Performing FAT and SAT

Concept and Definitions

What is a Risk Management ?

Risk Management is defined by Business dictionary.com as the identification, analysis, assessment, control, and avoidance, minimization, or elimination of unacceptable risks. Read more: http://www.businessdictionary.com/definition/risk-management.html#ixzz3BVbyA7DN

ISPE on GAMP 5 Guidance define Quality Risk Management Approach as a systematic process for the assessment, control, communication, and review of risks.

[ 4 ]

Concept and Definitions

What is a Risk Management ?

Two categories of Risk Management : QA Risk (GMP Risk). and Business Risk (Non-GMP).

Quality Risk Management Approach are based on clear process understanding and potential impact on patient safety. In other words, Quality Risk Management is based on GMPs systems/ equipment and how the patient safety could be impacted.

Business Risk Management is concentrated to ensure that business goals are accomplished. Therefore, make sense for business implement Quality Risk Management.

Risk Management – Performing FAT and SAT

[ 5 ]

How to classify Systems / Equipment

Pre-defined on SOPs and/or Validation Master Plan

Using Impact Assessments

Systems/Equipment Classifications:

Direct Impact

Indirect Impact

No Impact

Risk Management – Performing FAT and SAT

[ 6 ]

Direct Impact Systems/Equipment shall answer YES to one of the following questions:

Does the Systems/Equipment come in direct physical contact with the product?

Example: Filler, Tanks, Clean Steam, Filtered Compressed Air etc…

Does the Systems/Equipment produce or distribute a material that has direct contact with product?

Example: Bottles/Caps Molding machines, Form Fill & Seal (FFS) Machines, etc…

Is the Systems/Equipment used in cleaning, sanitizing or sterilizing?

Example: Clean Steam, CIP systems, Autoclaves, etc…

Risk Management – Performing FAT and SAT

[ 7 ]

Does the Systems/Equipment create or maintain a specified environmental condition required to preserve product quality?

Example: HVAC, Class A / ISO 5 rooms, etc…

Does the Systems/Equipment produce, monitor, evaluate, store or report data used to accept or reject product or GMP materials or data used to support Regulatory Compliance-Practices?

Example: PCS, EMS, Delta V, e-Batch Records, etc…

Does the Systems/Equipment perform a critical process step or operation in the manufacturing, processing, packaging, labeling testing or holding of the product?

Example: Inspections Systems, Vision Systems, Vacuum Systems, etc…

Risk Management – Performing FAT and SAT

[ 8 ]

Indirect Impacts Systems/Equipment shall answer YES to the following question:

Is the Systems/Equipment linked or support a direct impact systems?

Example: DI Water Systems, Chillers, Boilers, etc…

No Impact Systems/Equipment answer NO to all previous questions.

Example: Cooling Towers, Sand Filters, Office HVAC Units, OEE Systems, Palletizer, etc…

No impact Systems/Equipment are not critical for the product quality but for business.

Risk Management – Performing FAT and SAT

[ 9 ]

What is the difference between GMP critical and Business Critical

GMP Critical = Direct Impact :

Direct impact in the safety, identity, potency-strength, quality and/or purity of the product.

Direct impact to regulatory aspect such as labeling, and evaluation, generation, archiving, and reporting data that is used for release

Maintain, control, and monitor environmental conditions to preserve product quality

Used to clean, sanitize, or sterilize to preserve product quality

Risk Management – Performing FAT and SAT

[ 10 ]

Business Critical = Indirect Impact/No Impact:

Elements that affect the ability to manufacture and operate efficiently such as Boilers, Plant Steam, Compressed Air, Secondary / Tertiary Packaging

Elements that affect customer satisfaction such as cosmetic defects, failures in deliver on time, and missing promotional material

Systems that evaluate and monitor operation efficiencies

Equipment that help to improve operation efficiencies

Must comply with GEP (Good Engineering Practice):

Fitness to purpose, reliable, cost effective

Design follow industries guidance and statutory requirements

Take inconsideration GMP, Safety, Health, Environmental, ergonomic, operational, and maintenance requirements

Proof professional and competent design, construction, installation testing, and commissioning

Appropriate documentation such as design concepts, as build drawings, manuals, certifications, maintenance instructions, and test records.

Risk Management – Performing FAT and SAT

[ 11 ]

Concept and Definitions

When use Risk Management ?

Any time - It is a good process to ensure and demonstrate that any project is conducted properly.

Risk Management – Performing FAT and SAT

[ 12 ]

Risk Management – Performing FAT and SAT

Concept and Definitions

Factory Acceptance Test (FAT) – Formal testing performed at the vendor/factory site to determine if a system / equipment meet the specifications and acceptance criteria agree between the customer and the factory/vendor.

Site Acceptance Test (SAT) – Formal testing performed at the customer site to determine if once it is installed in real conditions the system / equipment still operating properly.

Commissioning – Engineering test (check list) as part of the system start up using real conditions to ensure that the system / equipment is performing as expected. It is used to release the system / equipment for production or to the validation group, as applicable.

[ 13 ]

Concept and Definitions

Who is responsible of FAT and SAT?

Vendor – Design, develop, and execute test that demonstrate that the system/equipment is meeting the specification as and acceptance criteria.

Customer Engineering Group – Review and approve FAT and SAT test, ensure that GEP (Good Engineering Practices) were followed.

Validation Group – Review and approve FAT and SAT test, ensure that GMPs (Good Manufacturing Practices) were followed.

Risk Management – Performing FAT and SAT

[ 14 ]

Risk Management – Performing FAT and SAT

Why Risk Management?

Benefits

Model

Requirements

[ 15 ]

Benefits:

Help to classify the system / equipment as GMP or Business Critical.

Provide a clear justification of the decision made based on risk criticality.

Identify and evaluate every single risk of the system / equipment.

Identify areas that will require more evaluation, adjustments , or re-design prior to the implementation.

Provide controls to reduce risk to an acceptable level.

Avoid waste effort and duplication while minimize down time.

Risk Management – Performing FAT and SAT

[ 16 ]

Risk Management Model

System / Equipment URS

Impact Assessment

GMP or Business

Critical

Abbreviated

Qualification

Commissioning / SAT

FAT

(Equipment Only)

FAT

(Equipment Only)

GEP

Business GMP

Risk Assessment

PFMEA/FTA/HAZOP/Risk

Priority/etc...

Performance Testing

Risk Assessment

Review

(as applicable)

Risk Assessment and

Traceability Matrix Review

Final Report

Risk Management – Performing FAT and SAT

[ 17 ]

Risk Management Model:

GMP Systems / Equipment:

User Requirements, Functional Requirements, and/or Design Review

Risk Analysis

Validation Strategy based on complexity

FAT (All configurations/capacities/functionalities)

Commissioning

Risk Analysis Review (as applicable)

Qualification (Abbreviated)

Risk Analysis Final Review

Traceability matrix

Final Report

Risk Management – Performing FAT and SAT

[ 18 ]

Risk Management Model:

Non-GMP (Business Critical) Systems / Equipment:

User Requirements, Functional Requirements, and/or Design Review

Project Plan

Risk Analysis

FAT / SAT (All configurations/capacities/functionalities)

Commissioning (Assurance at real conditions)

Risk Management – Performing FAT and SAT

[ 19 ]

Requirements:

Quality shall be involved from the beginning of the project.

Process Owner and Project Engineer shall provide a detailed requirements.

All deliverables and risk assessments shall be revised and/or approved by quality.

GEP must be followed all the time.

Periodic Risk Assessment review shall be performed.

Validation strategy shall be revised as necessary.

Any test or requirement leverage from FAT or Commissioning shall be justified.

All requirements shall be tested in at least one of the executable deliverables

Risk Management – Performing FAT and SAT

[ 20 ]

How to implement risk management to FAT and SAT?

Task and Responsibility

Tools

Role of the User Functional and Design specifications

How to measure Success

Risk Management – Performing FAT and SAT

[ 21 ]

Task and Responsibility

Legend:

D = Develop

E=Execute

R=Review

A=Approval

Process

Owner

Project

Engineer

Commissioning /

Executer Leader

QA-

Validation

Vendors/

Contractor

Project Proposal / Request D D R

User Requirement Specification D R/A R/A

Impact Assessment R/A R/A D

Risk Assessment (URS and Process) D R/A R/A

Validation Strategy R/A R/A D

Functional Requirements specification R/A R/A R/A D

Design Specification R/A D R/A R/A

Risk Assessment (Functional and

Design Requirements) D R/A R/A D

Traceability matrix development R/A R/A D R/A D

Design Review (Enhance Design

Review) R/A D R/A R

Factory Acceptance Test Development R/A R/A D R/A D

Factory Acceptance Test Execution R/A R/A E R/A E

Risk Management – Performing FAT and SAT

[ 22 ]

Task and Responsibility Cont.

Legend:

D = Develop

E=Execute

R=Review

A=Approval

Process

Owner

Project

Engineer

Commissioning /

Executer Leader

QA-

Validation

Vendors/

Contractor

Commissioning Development R/A R/A D R/A D

Commissioning

Execution/Verification R/A R/A E R/A E

Risk assessment

review/update R/A R/A E R/A E

Installation Qualification R/A R/A R R/A D/E

Operational Qualification R/A R/A R R/A D/E

Performance Qualification R/A R/A R R/A D/E

Individual summary reports R/A R/A R R/A D

Final Risk Assessment

Update R/A R/A E R/A E

Traceability matrix update R/A R/A E R/A

Final Report R/A R/A D

Risk Management – Performing FAT and SAT

[ 23 ]

Tools

Options and Tools to justify Quality Risk Management approach strategy:

Vendor documentation

Industry Guidance such as ISPE

Statistical Analysis:

Anova: Analyze difference in performance

% of missing caps in Capper machines

Determine cold spot in a Autoclave

Control Chart: Demonstrate that the parameters are within the acceptable range

Fill Volume

Temperature Range

Cpk: Demonstrate that a process is in control and centered

Cap torque

Part or component measure

Risk Management – Performing FAT and SAT

[ 24 ]

Tools

Options and Tools to justify decisions on Risk Management approach

strategy:

User Requirements

Functional Requirements

Design Specification

Impact Assessments

Risk Assessments

Risk Management – Performing FAT and SAT

[ 25 ]

Role of the User, Functional and Design specifications:

User Requirements Specifications (URS) is the document that specify the minimum requirements that the user needs for the acquisition or modification of a system / equipment. This is considered the guidance for the vendor to develop and provide the system / equipment that will fulfill the user needs.

Functional Requirements Specifications (FRS) is the document that specify how the system / equipment will comply with the user needs. This document is typically develop by the vendor or an expert in the subject.

Design specifications is the document that establish what equipment, instruments, hardware, and software are required to achieve the functions required to meet the user requirements.

Risk Management – Performing FAT and SAT

[ 26 ]

Role of the Impact and Risk Assessments:

Impact Assessment (IA) is the document that define the system / equipment criticality (Direct , Indirect, or No Impact). This document is the key to develop the risk management approach for the entire project.

Risk Assessment (RA) is the documents that identify the potential risk associated with the system/equipment, process, and /or project constraints. Multiple risk assessments can be performed to cover different aspect of the project. Risk assessments should define the following items

System/Equipment description

Potential failure / risk

Severity

Probability

Detectability

Risk level

Risk Management – Performing FAT and SAT

[ 27 ]

How to measure Success

Traceability Matrix (TM) - This document breakdown all the requirements on the URS, FRS, and DS and tabulate where each one was addressed.

Create a document that breakdown that tests require to validate the system/equipment and assess the test criticality and the risk of leverage some test.

E.g.. Qualification Test Risk Based Approach Evaluation - this evaluation includes the following elements:

• Test description

• Criticality Level (Low, Medium, High)

• Rational for criticality level

• Testing area (FAT/SAT/IQ/OQ/PQ/PV)

• Verification Level (Mandatory, Abbreviated, Optional)

• Rational/assumptions for verification level decision

Risk Management – Performing FAT and SAT

[ 28 ]

Summary of how implement Risk Management to FAT and SAT

Impact Assessment by answer the critical determination questions

Write a URS document that describe all the process owner requirements.

Perform a risk assessment that covers URS

Most Common Models:

FMEA (Failure Modes & Effects Analysis), FTA (Fault Tree Analysis)

HAZOP (Hazard Operability Analysis), Risk Priority, Etc…

Validation Strategy (GMP Critical) – Establish the risk based approach strategy. How, when, and who will be testing and verifying the risk assessment outcomes

Develop FRS (for Equipment and Utilities that have control systems/software)

Perform a risk assessment that covers FRS

Risk Management – Performing FAT and SAT

[ 29 ]

Summary of how implement Risk Management to FAT and SAT

Perform Factory Acceptance test

Drawings and Specs , Material of Construction

Alarm & Interlocks, Power Failures, Loss Communication

EHS (Environmental Health and Safety) Evaluation

Critical Instruments and Major components

Performance (Cycles, set-ups, Range, sizes, etc…)

Programs and logics

Revise the risk assessment and/or validation strategy based on FAT/SAT results. Rational for change in strategy must be provided.

Perform commissioning to ensure that the installation was performed adequately, verify changes in software, confirm that all documentation was provided, and verify that the

system/equipment components work as specified.

Risk Management – Performing FAT and SAT

[ 30 ]

Summary of how implement Risk Management to FAT and SAT

Revise the risk assessment and/or validation strategy based on Commissioning results, as applicable. Rational for change in strategy must be provided.

Perform abbreviated qualification to demonstrate that the system/equipment is suitable for intended used.

Worst case scenarios or bracket approaches (Equipment only)

Critical Parameter verification under operation conditions

Environmental Monitoring

Area Classification, Air Flow pattern, Air Changes, pressure, and

Temp. & RH, etc…

Final Review to Risk Assessment to answer how the identified risk were mitigated, controlled or monitored.

Traceability Matrix to track all critical requirements (Recommended)

Final Report to close the qualification/validation

Risk Management – Performing FAT and SAT

[ 31 ]

Interactive Exercise:

Using real life examples, participants study how to evaluate a Systems / Equipment and develop a Risk Management Approach

Systems:

Environmental Monitoring Systems

Process Control System

Equipment:

Filling Machine

Capper bowl

Packaging Line

Autoclave

Risk Management – Performing FAT and SAT

[ 32 ]

Bonus Material

Impact Assessment Determination and Qualification Requirements Charts

Risk Assessments Templates

Qualification Test Risk Based Approach Evaluation

Risk Management – Performing FAT and SAT

[ 33 ]

Questions and Answers

?

Risk Management – Performing FAT and SAT