ABOUT THE AUTHORJames Oliver is a principal validation engineer in the validation department at Baxter Healthcare Corporation, Baxter BioScience Division, Thousand Oaks, CA. Mr. Oliver has a Masters in chemical engineering from University College London, England. He can be reached by e-mail at james_oliver@baxter.com.

For more Author

information,

go to

ivthome.com [

A 3-D Risk Assessment ModelJames Oliver

INTRODUCTIONThis article provides a description of a method for assessing risk to product quality during the develop-ment, manufacture, and processing of pharmaceutical and biological products. There are a number of pos-sible methods of assessing risk, as described in ICH Q9 Quality Risk Management (1). This article presents a two-phase approach to risk assessment that may be used in a quality risk management program. The approach is based upon an initial high-level assess-ment that determines the level of risk of a particular system, material, or method in relation to the overall manufacturing process. A more detailed failure mode and effects analysis (FMEA) style assessment may then be conducted for high-risk systems to identify high-risk functions within these systems and potential areas for risk elimination or mitigation.

ADVANTAGES OF PERFORMING A QUALITY RISK ASSESSMENTWhy do we try to quantify potential risk to product quality at various stages in our processing? From a quality systems perspective, risk assessments help determine which systems, materials, methods, or processes used have a potential to impact product quality. Quantifying risk helps provide rationale for the following:

Designing quality into a process or system design

The amount of validation testing to perform Determining how much re-testing to perform as

part of change control for a change to an existing validated system

The level of regulatory submission required for a change to an existing licensed process

Establishing frequency of calibration, mainte-nance, and revalidation

Categorization of exception reporting based on impact to product quality

Allowing for a risk-based approach to sys-tem documentation (e.g., level of lifecycle documentation, disaster recovery procedures, standard operating procedures, frequency of administration/maintenance)

Prioritizing facility inventory of processes, equip-ment, materials, and methods based on impact to product quality.

AIMS OF DEVELOPING A RISK ASSESSMENT PROCEDURE The following are some of the considerations that were addressed in the development of the proposed risk-assessment approach:

A procedure needs to focus on the quality of the product and not include operational inefficiency or financial expense. It is important to ensure that the patient receives product that is of a con-sistently high quality and not to dilute the assess-ment with financial risk concerns. Financial and operational risk tends to artificially elevate the risk of lower risk systems at the expense of focus-ing on those systems that truly present a higher risk to product quality.

The procedure needs to be easily used by the vari-ous skill levels and disciplines at the facility to arrive at a consistent score and should not involve

P E E R - R E V I E W E D

70 Journal of Validation technology [Autumn 2008] i v t home.comi v t home.com

J A M E S OL I V E R

drawn out meetings and discussions where the final system risk is inconsistent depending on who conducts the assessment.

The procedure has to be relevant to the manufac-turing process used at the facility.

The procedure should be based upon current industry guidances (1-5).

3-D REPRESENTATION OF RISK ASSESSMENT STRATEGYThe following three factors represent the three main areas of risk that a system, material, or method may present to product quality:

Where a system or method is used in the manu-facturing process

What it does The complexity or novelty of the system, mate-

rial or method.

Risk increases in three areas of the production cycle. As product gets closer to delivery to patient, the risk increases. This increase in risk is known as the dis-tance along the product stream. Risk also increases as the system impact to processing of product increases. This increase in risk is known as the distance from product stream. The level of system complexity also increases risk. This is known as the system complex-ity. Complexity is based on factors such as level of customization, novelty, degree of integration required, reactivity, toxicity, and the number of products pro-duced using the same production equipment.

Figure 1 depicts the overall approach of the three-dimensional risk assessment strategy. The distance along product stream dimension represents a generic manufacturing process for a recombinant biological parenteral product.

Distance Along Product StreamAt the early stages of the manufacturing process vari-ous quality control systems are in place that can detect potential defects in system performance (e.g., contamination in a bioreactor). At later stages in the process such as filling and packaging, the majority of in process testing has been performed, and although there are controls in place to ensure product quality, the final processing steps carry more risk as they are closer to the point of delivery to the patient. Generally, a defect at the last stages in the process has a lower probability of detection than a defect at the beginning of the process. Therefore the risk to product quality increases at the later stages of the process based on level of defect detection (see Figure 2).

Distance Along Product Stream ScoringThe distance along the product stream is provided a risk score from 1 to 5 (see Table I). If a system is used in more than one area, the higher of the risk scores should be assigned for the areas where it is used.

Additionally as this risk score dimension is based on level of detection, the personnel conducting the assessment may decide to assign a higher risk score for operations early in the manufacturing process based on specific process concerns. An example of this may be viral contamination, in which specific testing for contaminants may not be conducted or be effective later in the process. In this case a viral inactivation step early in the process may be assigned a high-risk score to account for low level of defect detection sub-sequent to this step.

Distance From Product StreamThe closer the system is to the product (i.e., the active pharmaceutical ingredient [API] manufacturing process), the higher the risk associated with the system (see Figure 3). This dimension assesses how greatly the system, method, or material is interacting with the processing of product. In the case of packaging and shipping, product processing includes the identification of product and shipping the correct packaged materials.

Figure 1: Overall approach to risk assessment strategy.

1) Working cell bank thaw and inoculum cell culture

2) Bioreactor cell culture and harvest

3) Purification

4) Bulk formulation and sterile filtration

5) Post sterile filtration, filling,lyophilization, packaging, patient information,delivery to patient

Distance alongproduct stream

Area ofgreatest risk

System complexity

Distance fromproduct stream

1

2

3

4

5 4 3 2 1

1

2

3

4

Journal of Validation technology [Autumn 2008] 71

P E E R - R E V I E W E D

Distance From Product Stream ScoringThe distance from the product stream is provided a risk score from 1 to 5 (see Table II).

System ComplexityA greater level of system complexity or customization increases the probability of potential system failure; resulting in increased level of risk to the product. The complexity score also accounts for how novel the system or process is. Is this the first time the process, method, or material will be used in industry for this application? In this case process understand-ing needs to be developed either in house or with vendor assistance to ensure product quality. Or, is this an industry standard with a high level of process understanding already developed and documented (e.g., United States Pharmacopeia methods).

System Complexity ScoringTable III presents the scoring for system complexity. For computer systems and software, the complexity score can also be developed from software category numbers 1 thru 5 as described by good automated manufacturing practice (GAMP) (5).

Overall Risk ScoreTo obtain the overall risk score for the system, the distance along product stream, distance from product stream, and the system complexity scores are multi-plied together. Table IV compares the overall risk level to the overall risk score.

SECONDARY FMEA FOR HIGH RISK SYSTEMS For systems identified as high risk using the 3D risk assessment, a standard FMEA approach may be used to identify the most likely failure modes within the system. These likely failures may then be designed out of the system or anticipated and a contingency developed. As the FMEA is a more detailed assess-ment with the aim of risk mitigation, it is reserved for higher risk systems, rather than trying to mitigate the risk of systems that already provide a low risk to product quality. The FMEA approach is most valuable during the system design phase when it is possible to proactively design out functions that have a high risk of failure or product impact, rather than reactively perform mitigation once the system or material has been ordered and installed.

Figure 2: Increasing risk based on level of defect detection.

Increasing risk

WCB Thaw and inoculum cell culture

Bioreactor cellculture andharvest

Purification

Bulk formulationand sterilefiltration

Post sterilefiltration filling,lyophilization,packaging,patient

1 2 3 4 5

Table I: Distance along product stream.Process Area Where System is Used

Quantitative Risk Score

Qualitative Risk

Working cell bank thaw and inoculum cell culture

1 Low

Cell culture 2 Low/Medium

Purification 3 Medium

Bulk formulation and sterile filtration 4 Medium/High

Filling, lyophilization, packaging, shipping, patient related data

5 High

72 Journal of Validation technology [Autumn 2008] i v t home.com

J A M E S OL I V E R

Figure 3: Risk associated with the distance from product stream.

Increasingrisk

1

2

3

4

5

Plant steam andglycol system

Monitoring ofenvironment formanufacturingprocess

Environment formanufacturingprocess

Environment forproduct

City water and ROpretreatment

RO systems

WFI/cleansteam systems

Sterility and cleanlinessof in-direct productcontact materials/surfaces

Sterility and cleanliness ofproduct contact materials/surfaces

Raw materialsdelivery and storage

Raw materialspreparation

Raw material/productcontact materialsquality testing

Materials/components

Product qualitytesting

Environment andmaterials for lab testing

Receiving environmentfor lab testing materials

Productprocessing

Table II: Distance from product stream scoring.System Distance from Product Stream (Impact to Process) Quantitative

Risk ScoreQualitative Risk

Systems that provide minimal support of manufacturing pro-cesses.

1 Low

Systems that, though providing some degree of support for quality operations, system failure here does not necessarily lead to product quality issues.

2 Low/Medium

Systems that are used in support of the processing operation, where a system failure could lead to a potential quality issue in the processing operation itself.

3 Medium

Systems that feed directly into the processing of product. A sys-tem failure here is likely to affect product quality.

4 Medium/High

Product Processing. System failure will lead to product being incorrectly processed. Product quality is affected.

5 High

Journal of Validation technology [Autumn 2008] 73

P E E R - R E V I E W E D

Consider the Following Three Systems:

1. Laboratory refrigeratorTo be used in microbiology laboratory to store testing components used in analysis of in-process samples from cell culture and purification areas. System uses a basic non-programmable temper-ature controller, is industry standard, and is monitored by a chart recorder. System needs to be plugged into an electrical outlet.

2. AutoclaveTo be used to sterilize product contact components used in the bulk formulation operation (prior to sterile filtration). System is controlled by a PLC and the autoclave vendor supplies this model to multiple companies throughout industry. Some integration is required for connection to facility utilities. Provides a paper printout of cycle data.

3. Final product filling machineTo be used to asepti-cally fill final product into vials. System is integrated with multiple pieces of equipment upstream and down-stream, control system is custom programmed for the facility, includes electronic reporting of fill data, alarms, and events.

Distance Along Product Stream Score Laboratory refrigerator related to cell culture and purifi-

cation = 3 Autoclave related to bulk formulation area = 4 Filling machine = 5

Distance From Product Stream Score Laboratory refrigerator for lab testing materials = 2 Autoclave for product contact components = 4 Filling machine = 5

System Complexity Score Laboratory refrigeratorSystem uses a basic non-pro-

grammable temperature controller, is industry standard, and is monitored by a chart recorder. System needs to be plugged into an electrical outlet. System Complexity = 2

AutoclaveSystem is controlled by a PLC and the auto-clave vendor supplies this model to multiple companies throughout industry. Some integration is required for connection to facility utilities. Provides a paper printout of cycle data. System Complexity = 4

Filling MachineSystem is integrated with multiple pieces of equipment upstream and downstream, control system is custom programmed for the facility, includes electronic reporting of fill data, alarms and events. System Complexity = 5

Overall Risk Score and Level for the Example SystemsThe overall risk score is calculated by multiplying the distance along product stream, distance from product stream, and system complexity scores.

System Distance Along Product Stream

Distance From Product Stream

System Com-plexity

Overall Score

Lab refrigerator

3 2 2 12

Autoclave 4 4 4 64

Filling machine

5 5 5 125

The overall risk score is used to assign a risk level to the system as follows:

Risk Score Risk Level

1-25 Low

26-50 Low/Medium

51-75 Medium

76-99 Medium/High

100-125 High

Lab refrigerator = Low riskAutoclave = Medium riskFilling Machine = High risk

For the high-risk filling machine the design team may con-duct an FMEA to identify areas of risk that could be elimi-nated or mitigated prior to completion of system design. From a validation perspective an FMEA may help to focus validation testing of critical functions during validation planning and subsequent testing.

Example

74 Journal of Validation technology [Autumn 2008] i v t home.comi v t home.com

J A M E S OL I V E R

A FMEA analyzes each major system function and provides a quantitative assessment for the probability of failure, the impact of the failure, and the level of detection of the failure. An overall risk priority number (RPN) is calculated from these factors (i.e., RPN = severity of failure x probability of failure level of detection). If the RPN is high, it shows a high risk of system function failure that would impact the quality of the system output and would not be easily detectable. Functions that have a high RPN would be considered for redesign in order to mitigate or eliminate the risk of system failure. Table V

shows an example format that could be used for the FMEA. Figure 4 provides examples of how the risk assessment may be incorporated into systems design and validation planning.

CONCLUSIONOnce an overall assessment of risk has been made for a system, method, material, or process, the appropriate level of quality systems may be implemented to ensure product quality during the lifecycle of the manufac-turing process. Using a risk-based approach early in

Table III: System complexity scoring.System Complexity Quantitative

Risk ScoreQualitative Risk

The system, method, material or process is very simple, industry standard and non-customized. Widely used in the industry. Very basic design principles. Analytical methods are simple and are industry standard (e.g., USP testing for conductivity). Very low level of complexity.

1 Low

System, method, material, or process is based on industry standards or off-the-shelf applications. No customization, may be easily integrated into end user manufactur-ing operation. Low level of complexity.

2 Low/ Medium

System, method, material, or process is based on industry standards or off-the-shelf applications and some degree of customization is required to integrate into the end user manufacturing operation. Medium level of complexity.

3 Medium

System, method, material, or process is based on industry standards or off the shelf applications but has been highly customized for the end user manufacturing operation. High level of complexity.

4 Medium/High

System, method, material, or process is specifically designed and created for the end user manufacturing operation. Fully customized or novel or very high level of complexity. System requires high degree of integration into upstream and down-stream operations.

5 High

Table IV: Overall risk level based on the overall risk score.Overall Risk Score Overall Risk Level

1 25 Low

26 50 Low/Medium

51 75 Medium

76 99 Medium/High

100 125 High

Table V: An example format that could be used for the FMEA.System Function

Failure Mode Severity of Failure (1 5)

Probability of Failure (1 5)

Level of Detection (1 5)

RPN = (Sever-ity x Probabil-ity/Detection)

Actions or Comments

Journal of Validation technology [Autumn 2008] 75

P E E R - R E V I E W E D

a project may focus development in the areas that have the most impact on product quality; therefore, risk mitigation or elimination may be incorporated at the design stage rather than through repeated test-ing and rework after the design has been finalized. Early identification and mitigation of risk to prod-uct quality reduces potential risk to patient safety.

REFERENCES1. ICH, Q9 Quality Risk Management, November 9, 2005.2. ICH, Q8 Pharmaceutical Development, November 10,

2005.

3. ASTM E250007, Standard Guide for Specification, Design, and Verification of Pharmaceutical and Biopharmaceutical Manufacturing Systems and Equipment, August 2007.

4. FDA, Pharmaceutical cGMPs for the 21st CenturyA Risk

Based Approach, Final Report, September 2004.5. ISPE, GAMP 5: A Risk Based Approach to Compliant GxP

Computerized Systems, February 2008. JVT

ARTICLE ACRONYM LISTINGAPI Active Pharmaceutical IngredientFMEA Failure Mode and Effects AnalysisGAMP Good Automated Manufacturing PracticeRPN Risk Priority Number

Figure 4: Risk assessment incorporated into system design and validation planning.

Risk assessmentduring system design

System need identified

System designedaccording to

process and user requirements

Perform riskassesment

Is systemhigh risk?

Perform FMEA

Update design toeliminate/mitigatehigh risk functions

Complete systemdesign and

review process

YES

NO Complete systemdesign and

review process

Risk assessmentduring validation planning

Need to validateidentified

Perform riskassessment for what

is to be validated usingspecifications for

system

Is systemhigh risk?

Perform validationper validation plan/

protocol

NO

YES

Perform FMEAto identify highest

risk functions

Incorporate increasedtesting of high risk

functions in validationplan/protocol

76 Journal of Validation technology [Autumn 2008] i v t home.comi v t home.com