Evaluating Product Characteristics for Acceptance Mark D. Johnson, PMP ASQ CMQ/OE & CQE AbbVie Inc.

Evaluating Product Characteristics for Acceptance

Mark D. Johnson, PMPASQ CMQ/OE & CQE

AbbVie Inc.

Midwest Biopharmaceutical Statistics Workshop

2

Agenda

• Product Characteristics• Defect Rating Criteria• Sampling Techniques• Questions

20 May 2015


3

Product Characteristics

• Identify characteristics important to confirming product performance

• These might be items such as:Component dimensions for device componentsContent uniformity for tabletsPotency and impurities for a drug product lotForce to activate an autoinjectorCorrect tablet coloring for different dosage

strengths of same product

20 May 2015


4

Product Characteristics

Product characteristics can be based on:• Compendial performance expectations• Current quality attribute testing for a similar

product• New tests required to assess unique features of a

new product• Product features designed to address specific user

needs

20 May 2015


5

Defect Rating CriteriaWhen a pharmaceutical, medical device or combination product is defective, three defect types are possible:

• Design - the pharmaceutical was manufactured correctly, but anticipated or unanticipated side effects caused by the drug use resulted in patient harm or injury; the medical device was poorly designed or not properly tested, creating a situation where the products might be defective and dangerous.

• Manufacturing - the product was designed to meet requirements, but manufactured improperly or contaminated somehow during the manufacturing process, potentially exposing the patient to harm.

• Marketing – the product’s instructions, warnings, or recommendations for the use of the drug are not sufficiently clear or appropriate to help consumers use the product correctly.

Reference: HG.org Legal Resources Website

20 May 2015


6

Defect Rating CriteriaExamples of the three defect types are:

• Design – An infusion pump screen design is cumbersome or confusing to users, which causes a delay in therapy. For example, the “Start Infusion” key may be located next to the “Power” key, and a user may turn off the infusion pump and lose programmed pump settings instead of initiating infusion. In some cases, programmed settings are lost when a user turns the pump off, and the infusion settings have to be re-entered after the pump restarts.

• Manufacturing – Excess water has entered a drug substance batch manufacturing run, due to a faulty valve in the purified water system. This resulted in elevated impurity and reduced potency levels within the batch.

• Marketing – A patient does not adequately comprehend the medication’s use instructions that require him to eat a meal and drink water prior to taking his daily dose.

20 May 2015


7

Defect Rating Criteria• Product characteristics need to be examined to determine the

impact of not meeting their acceptance criteria• These characteristics might be based on similar attributes

assessed for a comparable product• Likewise, they may be identified and confirmed during the

Process Validation Lifecycle activities recommended to document development, verification, validation and future manufacture of the product

• This examination may be based on evaluating the impact of the defect upon product functionality or clinical effect to a user

• Also, risk-based evaluation can be used to identify the impact of a product defect

20 May 2015


8

Defect Rating Criteria• Product sampling based on identification of defects was initially

used in the Aerospace industry• This approach was adopted within the pharmaceutical industry in

the 1970s and 1980s• There is no industry-wide consensus on a specific percent defective

level used to characterize Critical, Major, Minor or Cosmetic defects• Dr. Wayne Taylor, a recognized sampling expert, cited the following

percent defective levels were commonly used to assess medical devices for product defects:Critical (Health and Safety): 0.065%, 0.1%, 0.25% Major (Functional): 0.25%, 0.4%, 0.65%, 1.0%Cosmetic: 2.5%, 4.0%

20 May 2015


9

Defect Rating CriteriaProduct Attribute Typical Assigned AQLsCritical 0.04%, 0.065%, 0.1%, 0.15%, 0.25%Major Functional 0.25%, 0.4%, 0.65%, 1.0%Minor Functional 0.65%, 1.0%, 1.5%Cosmetic Visual 1.5%, 2.5%, 4%, 6.5%

• Acceptance Quality Limits (AQLs) in Pharma / Devices• These attribute categories help identify the foci of our sampling activities.• Are we only interested in assessing superficial product characteristics, or do

we need to assess critical product operations or functions as well?• We use these attribute categories to assure our processes operate to meet or

surpass our expectations for observing defective product.

20 May 2015


10

Defect Rating Criteria

Defect rating criteria may be defined as:• Critical: A nonconformity likely to present a hazard to

health • Major Functional: A nonconformity which may cause the

product to be unfit for use, significantly degrades its function or performance, or is likely to generate a complaint

• Minor Functional: A nonconformity which may cause the product to function poorly or a user inconvenience, but may still be fit for use, or possibly generate a complaint

• Cosmetic Visual: A nonconformity detrimental to the high quality image of a product that will not affect usability or functionality, but affects only appearance of the product

20 May 2015


11


These criteria can be based on:• Similar defects experienced with a similar product• Historically observed defects for the product’s dose

modality (e.g., oral solution, tablets, capsules)• Risk-based assessment of hazardous situations/failure

modes experienced with the product• Means of evaluating product performance vs. current

test methods or other necessary standards

20 May 2015


12


Assessment of these criteria is completed by:• Laboratory testing of product vs. required quality

attributes• Functional testing of product to meet operational

requirements• Visual examination for any noticeable defects

20 May 2015


13

Sampling Techniques

• To assess product characteristics, an adequate sample of sufficient confidence must be used

• Based on the defect level of the characteristic of interest, different confidence levels might be considered for validation purposes:Critical (e.g., % defective ≤ 0.4%), validate to ≥ 90%

confidenceMajor (e.g., 0.40% < % defective ≤ 2.5%), validate to ≥ 50%

confidenceMinor (e.g., % defective > 2.5%) validate to ≥ 20%

confidence

20 May 2015


14

Sampling Techniques

Similarly for process validation activities, an adequate level of confidence is also required:• New process/product or major process change,

validate to ≥ 90% confidence• Routine revalidation, validate to ≥ 50% confidence• Minor process change or accelerated testing,

validate to 20-50% confidence

20 May 2015


15

Sampling Techniques

There are different methods available to sample product for inspection: • Simple Random Sampling – Selecting samples so that each

unit has an equal chance of being selected• Stratified Random Sampling – Selecting samples

deliberately from each time period or location in a batch• Nested Sampling – Selecting units from locations within a

batch and obtaining multiple samples from each location• Systematic sampling – Selecting units periodically over time

20 May 2015


16

Sampling Techniques

6.04.02.00.40.0

1.0

0.8

0.5

0.1

0.0

n sample sizec acceptance number

Lot Percent Defective

Prob

abili

ty o

f Acc

epta

nce

55 0175 0575 0

n c

Operating Characteristic (OC) Curve, 0.4% Defective Sampling Plans

Plan Confidence = (1-Probability of Acceptance)*100%

20 May 2015


17

Sampling TechniquesValidation vs. Product Acceptance Sampling Plans – What are the differences?• Validation Sampling

Cannot assume process is good, because it has not been validated yet Need to prove it's good "Guilty until proven innocent“ Need to use tighter sampling plan than one used for product acceptance (e.g., will reject

when data shows high confidence (e.g., 90% or 95%) that product performance vs. acceptance criteria is worse than acceptable % defective)

• Product Acceptance Sampling Assumes process is good, since process successfully validated "Innocent until proven guilty“ Will only reject when data shows low confidence (e.g., 5-10%) product performance vs.

acceptance criteria is worse than the AQL

20 May 2015


18

Sampling Techniques

If you pass the “n=575, a=0” PPQ sampling plan, you have 90% confidence (e.g., (1-0.1 Pr(Accept)*100%) you will accept a lot ≤ 0.4% Defective.

If you pass the “n=32, a=0” Product Acceptance sampling plan, you have 12% confidence (e.g., (1-0.88 Pr(Accept)*100%) you will accept a lot ≤ 0.4% Defective.

10.08.06.04.02.00.40.0

0.95

0.50

0.10

0.00


% Defective

Pr(A

ccep

t)

575 032 0

n c

Operating Characteristic (OC) Curve0.4% Validation Sampling (Blue) vs. Product Acceptance Sampling (Red)

20 May 2015


19

Sampling Techniques

• Once the process has been validated, the confidence of the sampling plan can be relaxed to routinely assess batches for large quality defects.

• ANSI Z1.4 provides attribute sampling plans to be used for routine lot inspection and acceptance

• ANSI Z1.9 provides variables sampling plans to be used for routine lot inspection and acceptance

• Both plans provide switching rules for transitioning from normal to tightened, tightened to normal, normal to reduced and reduced to normal sampling practices, dependent on the continuing documented quality history of the product

20 May 2015


20

Sampling Techniques

• Once you’ve determined the product characteristics to assess vs. their required acceptance criteria, you can build a sampling approach

• Two types of sampling plan approaches typically used:Attributes: units assessed within each sample can only

pass or fail the acceptance criteriaVariables: mean and standard deviation are calculated

for a sample of units, then a range is calculated using a factor suitable to the defect level and sample size for comparison with the acceptance criteria

20 May 2015


21

Sampling Techniques

• Attribute characteristic examples:Fill volume: >20 mLTablet defects: cracked, chippedExtraneous matter: visible particulate, rubber, hair

• Variables characteristic examples:Component dimensions: Length = 2 ± 0.25 mmEjection force: ≤ 10 NewtonsCrush test for tablet: ≥ 100 Newtons

20 May 2015


22

Sampling Techniques

• Attribute Sampling:Easy to conclude result, either pass/failMany samples might be required to provide adequate

confidence in pass/fail decision• Variables Sampling:

Variables data contains more information than percent nonconforming

More descriptiveMore statistical information and powerFewer samples required for same statistical power

20 May 2015


23

Sampling Techniques

8.06.04.02.00.40.0

0.95

0.50

0.20

0.10

0.00

Lot Percent Defective

Pro

bability

of A

ccepta

nce


90% conf 575 050% conf 175 020% conf 55 0Tightened 50 0Normal 32 0

n c

Operating Characteristic (OC) Curve0.4% Defective Attribute Sampling Plans

20 May 2015


24

Sampling Techniques

20 May 2015


25

Questions…

20 May 2015


26

Thank You!

20 May 2015

Evaluating Product Characteristics for Acceptance Mark D. Johnson, PMP ASQ CMQ/OE & CQE AbbVie Inc.

Documents

Transcript of Evaluating Product Characteristics for Acceptance Mark D. Johnson, PMP ASQ CMQ/OE & CQE AbbVie Inc.