Introduction to Statistical Applications for Process Validation

Eugenie KhlebnikovaSr. Validation Specialist, CQEMcNeil Consumer Healthcare

1

AGENDA

2

Regulatory Expectations for Statistical Analysis

Statistical Tools

Six Sigma and Process Validation

Common Mistakes to Avoid

REGULATORY EXPECTATIONS

3

PV GUIDELINES

• Emphasis on process design elements, and maintaining process control based on knowledge gained throughout commercialization

• Emphasize to have good knowledge to detect and to control variability through use of statistical analysis

• statistical tools to be used in the analysis of data

• the number of process runs carried out and observations made should be sufficient to allow the normal extent of variation and trends to be established to provide sufficient data for evaluation.

4

PROCESS VALIDATION LIFE CYCLE

5

Statistics to analyze and optimize results (DOE, variation analysis, etc)

Variation analysis, capability, stability analysis

Process CapabilityControl Charts

Stage 1: Process Design

Stage 2: Process Qualification

Stage 3: Process Monitoring and Improvement

PROCESS UNDERSTANDING

6

Testing the final product and passing specifications does not give knowledge

of the process

Variation at each production stage

Knowledge of stability and capability

PROCESS UNDERSTANDING – KNOW VARIATION

“Understanding variation is the key to success in quality and business” W. Edwards Deming (Father of Modern Process Control)

The customers “feel” variation and lack of consistency in a product much more so than the “average” (Jack Welch)

7

FDA PV GUIDANCE RECOMMENDATIONS

INTEGRATED TEAM APPROACH

8

process engineering and manufacturing analytical

chemistry

microbiology

industrial pharmacy

quality assurance

statistics

Recommended that a statistician or person with adequate training in statistical process control technique develop the data collection plan and statistical methods and procedures used in measuring and evaluating process stability and process capability.

DESCRIPTIVE VS INFERENTIAL STATISTICS

9

The Division Between Descriptive and Inferential Statistics

This distinction is based on what you’re trying to do with

your data

DESCRIPTIVE STATISTICS

10

• Summarizing or displaying the factsMean = Sum of all observations/ # of observations

Range = Max - Min

Standard DeviationVariance = std dev2

Relative Standard Deviation or CV = std dev*100/mean

RELATIVE STANDARD DEVIATION

Example 1:

Group Size Avg St Dev RSD

1 10 80 0.8 1.0

2 10 90 0.9 1.0

3 10 100 1.0 1.0

4 10 110 1.1 1.0

5 10 120 1.2 1.0

Example 2:

11

Group Size Avg St Dev RSD

1 10 80 1.0 1.4

2 10 90 1.0 1.1

3 10 100 1.0 1.0

4 10 110 1.0 0.9

5 10 120 1.0 0.8

Standard deviation is proportional to the average and the %RSD is unchanged

%RSD is changing because the average is changing, not the standard deviation

EXAMPLE: BLEND UNIFORMITY

Tote

Location

Batch 1 Batch 2 Batch 3

1 101 100 102

2 98 99 104

3 99 101 99

4 100 103 97

5 103 97 101

6 102 102 100

7 101 100 102

8 100 101 98

9 102 102 103

10 104 99 102

12

Specification:90-110%RSD ≤ 5.0%

EXAMPLE: BLEND UNIFORMITY

Tote

Location

Batch 1 Batch 2 Batch 3

1 101 100 102

2 98 99 104

3 99 101 99

4 100 103 97

5 103 97 101

6 102 102 100

7 101 100 102

8 100 101 98

9 102 102 103

10 104 99 102

13

Minitab Output:

Descriptive Statistics: Batch 1, Batch 2, Batch 3

Variable Mean StDev CoefVar Minimum Maximum

Batch 1 101.00 1.83 1.81 98.00 104.00

Batch 2 100.40 1.78 1.77 97.00 103.00

Batch 3 100.80 2.25 2.23 97.00 104.00

EXAMPLE: BLEND UNIFORMITY

14

INFERENTIAL STATISTICS

15

INFERENTIAL STATISTICS

• A decision about the batch is based on a relative small sample taken since it is not realistic to test the entire batch.

• To confirm that the data is representative of the batch, inference statistics (confidence and tolerance intervals) can be used to predict the true mean.

16

CONFIDENCE INTERVAL

• A confidence interval is an interval within which it is believed the true mean lies

where is sample mean, s is sample standard deviation, N is the sample size, and t value is a constant obtained from t-distribution tables based on the level of confidence. Note the value of t should correspond to N-1.

17

CI = ±

TOLERANCE INTERVAL

• A tolerance interval is an interval within which it is believed the individual values lie,

TI = ± k*s where is sample mean, s is sample standard deviation, N is the sample size, and k value is a constant obtained from factors for two-sided tolerance limits for normal distributions table believed the true mean lies.

18

EXAMPLE

A batch of tablets was tested for content uniformity. The mean value of 10 tablets tested was 99.1% and a standard deviation was 2.6%.

19

EXAMPLE: Confidence Interval

• t from a table

• N-1=10-1=9

• t=3.25

• probability of 99% covering 99% of data

20

EXAMPLE: Confidence Interval

• CI = ± = 99.1 ± =96.4 to 101.8

• Then we can say that we are 99% certain that the true batch mean will be between 96.4%and 101.8 %.

21

EXAMPLE: TOLERANCE INTERVAL

22

N=10,

k =5.594 probability of 99% covering 99% of data

EXAMPLE: TOLERANCE INTERVAL

23

• N=10, mean=99.1, s =2.6, k =5.594

TI = ± k*s

• Probability of 99% covering 99% of data:

TI =99.1 ± (5.594*2.6)

TI = 84.6% to 113.6%

EXAMPLE: Confidence and Tolerance Interval

• If a sample has the mean value of 10 tablets at 99.1% and a standard deviation at 2.6%.

• Then we can say that we are 99% certain that 99% of the tablet content uniformity lies between 80.6 and 117.6% and we are 99% certain that the true batch mean will be between 96.4 and 101.8 %.

24

SAMPLING

25

SAMPLING

• The cGMPs mention samples, sampling plans, or sampling methods repeatedly.

• Firms are expected:– To use a sampling plan that utilizes basic elements

of statistical analysis

– Provide a scientific rationale for sampling that would vary the amount of samples taken according to the lot size

– Define a confidence limit to ensure an accurate and representative sampling of the product

26

WARNING LETTER EXAMPLE

211.165 - Testing and release for distribution:

(d) Acceptance criteria for the sampling and testing conducted by the quality control unit shall be adequate to assure that batches of drug products meet each appropriate specification and appropriate statistical quality control criteria as a condition for their approval and release. The statistical quality control criteria shall include appropriate acceptance levels and/or appropriate rejection levels.

“For example, your firm's finished product sampling plan product A is not representative of the batch produced. A total of 13 units are sampled per lot, with 3 tested for bacterial endotoxin and 10 tested for bioburden. This sampling of 13 units is irrespective of lot size, which may vary from X to Z units (vials) per lot”

27

CHOOSING SAMPLES

Sampling

Method:

•Simple Random

•Convenience

•Systematic

•Cluster

•Stratified

28

SAMPLING METHODS

SYSTEMATIC

CONVENIENCE

SIMPLE RANDOM

0 min 30 min 1 hr

CLUSTER

top

bottom

middle

STRATIFIED

29

SAMPLING RISK

DISPOSITION IMPACT IF LOT GOOD

IMPACT IF LOT BAD

Lot is accepted Correct Decision Incorrect Decision(Type II or

Consumer’s risk)

Lot is rejected Incorrect Decision(Type I or Producer’s

risk)

Correct Decision

30

Expressed as Acceptable Quality Level (AQL): maximum average percent defective that is acceptable for the product being evaluated.

ACCEPTANCE SAMPLING

Acceptance Sampling is a form of inspection applied to lots or

batches of items before or after a process to judge

conformance to predetermined standards.

Sampling Plans specify the lot size, sample size, number of

samples and acceptance/rejection criteria.

Random sampleLot31

OPERATING CHARACTERISTIC CURVE

• The operating-characteristic (OC) curve measures the performance of an acceptance-sampling plan.

• The OC curve plots the probability of accepting the lot versus the lot fraction defective.

• The OC curve shows the probability that a lot submitted with a certain fraction defective will be either accepted or rejected.

32

OC CURVES

Ideal OC Curve

100908070605040302010

1 1.5 2 2.5 3 3.5

33

Reject all lots with more than 2.5% defective and accept all lots with less than 2.5% defectiveThe only way to assure is 100% inspection

Pro

bab

ility

of

acce

pta

nce

(%

)

Percent defective (%)

An Operating Characteristic Curve (OCC) is a probability curve for a sampling plan that shows the probabilities of accepting lots with various lot quality levels (% defectives).

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 .05 .10 .15 .20

Pro

babi

lity

of a

ccep

ting

lot

Lot quality (% defective)

Under this sampling plan, if the lot has 3% defective

the probability of accepting the lot is 90%

the probability of rejecting the lot is 10%

If the lot has 20% defective

it has a small probability (5%) of being accepted

the probability of rejecting the lot is 95%

OCCs for Single Sampling Plans

34

SAMPLING PLANS

Sampling plans involve:

Single sampling

Double sampling

Multiple sampling

Provisions for each type of sampling plan include1. Normal inspection2. Tightened inspection3. Reduced inspection

35

SWITCHING RULES

36

Reduced

Tightened

Normal

“and” conditions:Production Steady10 consecutive lots

acceptedApproved by responsibility

authority

“or” conditions:Lot rejected

Irregular productionA lot meets neither the accept nor the

reject criteriaOther conditions warrant return to normal inspection

2 out of 5 consecutive lots rejected

5 consecutive

lots accepted 10 consecutive

lots remain on tightened inspection

Start

Discontinue inspection

SAMPLING BY ATTRIBUTES: ANSI Z1.4 2008

• The acceptable quality level (AQL) is a primary focal point of the standard

• The AQL is generally specified in the contract or by the authority responsible for sampling.

• Different AQLs may be designated for different types of defects (critical, major, minor).

• Tables for the standard provided are used to determine the appropriate sampling scheme.

37

ANSI Z1.4 2008

PROCEDURE:1. Choose the AQL2. Choose the inspection level3. Determine the lot size4. Find the appropriate sample size code

letter from Table I-Sample Size Code Letters5. Determine the appropriate type of

sampling plan to use (single, double, multiple)

6. Check the appropriate table to find the acceptance criteria.

38

SAMPLE SIZE DETERMINATION

39

Table I - Sample Size Letter CodesSpecial Inspection Levels General Inspection Levels

Lot or Batch Size S-1 S-2 S-3 S-4 I II III2 to 8 A A A A A A B

9 to 15 A A A A A B C

16 to 25 A A B B B C D

26 to 50 A B B C C D E

51 to 90 B B C C C E F

91 to 150 B B C D D F G

151 to 280 B C D E E G H

281 to 500 B C D E F H J

501 to 1200 C C E F G J K

1201 to 3200 C D E G H K L

3201 to 10000 C D F G J L M

10001 to 35000 C D F H K M N

35001 to 150000 D E G J L N P

150001 to 500000 D E G J M P Q

500001 to over D E H K N Q R

SAMPLE SIZE DETERMINATION

40

SINGLE SAMPLING PLAN - EXAMPLE

Defect: any color except of red

N = lot size = 25 apples

From Sample Size Code Letters:

From Normal Single Level Inspection

n = sample size =3

C=acceptance number = 0 Accept/1 Reject

Lot or batch size General Inspection Level

16-25 B

Sampling Size Code

Letter

Sample Size AQL 0.010

B 3 0/1 Scenario 1:0 defectsAccept

Scenario 2:2 defectsReject

41

SINGLE SAMPLING PLAN - EXAMPLE

N = lot size = 120,000

From Sample Size Code Letters:

Normal Inspection

From Normal Single Level Inspection

Lot or batch size General Inspection Level

35,001-150,000 N

Sampling Size Code Letter

Sample Size

CriticalAQL 0.010

MajorAQL 0.65

MinorAQL 4.0

N 500 ACC 0 / REJ 1 ACC 7/ REJ 8 ACC 21 / REJ 22

42

STATISTICAL PROCESS CONTROL

• The principle of SPC analysis is to understand the process and detect the process change.

• Statistical Process Control (SPC) charts are used to detect process variation.

43

STATISTICAL PROCESS CONTROL

• The Current Good Manufacturing Practices for Process Validation published by the FDA in January 2011 states "homogeneity within a batch and consistency between batches are goals of process validation activities." Control charts explicitly compare the variation within subgroups to the variation between subgroups, making them very suitable tools for understanding processes over time (stability).

44

n = 1 2 < n < 9median

n is ‘small’3 < n < 5

n is ‘large’n > 10

X & RmX & R X & R X & S

VARIABLE CONTROL CHARTS

Used for measured data

45

CONTROL CHART SELECTION: ATTRIBUTE DATA

Defect orNonconformity Data

Defective Data

Variable n > 50

Constant n > 50

Constant Sample Size

Variable Sample Size

C chart u chart p or np chart p chart

Used for count (attribute) data46

Stable and Unstable Processes

A stable (or “in control”) process is one in which the key process responses show no signs of special causes.

An unstable (or “out of control”) process has both common and special causes present.

47

UCL

LCL

UCL

LCL

CONTROL CHART

305

303.7

302

300

298.0

296.3

285

280

0 min 30 min 1 hr1 hr 30

min 2 hr2hr 30

min

48

mean

UCL

LCL

Tablet Weight

PROCESS CAPABILITY

• Is the process capable of consistentlydelivering quality products?

• Is the process design confirmed as being capable of reproducible commercial manufacturing?

• Process capability is expressed as a ratio of specifications/process variability

49

PROCESS CAPABILITY INDECES

50

5.334.02.671.33-1.33-2.67-4.0-5.33 0

0 .4

0 .3

0 .2

0 .1

0 .0

Lower Spec. Limit

Upper Spec. Limit

Cust. Tolerance

0

0 .4

0 .3

0 .2

0 .1

0 .0

5.334.02.671.33-1.33-2.67-4.0-5.33

Lower Spec. Limit

Upper Spec. Limit

Cust. Tolerance

Cpk < 1 - not capableCpk = 1 - marginally capableCpk > 1 - capable

PROCESS CAPABILITY

Accurate and precise Accurate but not precise Precise but not accurate

Desired

Current

Situation

LSL USLT LSL USLT

Current

Situation

DesiredDesired

LSL USLT

51

PROCESS CAPABILITY INDECES

• Short-term (Cp and Cpk) and/or long term (Pp and Ppk) are commonly used to evaluate process performance.

• Cpk attempts to answer the question "does my current production sample meet specification?"

• Ppk attempts to answer the question "does my process in the long run meet specification?"

52

EXAMPLE: PROCESS CAPABILITY

53

10987654321

20.0

19.5

19.0

Sa

mp

le M

ea

n

__X=19.599

UCL=20.239

LCL=18.959

10987654321

1.2

0.8

0.4

Sa

mp

le S

tDe

v

_S=0.656

UCL=1.126

LCL=0.186

108642

21.0

19.5

18.0

Sample

Va

lue

s

2322212019181716

LSL USL

LSL 16

USL 23

Specifications

22.521.019.518.0

Within

Overall

Specs

StDev 0.674453

Cp 1.73

Cpk 1.68

Within

StDev 0.673974

Pp 1.73

Ppk 1.68

Cpm *

Overall

Process Capability Sixpack of Hardness

Xbar Chart

S Chart

Last 10 Subgroups

Capability Histogram

Normal Prob PlotAD: 0.304, P: 0.564

Capability Plot

PROCESS CAPABILITY

• At a minimum, 50 individual values or 25 subgroups for sub-grouped data are required to calculate process capability; and 100 individual values provide a stronger basis for the assessment.

• Use SPC charts to check if the process is stable

• Check the distribution (normal vs not normal)

• Use the Cpk value which represents the process under consideration

54

PROCESS CAPABILITY EXAMPLE

• A client had to meet Cpk requirement of ≥ 1.20.

• When data was assumed to be normally distributed, the Cpk =0.8

• When the non-normal behavior was accounted for, the Cpk = 1.22

55

SIX SIGMA AND PROCESS VALIDATON

• Six Sigma and Process Validation

• Use the process knowledge to make improvements

56

SIX SIGMA AND PROCESS VALIDATON

Six Sigma – process improvement methodology

DMAIC

Define Objective To improve compression process

MeasureMeasure hardness during PV

Analyze Statistical analysis, calculate Cp/Cpk

Improve Decrease variation

Control Control variation

57

Cpk and SigmaSigma 1, Cpk = 0.33

Sigma 2, Cpk = 0.67

Sigma 3, Cpk = 1

Sigma 5, Cpk = 1.67

Sigma 4, Cpk = 1.33

COMMON MISTAKES

• Incorrect use of statistical tools:

– ANSI Attribute Sampling for measurement data (pH)

– Incorrect sampling size

– Distribution is not checked

– Process in not stable

– Incorrect uses of Cpk (equivalency between equipment, large specification limits, etc)

59

WARNING LETTER: EQUIPMENT COMPARABILITY AND CAPABILITY

• The firm referenced the Cpk values for processes using a double-sided tablet press and the single-sided tablet press to demonstrate statistical equivalence.

• FDA evaluation :

– The Cpk value alone was not appropriate metric to demonstrate statistical equivalence. Cpk analysis requires a normal underlying distribution and a demonstrated state of statistical process control.

– Statistical equivalence between the two processes could have been shown by using either parametric or non-parametric (based on distribution analysis) approaches and comparing means and variances.

– Firm did not use the proper analysis to support their conclusion that no significant differences existed between the two compression processes.

60

STATISTICAL EVALUATION

• Is required by statute

• Is an expectation of the regulatory inspector during inspection of the firm as it relates to process validation of products

• Use statistical tools that are meaningful and useful to understand the baseline performance of the process

• Is invaluable as a troubleshooting tool post validation

61

QUESTIONS

62