Preventing OOS Deficiencies

05/03/23 02:08 Preventing OOS Deficiencies 1

Preventing OOS DeficienciesLynn Torbeck


List of TopicsBriefly review: Barr Case FDA OOS Guidance Able Laboratories Story PDA Scientific Advisory Board Committees

Troublesome fundamentalsUnresolved issuesPreventing OOS deficienciesFinal recommendations


Barr CaseAudited in 1989, 1991 and 1992.Refused to accept a consent decree.FDA was forced to go to court.Civil action taken June 1992.Decision in favor of the FDA on February 4, 1993.


Barr and Statistical IssuesInitial investigationsFull investigationsTestingRetestingAveragingOutliers techniques


Test

Out of Spec?Report out

resultsNo

Yes

Known PhysicalReason?

USP allowsresamples?

No

Yes2 nd Stage

(i.e. ContentUniformity) Passes?

Yes

Begin Lab InitialInvestigation

No

No

Resample asneeded

Invalidate data,Document,Start Over

Document

Original samplerepresentative?

No

Yes

Lab orAnalysterror?

Yes

Retest Passes?Yes

No

Otherreason?

Yes

FinishInvestigation

Yes

Documentand report

FullInvestigation

No

USP <111> Outlier IDHistorical Data

Review isinconclusive

Retestjustified?

NoRetest n times

Documentjustification

All pass?

No

Finishinvestigation

Yes

Yes

OOS Logicby Lynn Torbeck

No

March 1999

Copyright 1999 byL. Torbeck


Barr: Lessons LearnedFDA takes OOS issues very seriously.OOS SOP’s, laboratory logs and documented investigations will be part of any Quality System review.Companies are still getting Form 483 observations for not having an adequate SOP or for not following the SOP.


Barr: OOS PreventionAnalysts, supervisors and managers should read and discuss the Barr case and understand the OOS issues in context.


FDA Guidance“Investigating Out of Specification (OOS) Test Results for Pharmaceutical Production.”Issued as a draft in September 1998.Still in draft as of today.FDA has sent it to the attorneys.Final version could be out this year.


Draft: OOS PreventionAll laboratory personnel, analysts, supervisors and managers should read, study and discuss in-depth, sentence by sentence if necessary, the draft OOS guidance.Then do it again when the final guidance is released.


Able Labs – Cranbury, NJMassive number of OOS errorsRecall of all 46 products 3,184 lots recalledFive ANDA’s withdrawnHundreds of staff laid offSold to Sun Pharm in December 2005www.ablelabs.com

http://www.ablelabs.com/


Able Labs: Lessons Learned

It is still possible to have wide spread misunderstanding of the Barr case, the OOS guidance and OOS SOPs.Apparently the analysts felt they could not give an “incorrect result.”Management needs to instill and cultivate a “GMP Culture” in the analytical laboratory.


Able Labs: OOS PreventionReview the Able Labs web site.Discuss the Able Labs story with laboratory analysts, supervisors and managers.Discuss what a “GMP Culture” means in the analytical laboratory and how to develop and reward it.


PDA OOS CommitteesChemical OOS: Lynn Torbeck, Chair Eight members Draft technical report reviewed by the

FDA Planning a PDA/FDA conferenceMicrobial Data Deviations: Jeanne Moldenhauer, Chair Draft in revision


Troublesome Fundamentals:

OutliersReportable ValuesAveragingTesting into complianceFull consideration


Outliers - DefinedExtreme values vs outliers:


Outliers – Judge Wolin"The USP expressly allows firms to apply this test (outlier) to biological and antibiotic assays, ..., but is silent on its use with chemical tests.”

"In the Court's view the silence of the USP with respect to chemical testing and outliers is prohibitory."


Outliers - Investigation"In chemical procedures, where method accuracy variation is small, an outlier test may be appropriate as part of an OOS investigation, provided the sample and test procedure assumes homogeneity ... as in the composite strength assays. Our current thinking is that outlier tests are never appropriate where the purpose of the sample is to measure uniformity" Paul Vogel, September 10, 1993.


Outliers - TestsDixon's criteria, the test in USP<111>, is general in nature and not specific to biological issues. It can be used anywhere the statistical assumptions can be met.In general, statisticians agree with the philosophy that outlier tests should be used infrequently and with great caution.


Outliers - Recommendations

Don't use any outlier rejection test for rejection of chemical test results. But it can be used as supporting information in an OOS investigation to consider retesting.Keep all data, especially suspect data, for future review. Unusual data when seen in context and with other historical data often is not unusual at all, but in fact forms a known and well-behaved statistical distribution.


Reportable Values“Reportable Values for Out of Specification Test Results”

Lynn Torbeck Pharmaceutical Technology Vol. 23, No. 2, February 1999 Special Supplement


FDA R.V. Definition“It should be noted that a test might consist of replicates to arrive at a result. For instance, an HPLC assay result may be determined by averaging the peak responses from a number of consecutive, replicate injections from the same preparation. The assay result would be calculated using the peak response average.”


FDA R.V. Definition

“This determination is considered one test and one result.”


Implications of FDA Definition

A reportable value is the end result of the complete measurement method as documented.It is the value compared to the specifications.It is the value used for official reports.It is usually the value used for statistical analysis.


Figure 1Batch

Sample

Preparation

Figure 1

ReportableValue, RV

Inj


Figure 2Batch

Sample

Inj1

Preparation 3Preparation 2Preparation 1

Inj 2 Inj 3

Figure 2

Inj 7 Inj 8 Inj 9

ReportableValue, RV

Inj 4 Inj 5 Inj 6


Figure 3Batch

Sample Resample

Reinjection

ReportableValue, RV

Inj1

Preparation 1C

Preparation 1B

Preparation 1A

Inj 2 Inj 3 Inj 4 Inj 5 Inj 6

Repreparation2C

Repreparation2B

Inj 7 Inj8 Inj 9

Retest

Remeasure Remeasure

RV RV

Figure 3


InterpretationThe individual determinations do not have to meet the specification.Individual determinations are not reported out of the lab.However the variability of the determinations is a system suitability issue.Set a limit on the standard deviation or %RSD.


R.V.: OOS PreventionRecord in writing the operational definition of the Reportable Value for each test method in the method documentation, any protocols and any reports.Add “Only this reportable value can be compared to the specification criteria.”


AveragingSpecifically, the arithmetic mean; the sum of all of the numbers divided by the count of the numbers. More generally, it is a value that represents the central point of a data set. (In this sense, it can include the arithmetic average, the median, the mode, the geometric mean or the harmonic mean.)


Averaging"... as a general rule, firms should avoid this practice, because averages hide the variability among individual test results.“ "[Averaging] is particularly troubling if testing generates both out‑of‑specification and passing individual results which when averaged are within specification. "Here, relying on the average figure without examining and explaining the individual out‑of‑specification results is highly misleading and unacceptable."


Averaging"Averaging the results of tests intended to measure the uniformity of the test article is not current good manufacturing practice ... because it may hide the variability of the sample the test procedure is intended to detect. For this reason, all individual test results must be reported and evaluated on an independent basis" Paul Vogel, September 10, 1993.


Averaging: OOS Prevention

Do not average out of specification reportable values within specification reportable values to get an in specification result.Do not average reportable values for QA to make a decision. QA must see all individual reportable values, OOS and retests.


Testing Into ComplianceTorbeck, L., “Preventing the Practice of Testing into Compliance”, Pharmaceutical Technology, Oct 2002.Testing into compliance is the practice of ignoring valid information that should be used to make decisions.Such a practice is at best not scientific and at worst is fraudulent, illegal, and immoral.Such practices if found must be stopped.


Testing Into ComplianceAveraging OOS results with in specification results to get an in specification result.Physically averaging powers, granulations and liquids to get in specifications results. If not part of the validated process.

Discarding data or not recording data until is known to be in specification.Missing samples and rejected cans.Overwriting HPLC chromatograms.


Not Testing Into Compliance

Large initial sample sizes are acceptable if all data generated is reported.Large number of retests are acceptable if all data generated is reported.Failing system suitability is not an OOS.Out of limits for an in-process adjustment is not an OOS.


Compliance: OOS Prevention

Train all laboratory personnel, analysts, supervisors and managers to be able to identify specific situations of testing into compliance.Train to be able to defend situations that are not testing into compliance during an audit.


Full Consideration“For inconclusive investigations …. The OOS result should be retained in the record and given full consideration in the batch or lot disposition decision.This statement has caused some discussion as it is considered to be vague and undefined. It can, I think, be defined in a simple way.


Full ConsiderationFirst, all QA decisions are made with the Reportable Values, both OOS and retests.Second, QA looks at the magnitude of the retest values compared to the specifications.


Full ConsiderationIf the retest values are close to the target, the lot can be released.If the retest values are close to the limit that the OOS exceeded, technically the lot can be released, but QA should consider further investigation to determine why the retests are not at target.


Consideration: OOS Prevention

QA should detail and document the logic and rational for decisions based on retesting results after a OOS result is found.


Unresolved IssuesSpecification Limits for OOS?What size the retest sample?Second analyst?Statistical treatment of data?


Specification Limits for OOS?

Regulatory LimitsRelease: accept/rejectAction limits, Cpk=1.33Alert, Cpk=1.0 Warning limits Trend Validation limits


Specification Limits

Cpk=1.0

Cpk=1.33

Accept/Reject

Regulatory


Specification: OOS Prevention

Define in writing the levels of specification criteria.Justify in writing which specifications are considered applicable to OOS and why or why not.


What Size the Retest Sample?

“… a matter of scientific judgment,”“… retesting cannot continue ad infinitum.”“Such a conclusion cannot be based on on 3 of 4 or 5 of 6 passing results, but possibly 7 of 8.”“… will vary on a case by case basis … ““… an inflexible retesting rule … is inappropriate.”



“The number of retests … should be specified in advance …”“The number of tests should not be adjusted ‘on-the-fly’, as results are being generated.”“… a firm’s predetermined testing procedure should contain a point at which testing ends and the product is evaluated.”



This is an unresolved issue and the statisticians are still publishing journal articles and discussing it.Barr case n=7.Could be too much or not enough.Currently n= 3 to n=9.PDA OOS committee will recommend.


Retest ReferencesHofer, J., Considerations when determining routine sample size for a retest procedure, Pharmaceutical Technology, Nov. 2003.Anderson, S., An alternative to the ESD approach, Pharmaceutical Technology, May 2004.


Retest: OOS PreventionDefine in writing the sample size for retests or define the procedure to be used to determine the sample size.Provide scientific justification.


Second AnalystGuidance suggests a second analyst.Issues: Added complication and variation May not have a second analyst May not find the root cause Second analyst may not be as proficientRecommend that the manager decide and justify decision in writing.


Statistical Treatment of Data

“Statistical treatments of data should not be used to invalidate a discrete chemical test result.”“ … a statistical analysis may be valuable as one assessment of the probability of the OOS result …Another way to say outlier rejection.


Preventing OOS Deficiencies:

Setting specification criteriaStatistical ThinkingSources of variationCommon cause vs. special causeVariation reductionTrainingEducation


Setting Specification Criteria:

Two sides to the OOS issue.Incorrect limits are the major source of OOS.Many specifications were set early in the development process and may not be appropriate for the current process.Many specification were set using wishful thinking or incorrect approach.


Setting Specification Criteria:

Use historical dataUse distribution analysis Normal, log-normal, exponentialDon’t use X bar 3SUse Statistical Tolerance Intervals X bar K S for the alert limits where K is based on confidence and

percent of future values


Setting Specification Criteria

For action limits, permit the average to vary and widen the Tolerance Limits For accept/reject limits, add a further allowance for stability.Consider the clinical results as part of the justification for limits.


Statistical Thinking1. All work occurs in a system of

interconnected processes.2. All processes have variability.3. Process understanding and

variability reduction is the key to success.

Variation is the enemy.


Sources of Variation:Common cause variation: People Materials Methods Measurement Machines EnvironmentSpecial cause variation: One single factor changed


Common vs. Special Causes

A plot of the data with natural limits illustrates common cause variation.A value that is larger than would be expected by chance alone is assumed to be due to a special cause. Use CAPA to find it.

0 50 100

96

97

98

99

100

101

102

103

104

105

106

Observation Number

Ind

ivid

ual V

alu

e

I Chart for Yield%

1

Mean=100

UCL=103

LCL=97


Variability Reduction:Display boardsOperational definitionsWork to target, Target ( Low, High )Flexible consistencyHold constantMistake proofingHigh tech equipment


TrainingTraining is for a specific task or SOP.The goal is consistency.Freelancing causes problems.Little background is provided.An in-depth understanding is not needed to be in compliance if the SOP is followed.


EducationSomeone needs to: Learn and understand the basic

philosophy and principles. Know the background as it relates to

the topic. Understand the material well enough to

be able to make difficult decisions with confidence and be able to defend them.


Need for UnderstandingWhy was Able Labs out of compliance?Defend Reportable Values.Defend specifications applicable to OOSDefend not testing into compliance.Defend retest sample size.Why variability reduction is needed.


Final RecommendationsRead and understand the Barr Case.Read and study in-depth the OOS Guidance. Once is not enough.Audit the company SOP against the Guidance line by line.Have an active program to reduce OOS results.Keep management informed.


Thank YouThat ends my presentation.We are now ready for questions and answers.


References1. USA vs. Barr Laboratories, Inc. Civil Action No. 92-1744, US

District Court for the district of New Jersey, February 4, 1993.2. FDA, CDER, “Guidance for Industry, Investigating Out of

Specification (OOS) Test Results for Pharmaceutical Production,” September 1998.

3. WWW.AbleLabs.com4. Torbeck, L., “Reportable Values for Out-of-Specification Test

Results,” Pharmaceutical Technology, February 1999.5. Torbeck, L., “Preventing the Practice of Testing into

compliance,” Pharmaceutical Technology, October 2002.6. Hahn, G and Meeker, W., Statistical Intervals, John Wiley &

Sons, 1991.7. Torbeck, L., “Statistical Thinking,” Pharmaceutical

Technology, July 2001.

http://www.ablelabs.com/

Preventing OOS Deficiencies

Documents

Transcript of Preventing OOS Deficiencies