Importance of Proficiency Testing/Interlaboratory

Comparison

Dr. Erik J.M. Konings, Nestlé Research Centre, Lausanne, Switzerland

“International Workshop and Training Program on Good Food Laboratory Practices” to be held on November 15-19, 2016 in Mumbai.

Organized By

Food Safety and Standard Authority of India, Government of India

&

International Life Sciences Institute-India

&

Export Inspection Council, Government of India

Agenda

1 Introduction

4 Nestlé Proficiency Testing

5 AOAC INTERNATIONAL Proficiency Testing (to start as from March 2017)

2 General requirements for Analytical Quality Assurance

3 Introduction to Proficiency Testing (process, statistical design, benefits)

Poor Quality data can result in substantial problems and high costs

Consequences of reporting data of poor quality for the end-users?

• High costs involved in repeated measurements to correct poor data;

• Faulty decision-making;

• Damage to reputation and credibility;

• Possible loss of business;

• Legal and financial liability from the use of poor data.

The various tools helping laboratories to demonstrate the reliability of their analytical data

ISO/IEC 17025: General requirements for the competence of testing and calibration

laboratories

• Chapter 5.9: Ensuring the quality of test and calibration results

What is Proficiency Testing?

ISO/IEC 17043:2010 General requirements for Proficiency Testing

“Proficiency testing schemes (PTS) are interlaboratory comparisons that are organized regularly to assess the performance of analytical laboratories and the competence of the analytical personnel.”

Proficiency testing – The Process

Prepare samples

Homogeneity and Stability testing

Ship samples

Analyse

samples

Report results

Analyse results

Prepare report

Conduct Root cause

analysis

P-test Provider Testing laboratory

How often does a lab has to participate?

Proficiency testing – The Statistical designHow is the assigned value established?

The assigned value is the value selected as being the best estimate of the “true value” for the parameter under test. The method used to determine the assigned value may vary depending upon the particular scheme and test material.

As per ISO 13528 “Statistical methods for use in proficiency testing by interlaboratory comparisons”, there are five ways to determine the assigned value : • Formulation• Certified references values• Results from one laboratory• Consensus from experts laboratories• Consensus value from participants (e.g. median)

For Qualitative tests, participant’s results are compared against the intended result based on formulation.

For Microbiology samples, participant’s results are transformed by converting them to log10 before the statistical analyses is undertaken.

Proficiency testing – The Statistical design How is the standard deviation for proficiency assessment

(SDPA) determined?

The SDPA expressed the acceptable difference between the laboratory results and the assigned value. The method used to determine the assigned value may vary depending upon the particular scheme and test material.

As per ISO 13528 “Statistical methods for use in proficiency testing by interlaboratory comparisons” there are five ways to determine the SDPA: • Prescribed value• By perception• From a general model• From results of precision experiment• From data obtained on a round of proficiency testing scheme

Participant’s result converted in Z-score.How is the z-score calculated?

z-score is a scaling that makes the errors of equal magnitude in different situations, (normalized different errors in different methods) allowing to compare the performance of the laboratories observed in different tests and with different analytes.

As per ISO 13528, it is the ratio between the deviation of each individual result from the assigned value, and the standard deviation for proficiency assessment SDPA.

How do I interpret the z-score?

What to do with unsatisfactory results?

ISO/IEC 17025 Management Requirements – 4.9 Manage non conformities

If anything in the lab is not conform to what it shall be, a procedure must be activated to:

• Take immediate actions (repeat analyses, contact customer if necessary, remove an instrument out of order)

• Evaluate the cause and take corrective actions (train people, replace out of spec instrument, etc)

• Evaluate consequence of the work done

• Verify that the corrective actions prevent similar problems

Example of FAPAS report

If the following criteria is not met:u(xpt) < 0,3 *σpt

(standard uncertainty of the assigned value lesserthan 0.3 times the Standard Deviation of ProficiencyAssessment)

The uncertainty is not considered asnegligible and needs to be included inthe interpretation of the results. Inthis case other statistical designshould be considered (e.g. use of z’-score).

Example of FAPAS report

There are multiple benefits taking part to Proficiency Testing

Demonstrate good performance of the laboratories

Detecting method related issued

Raising trust in supplier’s certificates

Comparing the various analytical methods

Generating reference materials for methods monitoring

Example of Method Related Issues detected in P-test

Example of method performance comparison

Source: LGC report

Nestlé Proficiency Testing

• Is an accredited ISO/IEC 17043 Proficiency Testing Provider, by the Swiss Accreditation Service.

• Comprehensive testing programs offering a wide range of analytes and matrices

• Work with external P-test providers to offer the most suitable P-test programme adapted to Nestléproduct specifications (analyte, matrix, concentration)

• Participation to the Nestlé Proficiency Tests programme is mandatory for the Nestlé laboratories

• For parameters and/or matrices not covered by the Nestlé P-Test program, each laboratory must alsoparticipate in Proficiency test schemes provided by external organizations like FAPAS, LGC, Bipea etc.accredited as a P-test provider based on ISO/IEC 17043

Who are the Nestlé users of Proficiency Testing ?

900 laboratories• Internal laboratories • External laboratories (e.g. private,

governmental, supplier)

Worldwide located (90 countries)

Using around 600 test methods and 120 official methods

Delivering 128 million of analytical results per year

Nestlé P-test Program - key figures

41’000 analytical results /year

CHEMICAL NUTRIENTS MICROBIOLOGY(organised in collaboration with LGC)

CHEMICAL CONTAMINANTS(organised in collaboration with Fapas, LGC, Bipea)

Probiotics Pathogens Hygiene indicators

Allergens Antibiotics GMO Melamine& cyanuric acid Metallic contaminants Mycotoxins Nitrite/nitrate Pesticides Authenticity

Fats Sugars Vitamins Minerals Proximates Fatty acids

3 domains

244 analytical determinations

42 food products

Milk powder Lyophilized

Oil Spicy Fruit Purée Milk powder Cereals Meat

Infant formula Cereals Pet food Health care product Confectionery Sweetened/Unsweetened

condensed milk Bouillon Water Beverages

How do we treat non-satisfactory P-test results at Nestlé ?

• A root cause analysis (RCA) must be conducted after every non-satisfactory results i.e. for each unsatisfactoryand questionable result.

• Remedial action must be taken to solve the problem

• The investigation must be carried out immediately after each scheme round report has been published.

• Repeat samples are sent to the laboratory that failed to verify effectiveness of the corrective actions

• Two consecutive unsatisfactory results must trigger a specific action

• Key learnings and P-test best practices are shared with laboratories to prevent gaps reoccurrence

Example of key learning

The challenges from the Nestlé P-Test provider perspective

• Specific/exotic P-test demands

• Less than 8 results engaged, limitation of the statistical design

• Difficulties to meet criteria of homogeneity and stability check

Barrier to start new P-Tests

• Increasing number of documentation requested by the customs

Ensure samples delivery in every countries

• Testing multiple times and reporting an average

• Selection of senior technicians, not involved in everyday routine analysis, to perform the analysis of P-Test samples

Laboratory gives special treatment to P-Test sample

AOAC Proficiency Testing

• AOAC INTERNATIONAL is an A2LA accredited Proficiency Testing Provider.

• First accredited program for analytical food laboratories in the United States, in 2001.

• Comprehensive testing programs offering a wide array of analytes and matrices

• AOAC’s proficiency testing samples arrive at the laboratory ready to analyze like routine samples. The samples undergo rigorous homogeneity testing prior to shipment

Methods endorsed by CODEX

Vitamin A ISO 20633:2015 AOAC 2012.10

Total nucleotides ISO 20638:2015 AOAC 2011.20

Pantothenic acid ISO 20639:2015 AOAC 2012.16

Iodine ISO 20647 | IDF 234:2015 AOAC 2012.15

Cr, Mo, Se ISO 20649 | IDF 235:2015 AOAC 2011.19

Implementation of Infant Formula and AdultNutritionals PT Program

• As of March 2017

• Nutrients:• Fatty acids

• Vitamin E

• Vitamin B12

• Panthothenic acid

• Iodine

• Vitamin A

• Myo-Inositol

• Nucleotides

• Ultra Trace Minerals (Cr, Mo, Se)

• Quarterly • $650 per Shipment• 30 days for analysis

Example of Method Performance Information

Why is participating to Proficiency Testing/Interlaboratory comparison important?

• Proficiency test is a key element of quality assurance to assess the proficiency of laboratories

• Proficiency tests are pivotal to help demonstrate the reliability of laboratory results. P-test helps to identify problems in laboratories and initiate actions for improvement

• P-test plays a key role as it maintains confidence in laboratories capability to perform analysis, irrespectfully whether or not accredited to ISO 17025 (also interesting for third parties, e.g. authorities, customers)

Acknowledgement

Thanks to Laura Coisne, Nestlé P-test Manager, and the Nestlé P-test team for providing the information!

CONTACT:Erik.Konings@rdls.nestle.com