Operationalising a Performance Objective

with a Microbiological Criterion for a Risk-

Based Approach

prepared by

Jeff Farber and colleagues, Health Canada

modified and presented by

Tom Ross, Tasmanian Institute of Agriculture, University of Tasmania

2

Team Members

Brazil o Andrea Silva

France o Corinne Danan

o Olivier Cerf

India o Aditya Kumar Jain

Canada o Jeff Farber

o Helene Couture

o Anna Lammerding

o Aamir Fazil

o Penelope Kirsch

ICMSF o Jeff /farber

o Marcel Zwietering

o Leon Gorris

o Tom Ross

mailto:Andrea.Oliveira@anvisa.gov.brmailto:bzma.sdssa.dgal@agriculture.gouv.frmailto:ocerf@vet-alfort.frmailto:aditya@nddb.coopmailto:marcel.zwietering@wur.nlmailto:Leon.Gorris@unilever.commailto:Tom.Ross@utas.edu.au

3

the main purpose of risk assessment is to

provide support for risk management

decisions, i.e.,

how to minimise the risk in the most effective

manner

Risk Assessment

4

Principles And Guidelines For The Conduct of

Microbiological Risk Management (MRM)

CAC/GL 63-2007*

3. GENERAL PRINCIPLES FOR MRM

•PRINCIPLE 1: Protection of human health is the

primary objective in MRM

•PRINCIPLE 2: MRM should take into account the

whole food chain

• PRINCIPLE 3:…..

5

are conceptually linked to the FSO and ALOP

therefore

the effect of the steps in the food chain both

before and subsequent to a PO should be

considered in setting its value

Performance Objectives

6

exposure assessment

“in reverse”

evaluate how much growth and inactivation

could occur between production and consumption

reduce FSO appropriately to generate the

Performance Objective

7

The ICMSF Equation

Ho + ΣI - ΣR ≤ FSO

Ho = initial level of the hazard

ΣI = total increase (growth or recontamination)

ΣR = total reduction (inactivation or removal)

FSO = food safety objective

8

Rearranging the ICMSF Equation

PO ≤ FSO - ΣI(distribution/consumer) + ΣR(distribution/consumer)

FSO = H(consumption)

ΣI(disttribution/consumer) = total increase after production

due to growth or recontamination

ΣR(distribution/consumer) = total reduction after production due to

inactivation or removal

9

manufacturin

g

preparatio

n

distribution

and retail,

home storage

Setting Performance Objectives

H1

SR2, SI2

H2 = PO

H2

SI3

H3

primary

productio

n

processing

H1

H3

SR4, SI4

Hconsumed

(PO = H2) ≤ FSO –SI3 –SI4 + SR4

ALOP / no

public

health

burden

Hconsumption ≤FSO

10

manufacturing preparation public health

burden

distribution

and retail,

home storage

Control Options

primary

production processing

H1

ALOP / no

public health

burden

4

Hconsumed ≤FSO

H1

SR2, SI2

H2 = PO

Performance Objectives – Options

minimise H1

minimise SI2

maximise SR2

11

articulate outcomes that should be achieved by a

control measure or a series or a combination of

control measures

i.e., specifies the required ΣR and tolerable ΣI

Performance Criteria

12

primary

production

manufacturing public

health

burden

distribution

and retail,

home storage

Variability in Food Chains

processing

preparation primary

production

primary

production

processing

processing

distribution

and retail,

home storage

distribution

and retail,

home storage

distribution

and retail,

home storage

distribution

and retail,

home storage

preparation

preparation

preparation

preparation

ALOP / no

public health

burden

FSO

13

CCFH drafting group - Example 5A: Base Document: Operationalising a

Performance Objective with a Microbiological Criterion for a Risk-Based

Approach

Purpose

Who should establish a PO and who should apply it

Point in the food chain where the MC is applied

Establishment and implementation of an MC in relation to

a PO

Assumptions/decisions to be made for the establishment of an MC

Sampling plan

Organism(s) of concern

Method(s) of analysis

Interpretation of results

Actions in case of non-compliance

Other practical aspects

Reference documents

14

Purpose

A performance objective (PO) is a risk-based metric that allows government risk managers and food business operators to specify quantitatively, the required stringency of a food safety management system at a particular point in a food supply chain in consideration of the other control measures used in the food safety management system

Performance Objective (PO): The maximum frequency and/or concentration of a hazard in a food at a specified step in the food chain before the time of consumption that provides or contributes to a FSO or ALOP, as applicable

Establishing a microbiological criterion (MC) is one way to see if the PO has been met, i.e., is one way to “operationalise” the PO

Develop case scenarios (following the general approach described in the base document), to illustrate how one can derive an MC from a PO or from an FSO

15

Who should establish a PO and who should apply it?

A PO can be derived from a health target (e.g., ALOP) or a food safety objective (FSO) developed by a competent authority

Can also be established on the basis of a quantitative risk assessment developed for the relevant pathogen in a particular food for/by a competent authority

Food business operators can establish a PO on the basis of either an FSO set by a competent authority, or an evaluation (usually quantitative) of a hazard in the part of the food supply chain for which they are responsible

16

At what point in the food chain can a PO be established ?

A PO can be established at any point in a food supply chain (other than at the point of consumption)

Thus, one can have a PO for raw materials, ingredients, partial and final products within primary production, manufacture, distribution, as well as for products on the market and in foodservice operations

An MC established based on a PO, relates to the corresponding point in the food supply chain and may serve to verify by microbiological analysis whether the PO is met

MRM Metrics Framework

ALOP/FSO

Product

Step D

Step A

Step B

Step C

Agricultural Commodity

PO

MC PC PO

MC PC

Food Consumed

18

Establishment and implementation of an MC in relation to a FSO/PO A. Assumptions/decisions to be made for the establishment of an MC

Firstly, an assumption must be made regarding the distribution of

the pathogens in the lot of food

If no available data, a log-normal distribution is assumed, with a

default value of the standard deviation (SD):

o SD of 0.2 log10 cfu/g for foods with a “homogenous” distribution of microbes

(e.g., liquids with a degree of mixing)

o 0.4 log10 cfu/g for foods with “intermediate homogeneity” (e.g., ground semi-

solids)

o 0.8 log10 cfu/g for foods that are not homogenous (e.g., solid foods)

In certain cases, even greater non-homogeneity could occur, e.g., if

clumping occurs or if the contamination is restricted to surface

contamination of a food

-5 -4 -3 -2 -1 0 +1

Log (CFU/g)

0.0

0.1

0.2

0.3

0.4

0.5

Pro

ba

bil

ity d

en

sit

y

Food product has a log-normal distribution of pathogen “concentrations”

20

Assumptions/decisions to be made for the establishment

of an MC (cont)

The second requirement is to define the ‘‘maximum frequency and/or concentration” of the hazard that will be used to specify the PO

This would include what proportion (e.g., 95%, 99%, 99.9%, etc.) of the distribution of possible concentrations must satisfy the test limit, so that the PO is met

There are two limits: o x% of the distribution in the lot is above a PO

o test limit, m, that is the limit for a sample unit analyzed

21

B. Sampling plan

The sampling plan appropriate to assess an MC depends on the specific situation for which the PO is established.

Therefore, a PO can be set as:

A frequency (prevalence) limit (independent of concentration of the hazard

A concentration limit (independent of frequency), or

A limit for concentration and frequency (e.g., 99% confident that the mean log CFU/g is

22

G. Other Practical Aspects

Random sampling is often not possible and in these cases, the

calculations and interpretations of pathogen testing data may have only

limited value

In simple terms, what this means is that a positive finding (i.e.,

presence of a pathogen means something, while a negative one means

very little

Even when the necessary data are available to allow statistical

interpretation of the test results, the number of samples needed to

obtain a meaningful result may be too large to be practical

23

Summary - FSO/PO/MC

FSO is means of relating stringency of the entire farm-to-table system to public health outcomes

PO is the primary means for establishing the level of control needed at a specified step in the food chain

MC is a means of verifying that a PO is being achieved

FSO

Food Consumed

Product

PO

MC

24

Three Case Studies

Example 1: Deriving an MC from a PO that is set as a

numerical limit to the concentration of a pathogen

Example 2: Deriving an MC from a PO that is set as

the limit to the prevalence or proportion of a

microorganism

Example 3: Deriving an MC from an FSO for a

product supporting growth of the target pathogen

between PO and FSO

25

Example 1: Deriving an MC from a PO that is set as an actual limit to the number of a microorganism

PO established at a specific point in the value chain (i.e., pathogen level of ≤ 4 log cfu/g for 99.5% of the products in the batch)

Establish/decide on the concentration distribution in batch/lot and the standard deviation of the pathogen

concentration (i.e., lognormal; s.d. 0.8 log cfu/g)

Calculate a mean log concentration of the pathogen such that this batch/lot just complies

with the PO (i.e., 1.94 log cfu/g)

Decide on the microbiological limit m for the sampling plan of the MC (i.e., m = 2 log cfu/g)

From this follows how many samples would be needed to achieve the selected probability of rejection

(i.e., n = 5)

Decide on the probability with which a non-compliant batch/lot should be rejected

(i.e., > 95% confidence)

Calculate what the probability is for ‘n’ samples to be negative for a just compliant batch/lot

(i.e., n = 1, 53% up to n= 8, 0.6%)

1 2

3

5

4

6 MC: suitable sampling plan parameters m and n (i.e., m = 2 log cfu/g and n = 5)

(Figure 1.2)

Example 1: Deriving an MC from a PO that

is set as an actual limit to the number of a microorganism

26

The role of the PO for lot acceptability

and the m for sample acceptability

27

Regarding the choice of m, the following m and n values would give

alternative designs of the sampling plan that can detect/reject non-

compliant lots with the same confidence:

m n

(cfu/g) (log cfu/g)

4 0.60 1

19 1.28 2

100 2 5

500 2.7 16

1000 3 31

1738 3.24 57

The m values of 0.60 or 1.28 log cfu/g would be constrained by the method

for microbiological enumeration (e.g., by sensitivity, accuracy, standard deviation),

while m values of, e.g., 2.7 log cfu/g and higher, would require a very large number

of samples to be analyzed.

28

Example 2: Deriving an MC from a PO set as the limit to the prevalence of a microorganism

From this follows the number of samples that would need to be taken to achieve the selected

probability of rejection (i.e., n = 29)

Decide on the probability with which a non-compliant batch/lot should be rejected

(i.e., >95% confidence)

Calculate what the probability is for ‘n’ samples to be negative given the PO (i.e., n = 1, 90% up to n=30, 4.2%)

1

2

3

MC: suitable sampling plan parameter n (i.e., n = 29; testing for prevalence)

PO established at a specific point in the value chain (i.e., ≤10% of carcasses in a batch/lot are tolerated to be positive for the target

pathogen using enrichment and testing 10-g neck skin samples after chilling)

29

Example 3: Deriving an MC from a FSO for a

product supporting pathogen growth between PO and FSO

Establish/Decide on the concentration distribution in the batch/lot and the standard deviation of the pathogen

concentration at the point of the FSO (i.e., lognormal; s.d. 1.112 log cfu/g)

Decide on the microbiological limit m for the sampling plan of the MC (i.e., m = 0, in 25g samples)

From this, follows how many samples would be needed to achieve the selected probability of rejection (i.e., n = 15)

Decide on the probability with which a non-compliant batch/lot should be rejected(i.e., >95% confidence)

1 2

3

5 4

7 MC: suitable sampling plan parameter n

(i.e., n = 15; testing for presence/absence)

FSO established at the point of consumption (i.e., ≤0.2% of products have a pathogen concentration >100 cfu/g)

Calculate a mean log concentration so that the distribution with this mean log concentration and standard deviation

complies with the FSO(i.e., -1.2 log cfu/g )

6

Calculate what the probability is for ‘n’ samples to be negative for a just compliant batch/lot (i.e., n = 1, 18.5% up to n=15, 95.4%)

Derive a suitable mean log concentration and standard deviation of the distribution at the PO from the mean log concentration and standard deviation at consumption complying to the FSO (i.e., -2.5 log cfu/g; s.d. 0.8 log cfu/g)