General enquiries on this form should be made...

General enquiries on this form should be made to:Defra, Science Directorate, Management Support and Finance Team,Telephone No. 020 7238 1612E-mail: [email protected]

SID 5 Research Project Final Report

SID 5 (Rev. 3/06) of 29

NoteIn line with the Freedom of Information Act 2000, Defra aims to place the results of its completed research projects in the public domain wherever possible. The SID 5 (Research Project Final Report) is designed to capture the information on the results and outputs of Defra-funded research in a format that is easily publishable through the Defra website. A SID 5 must be completed for all projects.

This form is in Word format and the boxes may be expanded or reduced, as appropriate.

ACCESS TO INFORMATIONThe information collected on this form will be stored electronically and may be sent to any part of Defra, or to individual researchers or organisations outside Defra for the purposes of reviewing the project. Defra may also disclose the information to any outside organisation acting as an agent authorised by Defra to process final research reports on its behalf. Defra intends to publish this form on its website, unless there are strong reasons not to, which fully comply with exemptions under the Environmental Information Regulations or the Freedom of Information Act 2000.Defra may be required to release information, including personal data and commercial information, on request under the Environmental Information Regulations or the Freedom of Information Act 2000. However, Defra will not permit any unwarranted breach of confidentiality or act in contravention of its obligations under the Data Protection Act 1998. Defra or its appointed agents may use the name, address or other details on your form to contact you in connection with occasional customer research aimed at improving the processes through which Defra works with its contractors.

Project identification

1. Defra Project code PS2303

2. Project title

Addressing uncertainty and variability in pesticide risk assessments for birds and mammals (WEBFRAM-3)

3. Contractororganisation(s)

Central Science LaboratorySand HuttonYorkYO41 1LZUK

54. Total Defra project costs £ 303774(agreed fixed price)

5. Project: start date................ 01 January 2003

end date................. 31 December 2006

SID 5 (Rev. 3/06) of 29

6. It is Defra’s intention to publish this form. Please confirm your agreement to do so...................................................................................YES NO (a) When preparing SID 5s contractors should bear in mind that Defra intends that they be made public. They

should be written in a clear and concise manner and represent a full account of the research project which someone not closely associated with the project can follow.Defra recognises that in a small minority of cases there may be information, such as intellectual property or commercially confidential data, used in or generated by the research project, which should not be disclosed. In these cases, such information should be detailed in a separate annex (not to be published) so that the SID 5 can be placed in the public domain. Where it is impossible to complete the Final Report without including references to any sensitive or confidential data, the information should be included and section (b) completed. NB: only in exceptional circumstances will Defra expect contractors to give a "No" answer.In all cases, reasons for withholding information must be fully in line with exemptions under the Environmental Information Regulations or the Freedom of Information Act 2000.

(b) If you have answered NO, please explain why the Final report should not be released into public domain

Executive Summary7. The executive summary must not exceed 2 sides in total of A4 and should be understandable to the

intelligent non-scientist. It should cover the main objectives, methods and findings of the research, together with any other significant events and options for new work.

This project developed new and improved approaches for modelling risks to wild birds and mammals from the use of pesticides, which take account of variability and uncertainty in factors that influence risk. The models produced by the project are freely available for use on the internet at www.webfram.com.

Before a pesticide is approved for use, the risks to non-target organisms including wild birds and mammals are assessed. These risks depend on two main factors: the toxicity of the pesticide to birds and mammals, and the amounts of pesticide they are exposed to. Most current methods for assessing pesticide risks are deterministic – they use a single value for toxicity, a single value for exposure, and produce a single estimate of risk. In the real world, toxicity and exposure are not fixed; instead they vary both between species and between individuals. Furthermore, many aspects of risk assessment involve uncertainty – for example, when extrapolating toxicity from species tested in the laboratory to those exposed in the wild. Consequently, the effects of pesticides are both variable and uncertain. Deterministic methods cannot represent this variability and uncertainty. Instead, they provide a snapshot of one possible outcome. This may be sufficient for simple screening assessments, but for refined assessments a more complete picture of the possible outcomes and their uncertainty would be desirable.

Probabilistic methods take account of variability and uncertainty using probability distributions. Probability distributions show the range and relative likelihood of possible values for each input and output, and thus provide a fuller picture of the possible outcomes. This is a major advantage compared to deterministic methods, which use only single values for inputs and outputs. However, uptake of probabilistic methods has been hindered by a number of factors including lack of reliable data for input distributions, lack of standardised methods for probabilistic calculations, the complexity of the methods, and concerns about the validity of assumptions.

This project aimed to help overcome these obstacles by developing a set of probabilistic models for pesticide risks to birds and mammals, using the best available data, examining the validity of the assumptions, and evaluating the outputs by comparison with existing data from field studies. The models developed in this project were implemented as web-based, user-friendly software by a sister project The models cover five different scenarios, representing some of the crops and seasons where pesticides are most frequently used. For each scenario, the species likely to experience the highest exposures to pesticide were identified. The models estimate risk for these species and scenarios: skylark/cereals/summer, skylark/cereals/winter, yellowhammer/cereals/summer, blue tit/orchards/summer,

SID 5 (Rev. 3/06) of 29

http://www.webfram.com/

and wood mouse/cereals/summer.

Each model can be run for any sprayed pesticide that is used in the scenario it represents. Each model considers the sub-population of animals that visit fields treated with the pesticide and estimates their exposure for a single day (acute exposure), when residues of the pesticide are at their highest levels. The models estimate exposure for each species using the best available data on the diet and feeding behaviour of that species in UK conditions. They consider exposure only through the diet, as is normal in current EU regulatory assessments.

The models also estimate the dose-response relationship for the species of interest, i.e. how much exposure is required to cause mortality. This is then combined with the estimated exposures to produce two alternative measures of risk: a distribution of toxicity-exposure ratios (TER, as used in current deterministic assessments) and an estimate of percentage mortality for animals visiting the treated crop on the day of pesticide application.

The models quantify some important types of uncertainty: principally, the “sampling” uncertainty that is caused by having only limited amounts of data for each model input, and uncertainty about the difference between the dose-response relationship for species tested in the laboratory and those exposed in the wild. The effect of these uncertainties is shown by distributions or confidence intervals for the model outputs, showing how different the “true” TER or mortality could be. It was not possible in this project to quantify other types of uncertainty, so their effects are considered in a qualitative way.

This report provides an overview of the approaches and results of the project, including the modelling methods and data used, the nature of outputs produced and their interpretation, a comparison of model outputs with data from field studies, and a discussion of how the results could contribute to regulatory decision-making. More details including information on the data and input distributions used in the model are available online, as help screens provided with the software at www.webfram.com.

Project Report to Defra8. As a guide this report should be no longer than 20 sides of A4. This report is to provide Defra with

details of the outputs of the research project for internal purposes; to meet the terms of the contract; and to allow Defra to publish details of the outputs to meet Environmental Information Regulation or Freedom of Information obligations. This short report to Defra does not preclude contractors from also seeking to publish a full, formal scientific report/paper in an appropriate scientific or other journal/publication. Indeed, Defra actively encourages such publications as part of the contract terms. The report to Defra should include: the scientific objectives as set out in the contract; the extent to which the objectives set out in the contract have been met; details of methods used and the results obtained, including statistical analysis (if appropriate); a discussion of the results and their reliability; the main implications of the findings; possible future work; and any action resulting from the research (e.g. IP, Knowledge Transfer).

SID 5 (Rev. 3/06) of 29


ADDRESSING UNCERTAINTY AND VARIABILITY IN PESTICIDE RISK ASSESSMENTS FOR BIRDS AND MAMMALS

Andy Hart1, Willem Roelofs1, Joe Crocker1, Pierre Mineau2

1Central Science Laboratory, York, YO41 1LZ, UK2National Wildlife Research Centre, Ottawa, K1A 0H3, CANADA,

Introduction

EU and UK regulations require that, before a pesticide is approved for use, the risk to non-target organisms including birds and mammals has to be assessed. EU Directive 91/414/EEC specifies criteria to be used in an initial “first-tier assessment”, and states that pesticides which fail to meet these criteria may not be authorised for use unless an “appropriate risk assessment” shows that it will cause no unacceptable impact. Various options for these refined, higher tier risk assessments are identified in existing EU Guidance Documents, including both deterministic and probabilistic approaches.

Most current methods for assessing pesticide risks are deterministic – they use fixed values for toxicity and exposure, and produce a single measure of risk (e.g. a toxicity-exposure ratio). In the real world, toxicity and exposure are not fixed, but variable. Furthermore, many aspects of risk assessment involve uncertainty – for example, when extrapolating toxicity from test species to humans or wildlife. Consequently, the effects of pesticides are both variable and uncertain. Deterministic methods cannot incorporate variability and uncertainty directly. Instead, uncertain or variable factors are fixed to worst-case values, or dealt with subjectively using expert judgement, or simply ignored.

Probabilistic methods take account of variability and uncertainty by using probability distributions to represent them in risk assessment. Although this is a major advantage, uptake of probabilistic methods has been hindered by a number of factors including lack of reliable data for input distributions, lack of standardised methods for probabilistic calculations, the complexity of the methods, and concerns about the validity of assumptions.

This project sought to address these obstacles by developing probabilistic risk assessment models for a set of ecological scenarios commonly encountered in higher tier assessments for birds and mammals, using the best available data, examining the validity of the assumptions, and evaluating the outputs by comparison with existing data from field studies.

The models developed in this project were implemented as web-enabled software by a sister project (WEBFRAM 1, Defra project number PS2301), making them freely accessible for use on the internet at www.webfram.com.

This report provides an overview of the approaches and results of the project, including the modelling methods and data used, the nature of outputs produced and their interpretation, and a comparison of model outputs with data from field studies. More details including information on the data and input distributions used in the model are available online as help screens provided with the software at www.webfram.com.

Model outputs shown in this report are based on a hypothetical example pesticide, applied to cereals as a single application in summer at 1000 g a.s. ha-1, and were generated by the WEBFRAM model for the scenario “skylark, cereals, summer”. The software at www.webfram.com allows users to obtain analogous results for real pesticides by entering the relevant data and selecting an appropriate scenario. Because all of the ecological and biological data required for each scenario are pre-loaded, the minimum data required from the user are the application rate of the pesticide, the maximum number of applications, the minimum interval between applications, and the LD50 for a single species. In addition, the slope of the dose-response is required if the user wishes to obtain outputs expressed as % mortality.

Scientific objectives

The scientific objectives agreed for the project are listed below, together with a description of the extent to which they have been achieved.

1. Develop statistical/mathematical models using existing data and expert knowledge to take account of uncertainty and variability in the assessment of pesticide risks to birds and mammals.

SID 5 (Rev. 3/06) of 29




This objective was fully achieved for five specific scenarios involving acute risks to birds and mammals, chosen in consultation with the Pesticides Safety Directorate (PSD). These represent five of the scenarios that are most commonly encountered in regulatory assessment. Due to the challenges encountered in modelling acute risks, insufficient time and resource remained to develop models for long-term risks. Most elements of the acute models would also be relevant to long-term risks, but with substantial additions to model exposure and effects over time.

2. Test the operation of the models after they have been web-enabled.

This objective was fully achieved. Models were extensively tested by the project team, independent reviewers, PSD specialists, and other potential users including the pesticide industry. Feedback from testing was used to improve the models and also the performance and user-friendliness of the software.

3. Develop improved methods for quantifying uncertainty and variability in both acute and long-term toxicity.

This objective was partially achieved. Extensive research was done on methods for probabilistic modelling of acute toxicity, which is summarised below and incorporated in the final models produced by the project. However, further refinements could be achieved with further research (see later). No models were developed for long-term toxicity.

4. Evaluate the performance of the models for acute risk to birds, by comparing their outputs with existing data on impacts in field studies.

This objective was fully achieved (see results below). General approach

Model development in this project followed the general approach developed in the sister project WEBFRAM 1. This is described in more detail in the report of that project but may be summarised by the following steps:

1. Define the assessment scenario to address the information needs and protection goals of decision-makers. 2. Define the assessment endpoint, i.e. the measure of risk required. 3. Identify the key factors and mechanisms that influence the risk. 4. Decide which factors and mechanisms might contribute significantly to variability in the assessment endpoint.

These will be represented by distributions in the assessment model. 5. For each input variable, identify the appropriate statistical population. 6. Identify what data are available for each input. 7. Decide whether any extrapolations or adjustments are needed. 8. Identify possible sources of uncertainty, and decide which of these will be quantified. 9. Identify potential dependencies between variables. 10. Express the entire model as a set of mathematical equations or algorithms. 11. Select appropriate computational or graphical methods for calculations and outputs. 12. Specify distributions for the input variables. 13. Consider conducting sensitivity analyses. 14. Use appropriate software to carry out the computations and generate outputs15. Test the model and implement any necessary corrections or improvements.

Assessment scenarios

The EU guidance document (EC, 2002) states that higher tier assessments which refine the assumptions for dietary intake and foraging behaviour (PD and PT, see below) will usually focus on one or two key species that are considered to be of concern.

The WEBFRAM-3 project developed models for 5 different assessment scenarios, which were selected in consultation with PSD. All scenarios are focusing on the effects on bird and mammal species following acute (1 day) exposure to pesticides and can be identified by 3 characteristics: species, crop and season. The 5 scenarios modelled in the project are:

1. Skylark, Cereals, Summer2. Skylark, Cereals, Winter3. Yellowhammer, Cereals, Summer4. Bluetit, Orchards, Summer5. Wood mouse, Cereals, Summer

All scenarios are designed to reflect UK conditions, using UK data on bird diets and feeding behaviour.

SID 5 (Rev. 3/06) of 29

Each species is chosen to be a reasonable worst-case choice for the crop and season, in the sense that other species present in the scenario are expected to receive lower exposures based on the type and amount of food eaten and the proportion of this which they obtain in the crop. For example, the skylark (Alauda arvensis) is an omnivorous bird that feeds its chicks on insects, and nests and forages in cereal fields. It is a relatively small bird (and so has a high food intake to body mass ratio), and is likely to ingest pesticide-contaminated forage and insects (which have potentially high residue levels) as part of its diet. It is therefore likely to experience higher exposures than most other species of birds in cereals systems. Therefore, a risk assessment based on the skylark is expected to be protective of other species using the same crop.

More detail on all the scenarios may be found on the website www.webfram.com.

Assessment endpoints (measures of risk)

The EU Guidance Document on Risk Assessment for Mammals and Birds (EC, 2002) states “ecological risk assessors have long argued that except in the case of threatened or endangered species, the abundance and persistence of populations of organisms are more relevant as endpoints for assessment than are responses of individual organisms”. However, it goes on to state: “That does not preclude, that appreciable mortality without population level consequences may be judged unacceptable”.

It can be concluded that for birds and mammals, the protection goal includes the avoidance of both “appreciable mortality” and of effects at population level. The current regulatory approach addresses this goal by using toxicity-exposure ratios (TERs) to assess separately the risks of direct acute and chronic effects. However, given the specific interest of decision-makers in “appreciable mortality”, we also assess risk in terms of the level of mortality.

Therefore, the TER and percentage mortality are taken as alternative endpoints for the probabilistic assessment, with the aim of providing a more informative basis for decision-making.

Scope of models

For both TER and % mortality, the population considered comprises those individuals of a species that visit crop fields treated with the pesticide of concern. This includes animals that visit the fields for varying proportions of their time. It would be possible to enlarge the assessment to include a larger population e.g. all individuals of a species in arable land. That could be considered as a possible future refinement (with additional model inputs), if required.

Consequently, the first assessment endpoint is a distribution showing how the TER varies between individuals, while the second assessment endpoint is a point estimate for the percentage mortality in the exposed population of the species of concern. The models do not attempt to quantify population consequences of individual mortality.

Because the objectives of the assessment include estimating the level of mortality, it is necessary to model the exposure of individuals, since it is individuals which die or survive. The spatial scale for this assessment is therefore the foraging area of individual birds, which may contain a range of different crops and other habitats, rather than a single treated field. This is clearly more realistic than the standard first tier assessment.

The model considers exposure only through the diet, as is normal in current UK and EU regulatory assessments. The potential contribution of non-dietary routes of exposure should be considered when evaluating uncertainties affecting the assessment outcome (see later).

The models do not consider scale or timing of pesticide use: it is assumed that for each scenario all fields in each animal’s foraging range are treated with the pesticide on the same day. However, because the diet of the target species will vary throughout the year, the user may be asked to specify on what day the fields were treated.

Conceptual models

The conceptual models used for the 2 assessment endpoints are illustrated in Figures 1 and 2. The key difference is in the toxicity assessment: the slope of the dose-response is required for estimating % mortality, but not for the TER.

The model inputs and structure are the same as the assessment approach established in the Guidance Document for birds and mammals, except that avoidance is not considered because it is now considered inappropriate to model it as a multiplicative factor reducing exposure (EFSA 2004). Metabolism of the pesticide is

SID 5 (Rev. 3/06) of 29


also not considered, as in the standard acute risk assessment. Therefore the potential influence of avoidance and metabolism should be considered when evaluating uncertainties affecting the assessment outcome (see later).

FIR: Food intake per day (wet wt)

C: Concentration on food type

PD: Fraction of food type in diet

PT: Fraction of diet obtained in treated areas

AV: Avoidance -omitted

Exposure(mg/kg bw/day)

C: Initial concentration LD50

(mg/kg bw)

TER: Toxicity-exposure ratio

DEE: daily energy expenditure

BW: body weight

GE: gross energy content of food

M: moisture content of food

AE: assimilation efficiency

AppRate: application rate

RUD: residue per unit dose








(mg/kg bw)



BW: body weight













(mg/kg bw)



BW: body weight






Figure 1. Conceptual model for TER. Avoidance, metabolism and non-dietary routes of exposure are omitted (see text). All inputs are represented by distributions in the model except application rate (shaded).







C: Initial concentration

LD50(mg/kg bw)

%Mortality


BW: body weight






Slope ofdose- response








LD50(mg/kg bw)

%Mortality


BW: body weight














LD50(mg/kg bw)

%Mortality


BW: body weight







Figure 2. Conceptual model for % mortality. Avoidance, metabolism and non-dietary routes of exposure are omitted (see text). All inputs are represented by distributions in the model except application rate (shaded). One additional input is used (slope of dose response) that was not required for the TER.

Probabilistic method and software

Because it is desired to quantify variation in exposure and risk, and to provide confidence intervals representing sampling uncertainty, the assessment requires a probabilistic method that can separate variability and uncertainty. This was achieved using a 2D Monte Carlo procedure with random sampling and 1000 iterations in the inner loop (representing variability between individual skylarks) and 1000 in the outer loop (representing quantified uncertainty). The calculations were conducted in Matlab 6.5 with Statistics Toolbox® and were subsequently compiled for integration with the web interface (see later).

SID 5 (Rev. 3/06) of 29

Exposure assessment

Exposure model description

In first tier assessments, acute avian exposure is estimated using the following equation (European Commission 2002, p. 9):

where:ETE = estimated theoretical exposure (mg/kg bw/day)FIR = food intake rate (g fresh wt/day)BW = Body Weight (g)C = Concentration (mg ai/kg wt) of pesticide residue on foodAV = Avoidance (set to 1, see above)PD = Proportion (fresh wt) of Diet made up by food type (0-1)PT = Proportion of diet obtained from treated area (0-1)

Because for most species FIR is not known, the amount of food consumed is estimated using a different approach. This approach is based on the daily energy requirement (DEE) of the species, which is estimated from a regression relationship with body weight (an exception to this was skylark, for which direct measurements of DEE were available). DEE is combined with data on the gross energy (GE) contained in food items and the assimilation efficiency (AE), to estimate the amount of food required to fulfil the daily energy requirement. Because GE is expressed in terms of dry weight, the moisture contents (M) of food items must also be taken into account. Where the diet comprises only a single food:

In the first tier assessment, PD is set to 1 (single food). If multiple foods are considered in the refined assessment, then PD is set to values less than 1 and it is necessary to modify the calculations so that the total energy derived from the different foods eaten still sum to the daily energy requirement (European Commission 2002b, p. 32). For example, if the diet comprises multiple foods (e.g. arthropods, seeds and leaves), the equation for ETE would be expanded as follows:

Where: i = subscript for different food types (small insects, cereal seeds, weed seeds, broad leaf foliage, grass/cereal foliage) DEE = Daily Energy Expenditure (kJ) = 2.41*BW + 20.2 (see later)Ci = Concentration (mg ai/kg wt) of pesticide residue on food type iPD = Proportion (fresh wt) of Diet made up by food type i (0-1)Mi = Proportion Moisture in fresh food of type i (0-1)GEi = Gross Energy (kJ/g dry wt) provided by food type iAEi = Assimilation Efficiency of food type i (0-1)PT = Proportion of food eaten that is from the treated area (0-1)AV = Avoidance of treated food (set to 1, see above)

Pesticide residues (Ci) are estimated using the same method as was used to derive the default estimates used in the standard first tier assessment, using residue per unit dose factors derived from historical data from a range of crops and pesticides (for an overview of this approach see Appendix II of the guidance document, European Commission 2002). Therefore, residues Ci for a given food type i are estimated using the following equation:

whereAppRate = application rate (kg ai/ha)RUDi = residue per unit dose on food type i (mg ai/kg per kg ai/ha)

SID 5 (Rev. 3/06) of 29

Only peak exposure on the day(s) of application is considered, so degradation of residues is only relevant in case of multiple applications (apart from the possibility of partial decay within 24 hours). For those pesticides, the concentrations on food type i is calculated using:

where= application rate (kg ai/ha) on food type i after application n+1 (mg ai/kg). For the first application

= Ci.Ci = residue per unit dose on food type i directly after application (mg ai/kg per kg ai/ha)

= Concentration on food type i after application n (mg ai/kg). For the first application = 0.t = time between application n and application n+1 (d)DT50 = half life ( = ln(2)/k) where k is the first order degradation rate (d-1)

Specification of model inputs

An summary of the inputs used for the exposure model is provided in Table 1, using the skylarks/cereals/summer scenario as an example. Distributions were used to represent variability in all parameters of the exposure model except for AV, which was fixed to 1 as explained above.

For each variable in the exposure model, relevant data were collated from the literature and carefully evaluated for their suitability for the assessment scenario. In some cases, the statistical population of the available data (e.g. a distribution of values from different studies) did not match the population properly required for the exposure model (distribution for variability between individuals of the species under consideration). In those cases we decided to run multiple versions of the model with alternative assumptions so as to show the effect of the uncertainty involved (see below).

The form of each input distribution was evaluated by consideration of the intrinsic nature of each variable (e.g. limit values), by visual inspection of the data, and by considering goodness-of-fit statistics and plots. Adjustments and assumptions made to derive appropriate distributions are described and justified in the detailed documentation on the exposure pages of the website.

For variables where the available data showed seasonal variation, the values used in the model are those appropriate to the application date(s) entered by the user.

Table 1. Summary of inputs for exposure assessment. For details of the parameter values and justification of distribution choice, see Appendix. Quantified uncertainties are identified in the Table (for details of methods see Appendix). Other extrapolations and uncertainties affecting the parameters are evaluated qualitatively in Table 2. n=sample size (range for different sexes or food types).Input Data n Source of data Distribution Quantified

uncertaintiesUnquantified extrapolation and uncertainties

DEE – Daily Energy Expenditure (kJ/g)

Energy expenditure of skylarks as measured by Doubly-Labelled Water turnover

12 Tieleman et al (2004)

Linear regression on Bodyweight

Slope, intercept & error term in regression (mixture of uncertainty & variability)

DEE data were for smaller bodyweights than simulated population, and were estimated assuming a diet of 90% insects, 10% seeds.

BW - Body weight (g)

Body weight of male & female skylarks

37 & 22

Tieleman et al (2004) & Crocker et al. (unpublished).

Normal Sampling uncertainty

AEl - Assimilation Efficiency

Fraction of the energy present in leaves that is taken up by skylarks

5-20 Bairlein (1997) Beta Sampling uncertainty

Data for leaves and seeds represent variation between sites and may underestimate variation between individuals.

GEi (kJ/g dry wt)

Gross energy (dry weight) values of skylark foods

6-75 Smit & Crocker (unpublished),Gluck (1985), Green (1978)

Lognormal or normal depending on food type

Sampling uncertainty

Variance of gross energy for dicots may be over-estimated

Mi - food moisture

Moisture content of wildlife foods

7-21 Smit & Crocker (unpublished)

Beta Sampling uncertainty

Moisture content of dicot leaves estimated from crop

SID 5 (Rev. 3/06) of 29

Input Data n Source of data Distribution Quantified uncertainties

Unquantified extrapolation and uncertainties

content to enable conversion of dry weight energy data to wet weight

Gluck (1985) plants, weed foliage may have lower moisture content.

PT – Proportion of food from treated area

Measured as proportion of active time skylarks spent on cereal fields (ignoring birds who spent no time there)

26 Crocker et al (unpublished)

Beta Sampling uncertainty

Skylarks radiotracked for PT may not be representative sample of population visiting cereal fields. Time spent active in field used as surrogate for proportion of diet obtained in field.

PDi – proportion of foods in diet

Dry weight estimation from faecal samples of foods taken by skylarks on farmland during 5 months breeding season

6-9 Green (1978) Beta Sampling uncertainty

Data represent variation between sites and may underestimate variation between individuals. Mature and immature seeds have different moisture content but data for PD do not distinguish them. See later for how these issues are dealt with.

RUDi – residue per unit dose

Historical data on residues in various types of vegetation, seeds and insects.

18-96

European Commission (2002)Fletcher et al. (1994)

Lognormal Sampling uncertainty

Insect residues are estimated from data on forage and small seeds and the extrapolation is very uncertain. Preliminary results of new research appear to show some possibility of higher values but more of lower values.

AppRate – application rate

Fixed at the nominal rate (input by user)

n/a - - - Farmers sometimes apply pesticides at reduced rates.

Alternative models for dietary composition (PD)Published values for the dietary composition of mammals and birds often represent an average for an indeterminate number of individuals caused by the nature of their collection and/or analysis (e.g. faecal sacs). As a consequence individual birds cannot be linked to data and data often represents variability between methodologies, study site and month, rather than variability between individuals. For example, as a consequence of pooling faecal samples, each faecal sample may represent a number of individuals. The relation between the proportion of each food type in such a sample and the proportions eaten by different individual is uncertain: for example, if there are 2 foods and the pooled sample contains 40% seeds and 60% insects, this might result from 40% of the individuals eating only seeds and 60% of the individuals eating only insects, or at another extreme from every individual eating a mixture of 40% seeds and 60% insects. To explore the effect of this uncertainty on the assessment, the whole exposure assessment was repeated twice with the following alternative assumptions: Single foods model: each individual consumes only one type of food, PD estimates the proportion of

individuals eating each food Multi foods model: each individual consumes all foods in proportions estimated from PD.

An alternative approach might be to assign a distribution estimating relative probabilities for the range of possible true situations between the extremes of the “single food” and “multi food” assumptions. This was not done because we felt there was not enough information to estimate the relative probabilities, so using the two extremes more appropriately reflects the state of knowledge.

For the woodmouse scenario, the same problem also occurred within a food group (arthropods). The data that is available on the proportion of arthropod species in the diet also reflected variability within studies rather than individuals. Therefore, we do not know whether all wood mice are eating 40% Lepidoptera larvae, 30% Coleoptera larvae or whether 40% of wood mice are eating a wholly Lepidoptera larvae diet and 30% eat nothing but Coleoptera larvae. As a consequence, we run 2 versions of the models, one representing an all-mice-mixed arthropod assumption and one representing a some-mice-single-arthropod assumption:

SID 5 (Rev. 3/06) of 29

Single arthropod group model: each mouse consumes only one type of arthropods, PD estimates the proportion of individuals eating each food

Multi arthropod groups model: each mouse consumes all foods in proportions estimated from PD.

In total, up to 4 different models may be run per scenario, because of uncertainties in the data available for estimating dietary composition.

Energy content of seeds

The majority of literature on moisture and energy content of seeds derives from measurements on mature seeds (e.g. crop seeds at harvest). However, a single study (Gluck, 1985) provides data on immature seeds that were observed to be eaten by goldfinches, and these have much higher moisture and lower energy content.

It seems likely that both mature and immature seeds will be taken by other species, and that the proportions of each are likely to vary seasonally. Unfortunately there is little or no information on this. There was also not enough information to specify an uncertainty distribution for the proportion of consumed seeds that are immature. Therefore, as for PD, this uncertainty was represented by using alternative models representing the extreme possibilities (0% and 100% immature seeds).

Uncertainties and dependencies in exposure assessment

Sampling uncertainty was quantified for all the inputs that were represented by distributions, and the error term of the regression of energy expenditure on body weight was quantified (see help pages on website for details of methods). Other identified uncertainties are listed and evaluated qualitatively (see later).

The dependencies quantified in the exposure model were: the regression of daily energy expenditure on body weight, the negative dependency between PD for different food types, which results from the constraint that they

must sum to 1, and the dependencies between the uncertainty distributions for the parameters (e.g. mean and variance) of each

input distribution (see help pages on website for methods).

An important additional uncertainty concerns the daily food intakes simulated by the model. This uncertainty was evaluated by examining the estimates of FIR produced for the different assumptions on PD and M (see below).

Example output: estimated distributions for daily food intake (FIR)It is possible that the distributions used to represent BW, GE, M and AE, plus the regression relationship between DEE and BW, may generate some unrealistically high estimates for FIR (e.g. a skylark eating several times its bodyweight in a day). This uncertainty was evaluated by examining the distributions produced for FIR, expressed as a proportion of body weight.

Figure 3 shows four such distributions corresponding to the four combinations of assumptions about PD (single food vs. multi-food diets) and M and GE (for mature vs. immature seeds) for the scenario skylark/cereals/summer. The “single food” models (where each individual consumes only one food type on a given day) resulted in high food intakes that seem clearly unrealistic for skylark (>2 x body weight)1. This suggests that the true diet is closer to the “multi-food” model. Of the multi-food models, the one assuming 100% of consumed seeds were immature gave higher frequencies of high intakes, though mostly under 2x bodyweight. We conclude that, of the 4 models, the multi-foods/mature seeds model is most realistic. However, it must be remembered that there is a continuum of possible models between these 4 extremes, many of which may be as reasonable as the multi-foods/mature seeds model. Because of this uncertainty we continue to show results for all 4 models in the remainder of the assessment, although bearing in mind that the true situation is likely to be closest to the multi-foods/mature seeds model.

1 There are a few published records of birds eating over 2x body weight but this applies mostly to fully herbivorous species, and in our judgement this is unlikely for skylarks.

SID 5 (Rev. 3/06) of 29

0 1 2 3 4 50%

20%

40%

60%

80%

100%

% o

f sky

lark

s

Immature Seeds

FIR / BW

0 1 2 3 4 50%

20%

40%

60%

80%

100%

% o

f sky

lark

s

FIR / BW

Sin

gle

Food

Item

0 1 2 3 4 50%

20%

40%

60%

80%

100%

% o

f sky

lark

s

Mature Seeds

FIR / BW

0 1 2 3 4 50%

20%

40%

60%

80%

100%

% o

f sky

lark

s

FIR / BW

Mul

tiple

Foo

d Ite

ms

95% Confidence IntervalMedian

Figure 3. Example output for skylark/cereals/summer scenario. Cumulative distributions for the ratio of daily food intake to body weight of skylarks, for 4 separate models representing different assumptions about diet (single food/multiple foods) and seed preference (mature/immature seeds).

Example output: estimated distributions for exposure (ETE)Example results of the exposure assessment for the four alternative models in the skylark/cereals/summer scenario are shown in Figure 4. The confidence intervals represent the combined effect of the uncertainties quantified in the assessment.

For all 4 models, the deterministic tier 1 exposure estimate falls in the upper range of the probabilistic exposure distribution, as would be expected given the conservative assumptions made in tier 1 (PT and PD equal to 1). The multi-foods/mature seeds model gives fewer exposures above the tier 1 estimate than the other 3 models, as would be expected from the lower food intakes generated by this model.

The slight discontinuities that can be seen in the lower panels of Figure 4 result from the single-food assumption: they mark the boundaries between groups of individual birds consuming different food types (e.g. 100% seeds, or 100% insects). As discussed above, these models are considered less realistic than the multi-food/mature seeds model, which has more realistic food intakes.

SID 5 (Rev. 3/06) of 29

0.0001 0.01 1 100 100000%

20%

40%

60%

80%

100%

% o

f sky

lark

s

Exposure [mg / kg BW / d]

Mul

tiple

Foo

d Ite

ms

0.0001 0.01 1 100 100000%

20%

40%

60%

80%

100%

% o

f sky

lark

s


Sin

gle

Food

Item

0.0001 0.01 1 100 100000%

20%

40%

60%

80%

100%

% o

f sky

lark

s


Mature Seeds

0.0001 0.01 1 100 100000%

20%

40%

60%

80%

100%

% o

f sky

lark

s


Immature SeedsMedian Exposure95% Confidence IntervalDeterministic

Figure 4. Example output for skylark/cereals/summer scenario. Cumulative distributions for the exposure of skylarks, for 4 separate models representing different assumptions about diet (single food/multiple foods) and seed preference (mature/immature seeds). Vertical dashed lines show the standard tier 1 exposure estimate for this scenario.

Effects assessment

Effects model description

The assessment objective is to estimate TERs and % mortality for skylarks. Estimating the TER for skylarks requires estimating the LD50 for skylarks exposed to the pesticide of interest, and estimating % mortality requires estimates of both the LD50 and slope of the dose response curve, again for the pesticide of interest and the skylark. However, in practice, pesticide toxicity tests have probably never been conducted with skylarks, and it is unlikely (for ethical reasons) that this would be considered.

Because the dose-response function of the focal species is not available it has to be estimated from toxicity data available for other species. The first step is to determine the location of the dose-response function. For this, we need to know what the LD50 is for the focal species. The species sensitivity distribution describing variation in sensitivity towards the pesticide of interest can be used to estimate the LD50 of the focal species. Since the focal species could be anywhere on the distribution (e.g., it could be really sensitive or insensitive), the distribution can be used to predict an LD50. Because we do not know what the LD50 of the focal species is, predictions represent uncertainty.

The approach taken is therefore to estimate a distribution for between-species variation in the LD50, and another distribution for between-species variation in the dose response slope, and use these to represent (in the outer loop of the 2D Monte Carlo) the uncertainty about the LD50 and slope for skylarks. Both distributions are assumed to be lognormal; Aldenberg & Luttik (2002) have reported that a high proportion of pesticides show a

SID 5 (Rev. 3/06) of 29

good fit to lognormal for the distribution of LD50s, but there are too few data to test this for slopes (range 2-6 species per pesticide).

Between-species variation in LD50The distribution for the LD50 is estimated by adapting the approach of Aldenberg & Luttik (2002). This method uses existing toxicity data for many pesticides and species to estimate a pooled standard deviation for between-species variation in the LD50, i.e. the standard deviation of the species sensitivity distribution (SSD). This pooled estimate for the standard deviation is then combined with an estimate of the mean log LD50 (i.e. the mean of the SSD), which is assumed to vary between pesticides and is estimated as the geometric mean of the toxicity data available for the pesticide in hand. The mean and standard deviation together define the SSD for between-species variation in the LD50.

It is assumed by Aldenberg & Luttik (2002) and in this assessment that the SSD standard deviation is the same for all pesticides, and precisely known. In fact, the true standard deviation is obviously uncertain, and is likely to vary to at least some extent between pesticides. Refined statistical approaches to allow for both uncertainty and variability of the SSD standard deviations have recently been developed in an EFSA opinion (EFSA 2005), and similar approaches could be developed for use in our model. However, this development occurred too late for adaptation and implementation within the project. The influence of this limitation on the assessment outcome is considered qualitatively together with other uncertainties (see later).

The sampling uncertainty for the mean LD50 is quantified assuming a normal distribution on the log scale:

where is the mean of the log10 LD50s for the pesticide in hand, n is the number of LD50s for the pesticide in hand (from different species), and is the pooled standard deviation for the SSD (on log10 scale) as estimated by Aldenberg & Luttik (2002).

The advantage of using the Aldenberg & Luttik approach is illustrated in Figure 5(a) and (b). Figure 5(a) shows the SSD for the pesticide of interest together with confidence bounds, when estimated solely from the two LD50s available for a hypothetical pesticide, using the method of Aldenberg & Jaworksa (2000). Figure 5(b) shows the SSD for the same pesticide when estimated by the Aldenberg & Luttik approach, using the pooled standard deviation from other pesticides. It can be seen that the extra information used in the Aldenberg & Luttik approach greatly narrows the confidence bounds around the SSD.

The SSD estimated by the Aldenberg & Luttik approach (as illustrated in Figure 5(b)) is used in risk characterisation as an uncertainty distribution for the LD50 of the species of concern.

0.0001 0.01 1 100 100000 0

20%

40%

60%

80%

100%

Dose [mg / kg BW / d]

% o

f spe

cies

0.0001 0.01 1 100 100000 0

20%

40%

60%

80%

100%


% o

f spe

cies

LD50Median95% Confidence Interval

108 108 106 106

Mallard Duck

Bobwhite Quail

Mallard Duck

Bobwhite Quail

Method (a) Method (b)

Figure 5. Species sensitivity distribution (SSD) acute avian LD50 of a hypothetical pesticide. Method (a): using the two LD50s available for the pesticide to estimate both the mean and standard deviation (Aldenberg & Jaworksa 2000); Method (b): using the two LD50s available for the pesticide to estimate the mean, and a pooled standard deviation from other pesticides (Aldenberg & Luttik 2002).

Between-species variation in the slope of the dose-responseThe distribution for the slope is estimated by adapting the approach used for the LD50s. A historical dataset (71 slopes for 29 pesticides, with slopes for 2-6 species per pesticide) was used to estimate a pooled standard

SID 5 (Rev. 3/06) of 29

deviation for between-species variation in slopes, which was 0.21 with a 90% confidence interval of 0.20 – 0.23. This is then combined with an estimate of the mean slope, which is assumed to vary between pesticides and is estimated as the geometric mean of the slopes available from different species for the pesticide in hand. The mean and standard deviation together define the distribution for between-species variation in the slope.

As for the LD50s, it is assumed that the standard deviation is the same for all pesticides, and precisely known, although in fact it is uncertain and probably variable. Again, the influence of this limitation on the assessment outcome is evaluated qualitatively below.

Sampling uncertainty for the mean slope is quantified assuming that slopes are a normally distributed on the log scale, in the same way as for the mean LD50 (see above).

Estimating the dose-response for the species of concernIn order to estimate % mortality we need to model the dose-response for the species of concern. The distributions derived above for between-species variation in the LD50 and slope of the dose-response are sampled once in each outer loop of the 2D Monte Carlo. This LD50 and slope specify one realisation of the dose-response for the species of concern, for use in each inner loop. The 1000 outer loops generate 1000 realisations, which can be used to estimate a median estimate of the dose-response curve for the species of concern, together with confidence intervals.

An example of the estimated dose-response for the species of concern is shown in Figure 6, and is used when estimating % mortality (see later). When no slopes are available, only TERs can be generated as output. The confidence intervals in Figure 6 are wide because they represent the combined effect of the uncertainties quantified in the effects assessment, i.e. the sampling uncertainties for estimating the mean LD50 and mean slope from small datasets (n=2), plus the uncertainty regarding the location of the species of concern within the SSD and the distribution of slopes.

1 10 100 1000 10000 100000 0%

20%

40%

60%

80%

100%

% m

orta

lity


Dose-Response SkylarkMedian Dose Response95% Confidence Interval

Figure 6. Example of median estimate for the dose-response curve for the species of concern (e.g. skylark), together with 95% confidence bounds. The confidence bounds represent the combined effect of the uncertainties quantified in the assessment. Result for a hypothetical pesticide with LD50s and slopes for 2 species: species 1, LD50 = 400 mg a.s./kg BW, slope of dose response = 4.35; species 2, 2000 mg a.s./kg BW, slope of dose response = 9.66.

Uncertainties and dependencies

The following uncertainties are quantified in the effects model: Sampling uncertainty resulting from estimating the mean of the SSD from the LD50s available for the

pesticide in hand. Sampling uncertainty for the mean dose response slope from the slopes available for the pesticide in hand. Uncertainty about where the species of concern falls on the SSD. Uncertainty about where the species of concern falls in the distribution of dose response slopes.

SID 5 (Rev. 3/06) of 29

The following uncertainties were not quantified and are instead evaluated qualitatively (see later): Variability of the SSD standard deviation between pesticides Variability of the dose-response slope between pesticides Uncertainty about the shape of the SSD (assumed lognormal) Uncertainty about the shape of the distribution of dose-response slopes between pesticides (assumed

lognormal).

There was no evidence of dependency between the LD50s and slopes (Figure 7).

0.1

1

10

100

1000

-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4

log LD50

Slop

e

Figure 7. Scatter plot of relationship between LD50 and slope for different combinations of pesticides and species. Each point represents a single toxicity study.

Risk characterisation

Toxicity-exposure ratiosTERs were calculated in the usual way for acute avian assessments, as the ratio of the LD50 to the estimated exposure.

Percentage mortality To calculate percentage mortality requires an estimate for the slope of the dose response curve for the species of concern. The reciprocal of the slope was taken to obtain the standard deviation of the dose response:

Assuming a normal dose response curve (i.e. probit), the percentage of individuals dying in the species of concern can be estimated by calculating the cumulative percentage of the normal distribution defined by the mean log LD50 and the standard deviation 1/slope:

% Mortality = CDFNormal(log Exposure, log LD50, σlethal dose)

Uncertainties and dependencies

The uncertainties quantified in the exposure and effects assessments (see above) are propagated by the 2D Monte Carlo procedure to provide confidence intervals for the distribution of TERs, and for the point estimate of % mortality.

In the estimation of percent mortality, it is assumed that there is no dependency between the exposures of individual birds and their sensitivities to the pesticide. Such a dependency might arise if there was a significant avoidance response caused by sub-lethal intoxication. The impact on the assessment outcome of excluding avoidance from the model is considered qualitatively below.

SID 5 (Rev. 3/06) of 29

Example risk characterisation output: overlay plot

As well as modelling the primary assessment endpoints (TER and % mortality), an overlay plot can be generated to provide a visual impression of the degree of risk.

This also provides the opportunity to highlight some key differences between the probabilistic assessments for birds and aquatic organisms. For aquatic organisms, the overlay plot combines the SSD with a distribution of exposures, whereas for birds it combines a dose-response curve with a distribution of exposures. This is appropriate because the avian assessment focuses on a specific indicator species, in this case the skylark, and the exposure assessment is specific to that species. This contrasts with the aquatic assessment, where all species are assumed to experience the same distribution of exposures, because they inhabit the same water bodies.

Furthermore, the avian assessment needs to focus on individuals, partly because this is relevant for the protection goal, and partly because exposure varies widely between individuals in the same location (because of differences in behaviour, i.e. PT and PD). Because the avian exposure distribution represents variation between individuals, it should be compared with an effects distribution that also represents variation between individuals – i.e. a dose-response curve. This also illustrates the general principle, that the statistical populations of different distributions in a probabilistic assessment should be compatible with one another and with the assessment output.

An example overlay plot for one of the four models generated for the skylark/cereals/summer scenario is shown in Figure 8. The example presented is from the most conservative of the four models, using the single foods/immature seeds assumption. As discussed earlier the upper levels of food intake simulated in this version of the model are unrealistic. The overlap between exposure and effects is less in the other three models (not shown).

0.001 0.01 0.1 1 10 100 1000 10000 100000 0%

20%

40%

60%

80%

100%

% o

f sky

lark

s


Single Food Item - Immature Seeds

Median95% Confidence IntervalMedian Exposure95% Confidence Interval

100%

80%

60%

40%

20%

0%

% of skylarks

Figure 8. Example output from the skylark/cereals/summer scenario, assuming single foods and immature seeds. Overlay plot comparing the exceedance function (EXF) for exposure of individual skylarks with the estimated dose response curve for skylarks, for a hypothetical pesticide. The degree of overlap gives a visual impression of the level of risk (more overlap implies higher risk).

Example risk characterisation output: distribution of toxicity-exposure ratios

Figure 9 shows an example of cumulative distributions for the TER, for the skylark/cereals/summer scenario and a hypothetical pesticide. The distribution for the multiple foods/mature seeds model gives lower frequencies of TERs below the tier 1 decision criterion of 10, as would be expected due to its lower and more realistic food intakes (see earlier). For this model and the example pesticide used in the figures, the proportion of skylarks with

SID 5 (Rev. 3/06) of 29

TERs below 10 is estimated as 1.7% with a 95% confidence interval of 0.1 – 45%. The relatively wide confidence interval results from the combined effect of the uncertainties in the assessment inputs.

The confidence bounds in Figure 9 include an allowance for uncertainty in extrapolating toxicity from lab species to the species of concern (in this example, skylarks), which is usually considered to be allowed for within the normal TER trigger of 10. Therefore, it might be argued that the median estimate for % skylarks below 10 is relevant for comparison with TER=10. Alternative approaches to using TER distributions in decision making are explored later below.

The first-tier TER for a pesticide with the same toxicity data as the example used in the Figures would be 7.4. This is lower (i.e. higher risk) than over 95% of the TERs obtained from the probabilistic assessment when estimated by the “multiple foods/mature seeds” model (Figure 9). This reduction in the estimated risk is consistent with the refinements that were introduced, some of which (e.g. PT, PD) replaced extreme worst-case assumptions with realistic distributions. In addition, considering both toxicity values has a substantial effect, because the difference between them was large (400 vs 2000 mg/kg bw), so the mean LD50 is much higher than the minimum.

Each TER produced by the model combines an estimate of individual exposure with an estimate of the median lethal dose (LD50) for the species. This was done for consistency with the current concept of TERs in deterministic assessments, which combine an upper percentile for exposure with the LD50. If variation in toxicity between individuals were taken into account (using the slope of the dose-response), the distribution of TERs would be substantially wider. Variability in toxicity between individuals is, however, taken into account when modelling the alternative measure of risk, percent mortality (see below).

0.01 1 100 10000 0%

20%

40%

60%

80%

100%

% o

f sky

lark

s

Toxicity Exposure Ratio (TER)

0.01 1 100 10000 0%

20%

40%

60%

80%

100%

% o

f sky

lark

s


0.01 1 100 10000 0%

20%

40%

60%

80%

100%

% o

f sky

lark

s


0.01 1 100 10000 0%

20%

40%

60%

80%

100%

% o

f sky

lark

s


Median95% Conf. Int.TER = 10

Immature Seeds Mature Seeds

Mul

tiple

Foo

d Ite

ms

Sin

gle

Food

Item

Figure 9. Example output for the skylark/cereals/summer scenario and a hypothetical pesticide. Cumulative distributions for variation in toxicity-exposure ratio between individual skylarks visiting cereal fields on the day of pesticide application. The four panels show results for the four different scenarios for diet composition (single vs. multiple foods) and seed preference (immature vs. mature). Vertical solid lines represent the first tier decision-making criterion (TER=10).

SID 5 (Rev. 3/06) of 29

Example risk characterisation output: percent mortalityFigure 10 shows an example of uncertainty distributions for % mortality as four exceedance functions, one from each of the four models in the skylark/cereals/summer scenario. Curves close to the x and y axes imply a lower risk than curves further to the right. Inspection of the four curves shows that the risk is highest for the single foods/immature seeds scenario, and lowest for the multiple foods/mature seeds scenario, as expected. Because % mortality is a summary statistic for the population, the exceedance distributions represent uncertainty and do not themselves have confidence intervals.

0% 20% 40% 60% 80% 100%0

0.2

0.4

0.6

0.8

1

% Mortality

Exc

eede

nce

Pro

babi

lity

Multi Food - Immature Seeds Multi Food - Mature SeedsSingle Food - Immature SeedsSingle Food - Mature Seeds

Figure 10. Example output for the skylark/cereals/summer scenario and a hypothetical pesticide. Exceedance functions for the uncertainty in estimating % mortality for skylarks visiting cereal fields on the day of pesticide application. The four panels show results for the four different scenarios for diet composition (single vs. multiple foods) and seed preference (immature vs. mature).

Each exceedance curve can be used to obtain a median estimate for % mortality, together with any confidence interval that is desired. For example, the arrows in Figure 11 show that for the single food/immature seeds model, the median estimate for mortality is 0.6%, and the 95% confidence interval is approximately 0 – 15% (the upper bound varied from 14-16% in 3 runs of the model runs). For the more realistic multiple foods/mature seeds model, the risk would be lower.

The exceedance curves can also be used to estimate the probability of exceeding any given level of mortality, which may be useful to decision-makers if their protection goal is to avoid “appreciable mortality”. For example, in Figure 10 it can be seen that for the most realistic model (multiple foods/mature seeds) the chance of exceeding 5% mortality is less than 0.05 (i.e. less than 1 in 20).

SID 5 (Rev. 3/06) of 29

0% 20% 40% 60% 80% 100%

0

0.2

0.4

0.6

0.8

1

% Mortality

Exc

eede

nce

Pro

babi

lity

Single Food - Immature Seeds

Figure 11. Example output for the skylark/cereals/summer scenario and a hypothetical pesticide. Exceedance function for %mortality in the single foods/immature seeds model. The arrows show that the median estimate of mortality is 0.6%, and the 95% confidence interval is approximately 0 – 15%.

Sensitivity analysis

As both the effects and exposure assessments contain multiple uncertain and variable parameters, sensitivity analysis was conducted to quantify the contribution of each to input to variability and uncertainty in the outputs. This might be especially helpful in guiding additional data collection to reduce key uncertainties, if further refinement of the assessment was required.

A visual impression of the relative contributions of the exposure and effects assessment can be gained from the overlay plot, Figure 8. This shows that despite the larger number of uncertain variables in the exposure assessment, the confidence intervals for the effects assessment are wider, especially in the critical region where the curves overlap. This reflects the very low sample sizes for toxicity data (2 species tested), and the fact that the position of the species of concern on the SSD and the distribution of dose-response slopes is unknown. In addition it must be remembered that there are additional uncertainties in the toxicity assessment that have not been quantified (see earlier). These observations suggest that large reductions in uncertainty could in theory be obtained by conducting additional toxicity tests, however, this is considered inappropriate due to concerns over animal welfare (European Commission, 2002, p. 23).

Quantitative sensitivity analysis was conducted separately for variability and uncertainty. To analyse uncertainty, parameters sampled in the outer loop (e.g. the mean of the SSD, or the LD50 for skylarks) were correlated with summary statistics from the inner loop (e.g. 2.5th, 50th and 97.5th percentile of TER or % mortality).

To analyse variability, values sampled in inner loops (e.g. PT values for individual skylarks) were correlated with the individual outputs of the inner loops (e.g. TERs for the same individual skylarks). This produced one set of rank correlation coefficients for each outer loop. These were then averaged over outer loops, e.g. to obtain a distribution of rank correlation coefficients between PT and TER.

The results of these analyses (not shown) confirmed the visual result from Figure 8, that uncertainty in toxicity had a dominant effect on the assessment outcome. In addition, it showed that uncertainty in PT (proportion of diet from treated area) was also a major contributor to uncertainty in the outputs. Other exposure parameters had relatively little effect.

SID 5 (Rev. 3/06) of 29

Strengths and uncertainties of the WEBFRAM models

StrengthsThe key strengths of the probabilistic models developed in the project compared to deterministic assessment are:1. It provides two alternative assessment endpoints to help provide a fuller picture of risk for decision-makers: a

distribution of TERs (familiar) and estimates of the level of mortality (directly relevant to the protection goal of preventing appreciable mortality).

2. It replaces extreme worst-case assumptions about PT and PD with distributions reflecting the real variation in these parameters, and takes account of their uncertainty.

3. It quantifies variability and uncertainty in other exposure parameters, for which fixed average values are used in deterministic assessments.

4. It incorporates dependencies between daily energy expenditure and body weight, and between the uncertainty distributions for the parameters (e.g. mean and variance) of most input distributions.

5. It copes with inadequacies in data about dietary composition and seed preferences using 4 alternative scenarios, and addresses concerns about unrealistic estimates of food intake.

6. It makes more use of the toxicity data for the pesticide in hand (the LD50 and slope for both tested species, instead of only the most sensitive LD50).

7. It quantifies several major sources of uncertainty in the effects assessment, which may replace the standard uncertainty factors used in decision-making or provide a basis for reducing them.

8. It identifies which parameters are contributing most to uncertainty, which may help in targeting additional studies if more certainty is required.

Evaluation of unquantified variables, uncertainties and dependenciesEach part of the assessment was considered in turn to identify other variables, uncertainties and dependencies that might alter the outcome, and these are listed and qualitatively evaluated in Table 2.

Note that the Table is restricted to uncertainties that affect the estimation of acute risks from dietary exposures. There would be additional uncertainties in extrapolating from these endpoints to other measures of risk, e.g. population level. These issues do not affect the reliability of the assessment endpoints per se, but should be considered when interpreting the ecological consequences and in decision-making (see below).

As indicated in Table 1, there are many factors that may decrease or increase the true risk. Overall we consider that the balance of the identified factors is likely to decrease risk, i.e. the true risk is more likely to be lower than estimated. However, this is a qualitative evaluation and is inevitably uncertain.

SID 5 (Rev. 3/06) of 29

Table 2. List of unquantified variables, uncertainties and dependencies that may affect acute risk assessments using the WEBFRAM3 models and scenarios, together with an approximate qualitative evaluation of their potential influence on the outcome. The potential influences are evaluated subjectively for their direction and magnitude, i.e. whether they would be expected to make the real risk higher (+) or lower (-), and the magnitude of the increase or decrease (e.g. +++ represents a potential to make the true risk much higher). Assessments of magnitude are relative to other uncertainties in the Table. A subjective evaluation of overall combined influence of the uncertainties is provided at the foot of the Table.

Source of uncertainty Direction and magnitude

GeneralDistribution choice is always somewhat uncertain, and goodness of fit is questionable or not testable (due to insufficient sample sizes) for some parameters. Raw data were unavailable for some (e.g. body weight).

+/-(more for some

parameters)Measurement uncertainty applies to all variables but not quantified for any. +/-Exposure assessmentNot all cereal fields will be treated with the pesticide under assessment.* - -Those cereal fields that are treated, will not all be treated on the same day*. -/- -Uncertainty about estimation of PD and seed preference is represented by running models with different combinations of assumptions. True values are likely to be intermediate.

+/-

Assimilation efficiency data for leaves and seeds represent variation between sites and may underestimate variation between individual animals.

+/-

Assimilation efficiency data for arthropods are for different species and there is uncertainty in extrapolating to the species of concern.

+/-

Variance of gross energy for dicot leaves may be over-estimated -Moisture content of dicot leaves estimated from crop plants, weed foliage may have lower moisture content.

-

Insect residues are estimated from data on forage and small seeds and the extrapolation is very uncertain. Preliminary results of recent research appear to show some possibility of higher values but more of lower values.

+/- -

Animals radiotracked for PT may not be representative sample of population visiting cereal fields. Time spent active in field may not be a good surrogate for proportion of diet obtained in field. PT may vary between months.

+/-

Non-dietary routes of exposure are omitted. Could increase total exposure by 2-5 fold (e.g. Driver et al. 1997).

++/ .

Effects assessmentVariation of SSD standard deviation between pesticides, and uncertainty in its estimation. +/- Variation of standard deviation of dose response slope between pesticides, and uncertainty in its estimation.

+/-

Uncertainty extrapolating from toxicity in lab to field conditions (for same species) ++/- -Avoidance response (if applicable to pesticide under assessment) -/- - -Metabolism and depuration of pesticide under assessment. If %mortality is low, exposure probably requires most of a daily intake of food, and therefore provides a long time period for depuration and metabolism to reduce internal dose.

- /- - -

Overall: There are many factors which may decrease or increase the true risk. Overall we consider that the balance of the identified factors is likely to decrease risk, i.e. the true risk is most likely to be lower than estimated, especially for those pesticides with substantial avoidance responses and rapid metabolism. For pesticides with little or no avoidance and slow metabolism, the models are likely to underestimate risk.

+/- - -

* the sensitivity of the results to PT suggests these factors may have a large effect.

SID 5 (Rev. 3/06) of 29

Use of probabilistic risk estimates in decision-making

Annex VI of Directive 91/414/EC and the EU Guidance Document for birds and mammals (European Commission 2002) provide guidance on decision-making for TERs but not for other types of assessment endpoint. The lack of applicable criteria for decision-making is currently a major obstacle to the use of probabilistic approaches.

Possible approaches to decision criteriaThere are at least four possible approaches to this problem:1. Decisions could be made case by case: for each assessment, scientists and decision-makers would evaluate

the probabilistic risk estimate, consider its ecological implications and decide whether they were acceptable. Experience so far suggests this is very difficult in practice. Furthermore, it would add a lot of time to every assessment and the lack of an established standard might lead to unintended inconsistencies between decisions taken on different substances.

2. A preferable approach would be to establish general criteria, which could then be applied consistently and with less effort to different substances. This would require consideration of the ecological consequences of different levels of impact, their social and ethical acceptability, and the consequences of authorising more or fewer pesticides.

3. Another possibility, that could potentially be implemented more quickly, is to use the existing first tier as a reference point or benchmark. The idea is to produce a probabilistic estimate of what the risk would be, if the tier 1 assessment had produced a result exactly equal to the decision criteria in Annex VI of Directive 91/414/EC. This could then be used a reference point for comparing with the higher tier probabilistic assessment, e.g. by plotting distributions for the two assessments together. This avoids the need for establishing new criteria for acceptable risk, but depends on the existing criteria being appropriate.

4. A further, and potentially ideal, approach would be to calibrate the probabilistic model using data on effects in semi-field or field conditions, so as to identify what levels of probabilistic risk estimate correspond to acceptable impacts in the field. This again requires a judgement about what level of impact is acceptable. It also depends on the availability of reliable data on impacts in the field, which is not available for many types of risk. Some information of this sort is available for acute risks to birds (see below), but not for long-term or reproductive risks.

As part of this project a first attempt was made at the third approach (using the first tier as a reference point), and this is presented in the following sections, as a starting point for discussion.

Comparison with the probabilistic risk for TER=10We explored the application of this approach to both types of risk estimate: TERs and % mortality.

Figure 12 compares five different risk curves for percent mortality, using the same hypothetical pesticide and scenario as earlier figures: Four 2nd Tier risk curves. These are same curves as the exceedance functions for % mortality in Figure 10,

representing the four alternative models for diet. One 1st Tier curve. This curve is also an exceedance curve for % mortality, but calculated as if the

pesticide under assessment had a TER exactly equal to 10, based on the LD50 for a single species and first tier assumptions for PD and PT (both = 1).

The comparison between the actual curves for the pesticide of interest and the curve for Tier 1 would be interpreted as follows:1. If the Tier 2 curves are lower than the Tier 1 curve at every point, as in this case, then the risk for this

pesticide is lower than implied by a TER of 10. This is because the probability of any level of mortality would be lower than would be implied if the first tier TER was 10. In that case, it would be reasonable to argue that the risk of acute effects was acceptable.

2. If the Tier 2 curves were higher than the Tier 1 curve at every point, then the risk would be higher than implied by a TER of 10. This would not necessarily mean the risk is definitely unacceptable, because further refinement of the probabilistic assessment, or other sources of evidence (e.g. higher tier studies), might demonstrate that the true risk is actually lower than indicated by the Tier 2 curves.

3. If the Tier 2 curve crossed the Tier 1 curve, then the risk would be lower than implied by TER=10 for some magnitudes of effect, but higher for others. In that case the risk assessor and decision-maker would need to consider whether the magnitudes of effect, for which Tier 2 has higher probability, are of concern.

It is important to note that decision-making should also take account of the unquantified uncertainties that were evaluated in Table 2. In the hypothetical example used here, the result in Figure 12, together with the conclusion that the true risks are most likely to be lower, suggest that the acute risk may be acceptable.

SID 5 (Rev. 3/06) of 29

0% 20% 40% 60% 80% 100%0

0.2

0.4

0.6

0.8

1

% Mortality

Exc

eede

nce

Pro

babi

lity

Multi Food - Immature SeedsMulti Food - Mature SeedsSingle Food - Immature SeedsSingle Food - Mature SeedsFirst Tier

Figure 12. Comparison between the risk of mortality for a hypothetical pesticide (“Tier 2” curves, copied from Figure 10) and the risk of mortality implied by an acute avian TER of 10. Each curve is an exceedance function (EXF) for % mortality. The red curve (furthest to the right) represents the risk for TER=10.

Figure 13 shows the same approach applied to the other assessment endpoint in this study, the distribution of TERs. In this case, each curve is cumulative distribution for the TERs of individual skylarks. In this case, because lower TERs indicate higher risk, curves to the upper left side of the graph have higher risk than curves further towards the lower right. However, with three lines plotted for each distribution, it is difficult to interpret the outcome.

0.01 1 100 10000 0%

20%

40%

60%

80%

100%

% o

f sky

lark

s


0.01 1 100 10000 0%

20%

40%

60%

80%

100%

% o

f sky

lark

s


0.01 1 100 10000 0%

20%

40%

60%

80%

100%

% o

f sky

lark

s


0.01 1 100 10000 0%

20%

40%

60%

80%

100%

% o

f sky

lark

s


Median95% Conf. Int.TER = 10


Mul

tiple

Foo

d Ite

ms

Sin

gle

Food

Item

Figure 13. Comparison between cumulative distributions of TERs for the pesticide under assessment (black curves) and for the distribution of TERs implied by a deterministic first-tier TER of 10 (grey curves).

SID 5 (Rev. 3/06) of 29

The number of lines in the graphs can be reduced by combining the uncertainty and variability for each model run (by sampling the distributions representing variability and uncertainty by 1D Monte Carlo rather than 2D), so that only a single line is produced for each distribution. Each risk curve should then be interpreted as an uncertainty distribution for the TER of a single skylark selected at random. The resulting graphs are shown in Figure 14.

In Figure 14, the Tier 2 and Tier 1 curves cross in every case. This means the risk for the hypothetical pesticide would be lower than implied by a first tier TER=10 for some magnitudes of effect, but higher for others. Therefore the risk assessor and decision-maker would need to consider whether the magnitudes of effect, for which Tier 2 has higher probability, are of concern. However, in this example, it can be seen in Figure 14 that the probabilities are lower for all TERs below 10, for all four of the alternative models. As 10 is the normal threshold for concern, it might again be argued (after also considering the uncertainties in Table 2), that the risk for the hypothetical pesticide is acceptable.

0.01 1 100 10000 0%

20%

40%

60%

80%

100%

% o

f sky

lark

s


0.01 1 100 10000 0%

20%

40%

60%

80%

100%

% o

f sky

lark

s


0.01 1 100 10000 0%

20%

40%

60%

80%

100%

% o

f sky

lark

s


0.01 1 100 10000 0%

20%

40%

60%

80%

100%

% o

f sky

lark

s


MedianFirst TierTER = 10

Mul

tiple

Foo

d Ite

ms


Sin

gle

Food

Item

Figure 14. Comparison between cumulative distributions of TERs for a hypothetical pesticide (black curves) and for the distribution of TERs implied by a deterministic first-tier TER of 10 (grey curves). Here, each risk curve can be interpreted as an uncertainty distribution for the TER of a single skylark selected at random.

Methodology for constructing the Tier 1 risk curvesThe method used for estimating the risk curves for Tier 1 involves taking an LD50 that would give a TER of exactly 10 with the relevant exposure. This single LD50 is then used together with a generic estimate for the standard deviation of the SSD, as in the Tier 2 model. The estimate used is the pooled standard deviation for SSDs provided by Aldenberg and Luttik (2002). To apply the method to percent mortality, the pooled estimate for the slope of the dose response is also needed. This approach require the same assumptions as are used in the other models developed in this project: that avian SSDs are generally lognormal, and that the standard deviation is fixed and precisely known for all pesticides.

Other sources of evidence

Probabilistic risk estimates should always be considered together with any other relevant types of evidence that are available regarding the pesticide under assessment, e.g. field studies, monitoring data, specialised laboratory studies, etc. However, these lines of evidence will also be subject to the uncertainties, so it is essential to take the

SID 5 (Rev. 3/06) of 29

relative certainty of different lines of evidence into account when deciding how much weight to given them in reaching an overall conclusion.

Interpretation of ecological implications

As discussed earlier, the protection goal for birds and mammals considers population effects as well as individual mortality. Therefore, when deciding on the acceptability of any predicted level of mortality, consideration should be given to its potential consequences at the population level, in both the short and long term.

Comparison of model outputs with data on impacts in field studies

It is essential to assess the realism and credibility of the Webfram models. Sensitivity analysis, consideration of uncertainties, and expert peer review of the methods and data used in the models can contribute to this, but where possible model performance should be evaluated by comparing predicted risks to independent data on effects in field studies, to assess how well they agree. A comparison between model predictions and field data was conducted as part of the project, using information on field studies previously reviewed by Mineau (2002). This was limited to acute risks to birds, as the field studies provide much more limited information on effects on mammals. The key result is shown in Figure 15. Each point on the graph corresponds to a single field study. The horizontal axis shows the mortality predicted by Webfram, using the Webfram scenario that was considered most relevant to the crop and season of the field study, toxicity data for the pesticide concerned, and the application rates used in the field study. The vertical axis shows the level of impact that occurred in the field study, as evaluated by Mineau (2002) on a four-category scale.

Examination of Figure 15 suggests a positive correlation between the risk estimated by Webfram and the level of effect in field studies. Most of the field studies where evidence of mortality was detected are associated with Webfram estimates in excess of 20% mortality. Uncertainties quantified by the Webfram model are represented by horizontal confidence intervals, but it should be taken into account that the field study results are also subject to considerable uncertainty (e.g. variable efficiency in detecting mortality). Some of the predicted mortalities might be considered implausibly high, but it should be remembered that although non-dietary exposure is not considered (which will cause some degree of under-estimation of risk), avoidance and metabolism are also ignored (which will cause some degree of over-estimation of risk).

high mortality

some mortality

slight/sublethal effects

no impact

0 20 40 60 80 100

% Mortality

Fiel

d ef

fect

s sc

ore

Predicted % mortality (95% CL)

high mortality

some mortality

slight/sublethal effects

no impact

0 20 40 60 80 100

% Mortality

Fiel

d ef

fect

s sc

ore

Predicted % mortality (95% CL)

Figure 15: Mortality predicted by the Webfram model is shown on the x-axis, for a range of cholinesterase inhibiting pesticides used in a variety of different contexts and different application rates. As well as the median estimate for percentage of mortality (indicated by a dot) the 95% confidence interval for % mortality is also plotted (indicated by the horizontal lines). The y-axis classifies the field studies using 4 levels of potential risk, evaluated by Mineau (2002).

SID 5 (Rev. 3/06) of 29

Action resulting from this research

During the course of the project, the ongoing work and prototype models were presented and demonstrated at a series of events including scientific conferences (SETAC, BCPC, Fresenius, Agchem Forum), workshops (EUFRAM), and annual meetings with invited stakeholders. The models and supporting documentation were also subjected to peer review by independent experts from academia and industry. The feedback from all these activities was used to refine and improve the final versions of both models and software.

The final models produced by the project are freely available for use online at www.webfram.com. They are being evaluated by the UK authorities to decide on whether and how they should be used in regulatory risk assessments.

Possible future research

Although this project has made substantial advances, there are many areas in which further work could be considered. These include: Models for additional scenarios Models for long-term (reproductive) risks Collection of improved data on dietary composition and moisture content of seeds as eaten by birds (to

reduce the uncertainty demonstrated by different models shown in Figures 3 and 4) Refined modelling of toxicity (e.g. to take account of variation between pesticides in the standard deviation of

SSD, and its uncertainty) Modelling of avoidance, metabolism and non-dietary exposure Modelling of population effects Criteria for acceptable risk and decision-making.

Acknowledgements

The authors are grateful to the UK Pesticides Safety Directorate for funding; to Robert Luttik and Gerd Joermann for providing access to toxicity data; to Richard Shore for providing ecological data for the wood mouse scenario; to Franz Baerlein, Els Smit and Irene Tieleman for access to data for the exposure models; to Mark Clook of PSD for advice and feedback; and to the many other individuals who have provided valuable feedback and peer review on the models and software developed by the project. References

Aldenberg T & Luttik R. 2002. Extrapolation factors for tiny toxicity data sets from species sensitivity distributions with known standard deviation. In Posthuma, L., Suter, G.W.III. & Traas, T.P. (eds.) Species sensitivity Distributions in Ecotoxicology. Lewis, CRC Press, Boca Raton, pp 103-118.

Aldenberg T, Jaworska J. 2000. Uncertainty of the hazardous concentration and fraction affected for Normal species sensitivity distributions. Ecotoxicology and Environmental Safety, 46: 1-18.

Bairlein,F. (1999) Energy and nutrient utilization efficiencies in birds - a review. Proceedings of the 22nd International Ornithological Congress, Durban (eds Adams,N. & Slotow,R.), Birdlife South Africa,

Crocker,D.R., Prosser, P., Irving, P. V., Bone, P., & Hart, A. (2002) Estimating avian exposure to pesticides on arable crops. Birds and Agriculture (eds Boatman,N.D., Carter,N., Evans,A.D., Grice,P.V., Stoate,C., & Wilson,J.D.), pp. 237-244. Association of Applied Biologists, Wellesbourne

Driver, C.J., Ligotke, M.W., Van Voris, P., McVeety, B.D., Greenspan, B.J. and Drown, D.B. 1991. Routes of uptake and their relative contribution to the toxicological response of northern bobwhite (Colinus virginianus) to an organophosphate pesticide. Environ. Toxicol. Chem. 10: 21-33.

Dunning,J.B.Jr. (1993) CRC Handbook of Avian Body Masses. CRC Press, Boca Raton.

EFSA (2004). Opinion of the Scientific Panel on Plant health, Plant protection products and their Residues on a request from Commission related to the evaluation of methamidophos in ecotoxicology in the context of Council Directive 91/414/EEC. 14 December 2004.

EFSA, 2005. Opinion of the Scientific Panel on Plant health, Plant protection products and their Residues on a request from EFSA related to the assessment of the acute and chronic risk to aquatic organisms with regard to the possibility of lowering the uncertainty factor if additional species were tested. The EFSA Journal 301, 1-45.

European Commission, 2002b. Guidance document on risk assessment for birds and mammals under Council Directive 91/414/EEC. SANCO/4145/2000-final.

SID 5 (Rev. 3/06) of 29


Fletcher, J.S., Nellessen, J.E. and Pfleeger, T.G. (1994) Literature review and evaluation of the EPA food-chain (Kenaga) nomogram, an instrument for estimating pesticide residues on plants. Environmental Toxicology and Chemistry 13, 1383-1391.

Gluck,E.E. (1985) Seed preference and energy-intake of goldfinches Carduelis- carduelis in the breeding-season. Ibis 127, pp. 421-429

Green, R.E. (1978). Factors affecting the diet of farmland skylarks Alauda arvensis. J. Anim. Ecol. 47, 913-28

Mineau, P. 2002. Estimating the probability of bird mortality from pesticide sprays on the basis of the field study record. Env. Toxicol. Chem., 21, 1497-1506.

Tieleman,B.I., Williams, J. B., & Visser, G. H. (2004) Energy and water budgets of larks in a life history perspective: Parental effort varies with aridity. Ecology 85, pp. 1399-1410

References to published material9. This section should be used to record links (hypertext links where possible) or references to other

published material generated by, or relating to this project.

Models for the five scenarios developed in this project are freely available for use online at www.webfram.com. The software includes detailed instructions and also extensive help screens including more detailed information on the data and assumptions used in the models.

SID 5 (Rev. 3/06) of 29


General enquiries on this form should be made...

Documents

Transcript of General enquiries on this form should be made...