AEAT in Confidence -...

AEAT in Confidence AEAT/ED50287/Final Report

AEAT in Confidence

Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution

Final Report to European Commission Directorate-General Environment Ref: B4-3070/2003/359591/MAR/D.3

March 2004

AEAT in Confidence

AEAT in Confidence AEA Technology i

Title Mid-term evaluation on the implementation of

the framework for co-operation in the field of accidental or deliberate marine pollution – Final Report

Customer European Commission Directorate-General

Environment Customer reference

Reference: B4-3070/2003/359591/MAR/D.3

Confidentiality, copyright and reproduction

This report has been prepared under contract by AEA Technology plc on behalf of the European Commission Directorate-General Environment (ref. B4-3070/2003/359591/MAR/D.3). This report may not be used for any other purposes, reproduced in whole or in part, nor passed to any organisation or person without the specific permission in writing of the Commission.

File reference AEAT/ED50287/Final Report Reference number AEAT/ED50287 /Final Report Report number AEAT/ENV/1656 Report status Final Contact Details AEA Technology plc

Telephone +44 870 190 6528 Facsimile +44 870 190 6615

AEA Technology is the trading name of AEA Technology plc AEA Technology is certificated to BS EN ISO9001:2000 and ISO14001

Name Signature Date

Author Madeleine Mcdonagh

Reviewed by Nigel Pratten

Approved by Heather Haydock

AEAT in Confidence

AEAT in Confidence AEA Technology ii


Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report

AEAT in Confidence AEA Technology iii

Executive Summary

AEA Technology Environment has been contracted by the European Commission Environment Directorate General to undertake a study entitled ‘Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ (ref: B4-3070/2003/359591/MAR/D.3). This is the Final Report under that study to describe the work we have carried out, summarise the findings from our evaluation and make recommendations for the future. The Framework Programme has objectives related to supporting and supplementing Member States’ efforts to protect the marine environment, to contribute to improving their response capabilities, to facilitate efficient mutual assistance and to promote co-operation in order to provide for compensation for damage. The Commission is the only body that provides funding at an international level to promote co-operation between countries in the field of accidental or deliberate marine pollution. Since the Council Decision was adopted in 2000, the objectives of the Programme have been addressed by the development of a Community Information System and by actions within a three-year rolling plan. These actions have included Training and Information, Pilot Projects, Support and Information and Mobilisation of Experts. The mid-term evaluation has considered how the resources devoted to the programme have led to the various outputs, and how these in turn have led to the achievement of the objectives above. We have considered the linkages between the programme and other actions, policy instruments and previous related programmes. Linkages have included previous EC funding in this area, the activities of the International Maritime Organisation (IMO), relevant legislation /agreements/conventions and ongoing work within Member States. Accordingly, the key evaluation issues have been to establish: • Scope – to what extent the programme objectives are relevant to national and EU

needs; • Impacts – how successful the various actions have been and the value for money

they have provided; • Efficiency – how efficiently resources have been converted into outputs or impacts; • Effectiveness – how far the programme has contributed to achieving its objectives; • Sustainability – to what extent are the programme benefits effectively disseminated. Through examination of these issues, this study has also provided recommendations for continuing improvement, thus ensuring that the evaluation has been both retrospective and formative. In 2001, a separate Council Decision established a Community Mechanism to facilitate reinforced co-operation in civil protection assistance interventions, including marine pollution. This new instrument has covered the Mobilisation of Experts in particular and is therefore also considered within this evaluation. Our approach to obtaining the necessary information has involved evaluating data provided by the Commission and obtaining information from representatives from Member States and other stakeholders. A questionnaire has been used to assist in the collection of data and to ensure consistency. Visits have been made to 5 Member States,



AEAT in Confidence AEA Technology iv

other interviews have been carried out by telephone and additional information obtained via e:mail correspondence. Evaluating the impact of the Programme during a specific oil spill incident was felt to have the potential to provide an additional insight into the implementation and effectiveness of some key Programme Actions. The Baltic Carrier incident has been selected for this purpose because of the different ways in which the Programme and the Commission were involved and the accessibility of the various stakeholders. The Commission co-ordinated the mobilisation of both experts and observers in the early stages of the response operations and a project to assess the environmental effects of the spill was part-funded by the Framework Programme. This incident has therefore been used as a Case Study to assess the value of the interventions. The main findings of the study are summarised below. Scope and Rationale

• The Programme objectives are well-understood by stakeholders;

• The Programme complements activities at national, trans-national and regional levels, is perceived as adding value to these regional agreements and is recognised as unique in supporting co-operation at European level;

• Stakeholders believe that the Programme continues to be necessary;

• There are concerns from Member States about the evolving role of EMSA and the potential for duplication of effort; close co-ordination will be required.

Impacts

• While the impact of individual actions has not been measured, most stakeholders believe that the Programme is delivering benefit and good value for money;

• The level of co-operation between Member States is recognised as being very good;

• Stakeholders have different views over the most useful element of the Programme; the most commonly selected are training and workshops, exchange of information and enhancing co-operation between Member States;

• Views on the value of the Community Information System are highly polarised;

• Actions in Training and Information are valued, particularly by smaller countries that do not run their own national training programmes; however, some stakeholders have expressed some concerns over the variable quality of training courses;

• Stakeholders are positive about the value of Pilot Projects; however, there have only been 6 and it is too soon to draw clear conclusions on their impact and value for money;

• Stakeholders have a limited awareness of Support and Information Actions;

• Stakeholders are generally very positive about the Mobilisation of Experts through the Community Mechanism and the co-ordinating role of the Commission.

Process (Efficiency and Effectiveness)

• It is recognised that Commission staff have a high workload which is made worse by the Commission’s detailed procedures; however, against this context, Programme processes are felt to work well;

• Financing decisions are considered to be soundly based;

• Some Stakeholders would like the MCMP to have more influence in the evaluation of proposals.

Sustainability



AEAT in Confidence AEA Technology v

• Many of the Actions supported by the Programme are by their nature sustainable, e.g. training;

• Several Member States feel there is a lack of evaluation, feedback on benefits or critical appraisal of the outcomes of projects;

• Views are divided on the effectiveness of dissemination of project findings;

• Improvements in dissemination and learning will enhance the sustainability of the Programme.

Synthesis • The Programme is perceived as effective in meeting its objectives but has scope to

improve;

• Member States would generally like the focus on Training and Information actions to continue in future;

• There is a healthy list of topic areas that Member States would like to continue to be addressed within training and other actions;

• The current actions and coverage of the Programme are supported and there are no suggestions for major changes.

Baltic Carrier Case Study The Baltic Carrier Case Study has supported our general findings. All the Baltic Carrier actions have been favourably received by all parties and the mobilisation of experts and observers, in particular, was confirmed as very valuable. The need for improved dissemination of information to learn lessons from incidents has been reinforced by the Case Study. Key Recommendations. • The Programme should continue in its current format. • Consider the list of topics identified in this study when next reviewing priorities in the

three year rolling plan. • Ensure continuing involvement and dialogue with EMSA and the Regional Agreements

to ensure activities continue to complement. • Measure the impact of individual projects, follow-up after an appropriate time and

disseminate lessons to Member States. • Consider the future of the Community Information System and the possibility of

committing additional resources to support its maintenance and updating. • Review the need to hold a workshop to discuss and clarify the Community Mechanism

and the roles of experts and observers. • Review the procedures for letting training contracts and consider increasing

intervention to improve overall quality of courses. • Review all recommendations resulting from this mid-term evaluation and consider

priorities for action/implementation.



AEAT in Confidence AEA Technology vi

Contents

1 INTRODUCTION 1 1.1 OBJECTIVES OF THE STUDY 1 1.2 THE FRAMEWORK PROGRAMME 1 1.3 SCOPE OF THE EVALUATION 4 1.4 STRUCTURE OF THE REPORT 5

2 EVALUATION METHODOLOGY 6 2.1 DEVELOPMENT OF THE EVALUATION METHODOLOGY 6 2.2 ASSEMBLY AND ASSESSMENT OF INITIAL DATA 8 2.3 IN-DEPTH DATA COLLECTION 10

3 FINDINGS 12 3.1 OVERVIEW AND CONTEXT 12 3.2 IMPACTS 14 3.3 PROCESS (EFFICIENCY AND EFFECTIVENESS) 19 3.4 SUSTAINABILITY 21 3.5 SYNTHESIS 22 3.6 COMPARISON WITH PREVIOUS EVALUATION 24

4 BALTIC CARRIER CASE STUDY 25 4.1 BACKGROUND TO THE BALTIC CARRIER INCIDENT AND ACTIONS 25 4.2 MOBILISATION OF THE TASK FORCE 26 4.3 MOBILISATION OF OBSERVERS 29 4.4 PROJECT ON MONITORING AND ASSESSMENT OF ENVIRONMENTAL EFFECTS IN

GRONSUND 29

Appendices

APPENDIX 1 MAIN EVALUATION QUESTIONNAIRE APPENDIX 2 LIST OF STAKEHOLDERS PROVIDING INPUT APPENDIX 3 CASE STUDY QUESTIONNAIRES APPENDIX 4 TRAINING COURSE EVALUATION



AEAT in Confidence AEA Technology vii



AEAT in Confidence AEA Technology 1

1 Introduction

The European Parliament and the Council established a Community framework for co-operation in the field of accidental or deliberate marine pollution through its Decision no. 2850 of 20/12/2000. This framework set a legal basis for the role of the European Community in the field of response to marine pollution. The framework has been established for the period from 1 January 2000 to 31 December 2006 although, as the Decision was adopted in December 2000, actions started in 2001. In accordance with article 5 of the Decision, a mid-term evaluation should be carried out by 28/12/2003 and a report on this evaluation addressed to the European Parliament and to the Council. The European Commission Environment Directorate General has contracted AEA Technology Environment to undertake this study entitled ‘Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ (ref: B4-3070/2003/359591/MAR/D.3). This is the Final Report of the study. It describes the work carried out, summarises the findings from the evaluation and makes recommendations for the future.

1.1 Objectives of the Study

The stated objectives of the evaluation are to:

• assess whether the Programme has fulfilled the objectives as identified at the outset;

• measure if the budget invested so far, as well as the organisation devoted to the implementation of the Programme, have led to results contributing to achieving its objectives.

The objectives of the Programme are:

• To support and supplement Member States’ efforts at national, regional and local levels for the protection of the marine environment, coastlines and human health against the risks of accidental or deliberate pollution at sea;

• To contribute to improving the capabilities of the Member States for response in case of incidents involving oil spills or other harmful substances at sea and also to contribute to the prevention of risks;

• To strengthen the conditions for and facilitate efficient mutual assistance and co-operation between Member States in this field;

• To promote co-operation between Member States in order to provide for compensation for damages according to the polluter pays principle.

1.2 The Framework Programme

The Council Decision no. 2850 of 20/12/2000 that established a legal framework for the role of the European Commission in marine pollution also set an agreed level of funding




over a 7 year period. Prior to that, the Civil Protection Unit had bid for funding on an annual basis, which had led to difficulties in forward planning. A committee (Advisory Committee on Pollution by Hydrocarbons and other harmful substances discharged at sea) consisting of representatives from maritime Member States had provided advice to the Unit and assisted in the identification of suitable projects for Commission financial support. Since 2000, the Programme has been managed by the Civil Protection and Environmental Accident Unit with support from the Management Committee on Marine Pollution (MCMP), consisting of expert representatives from Member States and accession countries. The establishing legislation (in Annex II of the Decision) defines the types of eligible actions and the level of Community financial contribution. The actions, which have to be implemented in close co-operation with the relevant national competent authorities, are selected according to their capacity to contribute to:

• providing information and building capacity to deal with pollution incidents;

• improving techniques for response and rehabilitation after incidents;

• providing better public information to help clarify risks;

• helping local competent authorities and other groups to co-operate and to reduce risks and improve response;

• providing operational support by mobilising experts to assist Member States with emergencies.

The arrangements since 2000 have included the introduction of more detailed and prescriptive procedures for managing the allocation and commitment of funds. Priority areas within the three-year rolling plan have been reviewed annually by the MCMP and approved by other Departments within the European Commission. Formal calls for proposals have been issued in the priority areas and a system has been implemented for evaluating proposals and distributing funding between the various different types of action as listed in Annex II of the Council Decision. These cover:

• Actions in training and information (courses, workshops and exercises) – EC funding provided up to a maximum of 75% of total budget;

• Actions for improving techniques and methods of response and rehabilitation (pilot projects) - EC funding provided up to a maximum of 75% of total budget;

• Support and information actions (environmental impact, conferences and events);

• Mobilisation of experts – EC funding provided to cover expenditure generated by mobilisation of experts following marine accidents.

The Commission also provides a facilitation and co-ordination function for observers to visit the affected country during incidents. The Commission is the only body that provides funding at an international level to promote co-operation between countries in the field of accidental or deliberate marine pollution. 1.2.1 Calls for Proposals Calls for proposals have been published in the Official Journal of the European Communities on an annual basis and have provided guidance on the aims of projects




sought within each priority area and the type of action required. Proposals have been assessed using a standard approach as follows: Evaluation Criteria Measure Eligibility – Exclusion Criteria 12 criteria; Yes/No assessment for each Eligibility - Selection Criteria 9 criteria; Yes/No assessment for each Within Priority Fields 6 criteria; score for each from 0 (not

sufficient) to 4 (very good) The assessments have been made following a clear set of rules. An evaluation team consisting of senior staff from different units/directorates rates the proposals and external experts are asked to assess specific technical aspects of proposals in cases where there is insufficient internal expertise. The Commission then approves the proposals and finalises the contracts. The overall process is as shown in the flow chart below. There have been Calls for Proposals in 2000, 2001 and 2002. The next Call is scheduled for 2004. 1.2.2 Community Information System In addition to the actions above, the Council Decision also required a Community Information System (CIS) to be established for the purpose of exchanging data between Member States (and others) on the preparedness for and response to accidental or deliberate marine pollution. Annex I of the Decision specified that the CIS should use a modern automatic data processing system and provide general background information at Community level on a home page on an Internet site and information related to the national intervention resources on national home pages. In addition, a printed version should be maintained in the form of a Community operational loose-leaf booklet with information on emergency management in each Member State. Although the Commission was required to set up the web site and a Community home page, responsibility was given to each Member State to open (within six months from the

Commission/Member States propose and discuss priority areas for action

3-year rolling plan submitted to MCMP and adopted by Commission

Call for Proposals is published in the Official Journal

Proposals are evaluated by the Commission and shortlisted

Commission approves proposals and finalises contracts

Project commences

Input from specialist experts, if required




date the Decision entered into force) and maintain their national home page. The national home pages are required to include information covering: (i) a description of national structures and of links between national authorities in the

field of accidental or deliberate marine pollution, including focal points to be addressed in questions of emergency response;

(ii) general information on existing teams and equipment for emergency response and clean-up, in particular: - strike teams (seaborne) consisting of spill response vessels, - strike teams (land-based) to combat shoreline pollution and to organise

temporary storage as well as to conduct actions for the rehabilitation of sensitive coastal areas,

- expert teams to carry out environmental monitoring of pollution and/or the impact of the combating techniques used, including chemical dispersion,

- other mechanical, chemical and biological means for combating pollution at sea and cleaning up coasts, including systems for lightening of oil tankers,

- aircraft for aerial surveillance, - location of stockpiles, - emergency towage capacity, - emergency number(s) for public use;

(iii) conditions for offering assistance. Member States are also required to update their national home pages as soon as changes have occurred. 1.2.3 Community Mechanism On 23 October 2001 a Council Decision established a Community Mechanism to facilitate reinforced co-operation in civil protection assistance interventions. This new instrument covers both civil protection and marine pollution and provides for the following:

• the identification of intervention teams (and other intervention support), assessment teams and/or co-ordination teams in the event of emergencies;

• the setting up and implementation of a training programme for intervention teams, assessment experts, and/or co-ordination teams;

• workshops, seminars and pilot projects on major aspects of interventions;

• the establishment and management of a Monitoring and Information Centre (operational on a continuous basis);

• the establishment and management of a common emergency communication and information system;

• other support action such as measures to facilitate transport of resources .

As the mechanism includes marine pollution, some of the funds available to support the above actions can potentially be used for relevant marine pollution activities – although it is apparent that most of the actions are already included within the Framework Programme.

1.3 Scope of the Evaluation

This mid-term evaluation has considered how the resources devoted to the Programme have led to the various outputs, and led to the achievement of the Programme objectives. We have considered the linkages between the Programme and other actions




and policy instruments. Linkages have included previous EC funding in this area, the activities of the International Maritime Organisation (IMO), relevant legislation/agreements/conventions and ongoing work within Member States. Accordingly, the key evaluation issues have been to establish:

• Scope – to what extent the Programme objectives are relevant to national and EU needs;

• Impacts – how successful the various actions have been and the value for money they have provided;

• Efficiency – how efficiently resources have been converted into outputs or impacts;

• Effectiveness – how far the Programme has contributed to achieving its objectives;

• Sustainability – to what extent are the Programme benefits disseminated effectively.

Through examination of these issues, this study has also provided recommendations for continuing improvement, thus ensuring that the evaluation has been both retrospective and formative. Our approach to obtaining the necessary information has involved evaluating data provided by the Commission and obtaining information from representatives from stakeholders including Member States. A questionnaire has been used to assist in the collection of data and to ensure consistency. Visits have been made to 5 Member States, other interviews have been carried out by telephone and additional information obtained via e:mail correspondence. Evaluating the impact of the Programme during a specific oil spill incident was felt to have the potential to provide an additional insight into the implementation and effectiveness of some key Programme Actions. The Baltic Carrier incident has been selected for this purpose because of the different ways in which the Programme and the Commission were involved and the accessibility of the various stakeholders. The Commission co-ordinated the mobilisation of both experts and observers in the early stages of the response operations and a project to assess the environmental effects of the spill was part-funded by the Framework Programme. This incident has therefore been used as a Case Study to assess the value of the interventions.

1.4 Structure of the Report

This report is structured as follows:

• Section 2 provides a description of the evaluation methodology;

• Section 3 provides a description of the key evaluation findings;

• Section 4 provides a Case Study of actions carried out following the oil spill from the Baltic Carrier.

Additional data including questionnaires, list of stakeholders providing input, and detailed evaluation of training courses are provided in Appendices.




2 Evaluation Methodology

Based on this basic understanding of the Programme, the approach to the evaluation study can be summarised as:

• clarification of the intervention logic;

• development of the evaluation methodology;

• assembly and assessment of initial data;

• in-depth data collection;

• evaluation of key quantitative and qualitative data.

More information is provided in the following sections.

2.1 Development of the Evaluation Methodology

The Terms of Reference for this study require a number of principal questions to be addressed in order to provide an evaluation that is both retrospective and formative. In addition, the study is required to propose and test a set of feasible indicators that can be used both for this evaluation and for subsequent monitoring of future actions. We have therefore identified five key themes for the evaluation:

• Overview and context;

• Impacts;

• Process (efficiency and effectiveness);

• Sustainability

• Synthesis.

Each of the principal questions together with supplementary questions have been grouped and categorised according to these themes within an evaluation framework. The basic evaluation framework is shown in Figure 2.1. Indicators Measures Theme 1: Overview and Context

• Programme objectives;

• Interactions with other Programmes;

• Barriers to co-operation;

• Gaps in the Programme.

Qualitative, based on views of stakeholders.

Theme 2: Impacts

• Programme successes;

• Added value;

• Programme shortcomings;

• Value for money;

Both qualitative and quantitative, including an assessment of Programme outputs and views of stakeholders. Case Study used to provide additional information.




• Actions;

• Programme value.

Themes 3: Process (Efficiency and Effectiveness)

• Funding decisions;

• Funding split;

• Management processes.

Qualitative, based on views of stakeholders.

Theme 4: Sustainability • Dissemination. Qualitative, based on views of stakeholders.

Theme 5: Synthesis

• Overall assessment of the Programme

• The future.

Primarily qualitative, based on views of stakeholders, but also including a semi-quantitative assessment of the effectiveness of the Programme.

Figure 2.1 Evaluation framework The detailed questions developed for each of the evaluation themes is provided in Appendix 1. While it has been possible to take a quantitative approach to evaluating the training element of the co-operation framework, the nature of the other elements of the framework and the relative sparsity of quantitative data means that the evaluation has been largely qualitative. We have, therefore, made significant use of stakeholders’ perceptions. The information was gathered using a structured questionnaire, by e-mail, a telephone interview or a face-to-face interview. As much as possible, the information was collected in a common format to facilitate comparison and synthesis. Several of the e-mail surveys were followed up by telephone interviews and follow up questions were used in both face-to-face and telephone interviews to test and supplement answers provided in response to the structured questionnaire. Information provided by stakeholders was analysed using a matrix mapping approach. A table was produced listing the key evaluation issues and the responses of individual stakeholders were then mapped on to the evaluation issues as columns of the table. It was therefore possible to compare views by scanning across the matrix. Responses to issues tended to fall in to 3 categories:

• The view was held by many stakeholders. The view may have been expressed in different terms by different people, and some stakeholders stated their views more strongly than others but there was a general consensus over the issue.

• The views of stakeholders were split on an issue.

• One or two stakeholders held a strong view over an issue while others were relatively neutral.

We have not carried out a detailed statistical analysis of replies because of the relatively small size of the survey population, instead we have either reflected the above categories in our comments in the body of the report or referred directly to the number of stakeholders who have supported a stated view.




2.2 Assembly and Assessment of Initial Data

Various data sources were used as the basis for an initial assessment of the Programme. The team within the Commission’s Civil Protection and Environmental Accident Unit co-operated in providing much of this valuable information. A list of these information sources is shown in Table 2.1. Table 2.1 List of information sources Item Brief description Applicability to the

evaluation ‘Texts Relating to Community Cooperation on Marine Pollution’, Directorate General Environment, Civil Protection Unit, January 2001

Relevant Council Decisions between 1978 and 2000.

Detailed information about the evolution of the Commission’s involvement in marine pollution.

‘Council Decision of 23 October 2001 establishing a Community mechanism to facilitate reinforced cooperation in civil protection assistance interventions’

Details of the Community Mechanism.

Provides information to assess how the Community Mechanism fits with the Framework Programme.

‘Status Report on the Actions Undertaken in the Context of Decision 2850/2000/EC’ Working Document 5/2/1 Annex, 5th Meeting of the MCMP, October 2003

Summary of annual priority fields, actions supported, contracting party, funding provided, project dates, summaries of actions.

Basic information about the Programme outputs.

MCMP meeting documents Agendas, working papers, minutes etc for all 5 of the meetings held between March 2001 and October 2003.

Provide information about the workings of the MCMP including information about proposals and projects and the comments of Member States.

EC’s Marine Pollution web site Framework programme objectives and overview, texts and legislation, three-year rolling plan, Community Information System, contingency organisation, useful publications and reports, and international co-operation.

Background information about the programme; links to other regional conventions and agreements; links to project reports.

‘Evaluation of the Community Framework for Co-operation in the Field of Accidental Marine Pollution’, Peter Hayward Associates, March 1999

An evaluation of Community action in the field of marine pollution from 1994 to 1999 and a forward look to the proposed programme for 2000 to 2004.

Enables a comparison to be made of the results, impacts and trends between this evaluation and the previous one.

Helsinki Convention; Barcelona Convention; Bonn Agreement; Lisbon Agreement

Details of other relevant international conventions and agreements.

To assist in the assessment of the value that the Framework Programme adds to the existing regional conventions and agreements.

Evaluation forms from training and information projects

Feedback from participants. To assess the perceived value of the courses and workshops to the participants.

Procedure for letting contracts under the Framework Programme

Information about the process for calls for proposals, proposal evaluation and funding

To help in assessing the efficiency of the process of managing the Framework.

European Maritime Safety Agency (EMSA) web site

Summary of the role of EMSA. To help in assessing the objectives of the Framework taking account of the Agency.

European Maritime Safety Agency Work Programme 2004

The origin, work plan and future development of EMSA.

Enables the roles of EMSA and the Commission Programme to be compared.

‘Communication from the Commission to the Council and the European Parliament: towards a strategy to protect and conserve the marine environment’, CEC COM(2002)539 final, 2/10/2002

Description of the foundation upon which a thematic strategy for the marine environment can be built.

To help in assessing the objectives of the Framework taking account of the thematic strategy.




Item Brief description Applicability to the evaluation

‘The “Baltic Carrier” Oil Spill: Monitoring and Assessment of Environmental Effects in Gronsund (DK)’, Storstrom County, 2002

Report of the environmental assessment project part-funded by the Programme.

Used for the case study.

‘Accident of the Oil Tanker “Baltic Carrier” off the Danish Coastline’, Final Report of the European Task Force in Denmark from 1st to 5th April, 2001

Report of the mission and recommendations of the Task Force.

Used for the case study.

An initial assessment was carried out to provide an appreciation of the main areas of activity. Most of the funds have been allocated to projects within the agreed priority areas according to their eligibility and using an evaluation process based on well-defined selection criteria. Some additional funds have been provided for projects within the action areas but outside of a specific call for proposals. These include the funds provided for mobilisation of experts and invitations from other international organisations to co-finance training courses or conferences. The expenditure committed since 2000 as reported on the EC web site (Rolling Plan 2000-2002, updated 17/9/03) amounted to Euro 2,277,598 and has been divided as follows: The figure below shows the number of proposals received, projects for which funds have been committed and projects completed.

No. of Projects

176

4

2

% Funding Allocated

52%

35%

10% 3%

Training & Information

Pilot Projects

Support & Information

Mobilisation of Experts

34 28

20

13

23

75

40

10203040506070

No.

2000 2001 2002 2003

Year

Proposals and Projects

Projects completed

Projects for which fundscommittedProposals Received




Details of projects supported in the Rolling Plan are summarised on the EU web site and information is provided on the project title, contractor and budget (including EC contribution). The dates of courses, workshops and conferences are given where known and summary reports (in pdf format) of completed actions can be viewed or downloaded via links directly from the web site.

2.3 In-depth data collection

Face-to-face interviews were used to obtain in-depth information and it was agreed with the Commission that it would be appropriate to interview five country representatives in this way.

Countries were selected for the face-to-face interviews according to the following criteria:

• to give a reasonable geographic spread;

• to include some with recent spill experience;

• to include one with a significant involvement in actions under the Programme.

It had been hoped to include Spain as one of the selected countries because of their experience following the accident involving the Prestige. However, this did not prove possible and an alternative was substituted. The maritime countries not selected for the face-to-face interviews were sent the electronic version of the questionnaire and invited to complete it themselves. They were also offered the option of a telephone interview, if that was preferable. Other selected stakeholders were contacted by telephone. A separate questionnaire was developed to gather information for the Baltic Carrier Case Study. A full list of individuals interviewed and their organisations is provided in Appendix 2. More information about the questionnaires is given below. 2.3.1 Main questionnaire The questionnaire was aimed at the main Programme stakeholders, including the National Authorities within maritime Member States, Commission stakeholders, and project beneficiaries. The interview questions were based on the information requirements within the five themes identified in Section 2.1. Visits were made to five Member States (Denmark, France, Italy, Portugal and UK) for face-to–face interviews with National Authorities. Questionnaires were returned by e:mail from a further six countries (Germany, Greece, Ireland, Netherlands, Norway, Sweden) and a telephone interview was conducted for Belgium. It was not possible to obtain feedback from the remaining two countries contacted.

Further telephone discussions were held with:

• a project beneficiary in France

• the European Maritime Safety Agency

• the International Tanker Owners Pollution Federation

• the International Maritime Organisation.

In addition, discussions were held with Commission officials within the Civil Protection and Environmental Accident Unit.




2.3.2 Case Study questionnaires Questionnaires were also developed for the various stakeholders interested in the Commission’s involvement with the Baltic Carrier incident ie Danish Authorities, Task Force Members, Environmental Effects project beneficiary, and Member States. The questionnaires are shown in Appendix 3. Replies were received from representatives of all these stakeholders.




3 Findings

The findings have been evaluated according to the five themes within the evaluation framework. Particular weight has been given to the comments from the Member State representatives and their feedback and views are discussed in some detail below. At appropriate places, input from other stakeholders and comments based on our own observations are included. The Baltic Carrier case study has been used to provide a specific focus for views and opinions. In each theme area, the key points and recommendations are summarised at the end of the section.

3.1 Overview and Context

3.1.1 Programme objectives The Programme objectives are generally well-understood in a broad sense although not necessarily expressed in the same way as the Commission. Many stakeholders recognise the role of the Programme in promoting co-operation between Member States, exchanging information and developing skills, particularly through provision of training. The co-ordinating role of the Commission during incidents and the mobilising of the Task Force were mentioned specifically by some Member States. 3.1.2 Interactions of the Programme with other Programmes The work within the Commission Programme is consistently perceived as complementing that at national and trans-national levels. Various specific trans-national organisations or activities were mentioned by Member States as having relevance to the Programme. Other more or less formal arrangements with neighbouring countries were also mentioned. Some overlap of objectives with regional agreements, in particular, is acknowledged but duplication of effort with the Programme is not seen as a current problem. Nevertheless, there is a need for ongoing dialogue to ensure continued complementarity with the regional mechanisms. European Maritime Safety Agency (EMSA) 6 Member States have expressed uncertainty about the role of EMSA and have raised concerns about the need to avoid duplication. The general perception is that the role of EMSA is continuing to evolve and that care should be taken to avoid EMSA repeating work already carried out elsewhere and relearning lessons already well-understood. Officials within both EMSA and the Commission are clear that their roles are different and complementary and there seems to be a strong commitment from both parties to maintaining their complementarity. EMSA is concerned mainly with at-sea operational matters and has a focus on prevention of accidents. The Programme is concerned with response methods and co-operation, communication and co-ordination between countries. Nevertheless, there appears to be the potential for some overlap in identifying and spreading best practice and in mobilising expertise during incidents. Good communication between EMSA, the Commission’s Unit and other stakeholders will continue to be essential to avoid future issues. It has been suggested by one stakeholder that the most effective arrangement may be to bring the MCMP and the Programme under the umbrella of EMSA. However, EMSA is not




constituted in a way that allows the range of activities possible within the Framework Programme. 3.1.3 Barriers to co-operation Co-operation between Member States is generally felt to be very good and this is confirmed by the experience during incidents. Interacting with representatives of other Member States at MCMP meetings is valued by some, although others feel that more could be made of the opportunity. It is recognised that meetings at other fora also help co-operation. A concern was raised about a perceived lack of clarity under the Community Mechanism as to who will pay for what in the event that experts or resources are provided by one Member State to another during an emergency. With the involvement of new countries, it is particularly important to avoid this issue becoming a barrier to co-operation. 3.1.4 Gaps in coverage No comments or concerns were expressed about the overall scope of the Programme. Some stakeholders mentioned specific areas that they feel should be addressed or given more attention. These included:

• Funding of port state control activities; • Surveillance of illicit/accidental spills; • Development of better equipment and technology for oil spill response; • Response to hazardous and noxious substances; • Review of procedures on board vessels; • Standardisation of procedures for cost recovery for equipment provided to other

Member States and consistency in obtaining compensation from P&I clubs.

Most of these areas can be considered along with other suggestions when reviewing priorities within the three-year rolling plan. The cost recovery issue that was also raised by a different Member State as a potential barrier to co-operation (see Section 3.1.3) could be a topic for discussion at a future MCMP meeting and could include a representative from ITOPF or a P&I club.

Overview and Context Key Points • The Programme objectives are well-understood by stakeholders;

• The Programme complements activities at national, trans-national and regional levels;

• There are concerns from Member States about the evolving role of EMSA and the potential for duplication of effort;

• The level of co-operation between Member States is recognised as being very good;

• There is a perceived lack of clarity about the mechanisms for payment when resources are provided by one Member State to another during an incident.

Recommendations for Action • Ensure continuing involvement and dialogue between the Unit and officials

in EMSA and the Regional Agreements to avoid duplication;

• Discuss the issue of mechanisms for payment for resources provided by one Member State to another at a future MCMP meeting;




• Consider the list of identified gaps in coverage of the Programme when next reviewing priorities in the three-year rolling plan.

3.2 Impacts

3.2.1 Programme successes Actions in Training and Information are highlighted by 5 Member States and are particularly valued by the smaller countries that do not have their own national training programmes, thus indicating additionality brought by the Programme funding. Various aspects of the Commission’s role during incidents are mentioned by a further 5 Member States. The Prestige is viewed as particularly successful for the part played by the Commission, and several areas of support at incidents are valued:

• Co-ordination of offers of assistance; • Co-ordination of observers; • Encouraging co-operation between Member States; • Role of the Monitoring and Information Centre; • Identifying lessons learnt.

Other areas of the Programme mentioned by Member States include facilitating of networks, pilot projects, workshops, exchange of experts between Member States and harmonising procedures in prosecution of MARPOL offenders. 3.2.2 Added value The Programme is widely perceived as adding value to the regional agreements and is recognised as unique in supporting co-operation at a European level. If it did not exist, the consensus seems to be that the regional and trans-national agreements would be the next best instruments for promoting co-operation. However, these agreements are not set up to provide funding for Actions in the same way as the Commission Programme and each one covers only a few countries. The Training and Information actions are viewed as providing particular added value. 3.2.3 Programme shortcomings In one case it is felt that different standards of preparedness exist in different regions and that requirements from the Programme therefore also vary. The implication is that it is not possible for individual actions within the Programme to be of equal value to all Member States. This is almost inevitable and, indeed, is likely to become more exaggerated as some less-experienced countries become more involved in the work of the Programme. However, the Programme has an important part to play in spreading good practice across all countries and raising standards within the least experienced. It therefore has to be accepted that the best-equipped and most experienced countries may not benefit directly to the same extent as some others. The number of regulations within the Commission and the strict procedures are seen as slowing down some activities. 3.2.4 Value for Money Most stakeholders feel that the Programme is providing good value for money. Some reservations have been expressed about the value for money from a few training courses where the quality has been poor.




One stakeholder would like the benefits of the Programme to be critically assessed before judging the value for money. This mid-term report provides some indication of the perceived benefits. However, it is also recommended that more effort is spent in evaluating and disseminating the outcomes and benefits of individual actions. 3.2.5 Community Information System Views on the CIS vary widely. Some stakeholders value it highly and consider it to be a very useful tool during emergencies. Others feel it has value in exchanging information that helps in emergency planning. One stakeholder considers it to be a tool for the general public rather than for national authorities and experts. Several stakeholders mention other tools that provide information for emergencies or explain that they would refer first to their regional agreements rather than consulting the CIS during emergencies. There is a view amongst some stakeholders that the CIS is not regularly reviewed, updated or improved. Those Member States that are less convinced about its value find it time-consuming and burdensome to keep updated. Two Member States proposed that it would be preferable to have a web site that contained links to other relevant and useful sites. The majority of Member States do hold positive views about the CIS and it is clearly used regularly and helpful to some. However, each Member State has responsibility for its own site and there may continue to be difficulties in persuading those Member States that do not value the CIS to spend the time in updating their sites. On balance we believe that the CIS is a tool that should be retained, particularly if improvements can be made to the updating process. It is therefore recommended that the Commission should consider ways of convincing all Member States of the value of maintaining their national pages. There is also scope for the Commission to make improvements to those pages for which it is responsible. The potential benefits include:

• Updated authoritative information on all sites; • Greater consistency of information; • Improved ease of use; • Identification and dissemination of best practice related to emergency response; • Extension of functionality eg inclusion of links to other useful web sites. 3.2.6 Actions in Training and Information Almost all Member States appreciate and value the training supported by the Programme and there is a consistent view that it is an extremely important way of informing and developing individuals and thus indirectly improving national approaches to prevention and intervention. Training and workshops are also seen as a very useful way of building links between individuals and Member States - the Catania workshop (organised in order to review the Prestige incident) was specifically mentioned as valuable in this regard. Workshops generally have a greater potential to influence national authorities than training courses. Appendix 4 provides some details from our analysis of course evaluation forms. The feedback from participants is generally very positive (although it is accepted that participants who have not had to pay to attend may be less critical). In particular, the key ratings for quality of trainers and overall ratings for the courses are high. This finding is at odds, to some extent, with the perceptions of some Member States that have commented on the variable quality of training courses. It seems possible that




positive markings on the course evaluation forms may not always be reflected by participants’ comments on their return to their home base. One suggestion is that the Commission should send a representative to every course and the representative should remain throughout its duration to make an independent assessment of its quality. This approach has the potential to drive up standards but clearly there is also a cost implication for the Commission. One stakeholder perceives that there is an expectation that the available budget for actions should be spread around Member States and that this sometimes compromises quality. They cite a specific example where it is asserted that a well-respected and well-known organisation would be the best choice to run a basic course on oil pollution but they have not been selected. The Member State representative of the country in which the organisation is located has commented in a different way. Their concern is that there are no funds available within the host country to co-fund training and that the organisation is therefore not able to respond to Calls for Proposals. It is assumed that it is not possible for the Commission to fund training courses at the 100% level. The alternative of making up the additional funding required by charging participants to attend is not attractive, particularly as the smaller countries might then feel inhibited from sending representatives. The recommended approach is therefore to focus on improving the quality of courses and for the Commission to be more proactive in reviewing course objectives, course content and standard of presenters. Feedback and impact assessment of training and information can also be improved and it is recommended that the longer-term impact of the training is assessed. It is proposed that the contractor is required to obtain feedback some 3-6 months after the course and to ask participants to evaluate changes resulting from the training. A short questionnaire should be devised for this purpose. The difficulty of obtaining such feedback from participants coming from many different countries is recognised but there are various approaches that could be adopted to facilitate this. All feedback should be reviewed and action taken as appropriate to improve the quality of future courses. 3.2.7 Pilot Projects Only six pilot projects have been supported by the Programme and only three have been completed with final reports available. Care must therefore be taken in drawing conclusions from this small sample size. “The Certification of Oil Spill Response Equipment” project was mentioned by 2 Member States as likely to be of particular value when purchasing new equipment. One beneficiary Member State commented that the pilot projects provide the greatest value for the Member State that proposes the project because they enable know-how and technical capacity to be built up and lessons can be learnt directly. The Programme funding can be essential in enabling such projects to take place. Another stakeholder felt that one of the projects, in particular, did not appear to be of general relevance. At this stage, the Pilot Projects do not generally appear to be leading to changes in strategy at a national level but this will be a matter to keep under review as projects progress. Part of the assessment of Pilot Project proposals should include their general relevance within Member States and for the findings to be widely replicated. Some limitations have been identified in the evaluation and dissemination of key information from Pilot Projects (and other actions). For example, it would be useful for Member States to have more opportunity to discuss the findings from projects and for the key benefits and lessons of general importance to Member States to be distilled from the work. In some cases, it would be valuable to follow-up the project in, say, 6 or 12




months time to establish what further developments have taken place and the implications for Member States. 3.2.8 Support and Information Actions These actions cover a wide range of activities within the current headings of:

• “Environmental Impact” (1 project carried out in the aftermath of the Baltic Carrier incident and discussed later as part of the Case Study);

• “Conferences and Events” (3 conferences for which the Programme provided less than 30% of the total funding);

• “Other Support Actions” (1 pilot project and this mid-term assessment project).

It is apparent that specific activities within this action area are not readily brought to mind by the Member States. The following may explain the reasons for this:

- the non-specific title of the action area - the disparate nature of these projects - the lower funding profile of the Commission in its support for conferences

Most Member States have not commented clearly on these actions but have generally responded positively in the expectation that the action will lead to enhanced co-operation. A Finnish municipality heard about the Baltic Carrier environmental impact project and was keen to receive a copy of the report, indicating its perceived value outside the host country. 3.2.9 Mobilisation of Expertise through the Community Mechanism Not all Member States have experience of this aspect but the comments from Member States are generally very positive about the principle of mobilising experts to assist during incidents and the role of the Commission in co-ordinating this activity. The continuous operation of the Monitoring and Information Centre and the wider reach of the Commission to countries that are not accessible by individuals through their regional agreements are particularly valued. The possibility of sending observers to incidents was seen as extremely valuable and the co-ordinating role of the Commission in this and the other areas is also found to be very useful and effective. However, there is considerable confusion surrounding this area within some Member States for the following reasons:

• Uncertainty about the continued existence of the Task Force; • Uncertainty about what is covered by the Community Mechanism and what remains

directly within the Framework Programme in this area; • Issues about who pays for what (see Section 3.1.3); • Lack of broad objectives or terms of reference for experts or observers at incidents.

Some stakeholders were particularly interested in discussing experiences during specific incidents and held some interesting views, as outlined below:

Prestige – The Commission’s wider access to resources was useful. However, the request for equipment did not give enough detail about requirements and all offers were taken up by the host country without consideration of their suitability. Perhaps an expert could have assisted in the Monitoring and Information Centre during the incident. The Commission requested nominations for a panel but it was unclear whether the request was for experts or observers or both.

Baltic Carrier – Again, there was some uncertainty about whether the Commission was seeking nominations for an expert group or an observer group or both. The observer group included some individuals with spill experience who were keen to share their idea




and thoughts with the response authorities. This was presumably beyond their remit but may not have been made clear to them. It is understood that the recent joint meeting between the Civil Protection and Marine Pollution committees (26 January 2004) has identified some specific practical procedures for mobilising experts and observers and this should help to resolve some of the issues identified here. Bearing in mind that the joint meeting has already addressed some issues, it is recommended that the Commission reviews whether there is a need to provide Member States with the opportunity to discuss the Community Mechanism further in order to ensure a common understanding of the operation of the Community Mechanism and related activities and to identify possible areas for improvement. A workshop could be a suitable forum for such a discussion, if still needed. 3.2.10 Programme Value The responses indicate that the stakeholders are very positive about the value of the Programme. A range of aspects are identified as “most useful” but training, exchange of information and enhancing co-operation are each mentioned by several stakeholders. Some stakeholders view all aspects as equally useful and are reluctant to pick out any specific aspects! A very small number of aspects are identified as “least useful”. Most useful aspects include:

• Training and workshops

• Developing co-operation

• Raising awareness of people and resources

• Monitoring and Information Centre

• Meetings of the MCMP Least useful aspects flagged:

• CIS

• Imbalance between Member States benefiting by hosting funded actions

Impacts Key Points • The Programme is perceived as adding value to the regional agreements and

is recognised as unique in supporting co-operation at European level;

• Many key areas of the Programme have been identified as “most useful” by different Member States; the most commonly selected are training and workshops, exchange of information and enhancing co-operation between Member States;

• Views on the value of the Community Information System are highly polarised but some Member States do use it regularly and find it helpful;

• Actions in Training and Information are highly valued, particularly by smaller countries that do not run their own national training programmes;

• Feedback from participants attending training courses is generally very positive but several stakeholders have commented on the perceived variable quality of the courses;

• Stakeholders are positive about the value of Pilot Projects but it is too soon to draw clear conclusions on their impact and value for money;




• Support and Information Actions are not well-recalled by stakeholders;

• Stakeholders are generally very positive about the Mobilisation of Experts and the co-ordinating role of the Commission. However, there is considerable confusion about some aspects, perhaps because this action is now supported by the Community Mechanism.

Recommendations for Action • Consider ways of bringing about improvements to the Community

Information System;

• Increase intervention to improve overall quality of courses;

• Ensure feedback from a higher proportion of training course participants and review; assess the longer-term impacts of training;

• When assessing Pilot Project proposals, ensure the general relevance of each project to Member States is fully considered;

• Review the outcome of projects and distil the key benefits and lessons of general importance to feed back to Member States; follow-up projects 6-12 months after completion to establish and report any further developments;

• Revise listings of actions within the various action areas (see also Section 3.5.2);

• Review the need to hold a workshop or other forum to discuss the Community Mechanism.

3.3 Process (efficiency and effectiveness)

3.3.1 Funding Decisions Member States correctly perceive their role within the MCMP as deciding the priorities whilst the Commission assesses the financial aspects according to its administrative arrangements. Two Member States commented specifically about the current proposal assessment process and would like the MCMP to have more influence in evaluating proposals. In general, stakeholders believe that the financing decisions are soundly based although it is felt that funding constraints mean that some worthy projects cannot be supported. One stakeholder has questioned the rationale for sending a Task Force to the Galapagos and the selection process for the Task Force. 3.3.2 Funding Split The budget is not separately allocated to each type of action but is distributed according to the agreed priority areas as discussed within the MCMP. This has resulted in the funds being spent as follows (see also Section 2.2):

Type of action Percentage of total funds spent Training & Information 52% Pilot Projects 35% Support & Information 10% Mobilisation of Experts 3%




Stakeholders are generally content with the split of funding and there are no issues with Training & Information receiving the majority – indeed, one stakeholder suggests this action could be increased to 65% at the expense of Pilot Projects. One stakeholder has suggested that more funding could be made available for Mobilisation of Experts (again at the expense of Pilot Projects) and another suggests that there should be a special ‘crisis’ budget for this Mobilisation of Experts. 3.3.3 Management Processes Several stakeholders comment about the lack of evaluation, feedback on benefits or critical appraisal of the outcomes of projects (see also Section 3.2.7). One suggestion is for the Commission to report in between meetings on the results and recommendations of training and projects to allow the MCMP meetings to focus more on strategic planning. Two stakeholders would specifically like to see improvements in the management of quality assurance and would like the MCMP to be more proactive in following-up on reports. Several comments are made about the strict procedures that have to be followed within the Commission, leading to a high workload for the staff within the Unit and slowing down progress. However, it is also noted that administrative aspects such as issuing of minutes and notification of meetings work well. The good contacts that exist between staff in the Unit and the MCMP are valued. One stakeholder feels that the role of the MCMP is not well defined and would like a clear definition of competences and rights. There are strongly held feelings about the specific issue of the length of notice given for training courses and workshop and the difficulties for Member States when the notice period is short. It is recommended that the Commission should agree with Member States the minimum acceptable time to be given for nominating participants for courses. A similar potential issue for course organisers is also identified whereby they are given a short timescale for organising a course and that can result in them having to bear higher costs. It has also been suggested that greater clarity could be given on the type and level of training being provided, its objectives and the required experience/interests of the participants. This information should always be provided. It is recommended that the detailed procedures for letting training contracts and inviting nominations from Member States are reviewed. Three Member States would like longer meetings of the MCMP and two of these suggest more two-day meetings. Longer meetings would give more time:

• To consider important issues; • For the MCMP to give its views; • For reports to be presented and discussed; • For building relationships between Member State representatives.

It is recommended that the Commission seeks views more formally about the support from Member States for more two-day meetings. The MCMP could be involved in identifying and agreeing specific topic areas for more detailed consideration and discussion on Day 2.

Process (efficiency and effectiveness) Key Points • Financing decisions are considered to be soundly based;

• Some stakeholders would like the MCMP to have more influence in the evaluation of proposals;




• Stakeholders are supportive of the majority of funding being provided for training actions.

Recommendations for Action • Review the detailed procedures for letting training contracts, particularly to

ensure adequate notice of training course dates can be given both to Member States and contractors;

• Obtain views from all Member States about their potential support for more two-day meetings of the MCMP.

3.4 Sustainability

3.4.1 Dissemination Views seem to be equally split between those who are happy with project dissemination, those who do not seem to be aware of dissemination activities and those who would like to see improvements. Specific suggestions for improvement include:

• Better use of the web site;

• Executive summaries written to a common format giving lessons learnt and benefits (see also Section 3.2.7);

• Action plan following a final report (next steps, MCMP recommendations for follow-up);

• Emphasis on high quality of reports that are well-written and distributed to Member States in hard copy;

• Wider dissemination of procedures, manuals, rules etc that result from projects.

There is a suggestion that the MCMP is sometimes too politically correct, hinting at an unwillingness on the part of the Committee members to criticise work carried out in another Member State. Dissemination of project outcomes and recommendations for ways in which stakeholders can make use of the findings is extremely important if full value is to be obtained from funding invested. This is an aspect of the Programme that should be kept under regular review. It is therefore recommended that the Commission considers:

• Providing detailed guidance to contractors on delivery of a project summary in a standard format to ensure conclusions, benefits and lessons learnt are clearly identified and set down. This is particularly important for pilot projects but can also be required in modified form for other actions;

• Ensuring there is adequate time at MCMP meetings to discuss completed projects (it is noted that action has already been taken by the Commission in this area and that time has been allocated for such a discussion at the meeting to be held on 1st March 2004);

• Identifying follow-up actions and a procedure to ensure actions are closed out.

Sustainability Key Points • Several stakeholders feel there is a lack of evaluation, feedback on benefits




or critical appraisal of the outcomes of projects;

• Views are divided on the effectiveness of dissemination of project findings.

Recommendations for Action • Allocate resources to improving the quality, dissemination, and follow-up of

project deliverables.

3.5 Synthesis

3.5.1 Overall Assessment of the Programme 6 stakeholders gave a percentage figure for the extent to which each felt the Programme has met its objectives. The figures varied from 70% to 95% and averaged 80%. These values indicate that the Programme is perceived as successful but with some scope for improvement. The most effective of the action areas are seen as:

• Training and information • Mobilisation of experts • MIC • Observer missions • CIS • Building relationships between Member States (and others) • Coordination activities • Exchange of experts (based on previous experience before the Framework

Programme)

Once again, training is judged to be particularly successful and was selected by several stakeholders. 3.5.2 The Future All stakeholders consider the work carried out within the Programme is still necessary and several are very definite about this. However, 4 stakeholders raise questions about the future role of EMSA and the need to avoid overlap and duplication. It is emphasised that operational activities are the responsibility of the Member States and that the Programme and EMSA should provide only co-ordination support during incidents. One stakeholder believes the Programme to be necessary but would welcome a critical analysis of the benefits to be certain. Another mentions the importance of the Programme as a focal point in Europe, particularly with the involvement of new countries. There is a long list of key areas identified by the stakeholders to address next, adding further weight to the perceived value of the Programme. Many of these topics have already had some coverage within the Programme but stakeholders would like more. The list covers:

• Actions in training and information – courses, workshops and exercises; • More basic training; • More training on anti-pollution matters for decision-makers; • Pilot projects; • Environmental impacts; • Exchange of experts; • Surveillance by aircraft; • Use of satellite observation;




• Certification on new anti-pollution equipment; • Detection of chemical spills; • Response to hazardous and noxious substances; • Pollution from offshore wind farms; • Workshops on recent incidents, response activities, difficulties encountered, lessons

learnt. • Improved system of collection of evidence of illicit discharges to assist in judicial

cases and in support of international conventions as well as European regulators;

A number of these areas are worthy of consideration at the next review of priority areas within the Programme. Some stakeholders suggest ways in which the Programme should be improved in future. One proposition is to hold a workshop on the topic of ‘The EU Programme: how to proceed and how to improve’. On a similar theme, another suggestion is to review the types of action within the Programme with involvement and comment from the MCMP. Other suggestions for improvement include:

• Better dissemination and promulgation of lessons learnt; • Enhanced web site; • More effective participation of Member States.

The matter of dissemination and promulgation of lessons learnt is a recurring theme and recommendations in this area are made in Section 3.4. Improvements to the web version of the CIS are discussed in Section 3.2.5. There is scope for improvement to the Commission’s marine pollution web site more generally and it is recommended that the site is reviewed with a view to updating its content, providing more links to other web sites and improving user-friendliness. Lack of time in MCMP meetings appears to be the main limitation to the full participation of Member States. More two-day meetings as suggested in Section 3.3.3 are an option to reduce this problem, provided the majority of Member States are prepared to commit the time.

Synthesis Key Points • The Programme is perceived as effective in meeting its objectives but with

scope to improve;

• Stakeholders are positive that the Programme continues to be necessary;

• There is uncertainty about the activities to be carried out within EMSA and concern to avoid overlap and duplication;

• Stakeholders would generally like the focus on Training and Information actions to continue in future;

• There is a healthy list of topic areas that stakeholders would like to be addressed in future within training and other actions;

• The current actions and coverage of the Programme are supported and there are no suggestions for major changes;

• Some possible improvements to the Programme have been identified. Recommendations for Action




• The Programme should continue in its current format;

• Potential improvements should be prioritised for action and implementation, taking account of resources available;

• Improvements to the Commission’s marine pollution web site should be considered.

3.6 Comparison with Previous Evaluation

The most recent previous evaluation was completed in 19991 and covered three periods of Community action:

• An ex post evaluation of the Programme from 1994-1997; • An intermediate evaluation of the ongoing Programme for 1998-1999; • An ex ante evaluation of the proposed Programme from 2000-2004.

The action areas within the Programme from 1994 onwards were similar to the current ones and support was provided for training and seminars, pilot projects, a conference, a programme to exchange experts between countries and a post-spill survey. Many of the findings were similar to those reported here:

• The Programme was relevant to the needs of Member States; • The Programme was generally considered to have met its objectives; • Most Member States considered training to be the most valuable action; • Opinions on the usefulness of the CIS varied widely (at this time, a web-based

version of the CIS was not available).

Various recommendations were made, some of which have been addressed by the Commission, notably:

• ‘Focus on the development of more specialised training courses’ – there have been courses supported by the Programme covering most of the specific areas mentioned;

• ‘Involvement of professionals from those maritime states seeking accession to the EU’ – Norway and Iceland have attended MCMP meetings and were invited to training courses and Estonia, Hungary, Lithuania and Slovenia attended the MCMP meeting in October 2003 for the first time;

• ‘Papers should be sent out in good time for meetings’ – no adverse comments were received and one Member State has specifically commented that issuing of minutes and notification of meetings works well.

One recommendation that was made previously and that appears again here is the need for more discussion of the results of pilot projects and ensuring their effective dissemination.

1 Peter Hayward Associates, ‘Evaluation of the Community Framework for Co-operation in the Field of Accidental Marine Pollution’, London UK, March 1999.




4 Baltic Carrier Case Study

The following activities funded or co-ordinated by the Commission were carried out under the Framework Programme during the Baltic Carrier incident:

• Mobilisation of experts • Mobilisation of observers • Project on Monitoring and Assessment of Environmental Effects in Gronsund

The incident therefore provides an opportunity to assess some very specific examples of key areas of activity within the Programme and to obtain feedback from the range of stakeholders involved in the various different actions. The purpose of this case study was therefore to evaluate in more detail some aspects of the Programme that were investigated in a more general way during the main assessment. The case study is not intended to review the incident itself in any detail but only to assess the impact of the Programme actions. To this end, questionnaires were devised to obtain the perceptions of stakeholders as follows:

• Danish authorities (Danish Emergency Management Agency – DEMA); • Task Force members; • Environmental Effects project leader (Storstrom County, Aquatic Environmental

Division); • Member States.

These questionnaires are reproduced in Appendix 3. Responses were obtained from representatives of each of these stakeholders, although only three Member States completed and returned the questionnaire. No questionnaire was prepared specifically for the observers as their role only emerged during the course of the assessment. Nevertheless, the views of one observer were obtained via a Member State questionnaire and a follow-up telephone discussion. The following sections provide a very brief background to the incident and the actions together with the key points made by the stakeholders.

4.1 Background to the Baltic Carrier Incident and Actions

On March 29th 2001 the tanker Baltic Carrier collided with the freighter Tern in the Baltic Sea between Denmark and Germany. Approximately 2,400 tonnes of heavy fuel oil were released into the sea and, although a significant amount of oil was recovered at sea, there was considerable pollution of the islands and coastline in the Gronsund area. The clean-up operations extended well into 2002 until they were officially ended in April 2002. The oil spill was the largest in Danish history. Soon after it became apparent that Denmark was responding to a major incident, the Commission offered help to the Danish Authorities (DEMA) and, with their agreement, issued a call for experts from amongst Member States to form a Task Force to visit and assist the Authorities. At around the same time, it was agreed with the Authorities that they would be prepared to host observers from Member States and the Commission therefore issued a call for nominations of observers.




The Task Force was made up of three experts who visited Denmark between 1st and 5th April 2001. They were hosted by DEMA and visited 10 different sites at which they offered suggestions and advice related to collection of oil in near-shore waters, shoreline clean-up and treatment of oily waste. Soon after their return, the Task Force issued a report of their activities. This is available on the Commission’s web site. The observers group was made up of representatives from a significant proportion of Member States, predominantly those from countries to the north of Europe. They were briefed by the Danish Authorities, provided with maps and assigned a co-ordinator to accompany them on visits. They were taken to see a forward control base, a polluted salt marsh, a waste handling facility and a vehicle clean-up operation. Storstrom County was the authority responsible for monitoring of the affected coastal waters and shoreline. After the accident, the Aquatic Environmental Division of Storstrom County worked with the National Environmental Research Institute (NERI) and Roskilde University (RUC) to devise a monitoring programme. The study was approved by the Commission for part-funding (Euro 97.3k, 49% of the total project cost) from Framework Programme funds in July 2001. The study investigated the ecological impact of the spill and its economic implications. It also evaluated the clean-up operation, the monitoring programme and the environmental effects of the spill. A final report of the study was published in 2002 and is available on the Commission’s web site.

4.2 Mobilisation of the Task Force

Table 4.1 summarises the views and perceptions of the various stakeholders. It must be recognised that the incident occurred nearly 3 years ago and some recollections of details related to the work of the Task Force will doubtless have faded. Nevertheless, there are some useful and interesting comments. The objectives of the Task Force as perceived by the Danish authorities and the Task Force itself are significantly different. This could be an artefact of the passage of time and difficulty in remembering but, if a real difference existed, this would be of concern. It does appear there is a need for clarification about the role of experts, as suggested by a Member State not involved with the Task Force at the Baltic Carrier incident. Related comments about the role of observers described in Section 4.3 reinforce the need for such a discussion. In addition, it would facilitate the mobilisation process and provide the potential for greater acceptance of expert views if countries could commit to the inclusion within contingency plans of some positive reference to hosting experts (and observers) during spill incidents and listening to advice given. Clearly there are some significant issues to debate and resolve but there could be clear benefits to all parties if there was such a commitment. Although the final report of the Task Force is available from the Commission’s web site, it appears that the Member States responding to the questionnaire are not aware of (or have forgotten) this fact. Clearly the experiences and lessons learnt from any incident will be felt and retained most clearly by the affected country and by the members of any Task Force sent to assist. Nevertheless, it should be of considerable value to other Member States to hear about those lessons that are generally applicable. Wherever possible, adequate time should be devoted in MCMP meetings or in other fora to provision of such feedback. The workshop arranged recently in Catania in the aftermath of the Prestige incident is a good example of one approach that appears to have been very much appreciated by the MCMP.

AEAT in C

onfiden

ceAEAT/E

D50287/F

inal

Rep

ort

M

id-t

erm

eva

luat

ion o

n t

he

imple

men

tation o

f th

e fr

amew

ork

for

co-o

per

atio

n

in t

he

fiel

d o

f ac

ciden

tal or

del

iber

ate

mar

ine

pollu

tion’ – F

inal Rep

ort

AEAT in C

onfiden

ce

AEA T

echnolo

gy

27

Tab

le 4

.1 F

eed

back

on

th

e M

ob

ilis

ati

on

of

Exp

ert

s at

the B

alt

ic C

arr

ier

Inci

den

t

Dan

ish

Au

tho

riti

es

Task

Fo

rce M

em

ber

Oth

er

Mem

ber

Sta

tes

Under

standin

g o

f th

e pro

cess

for

iden

tify

ing a

nd m

obili

sing e

xper

ts

One

does

under

stan

d t

he

pro

cess

, one

does

n’t a

nd o

ne

does

but

would

fin

d a

n

outlin

e use

ful. T

her

e co

uld

be

a sk

ills/

exper

ience

data

base

main

tain

ed

to iden

tify

and s

elec

t ca

ndid

ate

s to

re

pre

sent

the

Tas

k Fo

rce

duri

ng

inci

den

ts.

Obje

ctiv

es o

f th

e Tas

k Fo

rce

To s

uper

vise

and g

ather

info

rmat

ion f

or

futu

re inci

den

ts.

To d

eter

min

e w

ays

in w

hic

h t

he

EC

could

hel

p t

he

Dan

ish a

uth

orities

to

min

imis

e th

e dam

age

from

the

spill

, to

as

sist

in t

he

reco

very

and t

o s

upport

fu

ture

work

ass

oci

ated

with t

he

spill

.

Wer

e th

e obje

ctiv

es a

chie

ved?

DEM

A w

as s

atisf

ied w

ith t

he

Tas

k Fo

rce

report

. H

opef

ully

yes

. D

ifficu

lt t

o p

rovi

de

tech

nic

al ex

per

tise

duri

ng r

esponse

oper

atio

ns

when

the

auth

ori

ties

had

ve

ry little

tim

e to

hea

r outs

ide

advi

ce.

Adeq

uate

bri

efin

g a

nd a

dm

inis

trative

su

pport

and c

o-o

per

atio

n f

rom

the

Com

mis

sion a

nd t

he

Dan

ish a

uth

ori

ties

D

efin

itel

y ye

s to

all

aspec

ts.

Any

feed

back

pro

vided

about

the

work

of

the

Task

Forc

e?

Ver

y lim

ited

am

ount

pro

vided

.

Any

spec

ific

contr

ibution o

f th

e Tas

k Fo

rce

to t

he

clea

n-u

p o

per

ations

The

Task

Forc

e gav

e ad

vice

. Sev

eral

asp

ects

incl

udin

g d

iscu

ssio

n o

f m

echan

ical

rec

ove

ry t

echniq

ues

, build

ing a

tem

pora

ry s

tora

ge

site

and

eval

uation o

f th

e ex

tent

of

the

pollu

tion.

Not

gen

erally

aw

are.

One

Mem

ber

Sta

te

knew

that

the

Tas

k Fo

rce

bro

ught

exper

ience

of

resp

onse

to s

pill

s of

hea

vy f

uel

oils

.

Wer

e an

y re

com

men

dations

imple

men

ted?

All

advi

ce w

as w

elco

med

on h

igh

visc

osi

ty H

FO.

How

ever

, th

e only

sp

ecific

pro

posa

l im

ple

men

ted

conce

rned

a w

arnin

g f

rom

the

Tas

k Fo

rce

about

a la

rge

oil

slic

k th

at w

as

about

to d

rift

aw

ay w

ith t

he

tide.

As

a re

sult,

the

oil

was

conta

ined

.

The

Task

Forc

e dis

cuss

ed e

nvi

ronm

enta

l m

onitori

ng a

t so

me

length

and,

subse

quen

t to

the

main

oil

reco

very

oper

atio

ns,

a r

esea

rch p

rogra

mm

e w

as

initia

ted b

y th

e au

thori

ties

.

AEAT in C

onfiden

ceAEAT/E

D50287/F

inal

Rep

ort

M

id-t

erm

eva

luat

ion o

n t

he

imple

men

tation o

f th

e fr

amew

ork

for

co-o

per

atio

n

in t

he

fiel

d o

f ac

ciden

tal or

del

iber

ate

mar

ine

pollu

tion’ – F

inal Rep

ort

AEAT in C

onfiden

ce

AEA T

echnolo

gy

28

Will

the

exper

ience

and les

sons

lear

nt

by

the

Task

Forc

e re

sult in m

ore

ef

fect

ive

inte

rven

tion in o

wn/o

ther

co

untr

y?

Co-o

rdin

ation b

etw

een r

esponsi

ble

au

thori

ties

has

impro

ved in D

enm

ark

thro

ugh b

ette

r co

ntingen

cy p

lannin

g.

For

exam

ple

, th

ere

are

now

pre

-ap

poin

ted p

lace

s fo

r te

mpora

ry s

tora

ge

of

oily

was

te.

(Note

: it is

not

clea

r th

at

this

im

pro

vem

ent

has

res

ulted

direc

tly

from

the

Tas

k Fo

rce

invo

lvem

ent)

.

Yes

. Le

ssons

lear

nt

are

alw

ays

use

ful

for

impro

ving p

repar

ednes

s in

the

affe

cted

countr

y an

d m

ore

wid

ely

thro

ugh info

rmat

ion f

rom

the

Tas

k Fo

rce.

The

exper

ience

has

bee

n

par

ticu

larly

use

ful fo

r th

e co

untr

y su

pply

ing t

he

exper

ts.

None

can c

om

men

t sp

ecific

ally

bec

ause

not

awar

e of

less

ons

lear

nt

from

this

in

ciden

t. H

ow

ever

, one

Mem

ber

Sta

te is

very

support

ive

of

the

pri

nci

ple

of

additio

nal ex

per

tise

and a

dvi

ce f

rom

the

Tas

k Fo

rce,

pro

vided

the

nat

ional

au

thori

ties

inci

den

t co

mm

and s

truct

ure

is

gea

red t

o r

ecei

ve a

nd d

eal

const

ruct

ivel

y w

ith t

he

advi

ce –

could

be

cove

red w

ithin

contingen

cy p

lans.

Anoth

er M

ember

Sta

te is

stro

ngly

in

favo

ur

of

the

use

of

spec

ific

exp

ertise

fr

om

countr

ies

that

hav

e par

ticu

lar

spec

ialis

t kn

ow

ledge

or

capab

ilities

.

What

was

the

most

succ

essf

ul as

pec

t of

the

Task

Forc

e m

obili

sation?

The

sugges

tions

conce

rnin

g t

he

monitori

ng p

rogra

mm

e as

re

com

men

ded

in t

he

final re

port

.

Exc

han

ge

of

info

rmat

ion o

n t

he

way

s of

dea

ling w

ith a

spill

such

as

this

. O

nly

one

com

men

t – b

ringin

g

exper

ience

fro

m p

revi

ous

spill

s.

What

was

the

leas

t su

cces

sful as

pec

t?

Rec

om

men

dat

ions

about

clea

nin

g o

f pollu

ted b

each

mat

eria

l.

Wea

k in

volv

emen

t in

the

follo

w-u

p,

nota

bly

the

envi

ronm

enta

l m

onitori

ng

that

could

als

o h

ave

iden

tified

les

sons

for

the

futu

re f

or

oth

ers.

One

Mem

ber

Sta

te t

hin

ks t

hat

the

curr

ent

pro

cess

could

ignore

sig

nific

ant

exper

tise

within

the

EU

. Anoth

er

men

tions

the

lack

of

report

ing o

f th

e w

ork

of

the

Task

Forc

e.

Should

the

Com

mis

sion c

ontinue

to

pro

vide

fundin

g f

or

mobili

sation o

f ex

per

ts a

t fu

ture

inci

den

ts?

Yes

bec

ause

larg

e oil

spill

s m

ay

thre

ate

n s

ever

al c

ountr

ies.

D

efin

itel

y ye

s. T

he

Task

Forc

e or

obse

rver

s ca

n p

rovi

de

oil

spill

ex

per

ience

to t

he

affe

cted

countr

y an

d

can f

eedbac

k le

ssons

lear

nt

to o

ther

co

untr

ies

at

a Euro

pea

n lev

el.

Yes

. O

ne

sugges

tion is

that

it w

ould

be

ben

efic

ial if t

he

Com

mis

sion p

rovi

ded

fu

nds

for

Tas

k Fo

rce

exper

ts t

o join

IT

OPF

exp

erts

at

inci

den

ts o

uts

ide

Euro

pe.

Oth

er c

om

men

ts

No

Rea

dy

to r

evie

w e

xper

ience

s in

Euro

pe

or

else

wher

e w

her

e ex

change

of

info

rmation c

an h

elp in r

espondin

g t

o

spill

inci

den

ts o

r pla

nnin

g f

or

futu

re

spill

s.

The

role

of

the

exper

ts s

hould

be

dis

cuss

ed in d

epth

at

an M

CM

P m

eeting.




4.3 Mobilisation of Observers

The mobilisation of observers at the Baltic Carrier was perceived as useful and successful by those Member States that have commented on it. The Danish authorities appear to have been very efficient and effective in organising the tour for the observers and to have been particularly helpful in co-operating and providing a co-ordinator from DEMA to accompany the observers. However, some interesting comments have been received about aspects of the mobilisation and role of the observers in-country. The call for nominations from the Commission for an expert group was apparently confused with calls for an observer group. It seems that this resulted in some observers trying to give advice to the response authorities - but not to those in charge of the operations who might (or might not) have welcomed the advice. Comments from others suggest this experience has been repeated in other spill incidents. Once again, it seems it would be helpful to discuss and clarify the role of observers at incidents within a MCMP meeting. Brief Terms of Reference and/or guidance notes could be written and a procedure put in place to ensure observers receive such information before any involvement with the affected country. The apparent confusion about whether Member States were being asked to nominate experts or observers should be easily remedied by a more careful choice of wording by the Commission in any call.

4.4 Project on Monitoring and Assessment of Environmental Effects in Gronsund

Feedback was obtained from the beneficiary of the Programme funding (Storstrom County), the Danish authorities (DEMA) and three Member States. 4.4.1 Beneficiary Feedback The questions for Storstrom County were intended to address aspects of the funding process, project management and additionality of the funding. The Programme funding was felt to be important because of the limited money available from the local or regional environmental authorities. So, although it is judged that the project would have proceeded without Programme support, it would have been significantly smaller in scope. There were meetings between the Storstrom County representative in Brussels and staff in the Unit before the proposal was finalised. However, the Commission was not kept informed of progress during the project lifetime. It appears, therefore, that there was no mechanism for the Commission to be informed about possible delays or interim findings to enable it to intervene in case of problems or to identify possible beneficial changes in direction. It is good practice to require regular, brief reports (usually monthly) from contractors of progress against milestones for all projects with a duration greater than 1 or 2 months. Certainly the Commission should automatically require this reporting for all Pilot Projects and, where appropriate, for Training and Information projects (at the discretion of the Commission). In this case, the project was perceived by Storstrom County to have achieved its original objectives and to have been completed to schedule. Storstrom County was paid on time by the Commission, although some costs (layout, printing and translation) were found to be higher than expected.




The findings from the project have been widely disseminated through distribution of the report to Member States, the availability of the report on the Commission’s web site and through presentations in Denmark, France and Finland. 4.4.2 Feedback from Danish Authorities DEMA is aware of the project, has received a copy of the final report and therefore knows about the findings. The findings of particular interest from the monitoring programme relate to the remaining oil in the sediments and in mussels. DEMA is also interested in the evaluation of the clean-up operations. DEMA is supportive of the Commission continuing to fund such environmental impact projects and is keen to exchange experiences in order to improve their response to spills in future. 4.4.3 Feedback from Member States Two of the 3 individuals responding are aware of the project but have not seen the report. One Member State was particularly interested in the specific issues associated with clean-up of a marsh/reed-bed whilst another was interested in the method of monitoring. On the question of the need for continued funding for projects such as this, one Member State believes that most Member States are capable of funding these projects themselves but suggests that funds could be made available, if a special request is made. Another Member State is clear that funds should only be granted if the project is carried out with the involvement of the authorities in the country affected. There is also a question about whether funding for such projects could be recovered from the IOPC Fund. It is suggested that the Commission considers discussing the principles of this (and the possibility of recovering Task Force costs) with the Fund managers. 4.4.4 Case Study Findings versus Findings from the Main Evaluation All the Baltic Carrier actions have generally been favourably received by all parties and the mobilisation of experts and observers, in particular, is confirmed as very valuable. Some recommendations made earlier in this report, such as the need for improved dissemination of information, have been reinforced. If at all possible, more time should be allocated at MCMP meetings for presentation and discussion of findings, experiences and lessons learnt from incidents. The Case Study has highlighted some specific issues that the Commission should consider:

• Discussion and clarification within the MCMP of the role of experts in the Task Force and of observers;

• Consideration of obtaining some further formal commitment in principle from Member States to accept advice from the Task Force and to host observers;

• The requirement for contractors to report regularly on the progress of pilot projects and possibly of some other actions;

• The possibility of recovering funding from the IOPC Fund for the costs of the Task Force and of environmental assessment projects following spill incidents.

AEA Technology

Appendices

CONTENTS

Appendix 1 Main Evaluation Questionnaire Appendix 2 List of Stakeholders Providing Input Appendix 3 Case Study Questionnaires Appendix 4 Training Course Evaluation

Appendix 1 Main Evaluation Questionnaire

AEA Technology

Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution Evaluation Questionnaire Theme 1 – Overview and Context 1.1 Your role What is your job and areas of responsibility? How do you link with the Programme? 1.2 Programme Objectives What is the Programme aiming to achieve?

1.3 Interactions of the Programme with other Programmes What other related activities are you aware of within your own country? What trans-national activities? How does this EC Programme link in with these other activities (eg regional agreements/conventions, European Maritime Safety Agency)? Is there any duplication? How can it be avoided? 1.4 Barriers to Co-operation Are there any barriers? What are they? 1.5 Gaps Are there any gaps in coverage that the EC Programme should fill? Theme 2 – Impacts 2.1 Programme Successes What have been the Programme’s greatest successes? Can you give specific examples and quantify the benefits, if possible? Why have you selected these? How was this success achieved? What role did the Programme play – did the EC have direct control or an indirect influence? 2.2 Added Value What would have happened if the EC Programme had not been in place? What has been the added value of the Programme? (refer back to answers given in 1.4 to check additionality) 2.3 Programme Shortcomings Are there any areas of the Programme that have not been successful? Why not? What could have been done to address the problems? 2.4 Value for Money Does the Programme give good value for money? Are the results obtained in proportion to the resources deployed? 2.5 Community Information System (CIS) Has the CIS helped the National Authorities when dealing with accidental or deliberate pollution incidents? How has it contributed?

AEA Technology

2.6 Actions in Training and Information Have these actions resulted in any modification of the national approach to prevention and intervention? To what extent? Was co-operation between Member States created or enhanced? Examples? What projects contributed directly to improving training and information? 2.7 Pilot projects Did any projects lead directly to investment in resources, procedures, equipment, or networking? To what extent were there improvements? 2.8 Support and Information Actions To what extent did they contribute to strengthening co-operation in risk prevention/response? 2.9 Mobilisation of Expertise To what extent did this lead to more effective intervention? 2.10 Programme Value What is the most useful aspect of the programme to you? What is least useful?

Theme 3 – Process (efficiency and effectiveness) 3.1 Funding Decisions Are the financing decisions soundly-based? Are the best proposals supported? Are the most necessary proposals supported? 3.2 Funding Split (currently 52% Training & Information; 35% Pilot Projects; 10% Support % Information; 3% Mobilisation of Experts) Is the allocation of funding between the various actions appropriate? If not, what changes should be made? 3.3 Management processes What have been the strengths and weaknesses of the overall management of the programme? In what ways did the Commission control or influence the outcome of projects? Are you aware of any data collected or indicators used to measure project performance compared with objectives? If so, are they appropriate? If not, what could be used? Should the Management Committee have a bigger role? Theme 4 - Sustainability 4.1 Dissemination Have project outputs been effectively disseminated? In what ways? Any suggestions for improvements? Theme 5 – Synthesis 5.1 Overall assessment of the Programme To what extent do you feel the Programme has met its objectives (give a percentage)? Which are the most effective of the action areas (CIS, Training and Information, Pilot Projects, Support and Information Actions, Mobilisation of Experts)? 5.2 The future Is the Programme still necessary? What are the key areas to address next? How should the Programme be improved in future?

AEA Technology

Appendix 2 List of Stakeholders Providing Input

NAME Status

Nature of Contact

Belgium

Thierry Jacques Belgian Representative, MCMP Email & telephone

Denmark

Peter Soeberg Poulsen Danish Representative, MCMP Email

Jorn Allan Pedersen Regional Commander, Danish Emergency Management Agency

Email

Jakob Lysholdt Sorensen Associated Head of the Aquatic Environmental Division, Storstrom Region

Email

France

Daniel Sylvestre French Representative, MCMP Face-to-face

Christophe Rousseau CEDRE Email & telephone

Stephane Le Floch Task Force Member at the Baltic Carrier

Email

Germany

Berthold Tiefenbach German Representative, MCMP Email & telephone

Greece

Yannis Karageorgopoulos

(with input from the Merchant Marine Ministry)

Greek Representative, MCMP Email & telephone

Ireland

Eugene Clonan Irish Representative, MCMP Email & telephone

Italy

Francesco Valentini Italian Representative, MCMP Face-to-face

Netherlands

Sjon Huisman Netherlands Representative on MCMP

Email & telephone

Theo Kramer Netherlands Representative, MCMP Email & telephone

Norway

Jan Nerland Norwegian Representative, MCMP Email

Portugal

Raul Valente Portuguese Representative, MCMP Face-to-face

Sweden

Thomas Fago Swedish Representative, MCMP Email

NAME Status

Nature of Contact

United Kingdom

Toby Stone UK Representative, MCMP Face-to-face

Brian Elliott Observer at the Baltic Carrier Email & telephone

European Commission

Pia Bucella Head of Unit, Civil Protection and Environmental Accident Unit

Face-to-face

Gilles Vincent Deputy Head of Unit, Civil Protection and Environmental Accident Unit and Chairman of MCMP

Face-to-face, email & telephone

Ioanna Sgourdopoulou-Karra

DG ENV Unit D3 Face-to-face, email, telephone

Thomas de Lannoy DG ENV Unit D3 Face-to-face, email

Horst Miska Monitoring & Information Centre Face-to-face

European Maritime Safety Agency

James Wood EMSA Telephone

International Tanker Owners Pollution Federation

Tosh Moller ITOPF Email & telephone

International Maritime Organisation

Jean-Claude Sainlos IMO Email & telephone

Appendix 3 Case Study Questionnaires

Questions for Danish Authorities at the Baltic Carrier Incident

A. Mobilisation of Experts

Please give your name and position within your own organisation.

What was your own role during the Baltic Carrier incident?

What was your involvement with the European Commission’s Task Force (Gilles Vincent, Stephane Le Floch, and Bernard Le Guen) during the Baltic Carrier incident?

What do you think were the objectives of the Task Force?

Do you think they achieved their objectives? If not, why not?

What specific contribution did the Task Force make to the clean-up operations?

Did you implement any recommendations made by the Task Force?

Do you think the involvement of the Task Force at the Baltic Carrier incident will result in more effective intervention in Denmark in any future incident? If so, in what ways?

What do you think was the most successful aspect of the Task Force mobilisation?

What was the least successful aspect?

Do you think the Commission should continue to provide funding for the mobilisation of experts at future incidents?

Do you have any other comments about the involvement of the Commission’s Task Force at the Baltic Carrier incident?

B. Environmental Assessment survey following the accident of the Baltic Carrier

Are you aware that the Commission part-funded a project in 2001/02 to assess the environmental impact of the oil spill from the Baltic Carrier?

Were you informed about the findings from the project? If so, how?

Were any recommendations from the project of particular interest or relevance to you? If so, can you remember which ones?

Do you think the Commission should continue to provide funding for projects following spill incidents? If so, why?

Thank you for taking the time to complete this questionnaire! Please return it to Madeleine McDonagh at [email protected] by Friday 23rd January.

Questions for Task Force Members at the Baltic Carrier Incident

Please give your name and position within your own organisation.

Why do you think you were selected to represent the Commission within their Task Force?

What were the objectives of the Task Force?

Did you achieve your objectives? If not, why not?

Did you receive adequate briefing and administrative support from the Commission?

Did you receive adequate administrative support and co-operation from the Danish response authorities?

Were you briefed adequately by the Danish authorities on your arrival?

Was there any resistance to your full involvement from the Danish authorities or others?

What specific contribution did the Task Force make to the clean-up operations?

Do you know if your recommendations were taken up by the Danish response authorities?

Do you think the involvement of the Task Force at the Baltic Carrier incident will result in more effective intervention in future:

(i) in Denmark? (ii) in your own organisation/country? (iii) in other countries?

If so, in what ways?

What was the most successful aspect of the Task Force mobilisation?



Do you have any other comments about your experience at the Baltic Carrier incident?


Questions for Member States about the Mobilisation of Experts at the Baltic Carrier Incident

Please give your name and country.

A. Mobilisation of Experts

Were you aware that oil spill experts were mobilised by the European Commission during the Baltic Carrier incident?

Do you understand the Commission’s process for identifying and mobilising experts?

Do you think that the Task Force sent to the Baltic Carrier incident was of an appropriate size and composition (there was one expert from the Commission and two from CEDRE)?

Was any feedback provided to you about the work of the Task Force?

Do you know if the Task Force made any specific contribution to the clean-up operations in Denmark? If so, what?

Do you think the experience and lessons learnt by the Task Force at the Baltic Carrier incident will result in more effective intervention in future in your own organisation/country? If so, in what ways?

From what you know, what do you think was the most successful aspect of the Task Force mobilisation at the Baltic Carrier incident?



Do you have any other comments about the involvement of the Commission’s Task Force at the Baltic Carrier incident?

B. Environmental Assessment survey following the accident of the Baltic Carrier

Are you aware that the Commission part-funded a project in 2001/02 to assess the environmental impact of the oil spill from the Baltic Carrier?

Were you informed about the findings from the project? If so, how?

Were any recommendations from the project of particular interest or relevance to you? If so,

can you remember which ones?



The Baltic Carrier Oil Spill Monitoring and Assessment of Environmental Effects in Gronsund AEA Technology is evaluating the European Commission’s marine pollution framework programme on behalf of DG ENV and has agreed with the Commission to use the Baltic Carrier incident as a case study. One way in which the Commission supported the activities in Denmark associated with the incident was to part-fund the project led by Storstrom County. We would like to understand your views about the support provided by the Commission for this project and would be very grateful if you could complete the short questionnaire below. Questions for Storstrom County, Aquatic Environmental Division

Please give your name and position within Storstrom County.

What was your role in the environmental assessment project?

Would the project have proceeded without the funding provided by the Commission?

Was the project completed within the original estimates of time and cost?

How was the Commission kept informed about progress?

Was there a meeting at any time between your organisation and a Commission representative to discuss the project?

Did the project achieve its original objectives?

Were you paid on time by the Commission?



Appendix 4 Training Course Evaluation

Introduction In total, there have been 18 courses/workshops run by 8 countries attended by an average of 23 participants /course representing an average of 10 countries/course. This Appendix provides an assessment of the evaluation forms collected from training course participants (not workshops) and provided by the Commission. Survey responses were received for all training courses EXCEPT the following:

• Analysis of recent major accidents

• Workshop – Defining the content of a course on Harmful Substances

110 individual responses were received from a minimum of 203 people who attended the 12 courses for which survey responses were received. For those courses where survey responses were received, the overall survey response rate is (at best) 54%.

This assessment involved designing an Excel spreadsheet to enable the aggregation and averaging of responses to be automated. The information on the evaluation forms was manually input to the spreadsheet together with individual’s comments where applicable. The findings are summarised below based on the questions in the forms. How well were the event’s objectives achieved?

• 75% of respondents answered this question

• 71% of those that answered this question thought that the event’s objectives were achieved “well”

• A further 20% thought the objectives were achieved “completely”.

• No respondents thought that the event’s objectives were “not very well” or “not at all” achieved

Were participants’ objectives achieved?


• 63% of those that answered this question thought that their own objectives were achieved “well”

• A further 21% thought that their objectives were achieved “completely”

• 13% thought their objectives were “partly” achieved.

• 2% of those that answered this question thought that their own objectives had not been very well achieved, whilst no respondents thought that their objectives had “not at all” been achieved

Relevance of training to job


• 53% of those that answered this question thought that the courses were “very relevant” to their job

• A further 46% thought that the courses were “relevant” to their job, whilst only 1% thought they were “of little relevance”

Content covered


• 85% thought the content covered was “about right”

• 9% thought there was “too much” content

• 6% thought there was “too little” content

Level of content


• 85% thought the level was “about right”

• 6% felt the courses were “too advanced”

• 8% felt the courses were “too elementary” Length of event


• 74% thought the length was “about right”

• 19% thought the courses were “too long”

• 6% thought the courses were “too short”

Quality of trainers / key speakers


• Those that answered this question gave the following grades:

* 0 = Poor; 3 = Very good

Quality of visual aids



1%0

6%1

33%2

61%3

Proportion of respondents awarding grade

Grade for quality of visual aids

0%0

1%1

41%2

57%3


Grade for quality of trainers*

Quality of handouts and course documentation



Overall rating for course events



Should these types of events be repeated?


• 97% of those that answered this question (105 out of 108 people) said “Yes”, these types of events should be repeated

• The three “No” responses referred to the following three courses:

- Human Environmental Life Protection

- Response to Oil Pollution on the Sea and on the Coast

- Airborne Pollution Surveillance

• In each case, only one participant thought the course should not be run again

0%0

5%1

35%2

61%3


Grade for quality of handouts

0%0

4%1

40%2

56%3


Overall grade for courses

AEAT in Confidence -...

Documents

Transcript of AEAT in Confidence -...