-1-

Negotiated Procedure:

Review of Accident Investigation Methodologies

D6: Final Report

(Document version 1.0: 9/1/2010) ERA/2009/SAF/NP/02

Overview of the Project: The Directive 2004/49/EC of the European Parliament and of the Council of 29 April 2004 on safety on the Community’s railways, establishes the conditions to ensure a high level of railway safety and equal conditions for all railway undertakings. To achieve this goal, every Member State must create a safety authority and an accident investigation body. In order to avoid recurrence and, where possible, to improve railway safety, this accident investigation body should investigate all serious accidents on the railway. These investigation bodies shall, herein supported by the European Railway Agency, also conduct an active exchange of views and experience for the purpose of developing common investigation methods, drawing up common principles for follow-up of safety recommendations and adaptation to the development of technical and scientific progress. To be able to fulfil this task and to provide structured and useful guidance to the network of National Investigation Bodies, the Agency needs an inventory of occurrence investigation methods and techniques both within as outside the railway industry. Executive Summary: This document presents the final deliverable for a project intended to provide the inventory and associated guidance, mentioned above. D1 provided an overview of the inception meeting. D2 provided an initial list of tools, methods and techniques that might be used to assist member states across all stages of the ERA generic occurrence investigation process. The present deliverable uses this enumeration to support a more detailed level of analysis. D3 assessed candidate methods against an agreed set of ERA validation criteria. D4 identifies a range of methods, tools and techniques that can support common practices across member states. In particular, D4.1 proposed a number of requirements for future investigation tools and describes how there criteria were validated by NIBs and with support from ERA staff. Deliverable 4.2 went on to identify three different approaches: a relatively simple causal sequence model involving cognitive interviews as well as Events and Causal Factors analysis; a more complex organisational approach including cognitive interviews, Accimaps and Change Analysis, and finally a Next Generation model based on either of the previous two approaches but also including Data Mining and Simulation/Training tools that are intended to help the dissemination of lessons learned between neighbouring member states. D4.2 also stressed that local variants on these tools might also be integrated into these more general models. D4.3 went on to present a detailed case study illustrating the application of the proposed tools to a major rail accident. A key element of the case study was that the Next Generation models provide a road map for integration with other ERA initiatives including the common causal classification work. D5 provided detailed observations from a site visit to an NIB. Finally, this report summarises each of the previous deliverables and presents brief findings from a final NIB meeting that was used to ‘close out’ the project. Prepared by: Prof. Chris Johnson, Department of Computing Science, University of Glasgow, G12 8RZ, Scotland, UK. Johnson@dcs.gla.ac.uk (Email), +44 141 330 6053 (Tel.), +44 141 330 4913 (Fax)

-2-

Contents: 1. Introduction to the ERA Review of Accident Investigation Methodologies

1.1 Purpose and Expected Results 1.2 Scope of Work: Objectives and Work Methodology 1.3 Terminology: Models. Methods and Tools

2. Detailed Summary of Previous Deliverables

2.1 Deliverable D1: Inception Meeting 2.2: D2: List of Tools, Methods and Techniques. 2.3: D3: Assessment of Tools, Methods and Techniques Against ERA Criteria 2.4 D4.1: Requirements for ‘Good Practice’ Models and Tools 2.5 D4.2: Selection of ‘Good Practice’ Models and Tools 2.6 D4.3: Case Studies of ‘Good Practice’ Models and Tools 2.7 D5: Minutes of Validation Meetings with NIBs

3. Introduction to this Deliverable for WP6

3.1 WP6 Description: 3.2 WP6 Methods: 3.3 WP6 Deliverables: 3.4 Revisions to WP6: 3.5 Comments from the February 2010 NIB Plenary

4. Summary

-3-

1. Background and Proposal Objectives European Directive EC/1991/440 provides the background to this project. It seeks to revitalise Europe’s railway infrastructures by establishing a framework for access and interoperability. The technical aspects of this harmonisation are described in more detail by Council Interoperability Directives EC/1996/48 and EC/2001/16. The European Railway Agency under Regulation [EC/2008/1335] is charged to support both the legal and technical integration of Europe’s railways. One aspect of this is a need to promote a common approach to questions concerning railway safety. The European Railway Safety Directive [EC/2004/49] emphasises the need to develop Safety Management Systems, Common Safety Indicators, Common Safety Targets and Common Safety Methods. This project will enable the European Railway Agency to provide world leading advice on a range of accident investigation methodologies that can be used to establish best practice across member states. 1.1 Purpose and Expected Results A number of generic phases are common in existing practices for occurrence (accident and incident) investigation and reporting in railway and other industries. For the purpose of this study, the following generic occurrence investigation process will serve as reference:

1: Safety occurrence notification

2: Immediate facts of the occurrence

3: Decision to investigate

12: Consultation

5: “Complete” factual information

4: Further factual information gathering

6: Reconstruction of the occurrence

7: Occurrence scenario

8: Analysis

11: Draft report

10: Recommendations

9: Causal factors

13: Final report

14: Publication and Monitoring

Figure 1: ERA Generic Occurrence Investigation Process1

1 The original version of the generic model in the call for tenders is a much better quality drawing. This copy is only introduced so that it can be edited in later sections of the analysis within this deliverable.

-4-

1.2 Scope of Work: Objectives and Work Methodology The aim of the project is to provide an extensive review, both within and external to the railway industry, to list and select what methods, tools and techniques are available and could be used in railway occurrence investigations. The work will range from theoretical methods to practical tools. There are 2 stages to the project, development of an inventory and the selection of recommended investigation methodologies. To structure the list of methods, tools and techniques and to improve readability, a template shall be created that indicates for each methodology a list of criteria enumerated in the following sections of this report. In a second stage, after the identification of all possible investigation techniques, a well-founded selection of recommended methods/tools/techniques shall be made for every phase in the generic occurrence investigation process. For this we will propose an appropriate selection process and criteria. Criteria to be considered should include: inappropriate or not suitable for railway occurrence investigations (e.g. specifically for nuclear or chemical process plants and not adaptable to railways) outdated, not used (anymore) superseded by another technique on the list too general/too specific, detailed or limited. Deliverable 2 provided an extensive review, both within and external to the railway industry, to list and select what methods, tools and techniques are available and could be used in railway occurrence investigations. It included theoretical methods as well as practical tools. WP2, therefore, developed an inventory that is then compared against the ERA criteria within Work Package 3. More than 100 further approaches were identified. However, these were argued to have broad similarities to techniques in this short list. Deliverable 3 assessed each of the methods, tools and techniques against the ERA requirements from the project specification document with reference to available evidence and links to requirements for Safety Management Systems derived from the European Railway Safety Directive.2 The fourth work package built on the findings of WP2 and 3. It was informed by requirements elicitation meetings held with individual member states and also from a joint meeting of the National Investigatory Bodies, ERA Offices, Lille, 13th-14th October 2009. These meetings helped to identify the properties that we might expect of tools, methods and techniques both in the short term and in the longer-term. These requirements were documented in the additional Deliverable 4.1. The ERA requirements were then applied to the evaluations from Deliverable 2 to identify a well-founded selection of recommended tools. These were described and justified in D4.2. Deliverable 4.3 then applied the selected techniques to a case study as a means of promoting the use of the recommended tools. Deliverable 5 reports on the results of an initial validation exercise. The interim findings from this project were discussed with the investigators and other staff in an NIB.

Finally, this deliverable provides an overview for the project and identifies directions for future work.

1.3 Terminology: Models, Methods and Tools Previous paragraphs refer to tools, techniques and methods in a general way. This reflects the interchangeable manner in which these terms have been employed by investigators and by the supporting literature. This also reflects the way in which terminology was used during the initial stages of the project. However, it can lead to confusion. For instance, a causal analysis tool such as Events and Causal Factors charting could also refer to the supporting methods that help to integrate this approach into wider stages of the ERA Generic Occurrence Investigation Process. To minimise confusion, we therefore follow a proposal made by several NIBs in response to the requirements presented in Deliverable 4.1:

• Models describe generic approaches to the analysis of adverse events. For instance, causal-sequence models represent an accident in terms of a chain of cause and effect relationships. Similarly, a process model looks at the ways in which different activities or operations interact during and immediately after an adverse event. Energy models look less at the operations or processes leading to an accident and instead focus more directly on the reasons why targets can be exposed to particular hazards. Logic (tree) models look at the conjunctions and disjunctions of events that form the necessary and sufficient conditions

2 UK HSE, Root causes analysis: Literature review, Prepared by WS Atkins Consultants Ltd for the Health and Safety Executive, Contract Research Report, 325/2001. http://www.hse.gov.uk/research/crr_pdf/2001/crr01325.pdf

-5-

for adverse events to occur. The human information processing model looks at the ways in which individual cognitive and perceptual resources are influenced by performance shaping factors in the environment to create the context in which human ‘error’ is likely to contribute to an accident or incident. This is a partial list but it illustrates the high-level categorisations that can be introduced by using models to describe a number of different methods.

• Methods describe particular approaches that have been developed to enable investigators apply models during different stages of the investigatory process, illustrated in Figure 1. For instance, the Events and Causal Factors (ECF) method begins by mapping out the events that lead to an accident. These are then associated with the causal factors, including management and safety culture concerns, that made those events more likely to occur. This approach can, therefore, provide means of implementing the cause-sequence model, mentioned above. Alternatively, other methods, such as the Multi-linear Event Sequencing (MES) reviewed in the previous deliverables associated with this project, could also be used to apply the cause-sequence model.

• Tool help implement the methods that are associated with methods. Hence, a company may sell a tool to conduct the Events and Causal Factors method. This, in turn, provides a means of applying the causal-sequence model for accident investigation. Similarly, an NIB may develop in-house tools that simplify the fault tree method that applies the logic model to accident investigation.

Other definitions are possible, reflecting the general lack of consensus on these distinctions. However, the following pages adopt these terms to reduce ambiguity and support the interpretation of particular recommendations for future tools.

-6-

2. Detailed Summary of Previous Deliverables

2.1 Deliverable D1: Inception Meeting

The inception meeting reviewed the timetable for the WPs associated with the project – only one change was made to the delivery of D2; as there has been a slight delay in holding the kick—off meeting. The other dates remain the same to enable the presentation of D3 at the next NIB meeting in October. The detailed rational for these decisions is presented in later sections of this inception report.

Date from Proposal Date Agreed at Inception D1 – Inception Meeting 16th July 2009 30th July 2009

D2 17th August 2009 28th August 2009 D3 1st October 2009 1st October 2009 D4 1st December 2009 1st December 2009 D5 15th December 2009 15th December 2009

The following action list was created to guide the remainder of the project:

• ACTION 1: CWJ to review the ERA documents provided by JR and BA at the inception meeting.

• ACTION 2: CWJ to consider the ways in which the severity of an incident might influence the selection of a technique as well as the maturity or resources available too NIBs in different member states within WPs 3 and 4.

• ACTION 3: CWJ and BA to liaise on the best ways to integrate our work with those of the Task

Forces – especially in WP5.

• ACTION 4: CWJ to update deadlines and distribute in this inception report (see previous table).

• ACTION 5: CWJ to ensure that D4 considers all aspects of the generic process outlined in the call for proposals issues by ERA.

• ACTION 6: ALL to identify potential case studies, CWJ to find ways of requesting case studies

from the NIBs in the October meeting.

• ACTION 7: BA and JR to discuss possible NIBs to visit as part of WP5 – preferably in time for the October network meeting in Lille so that CWJ can make contact with them there.

• ACTION 8: CWJ to draft a White Paper presentation for the February 2010 NIB to include, if

appropriate a training needs assessment – to be reviewed by the project team after D4 and D5 (ie in the later stages of the project).

However, the opportunity to interact directly with the NIB’s through the plenary meeting scheduled for February 2010 justified the subsequent extension of the contrast beyond the timescales that are summarised above.

-7-

2.2: D2: List of Tools, Methods and Techniques.

Deliverable 2 presented an initial selection of tools, methods and techniques that can be recruited encourage ‘common investigation methods’ and support member states by ‘drawing up common principles for follow-up of safety recommendations and adaptation to the development of technical and scientific progress’. These can be summarised as follows:

A Accident Analysis Framework, Accimaps Accident Investigation Training Course (UK Rail) Adverse Incident Tracking System Adverse Event Reporting System, (US Food and Drugs Administration) Australian Incident Monitoring System ATSB Aviation Safety Action Programme Aviation Safety Reporting System (ASRS, National Transportation Safety Board) ABCA Coalition Operations Lessons Learned Database, Australian Office of Transport Safety Investigations, Confidential Safety Reporting Information Scheme

B Barrier Analysis, Bayesian Analysis Bayesian Networks Bias,

analytical bias, author bias, availability heuristic, checklist bias, confidence bias, witness etc

Biomechanical models

C

Canadian National Defence General Accident Information System, and Safety Digest, Case-based reasoning Causal trees Counterfactual reasoning Cause-context summaries Cause-Consequence Models CD-ROM Chain of events Change Analysis Chat Rooms Checklists Confidential Incident Reporting System (CIRS) Cockpit Voice Recorders, Composite Risk Management (CRM) Computerised Accident Incident Reporting System (CAIRS) Conclusion, Analysis and Evidence diagrams, (CAE) Confidential Human Factors Incident Reporting Programme (CHIRP) Confidential Incident Reporting and Analysis System (CIRAS) cCnsequence assessment Cooperative Compliance Programme (OSHA’s)

-8-

CREAM Cryptography Current Reality Tree

D Databases Data Mining Data Recorders Data Reporting Analysis and Corrective Action System (DRACAS) Decision Theory DesktopVR Dynamic Querying Decision Trees Domino Theory

E Eindhoven Classification Model, Electronic mail Enhanced Cognitive Interviews for Rail Investigations European Space Agency Alert System, EUROCONTROL Risk Assessment Worksheets Event trees Events and Causal Factor Charts (ECF)

F

Failure Modes, Effects and Criticality Analysis (FMECA) Failure Reporting, Analysis and Corrective Actions (FRACAS) Fault trees Fax machines Five Whys Flight Operations Quality Assurance programmes, Flowchart Formal methods, FRA Highway-Rail Crossing Web Accident Prediction System, FRA Confidential Close Call Reporting System (C3RS)

G

GEMS, Generic Error Modelling Generic Occurrence Classification Global Aviation Information Network (GAIN) Goal Strucutured Notation (GSN)

H

HAZOPS HEIDI Heinrich ratio Human Reliability Analysis

I

Iceberg model, Incident Analysis Method for Railway Safety Management

-9-

International Nuclear Event Scale

J

Japanese Maritime Incident Reporting System Joint Center for Lessons Learned

K

Kepner-Tregoe Problem Analysis Kjellen's criteria

L Latent failure Likelihood Assessment Logic,

Causal Logic, Deontic Logic, Explanatory Logic, First Order Logic, Modal Logic, Temporal Logic,

M Major Hazard Incidents Data Service (MHIDAS) Management Oversight and Risk Trees (MORT) Manufacturer and User Facility Device Experience database (MAUDE) Multilinear Events Sequencing (MES) MTO (human, technology and organisation) Japanese Rail Accident Method

N

National Patient Safety Agency, see NPSA National Patient Safety Database Non-Compliance Analysis

O, P PARDIA (WBA) Performance Shaping Factors Petri Nets Peturbation Theory, P-Theory (part of MES/STEP) Physical Reconstructions Prevention and Recovery Information System for Monitoring and Analysis (PRISMA) PRISMA-Rail Precursor Indicator Model

Q QuicktimeVR

R

Rail-Program for Risk Informed Safety Managements Railway Technical Research Institute (RTRI) type accident analysis method Rail Data Recorders Reason Root Cause Analysis Tools

-10-

S

Safety Cases Safety Management Information System Sequentially Timed and Events Plotting (STEP) SHELL Simulations, Skills, Knowledge, Rules (Rasmussen) Skybrary Accident Information and Safety Information System SMORT Safety by Organisational Learning (SOL) Systems Theoretic Accident Model and Processes (STAMP) Systemic Causal Analysis Technique (SCAT) Systematic Accident Scenario Analysis (SASA) Systemic Safety Management System

T

Taproot Theory of Constraints Time-lines Toulmin's Argumentation Structures Technique for the Retrospective and Predictive Analysis of Cognitive Errors: TRACEr-rail version Tripod Tripod-Beta, Tripod-Delta

U

US Air Force Automated Security Incident Measurement US Army 5 stage model US Air Force 8-Step Problem Solving Methods

V

Virtual Reality VRML

W Why Because Analysis (WBA) Witness Guidelines, (US Department of Justice) Westrum's Taxonomy World Wide Web, Worst Plausible Outcome

X, Y, Z Yellow Book (Guidance on UK Rail Accident Analysis) Members of the NIBs and commercial organisations involved in accident investigations across the rail industry were then approached to ensure that there were no notable omissions. More than 150 additional techniques were identified in D2 but it was acknowledged that the selected list included examples of the leading approaches. The selection was then structured using phases of the ERA Generic Occurrence Investigation Process, illustrated in Figure 1. It is also important to stress the subjective nature of this allocation. The proponents of particular techniques often make strong claims about the general utility of their approaches across many different phases of an incident investigation. In consequence, we would argue that the existing classification in this deliverable represents a minimum allocation – in other words,

-11-

the absence of a technique from one phase does NOT imply that the approach or tool could not in principle support the associated activities during an investigation. However, those process phases that are associated with a technique are intended to represent the ‘usual’ application of the approach even though other may exist. This point will be reiterated in D3 which will deal in more detail with the support for each approach. Figure 2 provides an illustration of this allocation process that was used to ensure the coverage of the list across the ERA generic process.

1: Safety occurrence notification

2: Immediate facts of the occurrence

3: Decision to investigate

12: Consultation

5: “Complete” factual information

4: Further factual information gathering

6: Reconstruction of the occurrence

7: Occurrence scenario

8: Analysis

11: Draft report

10: Recommendations

9: Causal factors

13: Final report

14: Publication and Monitoring

Accident Analysis Framework, Accimaps (Rasmussen) Accident Investigation Training Course (UK Rail) Adverse Incident Tracking System, see AITS Adverse Event Reporting System, (US Food and Drugs Administration) Australian Incident Monitoring System, see AIMS ATSB Aviation Safety Action Programme Aviation Safety Reporting System (ASRS, National Transportation Safety Board) ABCA Coalition Operations Lessons Learned Database, Australian Office of Transport Safety Investigations, Confidential Safety Reporting Information Scheme Barrier Analysis, Bayesian Analysis Bayesian Networks Bias, Biomechanical models Canadian National Defence General Accident Information System, and Safety Digest, Case-based reasoning Causal trees Counterfactual reasoning Cause-context summaries Cause-Consequence Models1 CD-ROM Chain of events Change Analysis Chat Rooms Checklists Confidential Incident Reporting System (CIRS) Cockpit Voice Recorders, Composite Risk Management (CRM) Computerised Accident Incident Reporting System (CAIRS) Conclusion, Analysis and Evidence diagrams, (CAE) Confidential Human Factors Incident Reporting Programme (CHIRP) Confidential Incident Reporting and Analysis System (CIRAS) consequence assessment Cooperative Compliance Programme (OSHA’s) CREAM Cryptography Current Reality Tree Databases Data Mining Data Recorders Data Reporting Analysis and Corrective Action System (DRACAS) Decision Theory DesktopVR Dynamic Querying Decision Trees Domino Theory Eindhoven Classification Model, Electronic mail Enhanced Cognitive Interviews for Rail Investigations European Space Agency Alert System, EUROCONTROL Risk Assessment Worksheets Event trees Events and Causal Factor Charts (ECF) Failure Modes, Effects and Criticality Analysis (FMECA) Failure Reporting, Analysis and Corrective Actions (FRACAS) Fault trees Fax machines Five Whys Flight Operations Quality Assurance programmes Flowchart Formal methods FRA Highway-Rail Crossing Web Accident Prediction System, FRA Confidential Close Call GEMS, Generic Error Modelling Generic Occurrence Classification Global Aviation Information Network (GAIN) Goal Structured Notation (GSN)

HAZOPS HEIDI Heinrich Rratio Human Reliability Analysis Iceberg model Incident Analysis Method for Railway Safety Management International Nuclear Event Scale Japanese Maritime Incident Reporting System Joint Center for Lessons Learned Kepner-Tregoe Problem Analysis Kjellen's criteria Latent failure Likelihood Assessment Logic,

Causal Logic, Deontic Logic, Explanatory Logic, First Order Logic, Modal Logic, Temporal Logic,

Major Hazard Incidents Data Service (MHIDAS) Management Oversight and Risk Trees (MORT) Manufacturer and User Facility Device Experience database (MAUDE) Multilinear Events Sequencing (MES) MTO (human, technology and organisation) Japanese Rail Accident Method National Patient Safety Agency, see NPSA National Patient Safety Database Non-Compliance Analysis PARDIA (WBA) Performance Shaping Factors Petri Nets Peturbation Theory, P-Theory (part of MES/STEP) Physical Reconstructions Prevention and Recovery Information System for Monitoring and Analysis (PRISMA) PRISMA-Rail Precursor Indicator Model QuicktimeVR Rail-Program for Risk Informed Safety Managements Railway Technical Research Institute (RTRI) type accident analysis method Rail Data Recorders Reason Root Cause Analysis Tools Safety Cases Safety Management Information System Sequentially Timed and Events Plotting (STEP) SHELL Simulations Skills, Knowledge, Rules (Rasmussen) Skybrary Accident Information and Safety Information System SMORT Safety by Organisational Learning (SOL) Systems Theoretic Accident Model and Processes (STAMP) Systemic Causal Analysis Technique (SCAT) Systematic Accident Scenario Analysis (SASA) Systemic Safety Management System1 Taproot1 Theory of Constraints (TOC, Zotov, …) Time-lines Toulmin's Argumentation Structures Technique for the Retrospective and Predictive Analysis of Cognitive Errors: TRACEr-rail version Tripod Tripod-Beta, Tripod-Delta US Air Force Automated Security Incident Measurement US Army 5 stage model US Air Force 8-Step Problem Solving Methods Virtual Reality VRML Why Because Analysis (WBA) Witness Guidelines, (US Department of Justice) Westrum's Taxonomy World Wide Web Worst Plausible Outcome Yellow Book (Guidance on UK Rail Accident Analysis)

Figure 2: Stage 1 Safety Occurrence Notification Techniques 3

3 Highlighting is used to indicate perceived applicability of technique or tool to that phase of the ERA generic investigation and analysis process.

-12-

2.3: D3: Assessment of Tools, Methods and Techniques Against ERA Criteria WP3 took the techniques identified in WP2 and assessed them against a standard template, identified by the ERA Safety Unit that included the following criteria:

• name of the method/tool/technique • references to the method/tool/technique • other names or speciality names • primary objective of the method/tool/technique: the original purpose or function of the

method/tool/technique • a description of the process which must be followed to apply the method/tool/technique –

this description is a digest of information drawn from the references or subject matter experts

• an indication for which of the phases in the generic occurrence investigation process (C.2.2.1) it could be applicable

• has the method/tool/technique previously been applied in railway occurrence investigations, or could it be adapted to the railway context?

• alternative, overlapping or complementary method/tool/technique, e.g. methods/tools/techniques that can be used preliminary or successively to the method/tool/technique

• an indication whether the method/tool/technique is in use • computer tools that can support application of the method/tool/technique • evidence of successful application of the method/tool/technique • the required level of expertise to apply the technique: is it relatively easy to understand and

use? Is specific training needed? • the degree to which the technique lends itself to reviewable documentation • the consistency of the technique, such that if used on two occasions by independent

investigators, reasonably similar results are derived • any restrictions on application, e.g. problem scale, generality, accuracy, ease of use, cost,

availability, maturity, use of resources, data requirements, etc. As mentioned before, such an analysis was guided by the available literature, by expert interviews and by first hand experience in the application of many of the tools. The following pages provide an example of the several hundred pages of detailed analysis that were presented in this deliverable.

-13-

Accimaps Evaluation Criteria Assessment

Name of the method/tool/technique

Accimaps (Rasmussen)

References to the method/tool/technique

I. Svedung and J. Rasmussen, Graphic representation of accident scenarios: mapping system structure and the causation of accidents, Safety Science 40 (2002), pp. 397–417. C.W. Johnson and I.M. de Almeida, Extending the Borders of Accident Investigation: Applying Novel Analysis Techniques to the Loss of the Brazilian Space Launch vehicle VLS-1 V03, Safety Science, 46:1:38-53, 2008. http://www.dcs.gla.ac.uk/~johnson/papers/Ildeberto_and_Chris.PDF

Other names or speciality names ActorMaps, Generic AcciMap, InfoFlowMap, ConflictMaps (see below for details).

Primary objective of the method/tool/technique: the original purpose or function of the method/tool/technique

Accimaps provide a high-level overview of the actors, information flows and events that occur during adverse events.

A description of the process which must be followed to apply the method/tool/technique –this description is a digest of information drawn from the references or subject matter experts

Svedung and Rasmussen provide several tools designed to represent a particular accident(“Accimap”), a set of accidents (Generic AcciMap), as well as the various actors involved in an accident (ActorMap) and the information flow among decision-making bodies (InfoFlowMap). These can then be used to map out ConflictMaps between the parties involved in an accident (see below).

AcciMaps show the planning, management and regulating bodies creating the context in which an accident might occur. The Generic Accimap is designed to show the decisions that generated several possible accident scenarios within a particular domain. The ActorMap lists the actors involved in the Generic AcciMap, from the company management level to the highest level. The InfoMap deals with interaction between actors describing communications between various decision-makers.

An indication for which of the Step 4: Further factual information gathering

-14-

phases in the generic occurrence investigation process (Figure 1) it could be applicable

Step 5: Complete factual information Step 8: Analysis Step 9: Causal factors

Has the method/tool/technique previously been applied in railway occurrence investigations, or could it be adapted to the railway context?

These have been some attempts to use variations on Accimaps for Rail Accidents. In particular, Andrew Hopkins, Safety, Culture and Risk: The Organisational Causes of Disasters, CCH Australia, Sydney, 2005 deals with the Glenbrooke accident. This work has been compared to Why Because Graphs for Glenbrooke in http://www.rvs.uni-bielefeld.de/publications/Papers/Ladkin-Glenbrook.pdf. However, care is required to determine whether Hopkin’s is using Rasmussen and Svedung’s full approach.

Alternative, overlapping or complementary method/tool/technique, e.g. methods/tools/ techniques that can be used preliminary or successively to the method/tool/technique

Accident Analysis Framework, TRIPOD, ECF, Change Analysis, Tier Analysis.

An indication whether the method/tool/technique is in use

Accimaps remain in use, mainly within the research community.

Computer tools that can support application of the method/tool/technique

N/A

Evidence of successful application of the method/tool/technique

See references.

The required level of expertise to apply the technique: is it relatively easy to understand and use? Is specific training needed?

Medium level of sophistication, especially in the training required to follow the links between each of the respresentations – from ActorMaps to GenericAcciMaps etc.

The degree to which the technique lends itself to reviewable documentation

Very useful and accessible documentation with some engineering credibility as well as organisational focus.

The consistency of the technique, such that if used on two occasions by independent investigators, reasonably similar results are derived

Difficult to ensure consistency given the many different components of the approach and their inter-relationships.

Any restrictions on application, e.g. problem scale, generality, accuracy, ease of use, cost, availability, maturity, use of resources, data requirements, etc.

Relatively mature and has been applied on case studies ranging from fishing accidents to space mission failures. Scalable.

Do the tools and techniques provide equal benefits for both small and large member states?

Equally applicable across member states but with some initial training investment.

Do the tools and techniques provide support for all aspects of a failure (Human, organisational, technical) in equal measure or must they be integrated with other approaches?

Organisational and technical support developed by leading human factors expert. Of the three areas, technical failures are the least well supported

Can the tools and techniques provide credible support for the

Yes, extensible, flexible approach.

-15-

future requirements given increasing complexity and integration in railway operations? Do the approaches support the wider ‘safety strategy’ within an organisation? Can they both measure and control safety issues through links to risk assessment?

Yes, clear links to highest levels of policy making up from the basic events.

-16-

2.4 D4.1: Requirements for ‘Good Practice’ Models and Tools across the Generic Investigation Process As we worked on the criteria for the tools and models, it became clear that further consultation was required with the NIBs to ensure that we had identified the requirements for tools and models to support the investigation process. Working with ERA staff and a working group drawn from the NIB’s we created a new deliverable, not mentioned in the original contract, that identified both short term and long term requirements for the approaches studied in this project. These are summarised as follows. Short Term Requirements for Future Tools

The following table provides an initial set of requirements for future tools, methods and techniques that might be recruited to support National Investigatory Bodies (NIBs) in the short term.

No. Requirement Phase of the ERA

Investigation Process

(see Figure 1)

Justification

STR-1 There must be a good mapping between the proposed approach and existing techniques or skills within the NIB

All It is critical to build on existing good practice and not to recommend approaches that cannot easily be implemented with existing staff resources or budgets. (suggested by ERA)

STR-2 A rail accident is an organizational weakness. The tools are instruments to identify this from the first phase of the accident until the recommendations.

All It is critical that we provide some support for all stages of the investigation process considering the underlying organisational causes (suggested by ERA).

STR-3 Most accidents involve a range of commercial and non-commercial organisations. Tools must help identify the role played by these organisations in the context of an accident.

All Appropriate approaches will help to identify different organisational involvement in the context of an accident (suggested by ERA).

STR-4 The investigation should include tools to ensure that the facts are identified correctly and completely. An important goal is to identify the facts (truth) in all the phases of the investigation. First the facts. The proposed tools must provide NIBs with appropriate support for gathering and safeguarding evidence in the immediate aftermath of an adverse event.

1,2 and 3. If we do not have appropriate and documented techniques for the initial phases of an investigation then the other stages will be adversely affected. (suggested by ERA)

Table 1: Short Term Requirements for Future Tools (Cont.)

-17-

No. Requirement Phase of the

ERA Investigation

Process (see Figure 1)

Justification

STR-5 The tools for the investigators should be user friendly and limited in number. This increases the chance that they will be used and creates the possibility to get some experience and routine with the instruments. The tools are supported with user software packages (including instructions for use). The tools can be applied by individual accident investigators or investigation teams.

All User friendliness and tool support will determine the up-take of proposed approaches (suggested by ERA)

STR-6 The proposed accident investigation and analysis tools must help to review the existing risk assessment processes within operating companies.

13 and 14 If the output of the investigation process cannot be used to inform future risk assessment then these is a danger that safety management systems will not deliver their intended benefits of learning from previous failures. (suggested by NIB)

STR-7 All tools must be supported by appropriate documentation.

All There are increasing numbers of stakeholders in an investigation process – including judicial authorities in many member states. It is, therefore, important that we can show the output of various stages of an investigation both to co-workers in an NIB and to other appropriate stakeholders, including NSAs.

STR-8 As a development of requirement STR-4, it is particularly important to provide tools that can be used to support the generation of recommendations and to justify them.

9 and 10. It may not be possible to ensure absolute consistency in terms of the recommendations made by different investigators and different accidents; however, the provision of methods that help to justify the identification of particular justifications can encourage a degree of agreement by explaining why a finding was proposed.

Table 1: Short Term Requirements for Future Tools (Cont.).

-18-

No. Requirement Phase of the

ERA Investigation

Process (see Figure 1)

Justification

STR-9 The tools should focus on the various phases of the rail accident with the goal of identifying the human, technical and organizational shortcomings. With an emphasis on the organizational shortcomings.

All There is an increasing recognition that accident investigations must consider a broad range of causal and contributory factors. (Suggested by ERA)

STR-10

Proposed tools should encourage consistency but also support the identification of a diverse set of causal factors.

All Ideally we might like to ensure that different investigators would identify the same causes in similar incidents. This remains a longer term aim, given the differences in background and experience as well as the complex nature of some accidents can introduce differences in the investigation process. As well as encouraging consistency, it is important that techniques should not do this by focussing on a narrow set of issues.

STR-11

Proposed tools should make the common investigations easy and the more difficult ones possible.

All Tools must support the less complex investigations as well as the more complex failures that we can envisage in the future.

Table 1: Short Term Requirements for Future Tools (Cont.).

-19-

Longer Term Requirements for Future Tools

A further set of requirements can be identified for future tools that might be recruited to support National Investigatory Bodies (NIBs) in the longer term. This list, therefore, includes objectives that require additional research or for which there may be a lack of consensus both within the rail industry and in other safety-critical organisations. No. Requirement Phase of the

ERA Investigation

Process

Justification

LTR-1

Tools should provide support for the analysis of safety-culture within the organisations and groups involved in an accident.

All Safety culture is an increasing focus of concern in some member states. However, there is some disagreement about the attributes of safety culture and the metrics that might be used to assess it (suggested by an NIB)

LTR-2

Support should be provided for the consideration of software engineering in accidents and incidents in addition to the human factors, engineers and organisational issues.

All Software plays an increasingly important role in the running of high-capacity rail systems. However, many existing tools cannot be applied to understand the complex interactions between software components.

LTR-3

Tools should provide direct links to simulation and training for dissemination via web sites.

12, 13 and 14 There is a growing range of visualisation tools that can be used to help disseminate the findings of accident and incident investigations. However, many of these systems have not been used within the European rail industries (suggested by an NIB)

LTR-4

Tools should provide automatic support for the identification of common patterns of incident across European industry.

13 and 14 One of the greatest benefits in using recognised approaches is that we can define common mappings between different tools so that automated information retrieval tools can find common causes or recommendations between different member states within minimal intervention and at low costs providing best practices are extended from the commercial IT industry.

Table 2: Longer Term Requirements for Future Tools

-20-

No. Requirement Phase of the

ERA Investigation

Process

Justification

LTR-5

Tools should demonstrate consistency and repeatability between investigators.

All Ideally we would like to ensure that the recommendations derived from an investigation were not determined by the individual or team allocated to the investigation. In practice, it is hard to identify appropriate evidence for supporting such assertions. However, unless we can conduct this forms of analysis then there is little prospect for ensuring common standards both within and between member states.

LTR-6

Tools should support the analysis of incidents involving interfaces between organisations.

All Many incidents and accidents stem from communications problems between the organisations that are involved in operating infrastructure and maintaining them. Increasingly deregulation is increasing the use of sub-contractors and often there are patterns of incidents that indicate problems well before major accidents occur. Incident reports typically focus on solving the particular issue at stake rather than the underlying communications barriers with sub-contractors who may not share the same safety culture.

LTR-7

Tools should support the analysis of cross border incidents and accidents.

All The longer term future of work in this area is to ensure that different NIBs recognise and integrate with the tools of their neighbours.

Table 2: Longer Term Requirements for Future Tools

-21-

2.5 D4.2: Selection of ‘Good Practice’ Models and Tools across the Generic Investigation Process The criteria presented in D4.1 and validated by the NIBs were then used to identify candidate approaches for use across the ERA generic investigation process. Deliverable D4.2, therefore, identified ‘best practice’ tools and models to encourage ‘common investigation methods’ and support member states by ‘drawing up common principles for follow-up of safety recommendations and adaptation to the development of technical and scientific progress’. As mentioned in previous sections, accident models describe generic approaches to the analysis of adverse events. The first proposed approach, therefore, supports the development of causal-sequence models. These represent an accident in terms of a chain of cause and effect relationships. In contrast, the second proposal adopts a more organisational view of accidents looking less at the chain of events and more at the interactions or interfaces between organisations involved in an adverse event. The final proposal focuses on a future accident model that is intended to support a more integrated approach to accident investigation between member states. The intention is to map out ways that technological support might build upon existing models to help exchange lessons learned, for example between adjacent states that offer similar or interconnecting services across their rail infrastructures.

A number of more specific methods are associated with each of these accident models. Methods describe particular approaches that have been developed to enable investigators to apply models during different stages of the investigatory process, illustrated in Figure 1. For instance, the Events and Causal Factors (ECF) method begins by mapping out the events that lead to an accident. These are then associated with the causal factors, including management and safety culture concerns, that made those events more likely to occur. This approach can, therefore, provide means of implementing the cause-sequence model, mentioned above. Similarly, the Accimap method supports an analysis of the organisational interactions that can lead to an accident – for example between rail infrastructure providers and particular operation companies or between maintenance teams and operational staff.

-22-

Figure 3: Overview of the Proposed Approaches and Roadmap for Future Harmonisation

A great many methods which could be used to implement each of these generic accident models. For example, the ECF approach in our proposal for the cause-sequence model might be replaced by techniques such as Multi-linear Event Sequencing (MES) or Sequential Timed Event Plots (STEP). The intention in this deliverable is not to contribute to an academic debate about the relative strengths of each of the 200+ methods identified in previous stages of this project. In contrast, the intention is to show how selected approaches might be applied by NIBs to support rail accident investigations.

In addition to the tools that are recommended in this report, the site visits described in deliverable D.5 identified a number of informal and local ‘good practices’ that are widely employed by NIBs in particular member states. These include peer review mechanisms that can help to improve the quality of a final report before it is published; they also include interview techniques that are specifically tailored to the characteristics and culture of rail employees in a particular member state. The intention is not to replace or undermine these existing local ‘good practices’. In contrast, the intention is to encourage discussion about whether the local approaches provide the same level of support across the various phases of the ERA generic occurrence framework, shown in Figure 1, and to help identify further areas for development within member states.

Events and Causal Factors Charting

Cognitive Interviews

Accimaps

Cognitive Interviews

Change Analysis

A: Causal Sequence

Model

B: Organisational

Model

C: Next Generation Model

Low cost approach

intended for simpler

mishaps.

Higher cost approach

suitable for more complex

mishaps

Additional analytical tools

for dissemination and integration

with other member states.

Data mining techniques

Advanced simulation tools

for reconstruction and training

OR

-23-

2.6 D4.3: Case Studies of ‘Good Practice’ Models and Tools across the Generic Investigation Process Deliverable 4.3 was a further report that was not originally envisaged in the contract. However, it quickly became apparent that NIB members required examples to help them identify the key strengths and weaknesses of the proposed models and tools. We, therefore, developed a series of detailed case studies showing the application of the approach to the Glenbrook rail accident4. This was chosen because the accident combined technical, organisation and human factors. By selecting an incident from outside of any member state, the analysis also avoids any bias towards particular national investigatory systems. However, the details of the accident as presented in this report are sufficiently generic for it still to have purchase in more European railway reporting systems. The Glenbrook collision occurred in New South Wales, Australia. An interurban passenger collided with the rear of an Indian Pacific long distance passenger train that has slowed after reaching a failed signal. A number of factors were involved, from equipment breakdown to poor phrasing of the rules and to deeper issues to do with the safety culture in rail operating organisations within New South Wales. Seven people were killed in the accident. The subsequent accident report makes many references to a ‘culture of on-time running’ that existed at the State Rail Authority of New South Wales. The concern to meet the timetable deadlines led to drivers being forced to operate trains without functioning radios or with defective brakes. The implications of this concern for ‘on time running’ are spelled out in later sections of the report “degraded modes of operation accidents are more likely to occur, particularly if employees acting under the imperative of on time running are trying to have the infrastructure perform more efficiently than it is capable of doing” [p.150]. As mentioned, the Glenbrook accident was used in D4.3 to show the application of the selected tools to support different accident models, illustrated in Figure 3. For example, Figure 4 shows the ECF modelling approach from D4.3 for the case study, as part of the Causal Sequence accident model. As can be seen, the rectangles denote events that can be identified from the eye witness material derived from cognitive interviews from a range of witnesses and from the technical analysis of logging devices. For instance, in Figure 3 there are events to denote that Driver Willoughby sees the red aspect of signal 41.6 and the slows the train to contact the signaller via the signal post telephone. These are connected using arrows to indicate knock-on effects leading to an adverse event, denoted using the diamond shape. In this case, the successive delays and slow movement of the India Pacific train combined with the event labelled ‘Driver of Interurban train passes signal 40.8 without realising that the India Pacific has slowed in front of them’ to cause the collision.

4 Special Commission of Inquiry into the Glenbrook Rail Accident-Final Report, Chaired by P. A. McInerney, Special Commission of Inquiry into the Glenbrook Rail Accident, Currently available from the New South Wales Independent Transport Safety and Reliability Regulator, Sydney, Australia, April 2001. C.W. Johnson and C. Shea, The Contribution of Degraded Modes to Accidents in the US, UK and Australian Rail Industries. In A.G. Boyer and N.J. Gauthier (eds.), Proceedings of the 25th International Systems Safety Conference, Baltimore, USA, International Systems Safety Society, Unionville, VA, USA, 626-636, 0-9721385-7-9, 2007.

-24-

Key

Condition

Event

Continuation

Power unit fails for train sensing circuit covering signals 40.8 and 41.6.

Track-side telephone is only authorised way to

contact signaller

Driver of Interurban train passes signal 40.8 without

realising that the India Pacific has slowed in front of

them.

Signals 40.8 and 41.6 fail to safe with a stop

indication.

Diver Willoughby on the India Pacific sees red aspect of signal 41.6.

Diver Willoughby slows train and tries to contact signaller

via signal post telephone.

Diver Willoughby obtains permission from the signaller

and sets off with caution creating further delays.

Safeworking Unit 245 allows drive with

permission to pass singal with approval but only

showing extreme caution

Diver Willoughby on the India Pacific sees red aspect of

signal 40.8.

Diver Willoughby slows train and tries to contact signaller

via signal post telephone.

Initially the track-side telephone is locked so driver delayed while he fetches key from cab.

Diver Willoughby fails to contact the signaller, waits one minute then proceeds

with caution.

Interurban train collides with rear of India Pacific

train.

Outcome

Crew not authorised to use their on-board satellite

telephone to contact signaller

Driver Willoughby believes he cannot contact

signaller because press to ring button was broken

Figure 4: Initial Overview of the Glenbrook Accident Further examples were provided for all of the models recommended in Figure 3 of this report. Powerpoint presentations were then prepared and distributed to the NIBs for the plenary session scheduled in February 2010 following the extension of the contract.

-25-

2.7 D5: Minutes of Validation Meetings with NIBs D5 provided detailed notes from a site visit to an NIB. The purpose of the visit was to discuss initial requirements for tools that might support the ERA generic accident investigation process. Secondary aims were to identify the particular requirements of a new and dynamic National Investigatory Board. The Agency concerned was making significant steps to establish professional investigatory practices with finite resources. The small team of investigators worked closely together. An open and friendly atmosphere supported a range of formal and informal but effective processes. Their work has been guided by an impressive range of training courses so that the investigatory team are aware of many of the most. advanced tools available. A number of detailed insights were derived, in order of discussion during the meeting and not necessarily in order of priority:

1. Peer Review. Irrespective of the tools used, in small teams it is critical that peer review be used prior to the publication of incident reports.

2. Need for EC Judicial Support. More than tool support, NIBs need help in establishing judicial and legal relationships for their work. Other NIBs (Italy, Romania) also have complex relationships with Police and other agencies that can prevent access to evidence.

3. Need for ERA Review of NIB Investigation ‘Best Practices’. As identified in Lille, many

present practices are informal but well motivated – even though investigators ARE familiar with more advanced techniques. The RAIU argued that further site visits to NIBs might help collect ‘best practice’ as in point 1 above and clarify NIB requirements from ERA, as in point 2, that might support recommendations for particular tools.

4. NIB Concerns about Tools. As anticipated, several of the investigators argued that if tools are use prescriptively then “we will become little more than form-fillers and our task or training is to INVESTIGATE” that implies active analysis.

5. Need for ERA Guidance in Curriculum Development. The RAIU is a relatively new body. They had taken tra8ining courses from the UK (RAIB and Network Rail). From the US (NTSB) and from Canada (TSB). They had also taken University courses (Birmingham and Cranfield). These observations, in particular, the need to go so far afield illustrates the need for ERA to develop courses that might provide a one-stop, lower cost solution to NIB training requirements.

A subsequent finding from this project is that ERA should consider funding a further set of local site visits to NIBs as a means of identifying and exchanging local best practices across member states. This would provide a bottom-up consistency through good practices as well as the more top down recommendations of particular tools and models that are advocated in the deliverables of this project.

-26-

3. Introduction to this Deliverable for WP6 The following sections extract sections from the original proposal for this project that describe the context for WP6 and the associated deliverable, summarised in this document. 3.1 WP6 Description: The final stage of the project will be a ‘close-up’ meeting. This will take place at either at the ERA Headquarters in Valenciennes or at the Lille Conference Centre, as appropriate, on the 10th January 2010. The key objectives for this meeting will be to review the deliverables from all work packages and to ensure that the results of the review are well integrated with wider initiatives, both by the ERA and the Commission.

3.2 WP6 Methods: The close-up meeting will conduct a formal review of the project as a whole but also allow for less formal reflection on the contribution of the project to wider objectives for the European Railway Agency, for individual investigatory bodies, and also for railway industry safety across member states. This is significant because the challenges created by incidents and accidents change over time, for instance as technology and working practices change in response to Council Interoperability Directives EC/1996/48 and EC/2001/16. The implementation of ‘Technical Specifications for Interoperability’ often relies on new technologies and infrastructures. There is little prospect that new generations of complex systems will be 100% reliable. There will be times when systems have to be reconfigured, when upgrades must be installed or when new hardware is brought on-line. It is important to take the time to ensure that the deliverables from this project will be robust and usable for future generations of European railway applications.

3.3 WP6 Deliverables: The primary deliverable for work package six will be the minutes of the close-up meeting.

Electronic copies in PDF and in Word format will be provided by email to the project partners seven days before the associated review meeting. If appropriate, and in consultation with the project managers, we will also prepare a shorter conference paper to be presented jointly by all of the partners to help disseminate the work supported by the European Railway Agency and to provide an additional means of communicating our results back to the member states and other stakeholders that supported this project. 3.4 Revisions to WP6: At the inception meeting, it was decided that the output from this project would be presented at a final meeting of the NIBs to be held in Lille in February 2010. This meeting replaced the closed-out session described above but also involved the extension of the contract beyond the timescales initially envisaged in the contract. The legal arrangements were revised to allow for the work to be extended to include the February meeting. 3.5 Comments from the February 2010 NIB Plenary As mentioned above, the final project presentation was delivered to the NIB Plenary at the ERA offices in Lille, 9th-10th February 2010. Two presentations were made – the first focussed on the processes that were used to inform the selection of the tools and methods. Key slides from this presentation are included in Figure 5. The second presentation presented the case study material – this is illustrated in Figure 6 drawing on material from the work on Glenbrook.

-27-

Figure 5: Overview of the First NIB Presentation (March 2010)-

-28-

Figure 6: Overview of the Second NIB Presentation (March 2010)-

-29-

Figure 6: Overview of the Second NIB Presentation (March 2010)- [Cont.]

-30-

The work was well received by the delegates at this meeting. A number of issues were discussed and these are summarised below:

• Member states might find it useful to have a list of those techniques that are presently being used by leading agencies in other industries. This is an important point and deliverable D3 presents some of this information. However, experience in previous projects with NASAS and the NTSB have shown that often techniques are ‘recommended’ by leading agencies but they are not always used by investigators ‘in the field’. In particular, the techniques taught in training courses are often amended or abridged. This reinforces the comments made in the later stages of the present project that a ‘bottom-up’ analysis could be made of ‘good practice’ in NIBs at the moment. It might also be a useful way to support the questionnaires and survey materials being gathered by Rob Rumping within a parallel ERA project.

• The later stages of the project show better awareness of initiatives outside the Anglo-Saxon

countries. This is an important point and the initial feedback on the need to provide a balanced view has helped the project. In particular, the Lille meeting at the end of this project has shown that there are strong links between the ‘Next Gen’ model and concerns voiced at the German speaking meetings – where, for instance, some participants identified a need to look beyond high-level classifications such as ‘derailment’ to look for more detailed patterns. This is entirely consistent with the ERA work on causal taxonomies and with the data mining techniques advocated for future work within the agency. In other words, the initial taxonomy drafted by Bart Accou might be extended with input from the task forces and special groups, such as the German Speaking NIBs to reflect the differences of interpretation of common incidents such as derailments. Data mining techniques can then be used to collate and present information when these different incidents are reported. These approaches have also been extended to support accident reports written in a number of different European languages with automated partial translation.

• Are there any metrics for assessing safety maturity both within an SMS or looking more closely at the investigation process? This question was raised after the second presentation and related most closely to the early work packages, especially deliverable 3. Some of the information was provided in the overview but there is other relevant work being conducted by EUROCONTROL and the FAA in aviation and by the Canadian TSB. These issues go significantly beyond the focus of this project but they are strongly linked – because they also relate to the work on safety targets and safety indicators. Several other ERA projects can provide input on these issues – for instanmce by acknowledging that there may be different levels of safety culture for organisations at raically different levels of safety maturity. In other words some states may have a strong safety culture even though they have significant steps to take towards the implementation of a mature SMS. These issues might require some coordination across the various task forces and initiatives drawing on the safety KPI work being conducted under other commission projects.

• Can we link the bottom-up site visits recommended in future work in the second presentation to the ERA investigation questionnaires? Rob Rumping’s questionnaires address many of the recommendations that came out of the closing sections of this project by providing more detailed information about good practices already being used by NIBs but t might be that the results are qualitatively different when you compare the deliverable report from WP5 with the survey results. It would be easy to check – for instance, by comparing the responses from the pilot of the questionnaire from Hungary with the report D5 from this project to see how well the answers support each other or differ in content.

• Continuing concerns on the Legal Status of Investigations. Although this project focuses on tools and models, legal issues were consistently mentioned by the NIBs in discussions about the two presentations in Lille. Some states are concerned that the output from any investigation process should not be admissible in court. Others focus more on the memorandum of understanding as a

-31-

protection against the use of materials in court that were intended to be used ‘without blame’. At the end of the project, I have some concerns that the Accimap organisational approach raises considerable questions if it were to be presented in court because it looks beyond the operator and may include, for instance the financial regulator or NSA, as part of the analysis. I am unaware of any such materials ever being used in court of any member state – although a similar technique was used by barristers preparing for the Ladbroke Grove litigation. This question deserves wider consideration as part of the wider review of tools and techniques in the context of legal provisions for investigations.

• Barriers of Money or Technology? In the opening address for the second day of the meeting, Anders Lundström emphasised that the future of European cooperation in investigations can be supported by a range of different measures. However, information exchange and communication was identified as a key priority for ERA. In the breaks, I was asked how feasible it would be to use the more advanced data mining and simulation techniques in the NextGen model. The technology already exists but it is only widely used by organisations like Google or in the case of simulations driven by accident scenarios in NTSB research projects. In talking to the NIB representatives, I was struck by the diversity of maturity across member states. If we supported the implementation of these more innovative techniques, there may be a danger of moving well beyond the interests and capabilities of some newer member states. Equally, if these more innovative techniques are not considered then there is a danger that some NIBs may always feel that they are contributing to other states without being challenged to innovate in future ERA projects.

-32-

4. Closing Summary: This document presents the final deliverable for a project intended to guidance on appropriate tools and models to support the ERA generic accident investigation process. D1 provided an overview of the inception meeting. D2 provided an initial list of tools, methods and techniques that might be used to assist member states across all stages of the ERA generic occurrence investigation process. The present deliverable uses this enumeration to support a more detailed level of analysis. D3 assessed candidate methods against an agreed set of ERA validation criteria. D4 identifies a range of methods, tools and techniques that can support common practices across member states. In particular, D4.1 proposed a number of requirements for future investigation tools and describes how there criteria were validated by NIBs and with support from ERA staff. Deliverable 4.2 went on to identify three different approaches: a relatively simple causal sequence model involving cognitive interviews as well as Events and Causal Factors analysis; a more complex organisational approach including cognitive interviews, Accimaps and Change Analysis, and finally a Next Generation model based on either of the previous two approaches but also including Data Mining and Simulation/Training tools that are intended to help the dissemination of lessons learned between neighbouring member states. D4.2 also stressed that local variants on these tools might also be integrated into these more general models. D4.3 went on to present a detailed case study illustrating the application of the proposed tools to a major rail accident. A key element of the case study was that the Next Generation models provide a road map for integration with other ERA initiatives including the common causal classification work. D5 provided detailed observations from a site visit to an NIB. Finally, this report summarises each of the previous deliverables and presents brief findings from a final NIB meeting that was used to ‘close out’ the project.