Independent safety assessment by a CSM Assessment … A… · · 2016-04-21Independent safety...
35
Independent safety assessment by a CSM Assessment Body (RASBO) Planning, delivery, management and independent safety assessment report Presentation to ERA Arthur D. Little Limited Science Park Milton Road Cambridge CB4 0FH United Kingdom Tel.: +44 1223 427 100 www.adlittle.co.uk
Transcript of Independent safety assessment by a CSM Assessment … A… · · 2016-04-21Independent safety...
Independent safety assessment by a CSM
Assessment Body (RASBO)
Planning, delivery, management and independent safety
assessment report
Presentation to ERA Arthur D. Little Limited
Science Park
Milton Road
Cambridge CB4 0FH
United Kingdom
Tel.: +44 1223 427 100
www.adlittle.co.uk
2
Content
1 About Arthur D. Little
2 Introduction to independent safety assessment
3 Timing of independent safety assessment
4 Planning an independent safety assessment
5 Conducting the independent safety assessment
6 Management of findings
7 Contents of the safety assessment report
8 Cross acceptance
9 Value of independent safety assessment
3
Arthur D. Little is the world's first Management Consultancy,
focusing on technology intensive sectors
Innovating business – since 1886
Today we are the only premier global management consulting firm with a
125 year track record. Founded as a technology consultancy, throughout
our history we have contributed to numerous ground breaking innovations
Linking strategy, technology and innovation
We are acknowledged as a thought leader in linking strategy, technology
and innovation. We focus on technology intensive sectors delivering
business transformation
Working uniquely, different
Our people and their side-by-sideTM approach integrate cross-sector
knowledge and next level thinking seamlessly into your business.
How we work is innovative, what we deliver is positive change
1 Content – About Arthur D. Little
4
Safety and risk has been a part of the firm’s heritage for over a
century, highlighted by numerous landmark assignments
Waste minimization
services – Circa 1905
Warning agents for natural
gas – 1952NIOSH chemical
standards – 1979
Mass Transit Railway Signaling ISA –
1988 to date
Strategic Safety Management
Training – 1993 to date
Delhi Metro ISA - 2003 to
date
Prototype Safety Case supporting
UK Rail Privatisation – 1993-1995
EPA Risk Assessment
Support – 1979-1985
Texas City
investigation – 1947
EPA RCRA Program
Support – 1976-1980
Hooker Chemical
Investigations – 1979
Bhopal investigation – 1985
BP Forties Field Quantified Risk
Assessment – 1988-1990
Shell/Exxon UK Fife Plant Risk Studies -
1987 to date
ENI Refining Risk Studies – 1990 to
date
Swiss and Munich Re EHS
Management Systems – 1999-2000 UK Nuclear Liabilities Assessment –
1999-2000
Exxon Valdez –
1989
Greek Gas Pipeline Network
Risk – 1990 to date
Development of Hong Kong
MTR Safety Management –
1989 to date
Safety Development
Programs – 2008 to
date
Safety critical
systems – 1995 to
date
Major HAZOP/SIL
studies worldwide –
1970 to date
1 Content – About Arthur D. Little
5
We have undertaken signalling ISA assignments for many
clients around the world
MTR Corporation
ATC Replacement
Lantau Airport Railway
Quarry Bay Congestion Relief
Driverless Turnaround
Tseung Kwan O
LAR 4-tracking
YAM O Signalling
Disney Resort Line
West Island Line
KCR Corporation
East Rail Resignalling
European Signalling Products
Generic ERTMS (EVC and RBC)
CBTC GoA Level 4
Delhi Metro, Chennai Metro
Delhi Phase 1, 2 & 3
Chennai Metro
1 Content – About Arthur D. Little
EuropeanSignalling Projects
Betuweroute ERTMS L2 trackside
SA-NBS ERTMS L2
LTA
Singapore DTL1
6
Content
1 About Arthur D. Little
2 Introduction to independent safety assessment
3 Timing of independent safety assessment
4 Planning an independent safety assessment
5 Conducting the independent safety assessment
6 Management of findings
7 Contents of the safety assessment report
8 Cross acceptance
9 Value of independent safety assessment
7
An independent safety assessor provides additional assurance
that a given significant change achieves the necessary level of
safety
Conducts independent safety assessment of the risk assessment process
and safety demonstration of the significant change
Does not perform the risk assessment required by the CSM nor the safety
demonstration itself
Provides confidence that the risk assessment and risk management
activities have been properly conducted by the proposer
Conclusions are not binding on the proposer but are an important input to be
taken into account by them
2 Content – Introduction to Independent Safety Assessment
8
An independent safety assessor must be independent and
impartial
Must be independent from the design, risk assessment, risk management,
manufacture, supply, installation, operation/use, servicing and maintenance"
of the system under assessment
Must be free from any pressure or incentive which may affect their
judgement
Must not deliver advice or solutions on how to address non-conformances or
concerns identified by the assessment
2 Content – Introduction to Independent Safety Assessment
9
An independent safety assessor must be competent
Risk management competence
Technical competence
Management systems competence
2 Content – Introduction to Independent Safety Assessment
10
Independent safety assessment is widely used, in many
industries
2 Content – Introduction to Independent Safety Assessment
Independent Safety
Assessment
Rail
Nuclear
Defence
Aviation
Road
Transport
Medical
11
Content
1 About Arthur D. Little
2 Introduction to independent safety assessment
3 Timing of independent safety assessment
4 Planning an independent safety assessment
5 Conducting the independent safety assessment
6 Management of findings
7 Contents of the safety assessment report
8 Cross acceptance
9 Value of independent safety assessment
12
Independent safety assessment is best conducted throughout
the risk management process, from start to finish
Source: ERA
3 Content – Timing of Independent Safety Assessment
13
Content
1 About Arthur D. Little
2 Introduction to independent safety assessment
3 Timing of independent safety assessment
4 Planning an independent safety assessment
5 Conducting the independent safety assessment
6 Management of findings
7 Contents of the safety assessment report
8 Cross acceptance
9 Value of independent safety assessment
14
An independent safety assessment plan should describe, in
practical terms, how the assessment is to be completed
4 Content – Planning an Independent Safety Assessment
Independent SafetyAssessmentPlan
Scope of work
Assessment approach
Assessment team
Schedule of assessment activities
Planned deliverables
15
Content
1 About Arthur D. Little
2 Introduction to independent safety assessment
3 Timing of independent safety assessment
4 Planning an independent safety assessment
5 Conducting the independent safety assessment
6 Management of findings
7 Contents of the safety assessment report
8 Cross acceptance
9 Value of independent safety assessment
16
Content
5 Conducting the independent safety assessment
5.1 General approach
5.2 Gathering data
5.3 Working papers
17
The depth of assessment should be appropriately balanced,
according to the significance of the change
5.1 Content – Conducting the Independent Safety Assessment – General Approach
“Lightweight” assessment to
manage potential high risksBalanced approach
“Heavyweight” assessment to
manage potential lower risks
Vulnerable to surprises and
omissionsPretty much OK
Reflects an imprecise
understanding of potential risks
Significance
of change
Depth of
assessment
18
The depth of assessment can be guided by a thorough
understanding of the management systems and processes and
through an assessment of their strengths and weaknesses
5.1 Content – Conducting the Independent Safety Assessment – General Approach
Understand
Examine the management systems and processes in place
Verify
Test each system or procedure
Emphasis given to possible weaknesses
Focus on those elements that appear most critical
Assess
Analyse such processes for weaknesses
19
Sufficient understanding of all relevant management systems
should be obtained
5.1 Content – Conducting the Independent Safety Assessment – General Approach
Text
Technical
Management
Safety
Management
Quality
Management
Requirements
Capture
Software
Design
Hardware
Design
Verification and
Validation
20
Once the management systems are understood, their apparent
effectiveness should be assessed
5.1 Content – Conducting the Independent Safety Assessment – General Approach
Consider potential impacts
1Evaluate the management
system
2
Set inspection priorities
3
What are the potential impacts if the
management system does not
operate correctly?
Are the specified processes likely to
deliver their defined and/or
necessary objectives?
Are the systems and processes,
coupled with their controls, sufficient
to mitigate the potential impacts?
Ensure potential issues representing
high risk and/or weak management
controls receive sufficient attention
21
Verification should assess the correctness of the strengths
and weaknesses determined in the previous analysis
5.1 Content – Conducting the Independent Safety Assessment – General Approach
22
Verification should assess the correctness of the strengths
and weaknesses determined in the previous analysis (continued)
5.1 Content – Conducting the Independent Safety Assessment – General Approach
23
Content
5 Conducting the independent safety assessment
5.1 General approach
5.2 Gathering data
5.3 Working papers
24
Assessment data can be gathered through a mixture of
interviews and document review
5.2 Content – Conducting the Independent Safety Assessment – Gathering Data
Interviews are a highly effective tool for gathering evidence
25
Assessment data can be gathered through a mixture of
interviews and document review (continued)
5.2 Content – Conducting the Independent Safety Assessment – Gathering Data
Interviews are a highly effective tool for gathering evidence
Documentary evidence is also gathered from review of the
proposer’s documents and records
26
Content
5 Conducting the independent safety assessment
5.1 General approach
5.2 Gathering data
5.3 Working papers
27
Working papers should be maintained by each assessor,
throughout the assessment
5.3 Content – Conducting the Independent Safety Assessment – Working Papers
Can be paper or electronic
Record all assessment activities and their results
Written whilst conducting the assessment activities
Provide a basis for quality assurance
28
Content
1 About Arthur D. Little
2 Introduction to independent safety assessment
3 Timing of independent safety assessment
4 Planning an independent safety assessment
5 Conducting the independent safety assessment
6 Management of findings
7 Contents of the safety assessment report
8 Cross acceptance
9 Value of independent safety assessment
29
A log of findings should be maintained, tracking the status of
all issues identified during the assessment
6 Content – Management of findings
Documents all findings (non-conformities, inadequacies, etc)
Tracks all such findings to closure:
• Original finding, responses from the proposer, updates from
the assessor
• Dates of relevant updates / responses
• Current status (open / closed)
Updated as necessary throughout the assessment
All findings should be closed or non-blocking prior to issue of a
positive safety assessment report, or conditions noted
accordingly
Findings Log
30
Content
1 About Arthur D. Little
2 Introduction to independent safety assessment
3 Timing of independent safety assessment
4 Planning an independent safety assessment
5 Conducting the independent safety assessment
6 Management of findings
7 Contents of the safety assessment report
8 Cross acceptance
9 Value of independent safety assessment
31
The safety assessment report provides conclusions on the
fulfilment of the safety requirements by the significant change
7 Content – Contents of the safety assessment report
Identification of the CSM assessment body
Summary/reference of the independent safety assessment plan
Definition of the scope of the assessment, including limitations
Results of the assessment
• Details of the assessment activities performed to check
compliance with the CSM for risk assessment
• Identified non-compliances and recommendations
Details of cross acceptance
Conclusions
• Does the risk assessment performed by the proposer comply
with the requirements of the CSM?
• Are the risk controls sufficient to allow the change to safely
fulfil its intended objectives?
32
Content
1 About Arthur D. Little
2 Introduction to independent safety assessment
3 Timing of independent safety assessment
4 Planning an independent safety assessment
5 Conducting the independent safety assessment
6 Management of findings
7 Contents of the safety assessment report
8 Cross acceptance
9 Value of independent safety assessment
33
An independent safety assessment often relies on the results
of other assessments performed by third parties
8 Content – Cross Acceptance
Has the third party assessment been performed with the
necessary independence, impartiality and competence?
Is the artefact that is subject to cross-acceptance being used in
the same context as that assumed by the original assessment?
Does the assessment reach clear conclusions?
Are any restrictions raised by the assessment complied with?
34
Content
1 About Arthur D. Little
2 Introduction to independent safety assessment
3 Timing of independent safety assessment
4 Planning an independent safety assessment
5 Conducting the independent safety assessment
6 Management of findings
7 Contents of the safety assessment report
8 Cross acceptance
9 Value of independent safety assessment
35
Done well, independent safety assessment can significantly
increase confidence in the safety of a change
Good understanding & review of management processes allows significant
weaknesses to be identified and corrected
Effective review helps detect systematic errors
Focus on management systems encourages proposer organisations to
continuously improve
9 Content – Value of independent safety assessment
