Perspectives on Risk‐Informed Decision‐Making · 2020. 10. 31. · IBEX Aura GEMS Juno MAVEN...
Transcript of Perspectives on Risk‐Informed Decision‐Making · 2020. 10. 31. · IBEX Aura GEMS Juno MAVEN...
1
Perspectives on Risk‐Informed Decision‐MakingEnabling a Compelling Tomorrow
Michael A. JohnsonChief Technologist, Engineering and Technology DirectorateNASA Goddard Space Flight Center
2
NASA/SP-2010-576Version 1.0April 2010
Questions considered‐
• What is NASA’s view on risk and risk‐informed decision making?
• How does NASA incorporate risk insights into its day‐to‐day decision making?
• What are challenges and successes with integrating risk‐informed decision making at NASA?
3
National Aeronautics and Space Administration (NASA)capabilities are distributed across the Nation.
Background
4
Goddard Space Flight Center (GSFC) is distributed across five locations.
The main campus is in Greenbelt, MD.
GreenbeltGreenbelt
Goddard Institute for Space StudiesGoddard Institute for Space Studies
Independent Verification and Validation FacilityIndependent Verification and Validation Facility
Wallops Flight FacilityWallops Flight Facility
Goddard Space Flight Center
White Sands ComplexWhite Sands Complex
Background
5
TRACETRACE
ACEACE
SOHOSOHO
RHESSIRHESSI
WindWindVoyagerVoyager
GeotailGeotail
TIMEDTIMED
FASTFASTPolarPolar
StereoStereo
THEMISTHEMIS
IMAGEIMAGE
MMSMMSSolar‐BSolar‐B
QuikSCATQuikSCAT
ACRIMSATACRIMSAT
EO‐1EO‐1
COBECOBE
Landsat 7Landsat 7
TRMMTRMM
TDRSSTDRSS
AquaAqua
TerraTerra
CloudSatCloudSat
CALIPSOCALIPSO
GRACEGRACE
SORCESORCE
ICESatICESat
MessengerMessenger
CassiniCassini
New HorizonsNew Horizons
LROLRO
AquariusAquarius
RXTERXTE
ClusterClusterSDOSDO
NPPNPP
AIMAIM
LDCMLDCM
GPMGPM
TOMSTOMS
JWSTJWST
ComptonGRO
ComptonGRO
HSTHST
SpitzerSpitzer
SwiftSwiftFUSEFUSE
GALEXGALEX
FermiFermi
WMAPWMAP
Mars ScienceLaboratoryMars ScienceLaboratory
POESPOES
GOESGOES
WISEWISE
IBEXIBEX
AuraAura
GEMSGEMS
MAVENMAVENJunoJuno
LADEELADEE
RBSPRBSP
TWINS(Instrument)
TWINS(Instrument)
EUVEEUVE
SWASSWAS
NuSTARNuSTAR
IntegralIntegralIUEIUE
ERBSERBS
TOPEXTOPEX
Osiris‐Rex(Sample Return)
Osiris‐Rex(Sample Return)
PioneerPioneer
GalileoGalileo
Astro‐HAstro‐H
5
GSFC has flown over 300 Heliophysics, Earth Science, Astrophysics, and Planetary missions
GSFC provides essential technical and scientific support to address national and world societal needs through our work with and for other Federal agencies.
We derive and share information, solutions, and technology for NASA, the Nation, and the world.
We engage with other NASA centers, universities, and industry in search of opportunities to blend our competencies with their specialized assets in order to obtain knowledge that can be applied to addressing some of the most profound and compelling questions of our time.
Background
6
GSFC has a history of mission success:
No spacecraft failures since 1975.
Success has been derived in part, by enforcing a mostly static set of Mission Assurance Requirements (MAR).
1
3
5
12
8
13
9 9
13
11
9
5
78
4
8 8
2
5
8
32
5
23
4
1 1 1
32
4 45
4
6
87
5 5 5
7
4
6
4
2 2
7
2
45
2
43
7
23
21
3
1
1
1
1
3
12
12
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
Success Failure (due to LV) Failure (due to SC)
3
4
* GSFC does not develop or deliver launch vehicles
*
GSFC Mission Outcomes
301 SuccessesNumerous Launch Vehicle (LV) failures 0 Spacecraft (SC) failures since 1975
Background
7
NASA/SP-2010-576Version 1.0April 2010
• What is NASA’s GSFC’s view on risk and risk‐informed decision making?
• How does NASA GSFC incorporate risk insights into its day‐to‐day decision making?
• What are challenges and successes with integrating risk‐informed decision making at NASA GSFC?
8
It would be wrong to conclude that mission success derived from the MAR approach would obviate a need for change.
To effectively and efficiently meet today’s and tomorrow’s space exploration challenges, GSFC must not only make advancements in what we do; we must also advance how we do it.
1
3
5
12
8
13
9 9
13
11
9
5
78
4
8 8
2
5
8
32
5
234
1 1 1
32
4 454
6
87
5 5 5
7
4
6
4
2 2
7
2
45
2
43
7
232
1
3
1
1
1
1
3
12
12
Success Failure (due to LV) Failure (due to SC)
34
* GSFC does not develop or deliver launch vehicles
*
GSFC Mission Outcomes
What is GSFC’s view on risk and risk‐informed decision making?
9
Key Elements• Risk‐informed Framework• Risk‐informed Requirements Generation• Risk‐informed Decision Making• Risk‐informed Review and Audit
GSFC implements a Risk‐based Safety and Mission Assurance (SMA)‐ The process of applying limited resources to maximize the chance for safety and mission success by focusing on mitigating specific risks that are applicable to the project vs. simply enforcing a set of requirements because they have always worked.
Risk‐based Safety and Mission Assurance (SMA) began formal GSFC implementation in 2016
What is GSFC’s view on risk and risk‐informed decision making?
10
TRACETRACE
ACEACE
SOHOSOHO
RHESSIRHESSI
WindWindVoyagerVoyager
GeotailGeotail
TIMEDTIMED
FASTFASTPolarPolar
StereoStereo
THEMISTHEMIS
IMAGEIMAGE
MMSMMSSolar‐BSolar‐B
QuikSCATQuikSCAT
ACRIMSATACRIMSAT
EO‐1EO‐1
COBECOBE
Landsat 7Landsat 7
TRMMTRMM
TDRSSTDRSS
AquaAqua
TerraTerra
CloudSatCloudSat
CALIPSOCALIPSO
GRACEGRACE
SORCESORCE
ICESatICESat
MessengerMessenger
CassiniCassini
New HorizonsNew Horizons
LROLRO
AquariusAquarius
RXTERXTE
ClusterClusterSDOSDO
NPPNPP
AIMAIM
LDCMLDCM
GPMGPM
TOMSTOMS
JWSTJWST
ComptonGRO
ComptonGRO
HSTHST
SpitzerSpitzer
SwiftSwiftFUSEFUSE
GALEXGALEX
FermiFermi
WMAPWMAP
Mars ScienceLaboratory
Mars ScienceLaboratory
POESPOES
GOESGOES
WISEWISE
IBEXIBEX
AuraAura
GEMSGEMS
MAVENMAVENJunoJuno
LADEELADEE
RBSPRBSP
TWINS(Instrument)
TWINS(Instrument)
EUVEEUVE
SWASSWAS
NuSTARNuSTAR
IntegralIntegralIUEIUE
ERBSERBS
TOPEXTOPEX
Osiris‐Rex(Sample Return)
Osiris‐Rex(Sample Return)
PioneerPioneer
GalileoGalileo
Astro‐HAstro‐H
10
Developing space flight systems is GSFC’s “day‐to‐day”
• What is GSFC’s view on risk and risk‐informed decision making?
• How does GSFC incorporate risk insights into its day‐to‐day decision making?
• What are challenges and successes with integrating risk‐informed decision making at GSFC?
11
Ref: Jesse Leitner, Bhanu Sood, Eric Isaac, Jack Shue, Nancy Lindsey & JeannettPlante (2018) Risk‐based safety and mission assurance: Approach and experiences in practice, Quality Engineering, 30:4, 648‐662, DOI: 10.1080/08982112.2018.1473584
Printed circuit boards are a foundational component of most spaceflight electrical, electronic, and electromechanical systems
How do we assess the quality of the bare board and its manufacturing process?
How does GSFC incorporate risk insights into its day‐to‐day decision making?
Populated printed circuit board
Example: Printed Circuit Board Test Coupon Nonconformance Handling
12
Printed Circuit Board (PCB) PCB
PCB PCB
COUPON
COUPON
COUPON
COUPON
PCB PANEL
One PCB panel typically yields multiple bare PCBs. Test “coupons” inform the board/ fabrication process quality assessment.
Example: Printed Circuit Board Test Coupon Nonconformance HandlingHow does GSFC incorporate risk insights into its day‐to‐day decision making?
Populated printed circuit board
13
Traditional Approach: Evaluate test coupons to favored standard. Reject and refabricate all boards associated with nonconforming coupons.
Risk‐based Approach: Use requirements from one or more commonly‐used standards to define minimum requirements for a range of risk levels. Assess risk when nonconformance is found and determine whether risk is acceptable. Only refabricate if cause for nonconformance is understood and can be eliminated.
Risk‐based Results Summary: Out of 193 boards with nonconforming test coupons, over 85% had either no elevated risk or very low risk associated with the non‐conformance. The risk acceptance resulted in millions of dollars and hundreds of weeks of schedule saved relative to the traditional approach.
Copper wicking
Capped vias
Copper wicking and vias are two (of several) coupon elements evaluated during coupon analysis.
Example: Printed Circuit Board Test Coupon Nonconformance HandlingHow does GSFC incorporate risk insights into its day‐to‐day decision making?
14
Traditional Approach: Evaluate test coupons to favored standard. Reject and refabricate all boards associated with nonconforming coupons.
Risk‐based Approach: Use requirements from one or more commonly‐used standards to define minimum requirements for a range of risk levels. Assess risk when nonconformance is found and determine whether risk is acceptable. Only refabricate if cause for nonconformance is understood and can be eliminated.
Risk‐based Results Summary: Out of 193 boards with nonconforming test coupons, over 85% had either no elevated risk or very low risk associated with the non‐conformance. The risk acceptance resulted in millions of dollars and hundreds of weeks of schedule saved relative to the traditional approach.
Copper wicking
Capped vias
Copper wicking and vias are two (of several) coupon elements evaluated during coupon analysis.
Example: Printed Circuit Board Test Coupon Nonconformance HandlingHow does GSFC incorporate risk insights into its day‐to‐day decision making?
15
NASA/SP-2010-576Version 1.0April 2010
• What is GSFC’s view on risk and risk‐informed decision making?
• How does GSFC incorporate risk insights into its day‐to‐day decision making?
• What are challenges and successes with integrating risk‐informed decision making at GSFC?
16
What are challenges and successes with integrating risk‐informed decision making at NASA?
NASA is increasingly infusing small satellites (mass < 180 kg) into its missions and mission planning activities
• Increased cost and schedule efficiencies
• Greater spatial, temporal, and spectral coverage
• More frequent launch opportunities
• Greater potential for innovative mission implementation approaches
• Greater opportunities to leverage private sector innovation
Dellingr CubeSat deployment from the International Space Station, December 2017
Development and use of SmallSats raises multiple challenges and successes, many relevant to RIDM
17
Leveraging private sector innovation
The use of commercial off‐the‐shelf (COTS) components is increasing.
However, vendors may provide minimal mission assurance documentation, and components may have little flight heritage.
This presents an SMA challenge.
Dellingr 6U CubeSat‐ internal view (during development)
Dellingr CubeSat. 10 x 20 x 30 cm, ~11 kg
What are challenges and successes with integrating risk‐informed decision making at NASA?Example: The Dellingr CubeSat Mission
18
Consistent with this SMA challenge, the Dellingr spacecraft exhibited numerous component anomalies and failures during development and on‐orbit.
Yet, the mission is successful because the “mission system” was resilient to component failure.
Ref: Cutting Edge‐ Goddard’s Emerging Technologies, Fall 2018
What are challenges and successes with integrating risk‐informed decision making at NASA?Example: The Dellingr CubeSat Mission
19
Success Lesson:
RIDM should not be considered just a safety and mission assurance team activity.
The degree to which the approach is infused across the mission full‐lifecycle (formulation through operations) and throughout the mission team can provide mission resiliency and inform success.
Ground systems
Operations teamScience andengineering team
Spacecraft
The Dellingr “system” mitigated numerous on‐orbit component failures/anomalies and infrastructure challenges.
What are challenges and successes with integrating risk‐informed decision making at NASA?Example: The Dellingr CubeSat Mission
20
Other High‐level RIDM Integration Challenges
• The fear of failure: changing a “failure is not an option” dominant culture is not easy
• Organizational compartmentalization is a barrier to “mission system‐level thinking”
• Personal comfort and protection can be derived from strictly adhering to static SMA requirements
• Organizational inertia can be hard to overcome
• Organizations may need to re‐learn the “art of thinking”
An organization’s history of success can be its largest barrier to RIDM infusion
What are challenges and successes with integrating risk‐informed decision making at NASA?
21
In Closing
• GSFC had a history of success prior to implementing risk‐based SMA
• Yet, advancement essential and is not just informed by the “what”; it’s also the “how”
• GSFC has migrated to risk‐based SMA, yielding millions of dollars and hundreds of weeks of schedule saved relative to a static requirement‐based approach
• The degree to which a risk‐informed approach is infused across the mission full‐lifecycle (formulation through operations) and throughout the mission team can provide mission resiliency and inform success.
• An organization’s history of success can be its most significant barrier to RIDM infusion
Re‐learning the “art of thinking” can lead to a compelling tomorrow.
22
For further information:
1 Dezfuli, Homayoon & Stamatelatos, Michael & Maggio, Gaspare & Everett, Chris. (2010). NASA/SP‐2010‐576, NASA Risk‐Informed Decision Making Handbook.
23
2Jesse Leitner, Bhanu Sood, Eric Isaac, Jack Shue, Nancy Lindsey & Jeannett Plante (2018) Risk‐based safety and mission assurance: Approach and experiences in practice, Quality Engineering, 30:4, 648‐662, DOI: 10.1080/08982112.2018.1473584
For further information:
The paper Risk‐based safety and mission assurance: Approach and experiences in practice includes additional PCB case study details and additional risk‐based SMA examples and results.
url https://doi.org/10.1080/08982112.2018.1473584