OSMA Software Assurance Symposium

Mission Success Starts With Safety

OSMA Software Assurance OSMA Software Assurance SymposiumSymposium

(1)

Space Shuttle Return to Flight(The Safety Guy’s View)

Bryan O’ConnorBryan O’ConnorAssociate AdministratorAssociate Administrator

Office of Safety and Office of Safety and Mission AssuranceMission Assurance

"Mission success stands on the foundation of our unwavering commitment to safety"

Administrator Sean O'Keefe January 2003

30 July 2003


(2)

Knowledge and High Risk OperationsKnowledge and High Risk Operations

There are things we know that we know,There are things we know we don’t know,There are things we don’t know we don’t know.

Donald Rumsfeld, U. S. SecDef NATO HQ Press Conference, June ‘02

At NASA (and any other organization engaged in high risk activities)…There are also things we don’t know we know!


(3)

The Risk IcebergThe Risk Iceberg


(4)

The Four Levels of the Risk IcebergThe Four Levels of the Risk Iceberg• Known KnownsKnown Knowns

– Flight Data and demonstrated performanceFlight Data and demonstrated performance– Flight or test-validated analysis, simulations and modelsFlight or test-validated analysis, simulations and models– Operation within certification limitsOperation within certification limits

• Known UnknownsKnown Unknowns– Generic but undemonstrated failure modes and hazardsGeneric but undemonstrated failure modes and hazards– Risk analysis uncertaintiesRisk analysis uncertainties– Acknowledged test and analysis limitationsAcknowledged test and analysis limitations– Unverified modeling and simulation based predictionsUnverified modeling and simulation based predictions– Envelope expansion and operations within certification but out of familyEnvelope expansion and operations within certification but out of family

• Unknown UnknownsUnknown Unknowns– Bad assumptionsBad assumptions– Unfinished experimental researchUnfinished experimental research– Operation outside of certification limits (temperature, Q, tire speed, etc.)Operation outside of certification limits (temperature, Q, tire speed, etc.)

• Unknown KnownsUnknown Knowns– Miscommunicated test or analysis resultsMiscommunicated test or analysis results– Uneven understanding of data or environment across the teamUneven understanding of data or environment across the team– Poor documentation combined with loss of corporate memoryPoor documentation combined with loss of corporate memory


(5)

The Strategy for Safe Return to FlightThe Strategy for Safe Return to Flight

• Known Knowns: (Systems Engineering and Program Management)Known Knowns: (Systems Engineering and Program Management)– High quality program and mission management processes and peopleHigh quality program and mission management processes and people– ““Listen to the hardware”Listen to the hardware”

• Known Known UnUnknowns: (Risk Management)knowns: (Risk Management)– Reduce uncertainties with analysis, test and attention to flight resultsReduce uncertainties with analysis, test and attention to flight results– Recertify to the real environment (has it changed over the years?)Recertify to the real environment (has it changed over the years?)– Treat residual risk (including uncertainty) with conservative ground and flight Treat residual risk (including uncertainty) with conservative ground and flight

procedures, operating margins, contingency plans procedures, operating margins, contingency plans • UnUnknown Unknowns: (Continuous Research, Test and Evaluation)known Unknowns: (Continuous Research, Test and Evaluation)

– Do what we can to better understand the limitations of the system in the real Do what we can to better understand the limitations of the system in the real environmentenvironment

– Constantly challenge old assumptions, models and analysesConstantly challenge old assumptions, models and analyses• UnUnknown Knowns: (Continuous Process Improvement)known Knowns: (Continuous Process Improvement)

– Continuously assess and improve program communications, documentation, Continuously assess and improve program communications, documentation, workforce competenceworkforce competence

– Improve data analysis tools and techniques (e.g. trending)Improve data analysis tools and techniques (e.g. trending)


(6)

Return to Flight: Near Term ActivitiesReturn to Flight: Near Term Activities

• Columbia Accident Investigation Board recommendations to dateColumbia Accident Investigation Board recommendations to date– Develop and implement comprehensive inspection plan for RCCDevelop and implement comprehensive inspection plan for RCC– Modify NIMA memorandum of understandingModify NIMA memorandum of understanding– Develop in-flight TPS inspection and repairDevelop in-flight TPS inspection and repair– Upgrade NASA imaging of shuttle launch and ascentUpgrade NASA imaging of shuttle launch and ascent

• Other Activities focused on Return to FlightOther Activities focused on Return to Flight– External tank upgrades (foam shedding)External tank upgrades (foam shedding)– Improve FOD control at KSCImprove FOD control at KSC– Improve on-vehicle video/digital film downlink capabilityImprove on-vehicle video/digital film downlink capability– Assess KSC and MAF quality systems (including GMIPS)Assess KSC and MAF quality systems (including GMIPS)– Address public risk policy for shuttle over-flightAddress public risk policy for shuttle over-flight– Improve mission management and Certification of Flight ReadinessImprove mission management and Certification of Flight Readiness– Review shuttle requirements (waivers, deviations, exceptions)Review shuttle requirements (waivers, deviations, exceptions)– Validate / improve safety and mission assurance structure (in-line and Validate / improve safety and mission assurance structure (in-line and

independent)…”no there there” ?independent)…”no there there” ?– Improve problem tracking and anomaly resolutionImprove problem tracking and anomaly resolution


(7)

Safety and IndependenceSafety and IndependenceIn

depe

nden

ce

Organization

i Contractor SMA Xi NASA Project SMA X Xi Center (matrixed) SMA X X Xi Center Independent

Assessment; X X X XI HQ Code Q, NAC, ASAP,

IV&V, NESC X X X X X

I GAO, IG, Congress, OSHA X X X X X X


(8)

Typical Safety Program Interfaces Typical Safety Program Interfaces (Example: Headquarters Directed Program)(Example: Headquarters Directed Program)

NASA Administrator

OSMA AACode Q

EnterpriseAA

Center Director

Center SMA

Program

Functional Support

IndependentCommunications Path(re: Program)

Organizational Reporting

SMA support

Insight

Matrix support

Policy

NESCNESC


(9)

Typical Safety Project InterfacesTypical Safety Project Interfaces(Example: Center Managed Projects)(Example: Center Managed Projects)

NASA Administrator

OSMA AACode Q

EnterpriseAA

Center Director

Center SMA

Projects

Functional Support

IndependentCommunications Path(re: Program)

Organizational Reporting

SMA support

Policy

NESCNESC

ProjectsProjects

Projects


(10)

NASA Safety and Engineering CenterNASA Safety and Engineering Center

Training and Training and DevelopmentDevelopment

Safety & Mission Safety & Mission AssuranceAssurance

Chief Engineer Chief Engineer OfficeOffice

Program Program EnterprisesEnterprises

LaRCLaRC

NESCNESC

NASA NASA CentersCenters

IndustryIndustry

Nat’l Nat’l FacilitiesFacilities

AcademiaAcademia•Supplement to agency Supplement to agency system safety and system safety and engineering activitiesengineering activities

•Independent engineering Independent engineering assessments and testingassessments and testing

•Independent safety and Independent safety and mission assurance analysis, mission assurance analysis, surveillance, auditsurveillance, audit

$$

Training TasksTraining Tasks Engineering/safety tasksEngineering/safety tasks


(11)

The Planned Result: Ex Scientia Salus The Planned Result: Ex Scientia Salus (From Knowledge: Safety)(From Knowledge: Safety)

OSMA Software Assurance Symposium

Documents

Transcript of OSMA Software Assurance Symposium