TRUSTWORTHY DESIGN VALIDATION OF A SATELLITE PLATFORM

TRUSTWORTHY DESIGN VALIDATION OF A SATELLITE PLATFORM

Viet Yen Nguyen [email protected] this research is commissioned by: MDDays 2013, Eindhoven, Netherlands

OVERVIEW

1.  Space System Software Design & Validation 2.  Formal methods with COMPASS toolset 3.  Satellite case study 4.  Conclusions

TECHNOLOGY CONTEXT OF THIS TALK

Space Engineering

Formal Methods

Academia Industry

Requirements

Safety & Dependability

Control

Software Hardware

FDIR

Thermal

Systems

Cost

…

…

Logics

Model Checking

Markov Chains

Small-Step Semantics

Timed automata

Hybrid systems

Reachability

SAT-solving

Formal Semantics

…

…

CASE: SATELLITE OF ESA MISSION IN DEVELOPMENT

•  Platform keeps satellite in space, like car’s chassis. –  control & data unit, –  propulsion, –  telemetry, tracking & cmd. –  power, –  attitude & orbit control sys., –  reconfiguration module, –  etc.

•  Fault detection, isolation, recovery (FDIR) software and hardware: –  redundancies + recovery, –  compensation algorithms, –  failure isolation schemes, –  omnipresent in satellite.

Note: shown satellite is not the one of the case study.

FDIR

FDIR FDIR

FDIR

FDIR

FDIR

FDIR

FDIR

FDIR

DEMANDING REQUIREMENTS NEED EXTENSIVE V&V

•  “Satellite’s in-orbit lifetime shall be at least 5 years.” •  “Satellite’s platform reliability shall be equivalent or less than

1000 FIT.” •  “The probability that the satellite transits to safe mode shall be

lower than 0.005, considering 40 hours for ground time to repair.”

•  “In case of an over-current, a switch to the redundant coil is performed by OBDH SW.”

•  “While in nominal mode and an lost uplink, the transponder shall transit within 10 seconds to safe mode.”

•  … thousands of similar requirements!

•  In presence of space hazards like radiation, extreme heat/cold, radio latencies, gravitational forces, etc.

Note: shown requirements are not from the case study.

EXAMPLE OF V&V IN SPACE SYSTEMS ENGINEERING

ECSSȬMȬSTȬ10CȱRev.ȱ1ȱ6ȱMarchȱ2009ȱ

19ȱ

4.4 Project phasing

4.4.1 Introduction Theȱlifeȱcycleȱofȱspaceȱprojectsȱisȱtypicallyȱdividedȱintoȱ7ȱphases,ȱasȱfollows:ȱ

x Phaseȱ0ȱȬȱMissionȱanalysis/needsȱidentificationȱ

x PhaseȱAȱȬȱFeasibilityȱ

x PhaseȱBȱȬȱPreliminaryȱDefinitionȱ

x PhaseȱCȱȬȱDetailedȱDefinitionȱ

x PhaseȱDȱȬȱQualificationȱandȱProductionȱ

x PhaseȱEȱ–Utilizationȱ

x PhaseȱFȱ–ȱDisposalȱ

AȱtypicalȱprojectȱlifeȱcycleȱisȱillustratedȱinȱFigureȱ4Ȭ3.ȱȱ

Projectȱ phasesȱ areȱ closelyȱ linkedȱ toȱ activitiesȱ onȱ systemȱ andȱ productȱ level.ȱDependingȱ onȱ theȱ specificȱ circumstancesȱ ofȱ aȱ projectȱ andȱ theȱ acceptanceȱ ofȱinvolvedȱrisk,ȱactivitiesȱcanȱoverlapȱprojectȱphases.ȱ

Atȱ theȱ conclusionȱ ofȱ theȱ majorȱ activitiesȱ andȱ theȱ relatedȱ projectȱ reviewsȱconfigurationȱbaselinesȱareȱestablishedȱ(seeȱECSSȬMȬSTȬ40).ȱ

Figureȱ4Ȭ3:ȱTypicalȱprojectȱlifeȱcycleȱ

ȱ

•  Requirements analysis. •  Fault tree analysis. •  Review and inspection.

•  HW/SW co-testing. •  Early prototyping. •  Control simulation.

•  End-to-end IS testing. •  Operations readiness. •  Mission scenario tests.

•  In orbit testing.

SPACECRAFT IS BECOMING “FLYING SOFTWARE” Flight Software Complexity

3/5/2009 28

Growth in Code Size for Human and Robotic Missions

1

10

100

1000

10000

100000

1000000

1000000019

68

1970

1972

1974

1976

1978

1980

1982

1984

1986

1988

1990

1992

1994

1996

1998

2000

2002

2004

Year of Mission

NCSL

(log

sca

le)

unmannedmannedExpon. (unmanned)Expon. (manned)

Non

-Com

men

t Sou

rce

Line

s (lo

g sc

ale)

1969 Mariner-6 (30)1975 Viking (5K)1977 Voyager (3K)1989 Galileo (8K)1990 Cassini (120K)1997 Pathf inder (175K)1999 DS1 (349K)2003 SIRTF/Spitzer (554K)2004 MER (555K)2005 MRO (545K)

1968 Apollo (8.5K)1980 Shuttle(470K)1989 ISS (1.5M)

Robotic Human

Figure 3. History of flight software growth in human and robotic missions.

In the realm of human spaceflight, there are only three NASA data points (Apollo, Space Shuttle, and International Space Station), so there is much less confidence in projecting a trend. Orion flight software size estimates exceed 1 million lines of code, which means that it will be more than 100 times larger than Apollo flight software and will run on avionics with thousands of times the RAM and CPU performance of the Apollo avionics.

Not all NASA centers show a growth trend. Figures 4 shows flight software sizes for selected GSFC missions from 1997 through 2009, with NCSL ranging from 28,400 to 149,500. The uncharacteristically large amount of code for the 2001 MAP mission was due to the need for an ‘executive’ for the Remote Serving Nodes (RSN) processors, programmed in assembly language. The lack of a growth trend matched local expectations in that many of Goddard’s science missions are similar in nature, with routine variations in instruments and observation requirements. However, one mission currently in development is expected to have significantly more software, though size estimates are not yet available. Specifically, the Laser Interferometer Space Antenna (LISA) mission comprises three formation-flying spacecraft with distances among them measured to extraordinarily high levels of precision. Flight software will be larger due to a doubling of issues: twice as many control modes, twice as many sensors and actuators, and fault detection on twice as many telemetry points. Section 4.4 examines LISA in more detail as a case study in growth of software complexity.

From NASA Study Flight Software Complexity (2009). Same growth trend in: •  military aircrafts, •  cars, and •  airplanes.

CHALLENGES IN SPACE VERIFICATION & VALIDATION

More demanding requirements.

More FDIR software.

More system behaviors.

More issues during V&V

More pressure on meeting cost and schedule constraints!

Example: once every 175 year launch window for Voyager 2.

OUR SOLUTION: COMPASS Formal-methods based system software co-engineering

HISTORY: COMPASS CONSORTIUM

Formal semantics, performance

evaluation, model checking, etc.

Model checking, safety &

dependability analysis, etc.

Domain experts

Model-based engineering tools

COMPASS

INCREASE FORMALITY IN EARLY PHASE B


19ȱ

4.4 Project phasing













ȱ

Improve understanding of system software behavior with early V&V.

1.  Model-based. 2.  Formal methods. 3.  Automated analysis tools.

MODEL-BASED ANALYSIS FOR V&V USING COMPASS

Model Nominal Hybrid automata

Error Markov chains

Requirements Property Patterns CTL/LTL

CSL

Automated Analyses

Simulation State exploration

Model Checking Reachability

Fault Tree Analysis Reachability

FMEA Reachability

Probabilistic Risk Assessment

Probabilistic transient

Diagnosis Constraints

TwinPlant reachability

ARCHITECTURE & ANALYSIS DESIGN LANGUAGE: ONE LANGUAGE FOR ALL SYSTEM-LEVEL ASPECTS

•  Originated on MetaH from 1989 •  Standard by SAE (Society of Automotive Engineers) •  Models real-time and performance critical HW/SW architecture •  Hierarchical and component-based.

NOMINAL MODELING

system Power ! features! voltage: out data port real; !end Power;!!system implementation Power.Imp ! subcomponents! batt1: device Battery.Imp in modes (primary);! batt2: device Battery.Imp in modes (backup); ! connections! data port batt1.voltage -> voltage in modes (primary);! data port batt2.voltage -> voltage in modes (backup); ! modes! primary: initial mode;! backup: mode; ! transitions! primary -[batt1.empty]-> backup;! backup -[batt2.empty]-> primary; !end Power.Imp;!

Component type

Component implementation

Interface by ports

Behaviour

Dynamic reconfiguration

Modes

ERROR MODELING

error model BatteryFailure ! features!

!ok: initial state; !!dead: error state;!!resetting: error state;!!batteryDied: out error propagation;!

end BatteryFailure;!!error model implementation BatteryFailure.Imp! events!

!fault: error event occurrence poisson 0.001;!works: error event occurrence poisson 0.2;!!fails: error event occurrence poisson 0.8;!

transitions!!ok -[fault]-> dead;!!dead -[batteryDied]-> dead;!!dead -[reset]-> resetting; !!resetting -[works]-> ok; !!resetting -[fails]-> dead; !

end BatteryFailure.Imp;!

Failure modes

Fault propagations

Failure rates (based on FITs)

Fault behaviour

FAULT INJECTION AND MODEL EXTENSION

error model BatteryFailure ! features!

!ok: initial state; !!dead: error state;!!resetting: error state;!!batteryDied: out error propagation;!

end BatteryFailure;!!error model implementation BatteryFailure.Imp! events!

!fault: error event occurrence poisson 0.001;!works: error event occurrence poisson 0.2;!!fails: error event occurrence poisson 0.8;!

transitions!!ok -[fault]-> dead;!!dead -[batteryDied]-> dead;!!dead -[reset]-> resetting; !!resetting -[works]-> ok; !!resetting -[fails]-> dead; !

end BatteryFailure.Imp;!

system Power ! features! voltage: out data port real; !end Power;!!system implementation Power.Imp ! subcomponents! batt1: device Battery.Imp in modes (primary);! batt2: device Battery.Imp in modes (backup); ! connections! data port batt1.voltage -> voltage in modes (primary);! data port batt2.voltage -> voltage in modes (backup); ! modes! primary: initial mode;! backup: mode; ! transitions! primary -[batt1.empty]-> backup;! backup -[batt2.empty]-> primary; !end Power.Imp;!

Dead: voltage := 0!

…! -- nominal transitions!charged -[when _error.state != dead then voltage := 2.0 * energy + 4.0]-> charged;!charged -[empty when energy = 0.2 and _error.state != dead]-> depleted;!depleted -[when _error.state != dead then voltage := 2.0 * energy + 4.0]-> depleted;!-- nominal transitions with fault injection!charged -[when _error.state = dead then voltage := 0]-> charged;!charged -[empty when energy = 0.2 and _error.state = dead then voltage := 0]-> depleted;!depleted -[when _error.state = dead then voltage := 0]-> depleted;!-- error transitions !depleted -[_error.#works when _error.state = resetting]-> depleted;!charged -[_error.#works when _error.state = resetting]-> charged;!-- error transitions with fault injections!depleted -[_error.#fault when _error.state = ok then voltage := 0]-> depleted;!…!

Nominal

Erroneous

Fault injection

Extended model

BEHAVIOUR METAMODEL: FORMAL SEMANTICS

•  Operational semantics: –  System is tuple of variables.

(mode,alert,battery1.mode,…)!

–  System state is a valuation to tuple variables. <primary,false,charged,…>

–  Nominal/error transitions change current state to new system state. <primary,false,charged,…> ! ! -[empty]-> <backup,false,charged,…>

•  Fundamental to formal methods. –  Precise & unambiguous –  Automated analyses

FOR REVIEW ONLY 20120903115800

hprimary, [alert 7! ?]i khcharged, [energy 7! 1.0, tryReset 7! ?,voltage 7! 6.0]i khcharged, [energy 7! 1.0, tryReset 7! ?,voltage 7! 6.0]i k

hm0, [voltage 7! 6.0,alert 7! ?]i+ 30.0


hm0, [voltage 7! 6.0,alert 7! ?]i+ ⌧


hm0, [voltage 7! 4.8,alert 7! ?]i+ 10.0


hm0, [voltage 7! 4.8,alert 7! ?]i+ ⌧

hprimary, [alert 7! >]i khcharged, [energy 7! 0.2, tryReset 7! >,voltage 7! 4.4]i khcharged, [energy 7! 1.0, tryReset 7! ?,voltage 7! 6.0]i k

hm0, [voltage 7! 4.4,alert 7! >]i+ empty

hprimary, [alert 7! ?]i khdepleted, [energy 7! 0.2, tryReset 7! >,voltage 7! 4.4]i khcharged, [ 7! 1.0, tryReset 7! ?,voltage 7! 6.0]i k

hm0, [voltage 7! 6.0,alert 7! ?]i+ ...

Figure 3.2: An example trace of the Power system.

3.5 Graphical Notation

As an extension of the COMPASS project, we, together with Ellidiss, developeda graphical notation of SLIM and a graphical editor for it. Given SLIM’s

57

PATTERNS: LOGIC-FREE SPECIFICATION OF REQ’S PROPERTY SPECIFICATION: PATTERNS, NOT FORMULAS!

Patterns

• The system shall have a behavior where 80 voltage 90 globallyholds.

• The system shall have a behavior where with probability higher than0.98 it is the case that voltage � 80 holds continously within time

bound [0, 10] .

|(by automatic transformation)

#

Logic

• ⇤(80 voltage 90) (Linear Temporal Logic)• P

>0.98

�⇤[0,10]

(voltage � 80)

�(Continuous Stochastic Logic)

2012, Viet Yen Nguyen 20/43

CASE STUDY MODEL Detailed version of this is confidential.

STUDY PARALLEL WITH SATELLITE DEVELOPMENT


19ȱ

4.4 Project phasing













ȱ

Six months. Parallel with satellite development: 1.  Nominal design

architecture matured. 2.  FDIR architecture volatile

and in development. 3.  Many details were

yet to be decided.

CASE: SATELLITE OF ESA MISSION IN DEVELOPMENT

•  Platform keeps satellite in space, like car’s chassis. –  control & data unit, –  propulsion, –  telemetry, tracking & cmd. –  power, –  attitude & orbit control sys., –  reconfiguration module, –  …

•  Fault detection, isolation, recovery (FDIR) software and hardware: –  redundancies + recovery, –  compensation algorithms, –  failure isolation schemes, –  omnipresent in satellite.

Note: shown satellite is not the one of the case study.

FDIR

FDIR FDIR

FDIR

FDIR

FDIR

FDIR

FDIR

FDIR

DESCRIPTION: WHAT’S MODELLED - COVERAGE Most particularly: •  Voting algorithms on hot

redundant elements. •  Off-range checkers. •  Initialization sequences. •  Recovery sequences,

disabling/enabling system elements.

•  Flags: •  Status •  Commandability

•  Failure configurations. •  Evolution of physical quantities:

•  Time •  Pressure •  Heat

•  Timing constraints.

Platform

AOCS

Gyro

RW

Earth Sensors

Sun Sensors

CDU Processors

TT&C Receivers

Transmitters

EPS Batteries

Solar Panels

LARGEST SYSTEM-LEVEL MODEL EVER DEVELOPED

MODEL AND REQUIREMENTS METRICS

Scope Metric Count

Model

Components 86Ports 937Modes 244Error models 20Recoveries 16Nominal state space 48421100LOC (without comments) 3831

Requirements

Propositional 25Absence 2Universality 1Response 14Probabilistic Invariance 1Probabilistic Existence 1

2011, Viet Yen Nguyen FOR INTERNAL USE ONLY CASE STUDY - 35/45

CASE STUDY ANALYSES All analysis results are in the publication.

INJECTING FAULTS EXPLODE THE STATE SPACE

22

14

10

8

8

7

5

2

1

172

11

1372

10

16

35

11

3

213563

1 10 100 1000 10000 100000 1000000

Reactionwheel + earthsensor failures

Complete earth sensorfailure

Processor modulefailures

Single reactionwheelfailure

All reactionwheel failures

AOCS equipmentsfailure

Propulsion failure

Double earth sensorssignal failure

Single earth sensorsignal failure Multiplication of state space

Number of injections

Explosion correlates with scope of failure.

COMPASS TOOLSET: MAIN WINDOW ANALYSIS WITH COMPASS TOOLSET: MAIN [BOZ+09B, BOZ+09C, BOZ+09D,

BOZ+10A, BOZ+11]


Loading SLIM models

Adding fault injections

Adding requirements

Simulation states

BDD model checking of CTLformula for discrete case

SMT model checking of LTLformula for hybrid case

Counterexample whenproperty does not hold

Relate faults to higher-levelfailures e.g. “which faults leadto undervoltage?”

Fault tree contains Priority AND,AND, OR-gates and edges rep-resent triggers.

Computes probability of top-level event (e.g. system failure)

Fault tree to interactive Markovchain. Minimise to continuous-time Markov chain. Probabilisticmodel check it.

Determine effects on nominal modelupon combinations of failures.

“what happens when sensor dies?”

Determine system performancewhile under degraded modes.

Transforms full state space toa Markov chain. Probabilisticmodel check it.

Determine whether sufficientalarms are available to deducefaults.

Which alarms aretriggered on failure? Does the system recover from a failure?

COMPASS TOOLSET: SIMULATION ANALYSIS WITH COMPASS TOOLSET: SIMULATION



Choose a transition

Simulation states














COMPASS TOOLSET: MODEL CHECKING ANALYSIS WITH COMPASS TOOLSET: MODEL CHECKING



Simulation states

Verify a property e.g. “votingscheme on sensor valuesalways compensate doublesensor failures”














COMPASS TOOLSET: FAULT TREE ANALYSIS ANALYSIS WITH COMPASS TOOLSET: FAULT TREE ANALYSIS



Simulation states





Reachability analysiswith history variables Fault tree contains Priority AND,

AND, OR-gates and edges rep-resent triggers.









COMPASS TOOLSET: PROBABILISTIC RISK ASSESSM. ANALYSIS WITH COMPASS TOOLSET: PROBABILISTIC RISKASSESSMENT



Simulation states














COMPASS TOOLSET: FAILURE MODES & EFFECTS ANALYSIS WITH COMPASS TOOLSET: FAILURE MODES AND EFFECTSANALYSIS



Simulation states










“value range out of range!”





COMPASS TOOLSET: PERFORMABILITY ANALYSIS WITH COMPASS TOOLSET: PERFORMABILITY



Simulation states














COMPASS TOOLSET: DIAGNOSABILITY ANALYSIS WITH COMPASS TOOLSET: DIAGNOSABILITY



Simulation states














COMPASS TOOLSET: FDIR EFFECTIVENESS ANALYSIS WITH COMPASS TOOLSET: FDIR EFFECTIVENESS



Simulation states













Which alarms aretriggered on failure?

Which failures trigger an alarm?

Does the system recover from a failure?

EPILOGUE

FACT SHEET OF THE COMPASS TOOLSET FACTS SHEET OF THE COMPASS TOOLSET [Boz+11]

• Reused core software NuSMV [Cim+02], MRMC [Kat+11], Sigref [Wim+06]

• Subcontractors FBK (Italy), Thales (France), Ellidiss (France)• Budget ⇡ 750,000 Euro• Programmers ⇡ 12 persons• Lines of code ⇡ 100,000 lines of Python• Development time ⇡ 3 years• Reported Trac issues � 500• Used languages Python, C, C++• Licenses

• COMPASS License distribution in ESA member-states only• GNU Public License open source• FBK License binary-only non-distributable• CGM License binary-only non-distributable non-commercial-use• US transfer license for use by a specific US-based organisation

• Website http://compass.informatik.rwth-aachen.de/


BEYOND TODAY: ESA AND EU FUND FOLLOW-UPS

•  ESA projects –  AUTOGEF automated synthesis of FDIR logic –  FAME methodology for FDIR development lifecycle –  HASDEL enhancements for launcher/rocket development

•  EU projects –  D-MILS security aspects in system-level architectures

These project’s budgets account for over 4 million EUR.

CONCLUSION

•  Modeling language –  Based on AADL industry standard –  Graphical and textual –  Formal semantics

•  Discrete •  Real-time •  Hybrid •  Probabilistic

•  Extensive validation approach –  Correctness –  Safety & dependability –  Performability –  FDIR effectiveness

•  Engineer-friendly toolset –  State of the art (probabilistic) model

checking.

•  Spacecraft case studies –  Satellite platforms

•  Phase B •  Phase C

–  Satellite subsystems •  FDIR mode management •  Thermal regulation system

•  Follow-up projects –  Synthesis of FDIR logic –  Methodology and development

lifecycle of FDIR systems –  Launcher/rocket development –  Security architectures

State-of-the-art of validating spacecraft software designs for correctness, safety, dependability and performance aspects using a single system model.

MORE INFORMATION •  M.-A. Esteve, J.-P. Katoen, V.Y. Nguyen, B. Postma and Y. Yushtein.

Formal Correctness, Safety, Dependability and Performance Analysis of a Satellite. In 34th International Conference on Software Engineering (ICSE). ACM & IEEE.

•  COMPASS Toolset: compass.informatik.rwth-aachen.de–  Freely available for everyone in ESA-member states.

TRUSTWORTHY DESIGN VALIDATION OF A SATELLITE PLATFORM

Documents

Transcript of TRUSTWORTHY DESIGN VALIDATION OF A SATELLITE PLATFORM