DO254 DMAP Training 2011 Trailer

1

DO-254 Training

A comprehensive approach to the design assurance guidance for airborne electronic hardware

Presented by James Bezamat

JB 2010 2DO-254 Training

Topics

Training Introduction Trainer Presentation Objectives

Design Assurance Introduction Basics : CEH and design assurance Background DO-254 in a nutshell Rigorous structured development Limitations and expectation Design assurance Level Safety and System level considerations Independence Consideration Verification/Validation definitions DO-254 Scope Simple/Complex

Hardware Design Life Cycle Planning Process Hardware Design Processes

Requirements Capture Derived requirements Requirement standards Validation Process Traceability (part 1)

Conceptual Design Detailed Design

Top Level Drawing Hardware/Software Interface Data Detailed Design Document and HDL code Coding standards

Implementation Production Transition


Topics

Verification Process Independence Robustness Unitary Tests Code Coverage Verification Data

Hardware Verification Cases Procedures Hardware Verification Results

Verification completion Traceability (part 2) Automation

Configuration Management Process Process Assurance Certification Liaison Process

Hardware Accomplishment Summary

Additional Considerations Reuse COTS Service Experience Tool Assessment

Modulation based on DAL Special considerations for Level A & B Conclusion Glossary and acronyms

Training Objectives

To be prepared to manage a Design Project with the clear and complete understanding of what is request for certification.

To understand the actual impact of DO-254 introduction in Electronic Designs methods.

To be convinced that DO-254 guidance is the best way to improve quality in your designs.

To establish internal baselines for procedures, standards, practices and checklists to minimize cost impact and optimize of DO-254 introduction

In a word : to become DO-254 fluent



DO254/ED-80 in a Nutshell

Title : “Design Assurance Guidance for Airborne Electronic Hardware” Authors : RTCA (DO-254) / EUROCAE (ED-80) Date : April 2000 (first rev.) Recognized as a standard by FAA in 2005 (Advisor Circular AC 20-152) Clarification added by Certification Authorities in 2006 (CAST-27) Additional considerations in various CAST papers. FAA Guide on reviews and audits in 2007 (CEH job Aid) FAA Order 8110.105, July 2008 for certification staff Now a worldwide standard for Civil Aviation industry. Spreading into adjacent fields (defense, transportation, medical …)

• EASA – European Aviation Safety Agency, EU counterpart of FAA, pronounced“ee ah sa”

• FAA – Federal Aviation Administration, the US authority

governing aviation

• RTCA – Radio Technical Committee on Aeronautics, a private, not-for-profit corporation that develops consensus-based recommendations

RTCA functions as a Federal Advisory Committee.

• EUROCAE – European Organization for Civil Aviation Equipment, equivalent

of RTCA in the US, pronounced “Euro Kay”

Introduction


Avionic Basic : Design Assurance

Defensive systems such as design assurance, assurance oversight, operator intervention, system fault tolerance, built in tests, error capture

Vector of accident opportunity

Latent Failures, Local TriggersIntrinsic Defects, Atypical Conditions

Courtesy of James Reason

Introduction


Independence

DO-254 (April 2000) “All verification of Level A and B functions should be independent.” (Appendix A)

DO-254 + CAST-27 (June 2006) “For Levels A and B, the validation processes should be satisfied with independence.” (CAST-27

7.c)

Nota : independence of validation is also requested by CRI F-09

best practice : all verification and validation activities must be conducted with independence whatever the DAL is

CRI – Certification Review Item, additonnal considerations to use DO-254 for certification aspects , specific to a program (e.g. CRI F-08 and F-09 for A400M)

Introduction

DO-254 (April 2000) « Hardware Item - An item that has physical being. This generally refers to LRUs, circuit board assemblies,

power supplies and components. » (Appendix C Glossary) “These design processes may be applied at any hierarchical level of the hardware item, such as LRUs,

circuit board assemblies and ASICs/PLDs.” (chapter 5)

DO-254 + CAST-27 (June 2006) + FAA AC 20-152 (June 2005) “Therefore, the objectives and guidelines of DO-254/ED-80, together with the clarification of this CAST

paper, will provide the needed guidelines to be satisfied at the device level for those custom micro-coded devices “ (CAST-27 5.c)

“We don’t intend that you apply RTCA/DO-254 to every type of electronic hardware.” (FAA AC 20-152 2.c)


General Scope

Line Replaceable Unit

Introduction

System development life cycle (ARP4754)


PlansSpec.

Archit.

Code

Synth

P&R

Plans

Spec.

Archit.

Schem

P&R

Verif.

Integration

IntegrationTest

Plans

HW. Spec.

Arch.

SW/HW Integration

Test

PLD

CCA

LRU (hard)

Hard. Interf.

Detail.

System Spec. (ARP4754)

Safety Assessment

Airworthiness Constraints

LRU (soft)SW. Spec.

Certification Plan

Test

LRU

System Integration

Design Life Cycle


Requirements

• General Definition :

« Requirement - An identifiable element of a specification that is verifiable. » (Appendix C Glossary)

Req. AR_PLD1_HRS_5 Status Monitoring Status of all channels shall be monitored and be available through CPU interface

•More :A requirement should be correct and complete

• Correctness of a requirement statement means the absence of ambiguity or error in its attributes. (ARP 4754)

• Completeness of a requirement statement means that no attributes have been omitted and that those stated are essential. (ARP 4754)

Ident Title

Definition

Requirements Capture


Requirement Standards

Rule :Requirements shall not describe design solution

A conflict management shall be implemented in case of interrupt register update during processor read. In that case the PLD shall update the register after the processor read.

No interrupt information shall be lost.

Requirements Capture

Traceability (part 1) : general relations


Validation Process

System requirements

Hardware System requirements

Software System requirements

Hardware CCA requirements

Hardware PLD requirements

Safety requirements

Safety assessment


Conceptual design data example

Architecture description Each channel contains a Controller that calculates the instantaneous phase of a sinusoidal carrier. This carrier signal is mixed

with the input signal in the quadrature mixer to remove any frequency or phase offset on the input signal.

The Controller is built around a 32-bit accumulator. Each clock cycle the accumulator is incremented with the value IFreq, programmed by the register IFREQ.

Transmitter control

64x16RAM

CPU Interface

FIFOControl

Speed select_tx1

Tx2

_tx3

_tx4

_tx5

_tx6

_tx7

Tx8

activation

Conceptual Design Process


Design Life Cycle

Detailed Design Process


Top Level Drawing definition

According to CAST 28 and CAST 27 10 cin line with a configuration index HCI (as for DO-178B and ARP 4754) + HECI

HCI – Hardware Configuration Index : an Addendum to the HAS which identifies the specific hardware baseline configuration for technical data, part number, revision (10.9). It may also identify documentation baseline, Quality Assurance Records index

HECI – Hardware Environment Configuration Index, which describes all the tools and the environment used during design life cycle.

Nota : this definition is appropriate for a FPGA/ASIC design. It must be adapted to other type of hardware items (refer to definition dossier content)

Detailed Design Process

Production transition for LRU : activities

Definition dossier development for first prototype CCA definition dossier Design data, mechanical drawings, bill of materiel, optional equipment

Manufacturing and control dossier development for first prototype

Tooling for manufacturing and test

Produce the first LRU (prototypes) final assembly Qualification Test


Production Transition Process

Production transition for CCA : activities

Definition dossier development for first prototype Design data, electronic schematics, layout files, bill of materiel, optional equipment

Manufacturing and control dossier development for first prototype assembly instructions, Inspection Instructions and test instructions, calibration


Produce the first CCA (prototypes) PCB manufacturing Component Assembly (automated and manual insertion process) Test in-situ, integration test



Production transition for PLD : activities

Definition dossier development for first prototype Design data, netlist, layout files, configuration index, hardware/software interface data programming specification, component identification, marking form, checksum

To be included in CCA definition dossier Test pattern for production (ASIC only)

Manufacturing and control dossier development for first prototype Fabrication procedures, PLD on CCA assembly instructions, test specification


Produce the first PLD (prototypes) Bitstream generation Component programming Test in-situ, integration test Prototype fabrication and test by subcontractor (ASIC only)




Means of Independence Examples from appendix A :

1. Requirements or designs are reviewed by another individual.Should be used for Design Review team.

2. Test cases or procedures are developed by another individual.A clear separation between designer and verification/test engineer

3. Test cases or procedures developed by the designer are reviewed by another individual.Should be used for unitary tests developed by unitary function designer

4. An analysis performed by the designer is reviewed by another individual or a review team.

5. A different test is performed that confirms the results of testing by the designer, such as a test during flight test confirms a hardware item test or software verification tests, developed independently and performed on the target hardware item, confirm the results of testing by the designer.

Should add assurance credits

6. Test or analysis results are verified by a tool.

Some analysis (verification reports) may be automated to generate simulation report (passed/failed, parameters and environment extraction)

Verification Process


CCA Robustness

DO-254 (April 2000) “Requirements derived from the hardware safety assessment that have safety implications should be uniquely

identified. NOTE: Derived requirements may address conditions, such as:

Specific constraints to ensure that functions of a higher design assurance level can withstand anomalies of functions ..range of data inputs

reset states, Supply voltage time-related functions timing under normal and worst-case conditions. Signal noise and cross-talk. unused functions..” (chapter 5.1.2(4) Requirements Capture Process)

DO-254 + CAST-27 (June 2006) “both normal and abnormal operating conditions should be captured as derived requirements and addressed in

the tests. Where necessary and appropriate, additional verification activities, such as analysis and review, may have to be performed to address robustness aspects. “ (CAST-27 8.b)

best practices : Dedicated tests should be produced during qualification (temp. , supply)Deeper in-lab tests should be performed to check noise sensibility, critical timings …


Verification at PLD level


PlansSpec.

Archit.

Code

Synth

P&R

Plans

Spec.

Archit.

Schem

P&R

Verif.

Integration

IntegrationTest

Plans

Hard. Spec.

Arch.

Integration

Test

PLD

CCA

LRU

Hard. Interf.

Detail.

System Spec.

Safety Assessment

Airworthiness Constraints

Qualif.



Verification Process at PLD level by tests Simulation performed on HDL model

Standard simulation (log generated) Self-check cases (SUCCESS/FAILURE) Chronogram analysis (if needed) Unitary, integration and top level tests (depending on design) RTL and gate level simulations

Qualification Testing on prototypes Functional Testing

dedicated test board (reproduce simulation env.) Final/Mock-up CCA (actual application)

Environmental Qualification Testing Robustness (temp., supply conditions) Safety of flight (noise, vibration)

Characterization Robustness (timing, electrical, power margins)

Integration Functional Testing on CCA (part of CCA verification activities)


Traceability (part 2)

“Hardware traceability establishes a correlation between the requirements, detailed design, implementation and verification data that facilitates configuration control, modification and verification of the hardware item.”(10.4.1)

1. A correlation between the system requirements allocated to hardware and the requirements. 2. A correlation between the requirements and the hardware detailed design data. 3. A correlation between the hardware detailed design data and the as-built hardware item or

assembly. 4. A correlation between the requirements, including derived hardware requirements, and

detailed design data and the verification procedures and results. 5. The results of a traceability analysis.



Traceability activities

Mandatory From HRS/HDD requirements to verification procedures coverage (HVCP) From HRS/HDD requirements to verification results coverage (HVR)

Additional Correlation between verification cases procedures description and HDL simulation

code (conformity matrix) Correlation between proposed means of verification (justification plan) and

verification results (reports, analysis)



Traceability matrix example

Process Assurance

“Process assurance ensures that the life cycle process objectives are met and activities have been completed as outlined in plans or that deviations have been addressed.” (8.0)

“The process assurance activities are primarily focused on monitoring the development activities to ensure that they are proceeding in accordance with the approved plans. For this project, the emphasis is on ensuring the internal consistency of the set of controlled data we accumulate throughout the development effort.” (case study NASA 2002)

“Process assurance activities should be achieved with independence in order to objectively assess the life cycle process, identify deviations and ensure corrective action.”(8.0)


Process Assurance

Process Assurance Engineer activities

Process Assurance Engineer may be a member of the Quality Department dedicated to the CEH project

Audits are an effective method for performing Process Assurance activities

Plan, design review, verification review, requirements review are audited then approved by PAE. More generally, each checklist used during reviews are audited by PAE.

Design, Validation and Verification audits may be based on sampling data and assessment of the completion of the design objectives (from requirements to implementation)

PAE participates in the approval of each of the design documents (plans, design documents, verification documents, HAS ..)

Participation to Risk assessment board


Process Assurance

Reviews Progress review are schedule milestones

With customer (also called DRB, Design Review Board) IDR (initial Design Review / Kick Off Review), could include the Plan review PDR (preliminary Design Review), check the Design (validation, conceptual or detailed) CDR (Critical Design Review), check the Design and partial verification activities FDR (Final Design Review) or FQR (Qualification), ready for certification

With authorities SOI1 (Stage of Involvement), Plan review SOI2 , Design Review SOI3 , validation and verification activities SOI4, final review, ready for certification

Note : Design Review Boards may be used as a drafting review of SOI reviews

Best practices : reviews could be prepared with the support of had hoc checklists (increase review productivity)


Process Assurance

SOI checklists

PDR


Reuse of PLD The intention to use previously developed hardware should be stated in the PHAC. Initial baseline must be identified

Specification Design Data Implementation data Datasheet

Configuration Management and Process Assurance considerations should also be addressed for each use of previously developed hardware.

Configuration management process should include Traceability from the hardware product and life cycle data of the previous application to the new

application. Change control processes that can manage change requests from different applications of the common

item.

Additional considerations : reuse


COTS microprocessor usage

DO-254 + FAA AC 20-152 (June 2005) “Therefore, we don’t intend that you apply RTCA/DO-254 to COTS microprocessors.

There are alternative methods or processes to ensure that COTS microprocessors perform their intended functions and meet airworthiness requirements.” (CAST-27 12.b)

DO-178B

Additional considerations : COTS and IP


COTS usage

DO-254 (April 2000) “..typically the COTS components design data is not available for review.

The certification process does not specifically address individual components, modules, or subassemblies, as these are covered as part of the specific aircraft function being certified.

As such, the use of COTS components will be verified through the overall design process. (11.2 Additional considerations)

Additional considerations : COTS and IP

Tool assessment flow chart


Major question : output independently assessed ?

False good idea : relevant history ?

Additional considerations : tool assessment


Design assurance considerations for Level A and B functions

The longest and most dense section of the DO-254 guidance. DO-254 structured processes and strict development process objectives :

Detect introduction or existence of errors then eliminate them For complex functions of level A&B (critical functions) this is not enough to eliminate all errors For these additional architectural means to catch and detect errors are required

It is up to the applicant to select one or more of these methods or propose another method that would provide design assurance.” (appendix B)

Proposed additional methods : Functional failure path analysis and methods

Design assurance methods for level A & B functions Architectural mitigation Product service experience Advanced verification methods

Appendix B

It’s time to conclude



Why should we adopt DO-254 approach ?

DO-254 paves the way for better quality in your design. DO-254 processes may be used for all your projects, in order to guarantee a constant

level of quality E.g. the independence criteria should be mandatory in all hardware designs

(whatever the complexity) DO-254 compliant set of data is the best way to optimize the reuse issue It is also a powerful tool for subcontractor management


DO-254 complaints

1. Additional workload with no added value Additional documents

Loosing time writing formal docs Actual design work very limited (less than 10%) Proofreading and audit on documents are useless

Additional reviews

2. Project management issues Extra Delay on project schedule Will we have time to do it? Will it no be skipped due to strong project deadlines?

3. Follow-up, problem reporting and change request impacts on schedule A minor and trivial change may impact a huge amount of data and document (generally

with a redo of the design life cycle, from HDL code to component)


DO-254 Advantages(2)

Additional workload, with no added value Additional documents

All agreements and design decisions are put on paper Forces designer to think about the design Problems due to miscommunication or misunderstanding are solved early and rationale is visible

throughout the course of the project (important for years project) Facilitates transfer of design and knowledge to

Subcontractor, other team :Loosing less time in explaining how it works, complete, unambiguous and self-sufficient data set.

Documentation : Can reuse design data in other docs test division : Explain the functionality and how to test Production : Understand the board

Tools will help and automate certain activities

Lot of reviews Helps to discover design errors early in the process Team work : you’re not alone in the design Better ideas by peer reviews



Project management issues Additional Delays

Less problems during production or at customer site means less time lost in Solving the actual problem Production stops Customer discussion

Good requirement document with traceability Makes non-compliancy very clear and easier detectable Indicate clearly what still needs to be implemented

Skipped by time pressure Audits and review by QA will guarantee that the process is and will be followed Deviations need to be justified and agreed upfront Extra delay must be properly evaluated during the planning phase of the project



Change impact Additional workload

Avoid insignificant changes that occur at the project closure A management and organizational issue Each designer must be responsive for the quality of his production Who is able to decide that such change is insignificant in term of safety ? No exception allowed (process assurance responsibility) Cost can be minimized with intensive use of automatic procedures and checklists.

Change request from customer or from integration level A clear and documented process is preferable than a quick and undocumented patch. Preserve future reuse or release Extra cost and delay must be evaluated and planned at the very beginning of the project (Risk

management)



Conclusion All what is requested by DO-254 guidance is work design team have to do anyhow. All processes and standards are based on best practices and may be used for non DO-254

projects By formalizing it, we will avoid

Redundant work Useless discussions on not written statements Forgetting things Costly surprises at the end of project (not meeting specs) Hidden features

Avoiding all this will make Projects more predictable Projects more manageable Designs more reliable, robustness -> easier life for designers More confidence in the design Design more reusable

DO254 DMAP Training 2011 Trailer

Technology

Transcript of DO254 DMAP Training 2011 Trailer