ENGINEERING LESSONS LEARNED AND SYSTEMS

ENGINEERING APPLICATIONS

Paul S. Gill and Danny Garcia,NASA Technical Standards Program Office

William W. Vaughan,University of Alabama in Huntsville

43rd AIAA AEROSPACE SCIENCES MEETINGJANUARY 8-13, 2005

2

ENGINEERING LESSONS LEARNED AND SYSTEMS ENGINEERING APPLICATIONS

OUTLINE

• PERSPECTIVE

• INTRODUCTION

• SOME LESSONS LEARNED SOURCES

• SYSTEMS ENGINEERING LESSONS LEARNED EXAMPLES

• CONCLUDING REMARKS

3Infusion of Lessons Learned into Programs and Projects

Lessons Learned Implementation Process

ATPSRR

PDRCDR

DCR

FRRPost

Flig

ht

Generate and Communicate Lessons Learned Lessons Learned

Databases

Technical Discipline Working Group Review

PROGRAMS AND PROJECTS (NPG 7120.5)

IMPLEMENTATIONFORMULATION

Screened And

Approved Lessons

SYSTEMS ENGINEERING (NPG 71xx.x and SP-6105)

CONCEPTDEVELOPMENT

DESIGNPHASE

DEVELOPMENT& OPERATIONS

ProposedLesson

4

INTRODUCTION

• BACKGROUND

– NASA TECHNICAL STANDARDS PROGRAM INITIATIVE

– INTEGRATION OF LESSONS LEARNED WITH TECHNICAL STANDARDS

• LESSONS LEARNED INPUTS

– FACILITIES OPERATIONS

– AIRCRAFT DEVELOPMENT AND OPERATIONS

– ROCKET SYSTEM DEVELOPMENT AND OPERATIONS

– FAILURE ANALYSES

– CUSTOMER FEEDBACKS

– NEW TECHNOLOGY DEVELOPMENTS

• BENEFITS

– A VIABLE INTEGRATED AND LIFE-CYCLE BALANCED SYSTEM

– IMPROVED SYSTEM MODELS AND SYSTEM VERIFICATIONS

– DEFINITIONS AND REQUIREMENTS FOR DESIGN, DEVELOPMENT, AND OPERATIONS (STANDARDS, GUIDELINES, HANDBOOKS, AND SPECIFIC DESIGN REQUIREMENTS AND OPERATIONS PROCEEDURES DOCUMENTS, ETC.)

5

SOME SYSTEMS ENGINEERING LESSONS LEARNED SOURCES

• AIAA Satellite Mission Operations Best Practices

• Lessons Learned From The Tropical Rainfall Measurement Mission (TRMM) Mechanical And Deployable Subsystems  

• Lessons from the Shuttle Independent Assessment

• Space Engineering Lessons Learned

• Chandra Lessons Learned

• Systems Engineering Office Lessons Learned

• Flight Projects Lessons Learned Database

• Johnson Space Center Lessons Learned Database

• NASA Lessons Learned Information System (LLIS)

• Electronic Systems Branch Design Handbook Items Lessons Learned

• A History of Aerospace Problems, Their Solutions, Their Lessons 

• Working on the Boundaries: Philosophies and Practices of the Design Process

6

SYSTEMS ENGINEERING LESSONS LEARNED EXAMPLES

• Establish A Systems Engineering And Integration Role With The Authority To Integrate All Program Technical Elements, Organizations, And Other Affected Parties. (OSP Topic 4)

• Rigorous Requirements Development Processes Consistent With Accepted Systems Engineering Practices Are Critical To Establishing A Good Requirements Foundation For A Program/Project. (OSP Topic 6)

• Ensure A Sufficient And Experienced Systems Engineering Staff Is Available And Equipped With Effective Tools And Well-documented Processes To Lead Requirements Development. For Each Requirement, A Corresponding Rational And Verification Approach Should Be Maintained. (X-37 LL)

• The Systems Engineering Process Must Adequately Address Transition From Development To Operations. Develop And Execute Systems Verification Matrix For All Requirements. (LLIS 0641)

• Inadequate Systems Engineering Process And Oversight Is Especially Dangerous In Combination With Nonstandard Engineering Practices. (LLIS 1385)

7

• The Systems Engineering Plan Should Be Baselined In The Mission Definition Phase Of A Program To Avoid Unclear Direction Of How To Handle Milestone Event Success Criteria. (OSP 12.0001)

• All Systems Engineering Functions Need To Be Located In A Centralized Systems Engineering Organization With Well Defined Roles And Interface Responsibilities For The Various Systems Engineering Functions. (OSP 12.0002)

• It Is Important To Have Systems Engineering Follow An Established Systems Engineering Guideline And Maintain A Strong Capability And Presence In A Program. For Example, It Is Imperative That A Program-level Systems Engineering Management Plan Be Established And Distributed To All Lower Level Projects. (Exploration Systems Workshop)

• Human Factors Engineering Needs To Be Treated As A Sub-Discipline Within Systems Engineering And Integration And Be Included In The Development Of The Work Breakdown Structure For The Program. (Exploration Systems Workshop)

• Define Procedures To Ensure Effective Systems Engineering On Safety Issues And Institute Cost Control System For Safety And Reliability. (LLIS 0495)

SYSTEMS ENGINEERING LESSONS LEARNED EXAMPLES (Con’t)

8

THE LESSONS LEARNED PROCESS BEST PRACTICES

• Obtain Enterprise-Wide Commitment To A Formal Lessons Learned Process.

• Review And Prioritize Candidate Lessons Based On Their Applicability To Current And Future Projects, And Periodically Review Published Lessons For Their Continued Relevance.

• Do Not Limit Lesson Topics To “Screw-Ups;” Also Document Successes That Should Be Replicated.

• Charter A Lessons Learned Committee That Represents The Major Technical Organizations And is Charged With Real-Time Development Of Draft Lessons.

• Designate A Single Author To Interview Sources And Prepare Lesson Drafts.

• The Recommendations Drawn From The Driving Event Should Be Actionable.

• Innovate To Assure That The Lessons Learned System Is A Dynamic Resource--Not a “Data Morgue.”

* Source: The Lessons Learned Process: An Effective Countermeasure Against Avoidable Risk. J. Clawson and D. Oberhettinger. 2001 Proceedings of the Reliability and Maintainability Symposium, pages 94-97.

9

ENGINEERING LESSONS LEARNED AND SYSTEMS ENGINEERING APPLICATIONS

CONCLUDING REMARKS

• THE MODERN ERA – HIGH MOBILITY IN TECHNICAL WORK FORCE– ”ONE DEEP” TECHNICAL CAPABILITIES– MINIMAL MENTORING

• LESSONS LEARNED RECORDING AND TRACKING IMPORTANCE– MAINTENANCE AND IMPROVEMENT OF KNOWLEDGE– COUNTERMEASURE AGAINST AVOIDABLE RISKS– PUBLICATION AND INFUSION INTO THE ENGINEERING PROCESS

• ASSEMBLY OF LESSONS LEARNED DATA SETS– NASA TECHNICAL STANDARDS PROGRAM WEBSITE– RELATING TO CURRENT AND NEW AEROSPACE VEHICLE

DEVELOPMENTS

10

11

BACK-UP INFORMATION

1. ECSS-E-10A “SPACE ENGINEERING—SYSTEMS ENGINEERING” (APRIL 17, 1966)

2. ECSS-E-10 PART 6A “SPACE ENGINEERNG—SYSTEMS ENGINEERNG PART 6: FUNCTIONAL AND TECHNICAL SPECIFICATIONS “ (JANUARY 9, 2004)

3. NASA SP-6105 “NASA SYSTEMS ENGINEERING HANDBOOK” (JUNE 1995)

4. NASA LESSONS LEARNED INFORMATION SYSTEM (LLIS) http://llis.nasa.gov

5. LESSONS LEARNED/BEST PRACTICES DATABASES http://standards.nasa.gov

ECSS: European Cooperation for Space Standardization

12

Programs and Projects

Link ToRequirements

Infuse IntoEngineering Process Link the Lessons

Learned to System

Development Lifecycle

Requirements

Infuse the Lessons Learned Into the

Engineering Process

EstablishFeedbackSystem

Feedbacks Needed to Provide User Based

Enhancements

CO

MM

UN

ICA

TE K

NO

WLE

DG

E

Generate & Identify

Document

Categorize

Characterize

Evaluate

Generate & Identify Lessons Learned

Document Lessons Learned

Categorize Into Discipline Categories

Characterize into Programmatic Versus

Technical

Lessons Learned Validity is

Determined by the Working

Group

GE

NE

RA

TE K

NO

WLE

DG

E

Lessons Learned Closed-Loop Process