Page 1 Microgravity Program Support by E&TS and OSAT Surveillance Process for MRDOC Presentation to...

Microgravity Program Support by E&TS and OSAT

Surveillance Process for MRDOC

Presentation to Project Managers’ Working Group

August 7, 2000

Dennis Rohn (E&TS) and Gary Kelm (OSAT)


Agenda

• Introduction

• MSD Contract Management Approach

• Surveillance Organization and Roles

• Proposed Surveillance Process

• Surveillance Classification Guidelines

• OSAT/E&TS Organization and Estimated Staffing

• Feedback


Introduction

• MRDOC is a Performance Based Contract, but the government is still responsible for ensuring that contract and delivery order requirements are met.

• The process of doing this is called “surveillance”, which involves monitoring and communicating with the contractor to assure requirements are properly interpreted and verifying they are satisfied.

• A draft Surveillance Plan is currently part of MRDOC, but a more specific “Implementation Plan” was needed.

• A team comprising Tom St. Onge, Joyce Wanhainen (MRDOC SEB Chair), Dennis Rohn and Gary Kelm developed a proposed surveillance process.

• The team presented this process to Steve Simons (MRDOC COTR) and is now presenting it to the MSD Project Managers for feedback.


MSD Contract Management Approach

MSD Change Board (MCB)

• Approve Contract Changes • Define DO Allocations • Approve Delivery Orders• Approve Fee Requests• Issue Technical Direction• Approve Deliverables

Microgravity Science Program

(Directorate Level)• Workforce Planning• Make-Buy Decisions• Inter-directorate Issue Resolution

MSD Management Team (MMT)

• Information Exchange• Personnel Actions• Programmatic Strategy • Make-Buy Recommendations• ISO Management• Issue Resolution

MSD Surveillance Board (MSB)

• SRFA Review & Closeouts• Identify Trends/Estab Metrics• Initiate Special Studies• Monitor/Assess System Reqts & Program Risk• Recommend Actions to MMT• Support Project Surveil Teams

Project Surveillance Teams

• Generate/Closeout SRFAs• Evaluate Deliverables• Interpret Requirements• Identify Risk Areas• Support ERB/FRB• Attend Maj Milestone Rev


Surveillance Board

• Participants:

– COTR or Designee (1 or 2)

– Exhibit Representatives (5)

– E&TS System Engineer Representative (1)

– OSAT Representative (1)

– Business Management Representative (1)

– Secretary (1)

• Caucuses monthly (or as required).

• Evaluates problems, issues and concerns (e.g., Surveillance RFAs); requests additional surveillance activities, as needed.

• Reviews contractor metrics, trends and performance indicators and takes corrective or preventative action as required to reduce risk or bring contract into compliance.


Surveillance TeamsPrimary Goal: Ensure what the contractor proposes and builds meets contract requirements and

customer needs.

Teams are led by Project Managers. Team members support the PM and need to:

• Become familiar with MRDOC, including SOWs, Data Item Deliverables and technical requirements (i.e., science, safety, integration).

• Become thoroughly familiar with the MRDOC Exhibits and/or Delivery Orders (DOs) to be monitored.• Work closely with the Project Manager.

– Create mutual understanding and agreement on what is needed and how requirements should be interpreted.– Conduct independent analyses, as needed or directed, to assure validity of contract deliverables.– Identify areas of risk or concern and report to the Project Manager.– Conduct audits or other assessments as directed by the Project Manager or Surveillance Board.– Evaluate contractor performance against established metrics. Suggest new metrics that might promote

contractor performance.

• Work closely with the contractor team.– Help contractor interpret and understand requirements.– Offer suggestions, where appropriate, to help contractor understand requirements.– Serve as a consultant: propose potential options or alternatives for contractor consideration, without explicitly

telling the contractor how to do the job.– Participate in contract reviews, meetings, telecons, etc.– Monitor or witness contractor fabrication, assembly, test or other activities to assess progress and problems.

• Utilize personal expertise or recruit other NASA personnel with relevant skills and experience to evaluate/approve DIDs, investigate problems, etc.


Surveillance Team Roles

• What are our roles?– Extremely important!– Independent Verification and Validation– Requirements Interpretation (We are the Customer)– Providing Added-Value to the Contractor (they would like to know if they may not

meet a requirement as early as possible to avoid loss of P-R-O-F-I-T)

• Do’s– Attend Contractor meetings (you have the right!) that pertain to your area– Provide “advice”, add-value to their process (“consultant” model – they don’t have

to follow our advice)– Report to Project Manager on any identified problems (generate Surveillance RFAs,

when necessary)– Perform analysis and review contractor analysis for independent validation

• Don’ts– Argue with the contractor– Provide technical direction (we have the right to provide Technical Direction, but

this needs to be done through MRDOC Change Board and could involve $$$)– Perform contractor’s work – they are responsible – could be a problem later!


Surveillance is as Easy as ABC

CustomerRole

BuilderRole

AssociateRole

Works for

Works with

The three meeting purpose dimensions: (1) To generate and assess design options [formally organized but informally operated--no boss, tables-of-three, efforts]; (2); To present the results [by information briefings, lectures, conferences, and etc.] of the formally organized but informally operated option generation and assessment efforts [that include formally organized but informally operated "independent" assessments and reviews]; and (3) To confirm pre-published results and planned decisions

1) The customer invites the builder to nominate options

2) The builder formally nominates options, also known as design decision options, that they are willing to stand behind

3) The customer confirms the pre-coordinated design decision

Informally operated option generation and assessment{formally organized by the builder role]In which any and all may participate

All ideas only enter here - never from above

The associate (or surveillance team member) works here to make sure that the options offered up will meet the needs of the customer.


MSD Surveillance Process

Review contractor position and PM or designee make recommendation to issue. Continue discussions with contractor to try to resolve.

Project Team

Issue identified by team member

Project Issue Lead

Make Issue Known to contractor

Draft issue on RFA format, forward to PM for review

Update RFA and submit original to PM. Copies to PAM, CSE, Functional Mgr, contractor PM, MSB

PM reviews for clarity return to originator for update and distribution

Microgravity Change Board (MCB)

Microgravity Surveillance Board (MSB)

GRC MSD Management Team (MMT)

Define contractor position and make recommendation to issue

Project Contractor

Review Monthly Issue reports for trends

Cognizant of issues through monthly program discipline reviews

Clarification provided, issue goes away

START

A

Initiates investigation of issue and assesses risk (comparison to risk database). Reports status at monthly reviews.



PM requests ERB/FRB to assess options and recommend approach

PM requests in-house alternative study, prepare results

YES

NO

Supports in-house study.

ERB/FRB reports recommendation from review.

Project Team Project Issue Lead Microgravity Change Board (MCB)



Project Contractor

A

Agreeable solution to all parties, resolution reached

B



NO

YES

NO

Is the Issue

dismissed with no further action

Project Team Project Issue Lead Microgravity Change Board (MCB)



Project Contractor

YES

Submit RFA form to MCB for resolution

Resolved by MMT

Resolved by MCB.

NO

Review closeout reports.

B

Is there a minority view,

decision outside scope of project, increased risk, or

resource constraints?

Is there a cost, schedule, or performance

impact requiring a contract

mod., wavier, or task close-out?

END

YES

PM documents agreement on RFA and distributes copies to CAM, CSE, originator, functional Mangers, contractor PM, MSB Updates risk database as needed.


SRFA Form


Surveillance Activities Classification Guidelines• Class 1 (typical for FCF): >5 FTE E&TS/OSAT Support

– Engineering review of contractor deliverables– Independent engineering analysis and assessments– System requirements assessments– Support PM/contractor TIMs– Major review support– Change board support

• Class 2 (typical for Shuttle, Single-PI Experiments, Multi-PI Experiments, Exhibit 2, SAMS, Sounding Rockets): 2-5 FTE E&TS/OSAT Support

– Engineering review of contractor deliverables– System requirements assessments– Support PM/contractor TIMs– Major review support– Change board support

• Class 3 (typical for reflights, Gloveboxes, and PI Experiments for Multi-PI Experimental Apparatus): <1 FTE E&TS/OSAT Support

– System requirements assessments– Major review support– Change board support


OSAT

Su rveil lan ceIASafety Data P ackag eAp p roval/Develop m en tSafety Review S u p p ort@ JS C/KS CF CF Payload Safety ReviewProcess Su p p ortHazard An alysisDesig n Assessm en t& Ad vice

System Safety

Su rveil lan ceIAM&P CertificationM&P Assessm en tMaterial S election Ad viceDraw in g ReviewMIUL/MUAsAp p roval/Develop m en tW eld in g Q A

Materials & P rocesses

Su rveil lan ceIACon tin u ou s R iskMan ag em en t Train in gR isk An alysisR isk Assessm en tR isk Datab ase

R isk Man ag em en t

Su rveil lan ceIAF ME AF au lt Tree An alysisDefect Detection &Preven tion An alysisReliab ility P red iction s& Trad e S tu d iesTren d An alysisMain ten an ce Con cep tsLim ited Life Assessm en tsSp arin g An alysis& P lan sAvailab ility An alysis& Log istics

Reliab ility & Main tain ab il ity

Su rveil lan ceIAG IDEP Alert SearchF ailu re An alysisTren d An alysisRad iation E ffectsParts Selection & Screen in gDeratin gParts Con trol

EE E P arts

Su rveil lan ceIAAu d itsTest Mon itorin g& W itn essin gPRACA Review & C loseou tF ab rication Su p p ortD raw in g & P rocess P lanReviewProced u re Review& Assessm en tP rocess Con trol E valu ation(Con tam in ation Con trol,CM, ES D)Material Review Board

Q u ality Assu ran ce

Su rveil lan ceIAS /W Au d its(F CA/PCA)S /W F orm al In sp ectionS /W SafetyS /W CMS/W Test Mon itorin gor W itn essin g

Software P rod u ct Assu ran ce

O SATMicrog ravity P AM

MSD

EDAD/SED


OSAT Staffing

TASK AREA PAM IA Safety M&P CRM R&M EEE QA SPA TotalProgram PA 0.7 0.2 0.2 0.2 0.2 0.1 0.2 0.2 2Surveillance 5 5 Exhibit 1 0.6 0.3 0.2 0.4 0.2 0.3 0.4 2.4 Exhibit 2 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.7 Exhibit 3 0.6 0.2 0.2 0.3 0.2 0.3 0.4 2.2In-House 1 0.4 0.5 0.3 0.1 1.4 0.9 4.6Total 0.7 5 2.5 1.2 1.2 1.3 0.7 2.3 2 16.9Total CS 0.7 2 1.5 0.5 1 0.8 0.2 0.8 1.1 8.6Total SC 0 3 1 0.7 0.2 0.5 0.5 1.5 0.9 8.3

Discipline KeyPAM = Program Assurance Mgr. R&M = Reliability & MaintainabilityIA = Independent Assessement EEE = Electronic, Electrical, Electromechanical (parts)M&P = Materials & Processes QA = Quality AssuranceCRM = Continuous Risk Management SPA = Software Product Assurance

DISCIPLINE AREAOSAT FTEs ~ FY01


EDAD/SED Organization Supporting Surveillance

Prog ram Sys. En g r. (SE)F CF SESu rveillan ceIn d ep . Assessm en t (IA)

System s En g in eerin g F CF Ch ief En g in eer

P rog ram Sys. En g r. (SE)F CF SESu reveil lan ce & IA

Avion ics (C&DH)Lead

Prog ram Sys. En g r. (SE)F CF SESu reveil lan ce & IAAd van ced Tech n olog y

D iag n osticsLead

Prog ram Sys. En g r. (SE)F CF SESu reveil lan ce & IAEPCU Su p p ortW F CA Develop m en t

F lu id s/Th erm alLead

Prog ram Sys. En g r. (SE)F CF SESu reveil lan ce & IAEPCU Develop m en tW F CA Su p p ortEMI Testin gF CF Power Lab Bu ild -u p

PowerLead

Prog ram Sys. En g r. (SE)F CF SESu reveil lan ce & IAAd van ced Tech n olog y

SoftwareLead

Prog ram Sys. En g r. (SE)F CF SESu reveil lan ce & IAEPCU Su p p ortARIS /Rack Su p p ortAcou stic Lab DeveAcou stic Lab Testin gMEL Develop m en tMEL Testin gSDL (vib e) testin g

S tru ctu resLead

EDAD/SEDCh ief System s En g in eer

- Microg ravity

MSD

OSAT

FCF Project Projects

One person from the team will be assigned to provide primary surveillance, and coordinate team surveillance activities for each project


EDAD/SED Surveillance Staffing

• Level of effort dictated by Make/Buy Committee at about 21-22 Civil Servants for FY01

Task Area Systems Avionics DiagnosticsFluids/ Thermal Pow er Softw are Structures Total CS

Program SE 1 0.5 0.5 0.5 0.5 0.5 0.5 4 1 +TBDSurveillance & IA 1 1 1 1 1 1 1 7 1 +TBD

Sub-Total 2 1.5 1.5 1.5 1.5 1.5 1.5 11 2

FCF SE 1.1 0.3 0.3 0.3 0.3 0.3 0.5 3.1 TBDFCF GFE 0.5 (00) 1.5 0.1 1.6 TBDFCF Advanced Technology 1 1 2 TBDFCF Pow er Lab build-up 0.3 (00) 0 TBDTest lab build-up & FCF Tests 0.3 4 4.3 2 +TBD

Sub-Total 1.1 0.3 1.3 0.3 2.1 1.3 4.6 11 2

Totals 3.1 1.8 2.8 1.8 3.6 2.8 6.1 22 4 +TBD

SSC Support

EDAD/SED FTEs ~ FY01CS Discipline Area

Majority of above team w ould be full time, some of the Surveillance FTEs w ould include people brought in for specif ic purposes (review s/specif ic analyses)


EDAD/SED Work Authorization

• BMS requirement is to have an Engineering Request

• EDAD/SED responds with two additional forms that have to be filled out, one of which captures the monthly budget estimate.

• Proposal is to have a single Engineering Request (see form) to cover all surveillance needs – one JO, one set of BMS forms

• A Surveillance Engineering Support Request (see form) form has been developed to define what support is requested and when, for each project, including any special tasks

• Information for the form is developed between the Project Manager/Requester and the Engineering Surveillance Lead (or Chief Systems Engineer – Microgravity)

Page 1 Microgravity Program Support by E&TS and OSAT Surveillance Process for MRDOC Presentation to...

Documents

Transcript of Page 1 Microgravity Program Support by E&TS and OSAT Surveillance Process for MRDOC Presentation to...