DoD EVM Policy, Application, and Reporting...APPROVED FOR PUBLIC RELEASE DISTRIBUTION 1 Office of...

1APPROVED FOR PUBLIC RELEASE DISTRIBUTIONUNLIMITED: 20-S-0814 FEB 14, 2020Office of Acquisition Analytics and Policy

HSV FEB 2020

DoD EVM Policy, Application, and ReportingFebruary 18, 2020

Mr. John McGregorDeputy Director for EVMAcquisition Analytics and Policy

Huntsville EVM Training


HSV FEB 2020

Topics to be Discussed Introduction to AAP

Organization

Competency Model

Structure of Regulations

EVM Applicability

Policy/Guides – AAP

IPM Applicability

EVM-Central Repository Basics/Workflow

EVM-CR Data Extraction and Manipulation

IPMDAR Charts and Analytics


HSV FEB 2020

Organization

OUSD A&S

Acquisition Enablers

AAP

AAP EVM


HSV FEB 2020


HSV FEB 2020

Strategic partnerships across the EVM Value Stream

Working across DoD Services/Agencies, Federal Agencies, and Industry to facilitate the effectiveness of EVM for joint

situational awareness and program decision making

Our Mission Has Not

Changed!


HSV FEB 2020

Competency Model

Partnership with DAU

Course mapping

At a stand still

Potential for reduced career fields


HSV FEB 2020

Oversee competency needs of the EVM acquisition workforce and development requirements, and provide strategic approach to meet human capital needs

Develop and maintain EVM competency model

Review and certify DAU curriculum for EVM courses and continuous learning modules

Support course development for EVM and other functional area courses

AAP EVM is the functional lead for the EVM acquisition workforce

Current Initiatives: Competency Model Development; EVM Position Coding


HSV FEB 2020

An observable, measurable pattern of knowledge, skills, abilities, behaviors, and other characteristics needed to perform work roles or occupational functions successfully

(Adapted from DoD Instruction 1400.25)

Strategic workforce planning using a competency-based strategy to attract, assign, develop, and retain the best civilian workforce

What is a Competency and why use it?

Gap Assessment Training and Development

Career Roadmaps Recruitment and Selection


HSV FEB 2020

DOD Competency Management Process

SME involvement in the process

Draft list of competencies;

Pre-panel activities

Prioritize list of competencies;

Draft SME-validated model

Competency validation and gap

assessment

Competency reports for DOD use; proposed

model revisions

Conduct SME Developmental

Panels 1-4

Validate Model and Assess Competency

Gaps (in DCAT)

Conduct SMEPost DCAT

Panel

Report Findings and Prepare for

Model Refresh

DOD Component

Competencies Literature Review

Review Data

PHASE I: COMPETENCY DEVELOPMENT

PHASE II: COMPETENCY ASSESSMENT AND

VALIDATION

PHASE III:IMPLEMENTATION


HSV FEB 2020

Questions?


HSV FEB 2020

Structure of Regulations

OMB

FAR

DFAR

5000.02 Table 8/9 Changes


HSV FEB 2020

EVM’s primary value is its ability

to provide integrated, timely

performance status for joint

situational awareness, trend

analysis, and decision making


HSV FEB 2020

Office of Management and Budget (OMB)Establishes policy for planning, budgeting, acquisition and management of Federal funding for information technology (IT) and other capital investments

Federal Acquisition Regulation (FAR)Primary regulation for use by all Federal Executive agencies in their acquisition of supplies and services with appropriated funds


HSV FEB 2020

Defense Federal Acquisition Regulation Supplement (DFARS)Supplement to the FAR; providing DoD-specific acquisition regulations (note: Services have additional supplemental guidance)

DODI 5000.02DoD instruction written for PM’s regarding the management of acquisitions


HSV FEB 2020

DoDI 5000.02 Incorporating Change 3, August 10, 2017

Contract Value Applicability Notes Source

< $20MEVM not required; may be applied at PM discretion based on risk to the Government

Requires business case analysis and MDA approval Part 7 of OMB Circular

≥ $20M & < $100M

EVM Required; contractor is required to have an EVM system (EVMS) that complies with the guidelines in EIA-748*

The Government reserves the right to review a contractor’s EVMS when deemed necessary to verify compliance

A-11 (Reference (c)); DFARS 234.201 (Reference (al)); This instruction

≥ $100M

EVM Required; contractor is required to have an EVMS that has been determined to be in compliance with the guidelines in EIA-748*

The Contractor will provide access to all pertinent records and data requested by the Contracting Officer or duly authorized representative as necessary to permit initial and ongoing Government compliance reviews to ensure that the EVMS complies, and continues to comply, with the guidelines in EIA-748*.

Additional Information

For ACAT ID and IAM programs, OSD USD(AT&L) Performance Assessments and Root Cause Analyses (PARCA), in coordination with the CAE or designee, will review proposed contract work scope for EVM applicability and provide a recommendation to the DAE/MDA for a determination of EVM applicability. For all other ACAT levels, the CAE, or designee, will review and determine EVM applicability. If EVM is determined to apply, then threshold application in this table is utilized or a waiver from the CAE is required. If, based on the nature of the work, EVM is determined not to apply, then EVM is not placed on contract.Applying EVM outside the thresholds and criteria above, to include application on firm, fixed-price (FFP) contracts, FFP task orders, or FFP delivery orders, a cost-benefit analysis will be conducted, MDA approval is required, and the results provided to the contracting officer for documentation in the contract file.The term "contracts” includes contracts, subcontracts, intra-government work agreements, and other agreements.For indefinite delivery, indefinite quantity (IDIQ) contracts, inclusion of EVM requirements is based on the estimated ceiling of the total IDIQ contract, and then is applied to the individual task/delivery orders, or group(s) of related task/delivery orders, that meet or are expected to meet the conditions of contract type, value, duration, and work scope. The EVM requirements should be placed on the base IDIQ contract and applied to the task/delivery orders, or group(s) of related task/delivery orders. “Related” refers to dependent efforts that can be measured and scheduled across task/delivery orders.The Integrated Baseline Review is required when EVM is determined to be applicable.The initiation of an over-target baseline or over-target schedule must be approved by the Government program manager.Application thresholds are in then-year dollars.* EIA-748 = Electronic Industries Alliance (EIA) 748-C (Reference (au))


HSV FEB 2020


< $20MEVMS not required; may be applied at PM discretion based on risk to the Government.

Requires business case analysis and MDA approval.

Part 7 of Office of Management and Budget Circular A-11

FAR 52.234-4 FAR subpart, 34.2DFARS 234.201

DoDI 5000.xxc, Para. 3C.3.c.(3)

≥ $20M & < $100M

Required to have an EVM system (EVMS) that complies with the guidelines in EIA-748.*

The Government reserves the right to review a contractor’s EVMS when deemed necessary to verify compliance.

≥ $100M

EVMS Required; contractor is required to have an EVMS that has been determined to be in compliance with the guidelines in EIA-748.*

The Contractor will provide access to all pertinent records and data requested by the Contracting Officer or duly authorized representative as necessary to permit initial and ongoing Government compliance reviews to ensure that the EVMS complies, and continues to comply, with the guidelines in EIA-748.*

Additional InformationFor programs or contracts associated with programs where the USD(A&S) is the MDA, OSD USD(A&S Acquisition Analytics and Policy (AAP), in coordination with the CAE or designee, will review proposed contract work scope for EVM applicability and provide a recommendation to the DAE/MDA for a determination of EVM applicability. For all other ACAT levels, the CAE, or designee, will review and determine EVM applicability. If EVM is determined to apply, then threshold application in this table is utilized or a waiver from the CAE is required. If, based on the nature of the work, EVM is determined not to apply, then EVM is not placed on contract.

The term "contracts” includes contracts, subcontracts, intra-government work agreements, and other agreements.For indefinite delivery, indefinite quantity (IDIQ) contracts, inclusion of EVM requirements is based on the estimated ceiling of the total IDIQ contract, and then is applied to the individual task/delivery orders, or group(s) of related task/delivery orders, that meet or are expected to meet the conditions of contract type, value, duration, and work scope. The EVM requirements should be placed on the base IDIQ contract and applied to the task/delivery orders, or group(s) of related task/delivery orders. “Related” refers to dependent efforts that can be measured and scheduled across task/delivery orders.The Integrated Baseline Review is required when EVM is determined to be applicable.The initiation of an over-target baseline or over-target schedule must be approved by the Government program manager.Application thresholds are in then-year dollars.* EIA-748 = Electronic Industries Alliance (EIA) 748-Current Version

DoDI 5000.02, January 23, 2020 (Table 8 will be hosted on DAU)


HSV FEB 2020

DoDI 5000.02 Incorporating Change 3, August 10, 2017


< $20M Not requiredIPMR Should be used if cost and/or schedule reporting is requested by the PMO

IPMR DID DI-MGMT-81861A*

≥ $20M & < $50MRequired monthly when EVM requirement is on contract

Formats 2, 3, and 4 may be excluded from the contract data requirements list (CDRL) at program manager discretion based on risk

≥ $50MRequired monthly when EVM requirement is on contract

All Formats must be included in the CDRL

Additional InformationFor ACAT I contracts, task orders, and delivery orders, IPMR data will be delivered to the EVM Central Repository.The IPMR can be tailored to collect cost and/or schedule data for any contract regardless of whether EVM is required. For information on tailoring the IPMR, refer to the DoD IPMR Implementation Guide (Reference (cj)).Formats and reporting requirements for the IPMR are determined and managed by USD(AT&L) through the office of PARCA.Reporting thresholds are in then-year dollars.DI-MGMT-81861A = Data Item Management-81861 (Reference (av))


HSV FEB 2020


< $20M Not requiredIPMDAR Should be used if cost and/or schedule reporting is requested by the PMO

IPMDAR DID DI-MGMT-81861B*

≥ $20M & < $100MRequired monthly when EVM requirement is on contract

IPMDAR is required. All three components of the IPMDAR -Contract Performance dataset, Schedule, and Performance Native Report - can be tailorable based on pre-determined programmatic risk.

≥ $100MRequired monthly when EVM requirement is on contract

IPMDAR is required.

DoDI 5000.02, January 23, 2020 (Table 9 will be hosted on DAU)

Additional InformationFor all ACAT contracts, task orders, and delivery orders, IPMDAR data will be delivered to the EVM Central Repository.The IPMDAR can be tailored to collect cost and/or schedule data for any contract regardless of whether EVM is required. For information on tailoring the IPMDAR, refer to the DoD IPMDAR Implementation Guide (Reference (cj)).Formats and reporting requirements for the IPMDAR are determined and managed by USD(A&S) through the office of AAP.Reporting thresholds are in then-year dollars.DI-MGMT-81861B = Data Item Management-81861 (Reference (av))

Regulations Policy Guidelines

EIA-748-D

PM’s Guide to IBR

Process

NDIAIPM

Surveillance Guide

OMB A-11 Part 7 & CPG(OMB A-11 Part 7 is updated annually)

DFAR 234.201

IPMR DID

DoD 5000 series(undergoing updates)

IPMR CDRL

NDIA IPMEVMSIntent Guide

Defense Acquisition Guide

Processes

Toolkits

Government agrees only to section 2 (Guidelines Intent)

Industry agrees to all sections of the

IMP / IMS Guide

OTB / OTS Guide

GAO Cost Guide(N/A for DoD)

GAO IMS Guide(N/A for DoD)

AAP EVM website

IPMR DID Guide

DFAR 252.242-7005

DFAR 242.1107-70(production only)

Title 48 CFR

FAR 52.234-4

FAR Part 34

FAR 52.242-2(production only)

FAR 52.234-2

DoD Memo 2015-O0017 EVM

Threshold DeviationEVMSIG

DFAR 234.203

DFAR 252.234-7002DFAR 252.234-7001

EVMIG

Agile and EVM Guide

Title 10

(Chapter 11.3.1 EVM)


HSV FEB 2020

Questions?


HSV FEB 2020

EVM Applicability

Delegation Memos

5000.02 EVM Applicability

Contract type/value/scope

Conversations


HSV FEB 2020

Delegation Memo PARCA delegated

authority for EVM applicability reviews for ACAT ID and IAM


HSV FEB 2020

• Contract is Cost Plus or Incentive

• Contract > $20M (including known options)

• Contract has at least 18 months period of performance

Step 1. Contract Criteria Review

If all Yes

If any NoEVM Not Applicable*

EVM placed on contract; option to have

applicability determination based on nature of work (Step 2)

Step 2 (Optional). Work Attributes Review

Yes

No

EVM does not apply* EVM not on contract; no

deviation/waiver required

EVM does apply EVM is placed on contract; PMO

has option to seek waiver/deviation

* The PM has the option to make a business case to apply EVM outside the thresholds and application decision

• Does EVM apply based on review of SOW, PWS, WBS, & CDRLs (i.e., work discretely measurable & schedulable)

AAP & Services - ACAT ID & IAM for DAE/MDA decision–delegated to AAP/EVM

Services - other ACATs for SAE/CAE decision


HSV FEB 2020

Have the Conversation…

What are we buying?

How are we buying?

How do we manage?

Regardless of acquisition approach

Traditional Acquisitions

Middle Tier Acquisitions

Other Transaction Authorities

Agile Development Methods


HSV FEB 2020

EVM*: A program managementtechnique for

measuring program performance and

progress in an objective manner

Power Point Charts

Different Perspectives….

EVMS*: An integrated management system

that integrates the work scope, schedule, and cost parameters of a program in a manner that provides objective

performancemeasurement data

*Definitions from the DoD Earned Value Management System Interpretation Guide

…a system of systems

Data Analysis

People, Processes,

Tools

Compliance


HSV FEB 2020

Questions?


HSV FEB 2020

Game Time

Jeopardy


HSV FEB 2020

AAP EVM Policy/Guides

EVMSIG

EVMIG

MIL-STD-881

DoD Agile and EVM Guide

IPMDAR


HSV FEB 2020

DoD EVMSIG is used as the basis for the DoD to assess compliance to the 32 EVMS Guidelines

Involved Government and Industry stakeholders across EVM and compliance communities

Changes• Clearinghouse request• GL 6 graphic MRP/ERP• Formula and glossary item

for material cost variance

Published February 2018

https://www.acq.osd.mil/evm/assets/docs/DoD_EVMSIG_14MAR2019.pdf

https://www.acq.osd.mil/evm/assets/docs/DoD_EVMSIG_14MAR2019.pdf


HSV FEB 2020

DoD EVMIG provides guidance for the application and execution of EVM in the Department

Involved Government and Industry stakeholders across EVM community

Provides a description of EVMS concepts, as well as guidance for implementing EVM in the DoD

Published January 2019

https://www.acq.osd.mil/evm/assets/docs/DOD%20EVMIG-01-18-2019.pdf

https://www.acq.osd.mil/evm/assets/docs/DOD%20EVMIG-01-18-2019.pdf


HSV FEB 2020

Part 1 describes EVM Concepts and Guidelines

Part 2 provides guidance for Government use of EVM

The appendices contain additional reference material


HSV FEB 2020

MIL-STD-881 presents direction for effectively preparing, understanding, and presenting a Work Breakdown Structure Involved Government and

Industry stakeholders across EVM and cost communities Changes

• Added new appendices (Strategic Missiles Systems, Sustainment)

• Combined appendices (Manned/Unmanned Aircraft, Missile/Ordnance)

• Additional Standard Breakout for Common Elements

Published April 2018 • (currently being reworked)

http://quicksearch.dla.mil/qsDocDetails.aspx?ident_number=36026

http://quicksearch.dla.mil/qsDocDetails.aspx?ident_number=36026


HSV FEB 2020

A resource for DoD personnel who encounter programs on which Agile philosophies and Earned Value Management are applied1. Agile and EVMS System Compliance (March 2016)

2. Integrated Baseline Review (April 2018)

3. Reports, Metrics, and Analysis (April 2018)

Appendix: Agile and EVM Scenario (April 2018)

“EVM is not tied to any specific development methodology and does not prevent the use of other risk management techniques such as agile development. EVM and agile development are complementary and can be used on the same project. Agile development can be used to

incrementally deliver functionality to the customer while EVM provides a standard method for measuring progress.” -- OMB Circular A-11


HSV FEB 2020

Goal is to reduce time to receive actionable data and have the right conversations

IPMDAR Update Next Steps Resolve final items and revise DID language as needed Collect and adjudicate comments Technical infrastructure development and testing Release IPMDAR DID

• Complete DoD policy publication process/public review and comment• Adjudicate comments• Publish final DID


HSV FEB 2020

Questions?

Break


HSV FEB 2020

IPM Applicability Considerations

Adaptive Acquisition Framework

IPM Applicability

Scenarios


HSV FEB 2020


HSV FEB 2020

• Contract is Cost Plus or Incentive

• Contract > $20M (including known options)

• Contract has at least 18 months period of performance

Step 1. Contract Criteria Review

If all Yes

If any NoEVM Not Applicable*

EVM placed on contract; option to have

applicability determination based on nature of work (Step 2)

• Does EVM apply based on review of SOW, PWS, WBS, & CDRLs (i.e., work discretely measurable & schedulable)

AAP & Services - ACAT ID & IAM for DAE/MDA decision–delegated to AAP/EVM

Services - other ACATs for SAE/CAE decision

Step 2 (Optional). Work Attributes Review

Yes

No

EVM does not apply* EVM not on contract; no

deviation/waiver required

EVM does apply EVM is placed on contract; PMO

has option to seek waiver/deviation

* The PM has the option to make a business case to apply EVM outside the thresholds and application decision


HSV FEB 2020

EVM Reporting Required

TailoredIPM Cost, Schedule,

and Technical

Tailored IPM

Cost and Technical

Only

Urgent Operational

Need

Middle Tier Acquisition

Major Capability Acquisition

SW Acquisition

Business System

Services Acquisition

Tailored and Streamlined

IPMCost,

Schedule and Technical

Only

StreamlinedRequest Waiver for Burdensome Requirements

EVM Required

Compliance Clause

Required

Development

Sustainment

Does the work have definable deliverables?And Is the expected Period of Performance 18

months or longer?

Yes

No

Yes

Program and Contracting Scenario Assessment

CV > $100 M?

CV > $20 M?

What is the SW

Phase?


HSV FEB 2020

Urgent Operational

Need

Tailored and Streamlined IPMCost, Schedule,

Technical

DoDI 5000.02 Enclosure 13Objective:a) Can be Fielded < 2 years, ANDb) Not an ACAT 1 or 1A Program

- ACAT 1 - Dollar value for all increments of the program > $480

million RDT&E- or, for procurement, of more than $2.79 billion- MDA designation

- ACAT 1A- Value > $40M in single year- Value > $165M Material Solution through Deployment- Value > $520M Material Solution through Sustainment

- Delivery within Days or Months

Acquisition Approach: (Fixed Price)?- Tailored and Streamlined Program Strategies and Oversight.- Streamlined based on complexity and delivery requirements.- Strive for parallel activities vice sequential.- Acquisition Decision Making tailored to expedite delivery of capability- Limited Development.- MDA can authorize production at same time as development.

Program Management Approach:- Develop recommended courses of action- Assess Acquisition and operational risk in terms of cost, schedule,

and performance of each solution- Selected Course of action and management approach documented

in Acquisition Decision Memorandum.


HSV FEB 2020

EVM Required

Compliance Clause

Required



and Technical

Middle Tier Acquisition

NDAA 2016 Section 804

Objective: Rapid Prototype or Fielded Delivery within 5 years.- Rapid Prototype – Use of Innovative technologies with

residual operational capability within 5 yrs.- Rapid Feilding – Use of Proven Technology with

operational capability within 5 years.

Acquisition Approach: (Fixed Price?)- Flexible acquisition process tailored for expedience.- Encouraged to seek waivers of any statute or regulatory

requirements seen as burdensome.- Establish alternative acquisition paths.

Program Management Approach:- Nothing specific recommended- Implication is non-interference oversight focused

on technical delivery- Must identify applicable statutes and regulations

then decide to waive or not

Seek Waiver if Deemed Burdensome

SW Phase

Development

Sustainment

CV> $100M

CV> $20M


HSV FEB 2020

EVM Required

Compliance Clause

Required



and Technical

Major Capability

Acquisition

Major Capability AcquisitionDoDI 5000.UG

Variety of Lifecycles – What am I Buying?- Hardware Intensive Program- Defense Unique Software Intensive Program- Incrementally Deployed Software Intensive

Program- Accelerated Acquisition Program- Hybrid Program A (Hardware Dominant)- Hybrid Program B (Software Dominant)

Traditional Acquisition Approach – Acq strategy approved by MDA

Program Management Approach- Does Earned Value Mgt Apply or not- No alternative Mgt approaches described

SW Phase

Development

Sustainment

CV> $100M

CV> $20M


HSV FEB 2020

SW Acquisition

Software AcquisitionDoDI 5000.xx

Objective: - Acquisition of SW Only?- Iterative development- Continuous deployment

Acquisition Approach:- Product Roadmap- Select Agile processes and plan for establishing

Government competency- Definition of MVP and initial design constraints- Describe Budget- Define Contracting Strategy

Program Management Approach:- Establish cost projections to provide insight into

the cost of the work to be delivered- Maintain Roadmap including dependencies- Technical metrics for tracking progress

CV> $100M

EVM Required

Compliance Clause

Required

CV> $20M



and Technical

SW Phase

Sustainment

Development


HSV FEB 2020

Tailored IPM

Cost and Technical

Only

Services Acquisition

Acquisition of ServicesDoDI 5000.74

Objective: - Acquisition of Services with specified $ thresholds.

Acquisition Strategy:- Should provide incentives to industry to improve productivity and

performance consistent with DoD objectives.- Show Clinger Cohen Compliance- No private sector or Government source can better support this

function

Program Management Approach: (Cost and Technical Mgt)- Follow processes for the effective management, planning, and

execution of services - Allow for Forecasting requirements and budget.- Use data and metrics to support strategic management decisions and

documenting business trends and costs in the acquisition of services - Uses prudent business judgment to manage risk and structure a

tailored, responsive, and efficient services acquisition program - Use specific metrics and tripwires tailored to the specific needs of the

government customer for the service and the requirement.


HSV FEB 2020

Tailored IPM

Cost , Schedule

and Technical

Defense Business System

Defense Business System (DBS)DoDI 5000.75

Objective:- All DoD business capabilities and their supporting business

systems, including software-as-a-service, with the exception that only Appendix 4C of this issuance applies to business systems that are MDAPs.

- Information systems for business productivity and information technology (IT) infrastructure

Acquisition Strategy:- The procedures used to develop business capability requirements

and supporting systems will be tailored to the characteristics of the capability being acquired. Tailoring will focus on application of best practices to the totality of circumstances associated with the program, including affordability, urgency, return on investment, and risk factors.

Program Management Approach:- DoD acquisition of business systems will be aligned to commercial

best practices and will minimize the need for customization of commercial products to the maximum extent possible.

- Cost, Schedule and Technical are all important factors


HSV FEB 2020

Questions?


HSV FEB 2020

EVM Central Repository (EVM-CR)

Basics

Workflow


HSV FEB 2020

PURPOSE To establish a source of authoritative EVM data for the Department and to provide timely access for PMOs, Services, OSD, and DoDComponents.• Managed by OUSD (A&S) / AAP

• Hosted on the eBiz MilCloud

PROGRAMS ACAT 1A, 1C & 1D (MDAP) Programs with EVM reporting requirements on contract are required to submit to the EVM-CR.

DATA Standard reporting requirements are:• Cost and Schedule data (IPMDAR/IPMR) expected monthly

• Contract Funds Status Report (CFSR) expected quarterly

https://www.acq.osd.mil/evm/#/home



HSV FEB 2020

AIR FORCE148

ARMY52

DOD64

MDA57

NAVY192

Actively Reporting Efforts by Service

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

2010 2011 2012 2013 2014 2015 2016 2017 2018 2019

Submissions Over Time

• 625 total actively reporting efforts as of 12/31/2019

• 9868 submissions delivered to the EVM-CR in 2019


HSV FEB 2020

Reviewer

ACCESS EVM-CR via• External Certification Authority (ECA) certificate

• Certificates issued by major contractors Boeing, Northrup Grumman, Raytheon, and Lockheed Martin

Industry GovernmentSubmitter

RESPONSIBLE FORDelivery of reports

APPROPRIATE FORIndustry contractors

Industry Reviewer

RESPONSIBLE FOROversight of reports delivered by all submitters from their organization

APPROPRIATE FORIndustry contractors

ACCESS EVM-CR via• Common Access Card (CAC)

• NDAs: Support contractors must obtain and submit NDAs in order to gain reviewer or analyst permissions

RESPONSIBLE FORReviewing, approving, and publishing reports

Managing submitters and reviewers assigned to efforts

APPROPRIATE FORProgram Office

AnalystALLOWED TOView and download published reports

APPROPRIATE FORDoD

Request Access

Data access must be requested via the EVM Public Website https://www.acq.osd.mil/evm/#/home



HSV FEB 2020

DoD Access DoD and DoD support contractors can access data in the EVM-CR • Government users with an Analyst role can access all published reports in the EVM-CR• Contractors that support DOD HQ organizations can be granted similar access by

providing appropriate NDAs*

Industry Access Reporting contractors access data for assigned contracts only

Web Services Summary data is made available to external customers via API. (DAMIR, Navy RDAIS, AF PRMT & Army PMRT)

*Effective Aug 31, 2016, support contractors acting as EVM Reviewers are required to sign NDAs with the reporting contractor in order to access data within the system.


HSV FEB 2020

Government Program Office Provide Contract detail information to the EVM-CR Help Desk Team

Submit contract CDRL (form DD-1423) documents to Help Desk

Help Desk Identify Effort Numbers

Review contract CDRLs

Establish Contract/Effort Data Reporting Stream

Assign Initial Users

[email protected]

mailto:[email protected]


HSV FEB 2020

Review of DID & CDRLs to determine reporting requirements and Workflow process for Incremental Reporting.


HSV FEB 2020

Submit and Review Collaboration

INDUSTRYSUBMIT

PROGRAM OFFICEREVIEW & ANALYZE

PROGRAMOFFICEPUBLISH


HSV FEB 2020

Data View Reviewer and Submitter should verify data accuracy

Data Validation Report Provides listing of all data checks and indicates if file passed or failed

Performance Over Time Chart Provides visual rendering of PMB data received since effort reporting started.


HSV FEB 2020

Submission1. IPMR Cost and Schedule files delivered 12-17 days after the contractors

accounting period end date2. 10 day Auto-publish clock begins when submit button is pressed3. Communication via email between Reviewer and Submitter to provide corrected

files prior to publishing

Submission Review – Help Desk1. Data Quality Review ( common issues: PMB costs not explicitly reported or data

not in same order of magnitude throughout the file)2. Verify file tagging (total vs. component as well as human readable file types)3. Verify report date (human readable submissions only)4. Communication with Reviewer and Submitter teams to obtain corrected file when

data issues exist.


HSV FEB 2020

IPMDAR Delivery Requirement (16 business days)• “…. no later than sixteen (16) business days after the contractor’s accounting

period end date.”

Incremental Delivery Encouraged/Defined in CDRL• “The recommended incremental delivery process is the Schedule, followed by

the CPD with the Executive Summary, Government review of submittals, Government directed Detailed Analysis, Contractor Detailed Analysis delivery and all final data.”

Example of Incremental Delivery:Report Type Due (# of days after accounting

period close)• Native Schedule• Schedule Performance Dataset-SPD

5

• Contract Performance Dataset-CPD• Executive Summary

12

• Detailed Analysis 16


HSV FEB 2020

• What if a preliminary report is delivered and there are no changes? Can we publish it? When?

• Do we need to Auto-Publish? • If yes, when should the Auto-Publish timing begin with Incremental Delivery

requirements?• Would a “chat” capability on a submission be useful for submit/review teams to

collaborate?• Should we require justification in the event of rejection?• Should rejection be at the file level or submission level?• Is Reviewer voting useful? Would a collaboration state eliminate the need for

voting?


HSV FEB 2020

• XLS Export provides tabular view of JSON file settings and data

• CSV Export provides data in Excel pivot table

• XML Export converts the IPMDAR JSON file to legacy IPMR

UN/CEFACT file


HSV FEB 2020

Questions?

LUNCH


HSV FEB 2020

IPMDAR Initiative and Timeline

DID

Implementation Guide

Memo


HSV FEB 2020

EVM-CR Data and Manipulation

Data extraction

Pivot tables and manipulation

[email protected]


HSV FEB 2020

IPMDAR IPMRCPD-Contract Performance Dataset (JSON)

Formats 1-4 & 7 XML (UN/CEFACT)

SD-Schedule Dataset Format 6 XML (UN/CEFACT)Native Schedule Native SchedulePerformance Narrative • Executive Summary• Variance Analysis

Format 5

***Not Required*** Formats 1-4 (Human Readable)

Integrated Program Management Data and Analysis Report (IPMDAR)• Moving away from human readable formats and focusing on data for analysis

EVM-CR Tools have the capability to generate legacy formats


HSV FEB 2020

AT CA Level, Calculated by WBS

From BCWS & ETCs

at CA/WP

Encoded In FileDerived

Removed

Key

AT CA or WP Level, Calculated by WBS

Ancillary header values removed

Derived values Current period data

values Roll-up values in the

WBS and OBS Variances CPI/SPI BAC EAC

PMB data explicitly encoded in file for cross-check


HSV FEB 2020

AT CA Level, Calculated by WBS

From BCWS & ETCs

at CA/WP

AT CA or WP Level, Calculated by WBS


Removed

Key


Derived values Current period data

values Roll-up values in the

WBS and OBS Variances CPI/SPI BAC EAC

PMB data explicitly encoded in file for cross-check


HSV FEB 2020

Time-phased BCWS at CA or WP Level; derived by comparison to

prior submission


Derived values Beginning Of Period

Values

Baseline Changes derived by CA/WP

Current Period


Removed

Key


HSV FEB 2020

Hours and $, Time-Phased Future Forecast by Month thru end of contract/effort at the CA or WP level.

Values rolled up to OBS.

FTE values derived from hours and working hours defined in the contract calendar table

Derived values Current period values

Roll-up by OBS

At-Complete

FTE values derived from hours and calendar


Removed

Key


HSV FEB 2020

IPMR Strengths / Purpose

• Performance analysis at reporting level (e.g. WBS, OBS)

• Gold Card Analytics (CPI, SPI, VAC, TCPI, EAC)

Limitations / Weakness

• Visibility into management controls (WBS & OBS vs. CA/WP)

• Visibility into execution plan (partial future forecast reporting)

• Limitations for cost/schedule integration visibility

• Visibility into retroactive contract changes

IPMDAR Same Strengths / Capability• Generate performance analysis at reporting

level (e.g. WBS, OBS)• Gold Card Analytics (CPI, SPI, VAC, TCPI,

EAC)• Can generate legacy formats

And So Much More…• CA or WP visibility• Hours and dollars• Element of Cost visibility• Fully time-phased future plan• Positive traceability between cost and

schedule• Visibility into retroactive contract changes

with time-phased To Date

Greater Focus On Forward-Looking Analytics


HSV FEB 2020

IMPDAR Pivot Data

Flattened IPMDAR Contract Performance Data

IPMDAR Contract Performance Data

Pivot Table


HSV FEB 2020

Separating the Data from Printable Formats Enables Utilization of Modern Tools

Example Pivot Table Applications

Responsibility Assignment Matrix

Estimate to Complete by Control Account


HSV FEB 2020

Separating The Data From Printable Formats Enables Utilization Of Modern Tools


Budget at Complete by control account

CAM budget

Side-by-side dollars / hours


HSV FEB 2020


Example Pivot Table Applications Element of Cost by Control Account (Hours/Dollars)


HSV FEB 2020



Time-Phased Future Forecast by CAM (Dollars)

CAM Forecast (Dollars)


HSV FEB 2020


Time-Phased Forecast by CAM (Hours)


HSV FEB 2020

Example Pivot Table Applications Element of Cost by Control Account (Hours/Dollars)


HSV FEB 2020



Derived FTE Chart by Control Account

Derived FTE Chart by CAM


HSV FEB 2020

Questions?

Break


HSV FEB 2020

IPMDAR Charts and Analysis

Reporting Changes

Open communication required

Tools for asking the right questions

Tools for making program management decisions


HSV FEB 2020

Delivering data rather than printable formats

IPMR 1-4 and 7: Replaced with CPD

IPMR 6: Much the same (Schedule Dataset and Native Schedule)

IPMR 5: Performance Narrative Report (Executive Summary plus Detailed Analysis)

All ACAT levels to deliver to the EVM-CR


HSV FEB 2020

Final due no later than 16 business days after contractor accounting period close

Incremental delivery requirement defined in CDRL


HSV FEB 2020

Performance Narrative consists of: Executive Summary Detailed Analysis Report (Specific Variances at the Control Account level)

Variance thresholds should be established in the CDRL (flexible) Dollars and/or percentage Number of thresholds and/or directed


HSV FEB 2020

Data Structure Format: UN/CEFACT Zipped JSON Encodes cost/schedule data tables and relationships

Tools/Exportability (example: flat files)Spans start to end of contract period of performance Cumulative-to-date Future Forecast

Reporting at control account, both hours and dollars, and tagged to elements of cost

Maximum flexibility for analyzing data (no more Formats)


HSV FEB 2020

• Ability to convert IPMDAR data into trend charts

• Ability to look at control account trends and identify critical drivers

Programmatic CPI & SPI

Schedule Variance by Control Account Manager Schedule Variance by Control Account


HSV FEB 2020

• What is the plan to hire the number of people needed?

• Is this a realistic hiring plan?• What is the impact on cost

and schedule introduced by this identified risk?

• What are the current Staffing and hiring projections?

• What is the delta between past projections and actual hiring?

Programmatic View Individual WIPT View


HSV FEB 2020

• Forward looking: analysis of FTEs within a control account

• What does the trend line for BCWS look like?

• What is the delta between the BCWS and the ETC?

• Is there an imminent cost variance?

Time Phased BCWS (To-Complete) vs ETC by Control Account & EOC

Time-Phased FTE Plan by Control Account

Time Now

Time Now


HSV FEB 2020

Time Now

Time Now

Time Now

• Ability to isolate work streams that are contributing to Variance at Complete (VAC) at WBS and Control Account Levels

• Ability to recognize driving factors• Can be performed at the control

account and Work Package level

VAC

VAC

Month-Over-Month EAC


HSV FEB 2020

Baseline Forecast

• Not only does the IPMDAR contain a project’s cost data (CPD) but it also contains the project’s schedule data (SPD)

• Provides for cost schedule integration

ID Baseline Start Date Baseline End Date Forecast Start Date Forecast End DateTask 2 3/1/2020 3/26/2020 3/1/2020 6/9/2020Task 3 4/1/2020 4/13/2020 6/10/2020 6/21/2020Task 4 5/1/2020 5/23/2020 6/21/2020 7/12/2020

Native Schedule (IMS)

Schedule Dataset (SD)


HSV FEB 2020

VAC

Time Now

ID Baseline Start Date Baseline End Date Forecast Start Date Forecast End DateTask 2 3/1/2020 3/26/2020 3/1/2020 6/9/2020Task 3 4/1/2020 4/13/2020 6/10/2020 6/21/2020Task 4 5/1/2020 5/23/2020 6/21/2020 7/12/2020

• What specific Task / Work Package is causing the VAC?

• Cost and Schedule Integration Check - Are the CPD and the IMS/SPD telling the same story?

• Question to ask - Was more work added? Was the task planned incorrectly?

• Charts and analysis promote quick realization of cost and schedule impact and promotes a timely corrective actions discussion

• Is an OTB/OTS eminent?• Is there a scope creep?


HSV FEB 2020

Time Now

Time Now

Time Now

• What is one of the causes for the Budget versus Forecast cost discrepancy?

• IPMDAR provides the ability to dissect and isolate expended costs as well as planned/forecasted costs by element of cost (EoC)

• Should be noted in the Executive Summary

Time Phased Budget vs ETC by Control Account & Element of Cost (EOC)

• Indirect rate analysis – presenting changes by element of cost


HSV FEB 2020

Questions?


HSV FEB 2020

Acquisition Exchange Program (AEP)

Christine Kastl - Missile Defense Agency


HSV FEB 2020

Program Summary Acquisition, Analytics and Policy (AAP) Acquisition Exchange Program (AEP)

provides a unique career development experience for high-caliber individuals in acquisition, and acquisition-related career fields to serve as a Temporary Support Augmentee (TSA)

Program Objective Interact with senior officials within DoD and throughout the Federal

Government

Develop a thorough knowledge of EVM policy, development, and execution

Enhance the skills needed to prepare, advocate, and revise policy practices and procedures

Promote different perspectives within acquisition policy decision-making, while engaging in a career-broadening experience


HSV FEB 2020

Major policy updates and changes Integrated Program Management Data and Analysis Report (IPMDAR) Data Item

Description (DID) and Implementation Guide

EVM and Agile Desk Guide Work Group sessions – Integrated Program Manager’s Toolkit

Cost of Testing and Evaluation Working Group (CoTEWG) – proposing changes to the DoDI 5000.73

Travel and networking EVM Practitioners’ Forum – Roslyn, VA

IPMDAR Adjudication Team Meeting – Crystal City, VA

Naval EVM Symposium – Crystal City, VA

Army Black Belt Project Overview – Falls Church, VA

IPMDAR Adjudication Team Meeting – Melbourne, FL

National Defense Industrial Association (NDIA) Integrated Program Management Division (IPMD) – Melbourne, FL

Defense Acquisition University (DAU) Days – Huntsville, AL

IPMDAR Adjudication Team Meeting – Crystal City, VA


HSV FEB 2020

Real life application of Agile management processes Scrum Master training materials, Scrum framework knowledge, JIRA usage

Interact with senior leaders at the Pentagon Ms. Ellen M. Lord – Under Secretary of Defense of Acquisition and

Sustainment (OUSD A&S)

Ms. Stacy Cummings – Principal Deputy Assistant Secretary of Defense, Acquisition Enablers (AE)

Mr. David Cadman – Director of Acquisition, Analytics and Policy (AAP)

Mr. John McGregor – Deputy Director of AAP EVM


HSV FEB 2020


HSV FEB 2020

How do I apply? https://www.acq.osd.mil/evm/#/aep


HSV FEB 2020

AAP EVM Website:http://www.acq.osd.mil/evm/

AAP EVM Email:[email protected]

DoD EVM Policy, Application, and Reporting...APPROVED FOR PUBLIC RELEASE DISTRIBUTION 1 Office of...

Documents

Transcript of DoD EVM Policy, Application, and Reporting...APPROVED FOR PUBLIC RELEASE DISTRIBUTION 1 Office of...