Inside this

issue:

From the CEVM:

Assurance and Data

1

IPM Learning Culture 2-3

IMS Forecasting Metrics:

Current Execution Index 4-5

IBR: The Hidden Stories 6-7

Project Controls 8

EVM Training Materials

Available Online 9

Crossword Puzzle 10

Calendar 11

The Baseline Department of the Navy (DON)

Earned Value Management (EVM)

Quarterly Newsletter

April 2018

At its core, Earned Value Management (EVM) within the Department of Defense (DoD) involves two

main principles: assurance that the contractor has a disciplined approach to program

management and data needed for effective Government program management oversight.

When the cost, schedule, and/or technical risks are of a sufficient concern, it is in the best interest

of the Government that the contractor has an integrated approach to program management. An

Earned Value Management System (EVMS) integrates inputs from the planning, scheduling,

budgeting, work authorization, and cost accumulation processes with the program work

breakdown structure and organizational structure. The Federal Government has adopted the

Electronic Industries Alliance Standard 748 (EIA-748) as the industry standard to ensure the

program management system, or EVMS, used by the contractor has the necessary level of

discipline.

Within the DoD, the Integrated Program Management Report (IPMR) is used to define the program

management data needed for effective Government oversight. The IPMR Data Item Description

(DID) describes the data necessary to communicate program cost and schedule information

between the contractor and the Government. The IPMR DID will ensure the Government receives a

reliable output of the contractor’s EVMS. The reliability of the data report is vital to provide for

Government program management decisions based on an accurate account of the contracted

effort.

From the CEVM:

Assurance and Data

Written By: David Tervonen

Empowering Speed and Innovation

through an IPM Learning Culture

NAVAIR is a high-performing organization that leadership challenges to be fast, innovative, and increasing speed to the fleet

(Grosklags, 2017). One aspect of how NAVAIR-4.2.3, Integrated Project Management (IPM) Division, maintains this expectation is

by encouraging a learning culture.

A learning culture is a collection of organizational conventions, values, practices and processes. These conventions encourage

employees and organizations to develop knowledge and competence. An organization with a learning culture promotes

continuous learning and believes that systems influence each other. Since constant learning elevates an individual as a worker

and as a person, it opens opportunities for the establishment to transform continuously for the better. (Nabong, 2015)

High-performing organizations are three times as likely to have a strong learning culture (Mallon, 2009). In industries where

innovation is prized, such as military RDT&E, there is a high correlation between the drivers of innovative organizations and

learning cultures (Mallon, 2009), see Table 1. This continuous learning goal has persisted from NAVAIR leadership with the

development of NAVAIR University (NAVAIRU) and its nine colleges, RADM Eastburg’s emphasis on a learning organization since

2007 has led to VADM Dunaway’s 2013 NAVAIR Long-Range Strategy (Dunaway, 2013) and the current Navy Civilian Framework

(Richardson, 2016).

The IPM Division has more than 180 assigned analysts. Within the IPM Division, creating a learning culture for this many analysts

started with department and division leadership establishing: a training roadmap of well-designed training offerings, expectations

regarding professional development, and alignment between supervisory chains (branch heads) and Subject Matter Experts

(process group SMEs).

First, AIR-4.2.3 training offerings (Table 2, next page) were developed in coordination with professional instructional designers to

optimize courses for learner retention with continuous improvement from course evaluations and learner feedback. This

approach has led to a variety of course implementations based on the complexity of the subject matter and the optimal learning

environment, including: formal classroom offerings, practical supervised workshops, and self-paced workshops. This provides

our IPM teammates with the skills necessary for Earned Value Management (EVM) analysis, Integrated Government Schedule (IGS)

development/maintenance and other IPM services for our customers. Next, AIR-4.2 has included professional development as

one of three categories for employee ratings, recognizing the inherent value of a learning culture. Finally, this implementation

creates synergies between AIR-4.2.3 supervisors and the process groups resulting in a dialogue between analysts, developers,

SMEs, and leadership that proliferates lessons learned via white papers, toolkit amendments, tool improvements, one-hour “brown

bag” presentations, innovation captures and competency initiatives. In summary, IPM leadership was necessary to create the

environment where a learning culture could flourish, resulting in the wide-ranging expertise, experience, and processes that the

command values.

Written By: Matthew Wilkinson—NAVAIR

April 2018 Page 2

Value Drivers of Innovation

and Learning Cultures

(Mallon, 2009) Sample NAVAIR-4.2.3 Implementation

Values Time Spent Learning New

Knowledge and Skills High-Quality Instructional Design for Formal Classroom Offerings Training Roadmap and Expectations

Values Mistakes and Failures as

Opportunities for Reflection

Brown Bags and Sharing Lessons Learned Continuous Improvement through Training Feedback (Questionnaires, Online

Forms, and Examinations for Retention) Empowers Employees to have

Influence over Tasking Innovation Captures and Competency Initiatives

Rewards Learning Included in the Professional Development Rating for Analysts

Clearly Defines Decision-making

Processes Supervisory and Process Group Alignment Documented Standard Processes (Toolkits)

Encourage Questioning Program Office integration for IPM Analysis and IGS Development Processes Practical Workshops in lieu of Lecture-based Training

Promote Employee Responsibility

for Personal Development Self-paced Workshops Competency Initiatives as Targeted Growth Opportunities

Challenge Employees with Tasks

beyond Current Abilities Advanced Analytics to Support Customer Requests and Needs Advanced Scheduling Techniques to Support Customer Requests and Needs

Table 1: NAVAIR Integrated Project Management Implementation of a Learning Culture

(continued on Page 3)

Of course, the knowledge, skills, and attributes of an EVM analyst or IGS developer do not provide NAVAIR value until they are

implemented for project success. Therefore, establishing a learning culture was not only intra-divisional but required

enculturating lessons learned and best practices across the command. AIR-4.2.3 developed a variety of courses in coordination

with AIR-4.2’s Program Performance Team that target Integrated Project Team (IPT) members, Program Office leadership, and the

larger NAVAIR community for project planning and execution success. This series of courses includes Framing EVM for IPTs,

Foundation for Project Success, Managing with EVM, Managing Uncertainty for Project Control, and Integrated Schedule

Management Project Controls. All have been included in NAVAIRU, a command-wide initiative that provides a one-stop shop for

all NAVAIR-specific training that is integrated with official human resource training records. Additionally, AIR-4.2.3 analysts,

developers, and SMEs espouse these IPM methods, benefits, and products via targeted training, customer discussions, and

preparatory training before major reviews or risk assessments, such as Schedule Risk Assessments, Estimates to Complete, and

Integrated Baseline Reviews. Establishing a learning culture where technical and programmatic experts can cross-train with IPM

techniques produces a highly-functioning project management group across the command.

Following this approach, a learning culture has been established which has resulted in an 80% increase in demand for our

services, with the workforce growing from 105 in 2007 to 188 in 2017. Since 2010, AIR-4.2.3 training is consistently rated very

high, with a 94% average for internal training and a 91% average for enculturation training (Program Performance Team, 2018).

The quality of the training for learner retention and an enculturation focus have been key contributors to successfully establishing

a learning culture, which routinely reinforces itself and produces the benefits of high-performance and innovation resulting in

increased speed to the fleet.

Bibliography

Dunaway, D. (2013, September). NAVAIR’s Long-Range Strategy. Retrieved February 2, 2018, from Naval Air Systems Command: http://www.navair.navy.mil/osbp/index.cfm?

fuseaction=home.download&id=551

Grosklags, P. (2017). 2017 Commander’s Intent. Patuxent River: Naval Air Systems Command.

Mallon, D. (2009, May). Learning Culture and Innovation: How Your Organization’s Learning Culture Affects Your Company’s Ability to Get Ahead and Stay Ahead. Bersin &

Associates.

Nabong, T. A. (2015, April 7). Creating a Learning Culture for the Improvement of your Organization. Retrieved February 2, 2018, from Training Industry: https://

www.trainingindustry.com/articles/strategy-alignment-and-planning/creating-a-learning-culture-for-the-improvement-of-your-organization/

Program Performance Team. (2018, February 2). Survey Data. NAVAIR-4.2.

Richardson, J. (2016, October). Navy Civilian Workforce Framework, v1.0. Retrieved February 2, 2018, from Navy.mil: www.navy.mil/navydata/people/cno/Richardson/.../

April 2018 Page 3

IPM Learning Culture cont.

Table 2: NAVAIR-4.2.3 Training Roadmap

Careful Integrated Master Schedule (IMS) performance measurement is important to understanding the status of your

program. The IMS provides an ever-changing window into the contractor’s progress (or lack thereof). After all, not only is

your IMS the backbone of earned value metrics, the IMS is also your early warning system for potential problems. How the

contractor executes the baseline gives us the basis of all earned value metrics. However, as you get further away from your

Integrated Baseline Review (IBR) or your contract falls further behind schedule, the ability to forecast the upcoming work

accurately becomes paramount. This is where the Current Execution or CEI metric becomes valuable.

CEI is simply an IMS metric that measures how accurately the program is forecasting and executing to its forecast from one

month to the next. In other words, what did the contractor tell me he was going to finish this month (in last month’s IMS)

and how well did he do it! It measures the ability to forecast from month to month vice the ability to execute the baseline

plan.

Current Execution Index =

The National Defense Industrial Association (NDIA) has set the standard for industry at 75% efficiency. A program that is

doing well is always executing to their forecast. Those programs that are struggling are not.

One of the real benefits of using CEI is an increased program emphasis on ensuring the accuracy of the forecast schedule.

After all, if the contractor can’t forecast what they are going to do next month how well do you think they are forecasting

system delivery?

Typical monthly CEI tracking metrics look something like this:

In this example, we can see that over the past six months, the contractor has failed in all months to meet the 75% target.

Integrated Master Schedule (IMS)

Forecasting Metrics:

Current Execution Index (CEI)

Page 4 April 2018

Written By: John Collins—NELO

# of tasks completed in the month as forecast in prior month

# of tasks forecast to complete in the month per the prior forecast

(continued on Page 5)

The previous chart can be broken down even further by identifying problem Control Account Managers (CAMS) or the Work

Breakdown Structure elements. Both the contractor’s Organizational Breakdown Structure (OBS) and Contract WBS are

required to be traceable to relevant effort in the IMS by the Integrated Program Management Report (IPMR) Data Item

Description (DID). A breakdown of the current month’s performance by CAM could be presented in a simple table like the

one below.

In this example, CAMs Rogers, Wilson, Walters and Richards are failing to meet the 75% goals, and Rogers is the worst

performer by far. Use of the CEI metric at this level, identifies both positive and negative performance issues and drives

ownership and accountability behaviors at the contractor that are necessary for program success. If you find that one or two

CAMs are usually responsible for the contractor failing the forecasting metric, it’s a good bet that the CAM or CAMs may be

experiencing difficulty and not have a good handle on how his/her work will proceed.

Current Execution Index (CEI) cont.

Page 5 April 2018

CAM Number

Forecasted to Finish

Number Actually Finished

Number Missed Percentage Hit

Rogers 9 0 9 0.0%

Wilson 5 3 2 60.0%

Walters 12 8 4 66.7%

Richards 10 7 3 70.0%

Whitman 10 8 2 80.0%

Williams 5 4 1 80.0%

Parker 5 4 1 80.0%

Barton 7 6 1 85.7%

Walker 7 6 1 85.7%

Danvers 5 5 0 100.0%

Totals: 75 51 24 68.0%

As one of the Department of the Navy’s (DON) most experienced gate keepers of the IBR process, the NAVAIR Cost Estimating

and Analysis Department’s Integrated Project Management Division (AIR-4.2.3) discovered a hidden story in capturing,

maintaining and trending IBR data. Within the 4.2.3 Analysis Process Group (APG) and Management Systems and Training

Process Group, an IBR database has been established to capture IBR results including: the Control Account Managers (CAMs)

summary slides (~980 slides to date) along with data from 83 different programs, 19 defense contractors at 28 different

locations from IBRs covering the past ten years.

This information is used to monitor contractor IBR history, prepare for future IBRs, and use in EVM analysis. The hope is that

this tool will enable analysts to be proactive with the contractor by identifying potential risk factors early (before the next IBR

event) and examining potential areas of concern while developing monthly analysis.

The goal was to populate this database and use it to understand the findings from previous IBRs with the ability to sort, trend

and analyze by contractor/site etc. Data is gathered from the IBR documents—including the Appendix F (IBR Summary Form)

and CAM Interview Summary—to populate the database:

“Common Findings” were established to create some similarities amongst the data, reduce the total number, and create a

meaningful way to categorize the risks. The risks are bucketed into one of these common findings when input in the database.

Based on extensive IBR history and lessons learned, a list of “Typical IBR Findings” had already been established and was used

to create the common findings list.

Integrated Baseline Review (IBR):

The Hidden Stories

Page 6 April 2018

Written By: Victor Thomas—NAVAIR

List of Common Findings

IBR Database: Populated using data and findings from CAM summary slides

(continued on Page 7)

The development of the common findings is credited in part to the IBR training provided to analysts and IPT’s in preparation

for an upcoming event. Based on extensive IBR history and lessons learned, a list of “Typical IBR Findings” had already been

established and was used to assemble the common findings list. This enables the analysts to track and trend the effectiveness

of the contractor’s corrective actions after the IBR event is deemed closed and compare with current issues found during

monthly IPM analysis.

Below is a sample of how the data is used to support the IPM analyst while preparing for an SRA:

As a time saving measure, a proposal was put forward for the SRA to be performed internally, providing results for review and

approval. The IBR database was used to understand the Contractor and site IMS risk during past IBR’s for evaluation of this

proposal. After finding significant historical issues with the health of schedules, it was recommended not to allow for the

acceptance of an internal SRA from that site/contractor.

Below is another example of the data that can be gathered using our Excel raw data export tool:

This chart illustrates the results of many IBR’s at a particular site. The inadequate budgets finding was described the most at

this site during IBR’s. However, after examining the overall risk (as categorized by our established risk criteria), cost and

schedule carry the most risk at this particular site.

This is still a new product in the early stages, but it has helped us better understand the obscure hidden story issues seen in

analysis before the IBR (based on past history) as well after the IBR, addressing the propensity of effective corrective actions.

This data can be used:

• In preparing for follow-on reviews and IBR training

• In understanding site/contractor behavior

• To assist with understanding monthly IPM analysis

• To understand the effectiveness of the contractor’s corrective action plan suggested action(s)

IBR: The Hidden Stories cont.

Page 7 April 2018

Project Controls

Page 8 April 2018

Written By: Andrew Wargo—SSP

The development of a finely tuned process for project planning and delivery and rigorous program management practices is

paramount for successful delivery on all capital projects. The key to achieving this, although straightforward, is

simultaneously both unassailable and inaccessible to many organizations: project controls.

Project controls may seem too simple; however, successful planning and completion of a project require an understanding of

project scope, divided into manageable pieces, combined with a realistic schedule of discrete effort, and integrated with

insight into the resources available.

While many companies are adept at obtaining contracts for projects of staggering proportions, one specific thing enables the

delivery of each of those projects and will enable the delivery of future projects: the company is willing to assume risk due to

its confidence in project execution. As such, execution risk falls on the company and its partners, not the customer, in the

event of overruns or delays.

Similarly, understanding the scope desired within the required time frame is paramount to successfully executing resources.

Unbounded execution of resources without a plan can have significant consequences. Accordingly, the fundamental concept

in a disciplined project controls environment is to begin with the end in mind and to plan from the beginning.

Project controls help to identify deviations from the plan along the way, add structural stability to the management of the

project via change control, and contribute to effective risk and opportunity management by integrating the analysis of each

into management decisions.

Each major portion of large projects—technologies, capabilities, or pieces of construction—requires management at all levels.

Ultimately the configuration controls, change controls, coordinated limits, and management of the overall project must be

informed by a rigorous system of integrated planning, measurement, change control, management decisions, and customer

engagement. Customer engagement and trust on massive projects hinge upon the reputation of the performing organizations

and disciplined discussions of performance that rely on project controls to inform and drive sound decisions at all levels.

“Begin with the end

in mind.”

-Stephen Covey

Page 9 April 2018

EVM Training Materials Available Online

Defense Acquisition University (DAU):

EVM 101 Fundamentals of Earned Value Management

In a virtual classroom environment, professionals learn additional information about earned value management (EVM), which is

introduced in the prerequisite course. The course summarizes the language, data reports, metrics, graphs, and management

processes associated with EVM as they apply to DoD acquisition management. Professionals also learn the processes related to the

Performance Measurement Baseline, the Integrated Baseline Review, and the 32 Guidelines prescribed in Section 2 of the Electronic

Industries Alliance Standard (EIA-748) for EVM systems. Finally, professionals evaluate and compute basic EVM metrics and EVM metric

-based estimates at completion.

EVM 202 Intermediate Earned Value Management

Professionals taking this course work as members of an integrated product team for an ACAT I program. In the context of integrated

program management, participants review, develop, and experience the Earned Value Management (EVM)-related processes associated

with requirements generation, acquisition strategy development, Request for Proposal development, source selection, risk

management, Integrated Baseline Review, and analysis during program execution.

EVM 262 EVMS Guidelines and Compliance

The Earned Value Management System (EVMS) guidelines and compliance course provides knowledge needed to review integrated

management systems and to determine their compliance with the Electronic Industries Alliance EVMS (EIA 748) standard. Course

material, individual exercises, and group exercises review in-depth the 32 EIA 748 EVMS guidelines and the processes associated with

approval and surveillance of contractor and government integrated management systems.

EVM 263 Principles of Schedule Management

This course provides knowledge needed to interpret network schedules required by DoD policy and the Electronic Industries Alliance

(EIA) 748 Earned Value Management System (EVMS) Standard. Course material, individual and group exercises demonstrate the

schedule development, maintenance, and analysis processes. The exercises reinforce: the Precedence Diagram Method of scheduling;

schedule analysis using a schedule assessment model to analyze a complex; 700-line Microsoft Project network schedule; and

schedule risk analysis utilizing Monte Carlo simulation. Students will be required to create a Microsoft Project network schedule.

Other training and certifications are available from sources to include, but not limited to: the College of Performance

Management, AACE International, and Project Management Institute.

Page 10 April 2018

Crossword Puzzle

Page 11 April 2018

This publication was created by the Naval Center for Earned Value Management (CEVM) with contributions from the Systems

Commands. Please contact us at 703-695-0510 for any questions on the content of this publication. For more information

on EVM within the Department of the Navy: http://www.secnav.navy.mil/rda/OneSource/Pages/CEVM/CEVM.aspx

April May June

Su M Tu W Th F Sa Su M Tu W Th F Sa Su M Tu W Th F Sa

1 2 3 4 5 6 7 1 2 3 4 5 1 2

8 9 10 11 12 13 14 6 7 8 9 10 11 12 3 4 5 6 7 8 9

15 16 17 18 19 20 21 13 14 15 16 17 18 19 10 11 12 13 14 15 16

22 23 24 25 26 27 28 20 21 22 23 24 25 26 17 18 19 20 21 22 23

29 30 27 28 29 30 31 24 25 26 27 28 29 30

Calendar

EVM Events

30 Apr - 3 May NDIA Integrated Program Management Division Meeting

30 May - 1 Jun EVM World 2018, Fort Lauderdale, FL

Federal Holidays

28 May Memorial Day

Dec 18 Edition’s

Crossword Answers:

ASN(RD&A) Events

9 Apr The Athena Project (Deckplate Innovation Event) at Sea Air Space

11 Apr Small Business Facebook Live Event at Sea Air Space

12 Apr HASC Seapower Subcommittee Hearing

14 Apr LCS 17 Indianapolis Christening

17 Apr SASC Seapower Subcommittee Hearing

21 Apr LPD 27 Portland Commissioning

25 Apr Investing in the National Security Innovation Base

2 May Modular Open Systems Summit