X Drives Y - DAU

Anil DuaProf. of Contract ManagementDefense Acquisition University

[email protected] 2, 2018

X Drives Y

What it Means and How it Can Help Determine a Fair and Reasonable Price

May 15, 2019

2 Price Analysis-The Basics

Right Item Right Time Right Price OR Cost, Schedule, Performance

3 Things We All WantIncluding the Government


As a Reminder - Our Job

Our Job:

“Contracting Officers must purchase supplies and services from responsible sources at fair and reasonable prices.”

Very often we forget the “Responsible Sources” portion when we speak and simply state a fair and reasonable price.

Don’t Forget – The job is both so complete the sentence and do both.


Based on SPEEDING results in TICKETS

OR

Based on __X__ results in __Y__

If questioning what is my X and what is my Y, simply ask yourself:

Based on ___ results in ____ ?

Simply fill in the blanks accordingly – go with the one that makes sense.

X Drives Y – A Simple Example


X Drives Y – What Does it Mean?

Dependent Variable

Independent Variable

X (Speeding)

Y(Tickets)


Cause and Effect

Factors

Ratios

Overhead Rates or Indirect Cost Rates

Pool (Expenses)/Base (Direct Costs)

Improvement Curves

Regression Analysis

For purposes of Cost Estimating the above terms fall under an umbrella of what is called Cost Estimating Relationships (CERS)

Other Ways to Describe X Drives Y


From the CPRG (Contract Pricing Reference Guide): A cost estimating relationship (CER) is a technique used to estimate a particular cost or price by using an established relationship with an independent variable.

If you can identify an independent variable (driver) that demonstrates a measurable relationship with contract cost or price, you can develop a CER. That CER may be mathematically simple in nature (e.g., a simple ratio) or it may involve a complex equation.

“CER” Defined


We use CERs to forecast or predict the value of a dependent variable (Y), typically price, based on the value of a known independent variable (X).

In other words if you have history or collect data, where a relationship exists between X and Y you can possibly use it to help predict a price or some type of resulting (y).

Key thing to Understand


Other Examples of CERs

Indirect Cost PoolAllocation Base

Rate =

Eng OverheadDirect Labor $

Rate =

Mfg OverheadMachine Hours

Rate =

G&A ExpensesTotal Cost Input

Rate =

Mfg OverheadDirect Labor Hours

Rate =

G&A ExpensesValue Added Base

Rate =

Material O/H# Purchase Orders

Rate =


Identifies a relationship between the X and the Y variables

Causes you to Focus on the Key “Cost” Drivers of a Proposal

Allows you to question the Contractor’s proposed cost and defend your position.

How Can it Help Determine a Fair and Reasonable Price


What Does the FAR Say

How do we use cost “drivers” in our analysis?

Price Analysis – FAR 15.404-1(b)(iii) Use of parametric estimating methods/application of rough yardsticks (such as dollars per pound or per horsepower, or other units) to highlight significant inconsistencies that warrant additional pricing inquiry.

Cost Analysis – FAR 15.404-1(c)(c) Reasonableness of estimates generated by appropriately calibrated and validated parametric modelsor cost-estimating relationships; and(d) The application of audited or negotiated indirect cost rates, labor rates, and cost of money or other factors.


Steps to Developing CERs

1. Define the dependent variable2. Select independent variables to be

tested3. Collect data on the variables4. Explore the relationship between the

dependent and independent variables5. Select the relationship that best

predicts the dependent variable6. Document your findings


1. Define the Dependent Variable

• Will the CER be used to:– Estimate Price, Cost Dollars, Labor

Hrs, Mat’l Cost– Estimate Total Cost or Estimate a

component cost• Understanding and defining the

requirement is foundational to determining a fair and reasonable price.

• There is no substitute for good definition and no way to correct for a poor definition.


2. Selecting Independent Variables

• A good independent variable should:– Have a causal/beneficial relationship to the dependent

variable– Explain changes in the dependent variable– Be easily quantified and measured– Be available or obtainable– Be predictable for what you are pricing

• Independent variables often represent some measure of quantity, performance or physical characteristics. There are both cost-to-cost and product-to-cost relationships.


What “drives” costs?

• By definition, contract pricing with CERs requires us to know the independent variables, or “drivers” for every requirement. What are the “drivers” for cost or design in the following areas

Product Independent VariableBuilding Construction Floor space, roof surface area, wall surface

Passenger Car Curb weight, wheel base, passenger space, horsepowerTrucks Empty weight, gross weight, horsepower, number of driving axles,

loaded cruising speed

Sheet Metal Net weight, percent of scrap, number of rivets placed, inches of welding

Aircraft Empty weight, speed, useful load, wing area, power, landing speed


3. Collect Data on the Variables

• Data collection should not be limited to simply dollars or hours. Data should include all the factors that have an impact on the dollars or hours such as:

• Quantity• Year• Contract type• Terms and conditions• Market conditions• Business base


4. Explore the relationship between the dependent and independent variables

• Personal experience, subject matter experts, and graphical analysis can help you determine the underlying relationships.


5. Select the best estimating relationship

• In addition to expert opinion and visual analysis we have statistics such as, R-squared, T-statistic/Probability Not Zero, Standard Error (SEE) and Coefficient of Variation (CV).


6. Document your findings

• Allows others (and yourself) the ability to recreate your work and to properly apply the CER.

• Typical PNM comments:• The words in a PNM should convey the story about the

numbers. Do not depend on an attached spreadsheet to convey the story.

• The writer of the PNM needs to balance convenience and efficiency with the reader’s need for clear, logical, trackable and compelling documentation.

• Do not assume the data speaks for itself.


Issues and Concerns

1. Does the available data verify the existence and accuracy of the proposed relationship?

2. Is there a trend in the relationship?3. Is the CER used consistently?4. How current is the CER?5. Would another independent variable be better for

developing and applying a CER?6. Is the CER a self-fulfilling prophecy?7. Would a detailed estimate or direct comparison with actuals

from a prior effort produce more accurate results?

Several Demonstrations of How X Drives Y


Demonstrations of How X drives Y

House Price #BR $ / BR Acres $ / Acre Sq ft $ / Sq ft

1 $350K 4 .5 3500

2 $297K 3 2 3000

3 $409K 4 .5 3900

4 $307K 3 1 3200

5 $303K 4 5 3000

6 $306K 3 .5 3100

Range

Avg

Range/Avg

Residential House PricesMarket Research Data


Demonstrations of How X drives Y

Residential House PricesMarket Research Data

House Price #BR $ / BR Acres $ / Acre Sq ft $ / Sq ft

1 $350K 4 $87.5K .5 $700K 3500 $100

2 $297K 3 $99K 2 $149K 3000 $99

3 $409K 4 $102.25K .5 $818K 3900 $105

4 $307K 3 $102.33K 1 $307K 3200 $96

5 $303K 4 $75.75K 5 $60.6K 3000 $101

6 $306K 3 $102K .5 $612K 3100 $99

Range $26.6K $757.4K $9

Avg $94.81K/ BR $441K/ acre

$100/ sqft

Range/Avg .28, or 28% 1.72. or172%

.09, or 9%


Bedrooms Price4 $350,000.003 $297,000.004 $409,000.003 $307,000.004 $303,000.003 $306,000.00

Residential House PricesPrice Per Bedroom

Number of Bedrooms

Price

$0.00

$50,000.00

$100,000.00

$150,000.00

$200,000.00

$250,000.00

$300,000.00

$350,000.00

$400,000.00

$450,000.00

0 1 2 3 4 5


Acres Price.5 $350,000.002 $297,000.00.5 $409,000.001 $307,000.005 $303,000.00.5 $306,000.00

Residential House PricesPrice Per Acre

Acres

Price

$0.00

$50,000.00

$100,000.00

$150,000.00

$200,000.00

$250,000.00

$300,000.00

$350,000.00

$400,000.00

$450,000.00

0 1 2 3 4 5 6


Square Feet Price3500 $350,000.003000 $297,000.003900 $409,000.003200 $307,000.003000 $303,000.003100 $306,000.00

Residential House PricesPrice Per Square Feet

Square Feet

Price

$0.00

$50,000.00

$100,000.00

$150,000.00

$200,000.00

$250,000.00

$300,000.00

$350,000.00

$400,000.00

$450,000.00

0 1000 2000 3000 4000 5000


Another Example - Water Purifier

You have just received a new requirement to procure ten (10) mobile water purifiers. Accompanying the requirements letter is a purchase request (PR) in the amount of $80,000.

Your supervisor says that the PR represents the independent government cost estimate (IGCE) and that it should be included as part of your price analysis.


Water Purifier

While awaiting a more detailed technical description from the customer, are there any actions you should take with respect to the PR?

The budget person said that this particular water purifier was not in the inventory, so they collected prices on seven (7) water purifiers of a similar type and used the average price.

Determine what your expected pricing error would be if you used the average price as your objective.


Unit Price of Similar Systems

$ 9,475

$ 5,185

$12,499

$ 6,195

$ 2,499

$11,499

$ 8,499



$7,979 x 10 units = $79,790


Do We See a Potential CER?

Price (y)

GPH (x)

$12,499 5,500$11,499 4,500$9,475 4,000$8,499 3,250$6,195 2,500$5,185 1,500$2,499 600

Req’t 1,800


ASSUMPTIONS The price of the other system ($5,185) was itself, a fair

and reasonable price The only significant difference between the two units is

GPH (or all other differences are captured by GPH) The difference in the prices is directly proportional to the

difference in the GPH

Analogy–Cost Factor

= = $3.46 per GPHPriceGPH

$5,1851,500

Requirement x Factor = 1,800 GPH x $3.46 = $6,228


Price Directly Proportional to GPH

0

2000

4000

6000

8000

10000

12000

14000

0 1000 2000 3000 4000 5000 6000

Pri

ce

GPH

PRICE PER GPH

1800

$6,228



Is this CER/factor approach good?

Did this analysis support the PR estimate of $80,000?

oYesoNo


Let’s use the regression tool to plot the data from the seven buys.

o This will give us a linear representation of the data

o Below is the URL to the regression tool.

o Please access at your convenience after this webinar.


https://www.dau.mil/tools/t/SLR-Tool

Then Click Launch Tool



Plot Data Into Regression Tool

Price (Y)

GPH (X)

$12,499 5,500$11,499 4,500$9,475 4,000$8,499 3,250$6,195 2,500$5,185 1,500$2,499 600

Req’t 1,800


How Does Our CER (Price Related To GPH) Look Now Based On All The Data?


0

2000

4000

6000

8000

10000

12000

14000

0 1000 2000 3000 4000 5000 6000

Pri

ce

GPH

Price per GPH

Factor

General Trend

How Does Our CER (Price Related To GPH) Look Now Based On All The Data?


Let’s Continue With This Trend Line We Developed With the Regression Tool

0

2000

4000

6000

8000

10000

12000

14000

0 1000 2000 3000 4000 5000 6000

Pri

ce

GPH

Price per GPH

General Trend


Input the Requirement into the Regression Tool


Formula of a Line Y= A + BX

A = Y Intercept B= Slope X= The Requirement

Accuracy of the Equation – SEE and CV

Two Tests Needed to Use the Equation

o R- Squared o Probability Not Zero (T-Statistic)

ANOVA Table

Understanding Terms Used in Regression Analysis


Look at the estimating equationYc = A + B X

Price = 1559.22659 + (2.05659)(GPH)

Range is from $4071 to $6451 (at 90% confidence) for 1800 GPH purifier

Trend/Regression Analysis


Standard Error of the Estimate

Coefficient of Variation

How Accurate is The Equation for Estimating/Pricing Purposes


AVERAGE (MEAN) PRICE = 7979 PRICE = 1559.22659 + 2.05659 (GPH)Price

Price Y GPH Y YC Y - YC (Y - YC)2

12499 12499 7979 4520 20432983 5500 12499 12870 -371 13797711499 11499 7979 3520 12392412 4500 11499 10814 685 4694099475 9475 7979 1496 2238871 4000 9475 9786 -311 964558499 8499 7979 520 270697 3250 8499 8243 256 654686195 6195 7979 -1784 3181637 2500 6195 6701 -506 2557255185 5185 7979 -2794 7804840 1500 5185 4644 541 2925662499 2499 7979 -5480 30027269 600 2499 2793 -294 86541

0.000 76348707 0.000 1404141

GPH

0

2000

4000

6000

8000

10000

12000

14000

0 1000 2000 3000 4000 5000 6000

Y� Y − Y� (Y − Y�)2

Statistics to Regression TransitionMobile Water Purifiers

s2 = Σ(Y – Y�)2

n – 1=

76,348,7077 − 1

= 12,724,784.5

How would you put the standard deviation and CV into words?If we used the average price as our objective we could typically be off by give or take $3,567, or give or take 45%.




12499 12499 7979 4520 20432983 5500 12499 12870 -371 13797711499 11499 7979 3520 12392412 4500 11499 10814 685 4694099475 9475 7979 1496 2238871 4000 9475 9786 -311 964558499 8499 7979 520 270697 3250 8499 8243 256 654686195 6195 7979 -1784 3181637 2500 6195 6701 -506 2557255185 5185 7979 -2794 7804840 1500 5185 4644 541 2925662499 2499 7979 -5480 30027269 600 2499 2793 -294 86541

0.000 76348707 0.000 1404141

GPH

0

2000

4000

6000

8000

10000

12000

14000

0 1000 2000 3000 4000 5000 6000

Y� Y − Y� (Y − Y�)2

Statistics to Regression TransitionMobile Water Purifiers

s2 = Σ(Y – Y�)2

n – 1=

76,348,7077 − 1

= 12,724,784.5

How would you put the SEE and CV into words?If we used this equation, we could typically expect to be off by give or take $529.93, or give or take 6.64%.

Variance = Σ( Y – Yc )2

n – 2 =

14041417-2

= 280828.2

SEE = √Variance = √280828.2 = 529.93

CV = SEE

Y� =

529.937979

= .0664 or 6.64%


R-squared (Coefficient of Determination)

Should be greater than 80% in order to usethe equation

Test # 1 Explaining the Variations


Goodness of Fit Statistics


Explaining the Variation in the Prices Mobile Water Purifiers



12499 12499 7979 4520 20432983 5500 12499 12870 -371 13797711499 11499 7979 3520 12392412 4500 11499 10814 685 4694099475 9475 7979 1496 2238871 4000 9475 9786 -311 964558499 8499 7979 520 270697 3250 8499 8243 256 654686195 6195 7979 -1784 3181637 2500 6195 6701 -506 2557255185 5185 7979 -2794 7804840 1500 5185 4644 541 2925662499 2499 7979 -5480 30027269 600 2499 2793 -294 86541

0.000 76348707 0.000 1404141

GPH

0

2000

4000

6000

8000

10000

12000

14000

0 1000 2000 3000 4000 5000 6000

Y� Y − Y� (Y − Y�)2

Σ(Y – Y�)2 Σ( Y – Yc )2

Total Variation Unexplained Variation

Explained Variation = Total Variation – Unexplained VariationExplained Variation = 76348707 – 1404141 = 74944566Explained Variation

Total Variation7494456676348707 0.9816 or 98.16%= =

Explained VariationTotal Variation = r 2 (Coefficient of Determination)

0 ≤ r 2 ≤ 1We could say then that 98.16% of the variation in Price has been explained or

accounted for by the variation in GPH (capacity)


r2 = ~98% o Means the change in GPH

explains 98% of the variation in price

SEE (standard error of estimate) = 529.932o we could expect to be off by

give or take $530 CV = 6.64%

o our error is about 7%

Goodness of Fit Statistics (Strength of the Relationship)


Probability Not Zero = 100%o T-Stat tell us that we are essentially 100%

confident that there is a relationship between Price and GPH

o Go to the Coefficient Statistics Summary and find Variable – then GPH

o GPH must be equal to or greater than the chosen Confidence Level

o In this case since GPH is 100%and the Confidence Level is 90% ,we know there’s a relationship between the two variables that is acceptable to usand it passes Test #2.

Test # 2 T-Stat (Significance of the Relationship)


Coefficient Statistics Summary


ANOVA Table (Analysis of Variance)

Analysis of Variance (ANOVA) TableDue To: DF Mean Squares (SS/DF)Regression 1 74944566 (SSR) 74944566 (MSR)Residual (Error) 5 1404141 (SSE) 280828 (MSE)Total 6 76348707 (SST)

Sum of Squares (SS)

Total Variation = Explained Variation + Unexplained Variation

(sum of squares total) (sum of squares regression) (sum of squares error)(SST) (SSR) (SSE)


Perfect world you don’t need a range –contractor will accept what you offer so come up with the Objective only.

Real world you would want a range because it addresses the RISK involved, thus a Min, Objective and Max

The Regression Tool provides the range based on how much risk you are willing to accept.

The Fair and Reasonable Range


Prediction Interval and How Computed


Allows us to FOCUS on what truly drives cost or price

Gives us rationale to defend our position

Makes writing memorandums easier.

It’s how many things in the world work

Bottom line – Simply Think in Terms of X Drives Y


Questions


FAR 15.406-3(a) and DFARS PGI 215.406-3(a)

https://www.dau.mil/tools/t/Basic-Stats-Tool


References

https://www.dau.mil/tools/t/Basic-Stats-Tool


X Drives Y - DAU

Documents

Transcript of X Drives Y - DAU