The metrics that matter using scalability metrics for project planning of a distributed workforce ...
33
The Metrics that Matter Project Planning using Scalability Metrics for Distributed Development Dilber Mann, Development Director Relic Entertainment Inc.
-
Upload
game-connection -
Category
Technology
-
view
225 -
download
1
description
Have you expanded your organization across multiple locations, or are you a client that utilizes external partners that provide outsourcing services? Both have their "cost savings" challenge where cost savings analysis is often a topic well scrutinized. However, in the grand scheme of your organization, is it a metric that really matters? See actual analytics on multiple game projects and why cost savings isn't as important a metric when making informed decisions about project planning for scalable and distributed development. It's all about the Metrics that Matter.
Transcript of The metrics that matter using scalability metrics for project planning of a distributed workforce ...
The Metrics that MatterProject Planning using Scalability Metrics for Distributed Development
Dilber Mann, Development DirectorRelic Entertainment Inc.
• Establishing reliable metrics to help make informed decisions about organizing a distributed & scalable workforce.
Overview
• Context from multiple perspectives– Publishing / Central Group– Onsite Partner/Vendor management.– Studio Development
Understanding Content Source
Discussion Points
• Distributed Trends
• Establishing your own system of metrics
• Incorporating scalability metrics
• Discussing Real-World examples.
• Converting your Metrics into KPI’s
Identifying the Audience
• Two distinct groups under Distributed Development
1. Consolidated Resource Structure• scalability through Outsourcing.• development outsources when needed
2. Distributed Resource Structure• scalability through distributed workforce.• development is always distributed.
Distributed Trends
• Deciding between Distributed or Consolidated Organization.
– Option (Distributed) is available when region has a wide range for cost of living.
– Choice (for both) is driven by cost savings and scalability needs.
ExternalPartner
Distributed Trends
• Relation between Consolidated & Distributed Teams
Development Studio
Location A
Location B
Location C
Location D
Outsource
Distributed Workforce
• Advantages– Manage burn rate at lower tier/cost
locations.
– Maintain consistent man-day rates.
Distributed Workforce
• Wide Range for cost of living in Asia/Europe region.
• Tier 0/1: modern and Large Cities• Tier 2: modern but developing cities• Tier 3: developing cities• Tier 4: under developed, but progressively
developing Cities.
Distributed Workforce
• How to plan the distribution?– by using reliable metrics to make
informed decisions.
Establishing Foundation Metrics
• Cost Savings Analysis– Man-Day Rate comparisons– Scope– Impact of Studio Support
Cost Savings Analysis: Man-Day RateVendor Rate Studio Rate
Avg Studio Man-Day Rate $198.00 $700.00Mgm Overhead $22.00 $70.00Man-Day Rate $220.00 $770.00
Vendor Costs Studio CostsScope (Man-Days) 2500 2500Rate * Scope $550,000.00 $1,925,000.00
Vendor Costs Studio RateScope (Man-Days) 2500 2500Rate * Scope $550,000.00 $1,925,000.00
Studio Support $250,000.00
TOTAL $800,000.00 $1,925,000.00
****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****
71% SAVINGS
VENDORRATE
STUDIO RATE
Cost Savings Analysis: ScopeVendor Rate Studio Rate
Avg Studio Man-Day Rate $198.00 $700.00Mgm Overhead $22.00 $70.00Man-Day Rate $220.00 $770.00
Vendor Costs Studio CostsScope (Man-Days) 2500 2500Rate * Scope $550,000.00 $1,925,000.00
Vendor Costs Studio RateScope (Man-Days) 2500 2500Rate * Scope $550,000.00 $1,925,000.00
Studio Support $250,000.00
TOTAL $800,000.00 $1,925,000.00
71% SAVINGS
VENDORRATE
STUDIO RATE
****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****
Cost Savings Analysis: Studio SupportVendor Rate Studio Rate
Avg Studio Man-Day Rate $198.00 $700.00Mgm Overhead $22.00 $70.00Man-Day Rate $220.00 $770.00
Vendor Costs Studio CostsScope (Man-Days) 2500 2500Rate * Scope $550,000.00 $1,925,000.00
Vendor Costs Studio RateScope (Man-Days) 2500 2500Rate * Scope $550,000.00 $1,925,000.00
Studio Support $250,000.00
TOTAL $800,000.00 $1,925,000.00
71% SAVINGS
VENDORRATE
STUDIO RATE
****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****
Vendor Rate Studio RateAvg Studio Man-Day Rate $198.00 $700.00Mgm Overhead $22.00 $70.00Man-Day Rate $220.00 $770.00
Vendor Costs Studio CostsScope (Man-Days) 2500 2500Rate * Scope $550,000.00 $1,925,000.00
Vendor Costs Studio RateScope (Man-Days) 2500 2500Rate * Scope $550,000.00 $1,925,000.00
Studio Support $250,000.00
TOTAL $800,000.00 $1,925,000.00
58% SAVINGS
VENDORRATE
STUDIO RATESTUDIO
SUPPORT
Cost Savings Analysis: Studio Support
****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****
Resource Ramp-Up / Down
5% 9% 14% 17% 17% 17% 14% 9% 9%
0%2%4%6%8%
10%12%14%16%18%
1 2 3 4 5 6 7 8 9
• Approx. 50% of Manpower spread across 3 months during peak time.
****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****
Resource Compression
12% 23% 28% 23% 12% 12%
-5%
0%
5%
10%
15%
20%
25%
30%
1 2 3 4 5 6 7 8 9
• Approx. 50% of Manpower spread
across 2 months during peak time.
****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****
Resource Compression
12% 41% 41% 25%
-20%
-10%
0%
10%
20%
30%
40%
50%
1 2 3 4 5 6 7 8 9
• Approx. 50% of Manpower
spread across 1.25 months during peak time.
• Disclaimer: The sum of Month to Month % of work may exceed 100% total due to contingency buffer used for planning.
****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****
Resource Productivity
• 40 peak resources
• 50% efficient at start
• 80% efficient at peakResource Efficiency & Savings
Pilot Month 1 Month 2 Month 3 Month 4 Month 5 Month 6 Month 7
Group 1 50% 70% 80% 80% 80% 80% 80% 80%Group 2 50% 70% 80% 80% 80% 80% Group 3 50% 70% 80% 80% Group 4 50% 70% 80%
****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****
Importance of Cost Savings
• 35% - 50% Good Range for Cost Savings.• Is Cost Savings the best decision making metric?• Is it a metric that speaks to scalability?
Building a System of Scalability Metrics
• Man Effort Established– Internal Man Effort established externally.
• Added Value– Man Effort added to project.
• Budget Savings– Look at Internal cost of doing the same scope.
Scalability Metrics
****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****
Case Study
• Improving Performance over time.– Step 1: Conventional OS– Step 2: Distributed Development– Step 3: Dissolving the centralized role
Conventional OS
Project A(Conventional)
19% 8% 13%
24% 7% 19%
40% 20% 25%
Internal
External
Internal
External
Internal
External
Internal
External
Internal
External
Internal
External
Internal
External
Internal
External
Internal
External
Distributed DevelopmentProject A
(Conventional)
19% 8% 13%
Project B(Distributed Dev)35% less scope than
Project A
24% 7% 19%
40% 20% 25%
Internal
External
Internal
External
Internal
External
Internal
External
Internal
External
Internal
External
Internal
External
Internal
External
Internal
External
Dissolving the OS role
19% 8% 13%
24% 7% 19%
Project C(Optimum)
40% 20% 25%
Internal
External
Internal
External
Internal
External
Internal
External
Internal
External
Internal
External
Internal
External
Internal
External
Internal
External
Converting Metrics into KPI’s
• Why?– Keep track of your planned goals.
• How?• Continue to update your data.
– (planned & actuals).
• Convert Metrics into KPI’s.
Converting Metrics into KPI’s
• KPI’s (Key Performance Indicators)
– Metrics scaled to a 0 – 100% range.• +100% Out performing • 75% - 100% Normal Performance • >50% or <75% Under performing • <=50% Need Escalation
– Easy for multiple stakeholders to interpret.
Converting Metrics into KPI’s
• 20% Added Value Metric (Max Threshold)– 20% x 5 (Multiplier) = 100% KPI
• 25% Budget Savings Metric (Max Threshold)– 25% x 4 (Multiplier) = 100% KPI
• 40% Man-Effort Metric (Max Threshold)– 40% x 2.5 (Multiplier) = 100% KPI
Converting Metrics into KPI’s
ManEffort Established Added Value Metric Budget Savings Metric
Project A(Conventional) 47% 38% 54%
Project B(Distributed Dev)35% less scope than
Project A
59% 33% 76%
Project C(Optimum) 100% 100% 100%
• To establish reliable metrics that can help us make informed decisions about how we plan our distributed workforce.
Our Purpose
• Distributed Trends– Manage the impact of idle/burn rate by using Lower Tier/Cost locations.– Maintain consistent man-day rates by creating balanced distributed teams.– Identify reliable metrics to plan the distribution of resources.
• Cost Savings Analysis– Include Scalable Management & Studio Support.– Know that Schedule Compression has an impact.
• Decision Making KPI’s– Know what your KPI’s are telling you.– Agree as an organization on the KPI goals.– Track KPI’s throughout production.
Summary
