What you should know about software measurement platforms
description
Transcript of What you should know about software measurement platforms
IT Leader’s Guide to Enterprise Software Quality Solutions
What you need to know about the engine that will power your software quality initiative
CAST Copyright © 2014 - 2016 1
Contents
I. Taking the First Step
II. Software Measurement Essentials
III. Do-It-Yourself Solutions
IV. Enterprise Solutions
V. Total Cost of Ownership Analysis
CAST Copyright © 2014 - 2016 2
Taking the First Step
Congratulations! Your organization,
from developers up to executives, has
officially recognized that software quality
measurement is a key business value
driver. Not only is this a demonstration
of your company’s maturity, it is a
strategic inflection point that, if executed
properly, will propel you toward
sustainable returns.
Companies who have thoughtfully
deployed software quality measurement
platforms have experienced:
Improved collaboration between IT
and business
Reduced time-to-market
Expanded revenue generation
opportunities
Increased user and customer
satisfaction
Now, you just have to make sure that
your new strategic focus on software
quality has the platform to help you
accurately measure the most relevant
metrics and provide the right visibility
to the right people.
Sounds easy, right? Well, it depends
on what you use to power your
initiative.
3
Software Quality Measurement Essentials • Essential Capabilities
• Solution Breadth and Depth
CAST Copyright © 2014 - 2016 4
Software Quality Measurement Essentials
There are many offers today in the
code analysis market space. How do
you choose the right one?
Here’s a quick look at the most
essential capabilities of a software
quality measurement solution.
Solutions that fail to offer all of these
capabilities will not deliver the value
you are seeking.
A comprehensive software quality measurement platform must address all of your goals. A “good-enough” approach may give you “not-enough” results.
Measurement precision
Wide technology scope
Accurate analytical methods
Management visibility
Enterprise scalability
Measurements must be consistent over
time, and trends must be meaningful
Ability to analyze the technologies within
your most critical application
Accurate analysis must examine the
entire application system and in between
layers, beyond a single code unit
Easily scalable to all critical applications
and accessible by 100 or more staff
Have intuitive analytics for management
to quickly track performance & hotspots
Software Quality Essentials Description Consequences of “Good-Enough”
Approach
Ambiguous measurements cannot help
you make the right business decisions.
Exposes critical applications to high
risk, or incur costs for custom solutions
Inaccurate and irrelevant
measurements will force wrong and
costly decisions
Inefficiently run several instances,
reports are manually aggregated
Quality program will lose momentum
without management involved
Follow industry standards for quality
measurements and best practices (e.g.
OMG, CISQ, SEI)
Rooted in industry standards Program will be challenged by all
involved and quickly lose credibility
CAST Copyright © 2014 - 2016 5
Breadth and Depth of the Solution A truly meaningful and valuable software quality measurement platform must have breadth and depth.
Wide technology coverage that tidily addresses your most critical applications
Capability to analyze all layers of your critical applications; from code, to component, to system
Breadth
Depth
import java.applet.*;
import java.awt.*;
public class SecondApplet extends Applet {
static final String message = "Hello
World";
private Font font;
// One-time initialization for the
applet
public void init() {
font = new Font("Helvetica",
Font.BOLD, 48);
}
// Draw the applet whenever necessary.
Do some fancy graphics.
public void paint(Graphics g) {
// The pink oval
g.setColor(Color.pink);
g.fillOval(10, 10, 330, 100);
// The red outline. java doesn't
support wide lines, so we
Unit Level • Code quality at the developer
level
Component Level • Entire technology layer
• Inter-application invocation
• Single technology
System Level • Entire application
• Interaction between components
• Architectural cohesion
6
Do-It-Yourself Solutions • The Lure of “Simplicity”
• Tangible Costs of DIY
• Hidden Costs of DIY
CAST Copyright © 2014 - 2016 7
Beware of the Lure of “Simplicity”
Developing your own aggregation platform using statistics from single-technology code analyzers may appear to be a simple and frugal option to power your software quality program. But, they do not meet many of software quality measurement essential capabilities, and can cost you more long term.
Many technical organizations are drawn to
single-technology analyzers when they
begin to formulate their strategy around
software quality measurement. While its
low up front costs and seemingly endless
customization potential make these
analyzers an intriguing way to approach
software quality, there are a myriad of
downsides.
A Component of the Whole
To start, code analyzers are only a
component of a complete software quality
measurement platform. While licensing
the analyzers may be cheap, many
companies have quickly learned that they
have to either invest development
resources in creating a complex
aggregation platform or purchase an
commercially-available aggregator in
order to effectively track and report on
business-relevant metrics.
“Close-Enough” Measurements
Aggregating results from individual
analyzers may lead you down the wrong
path. While data form individual
analyzers can be aggregated, it does not
paint the whole picture. Issues at the
code level, do not have the same impact
as issues at the system-level, where
interactions between components can
cause catastrophic system failure. Only
looking at measurements at the code level
is a in accurate approach that may prompt
you to spend resources fixing a problem
that has very little business value.
Unfortunately, in this case, the whole is
not greater than the sum of its parts.
You Own the Platform - Completely
Analyzer components have to be updated
frequently when new versions are
released. New releases can destabilize
the platform, or worse – completely
change the results of the measurements
you have been taking.
You see, independent components do not
have to follow a coherent roadmap.
Product development is driven by a single
technology or language, and not based on
your overall enterprise needs.
CAST Copyright © 2014 - 2016 8
The Tangible Costs of DIY
$1.2M
INITIAL DEVELOPMENT 5 Technical Staff
$240K
PROJECT MANGEMENT 1 Project Manager
$300K
SOFTWARE QUALITY 1 Software Quality Consultant
BUILD
$340K / year
PLATFORM R&D 2 Technical Staff + Hardware
$120K / year
TRAINING & SUPPORT 1 Technical Staff
$180K / year
STRATEGY & MANGEMENT 1 Application Owner
$75K / year
SOFTWARE QUALITY 0.5 Software Quality Consultant
MAINTAIN
So, what are the costs of building and operating your own code analyzer aggregator platform? We did the math for you. These estimates are based on
resources located in the U.S.
We also assume that the
platform was built with a “good-
enough” approach. A platform
that satisfies the “software
quality measurement essentials
checklist” (p. 4) will likely cost
exponentially more.
$715K / year $1.74m over 2 years
CAST Copyright © 2014 - 2016
1. Project Scope Creep – Currently
available single-technology code
analyzers primarily address Java. If
your applications are in C, COBOL,
.Net, or other commonly used
languages, you may need to purchase
or extend additional custom
analyzers. This means a significant
expansion in the scope of the project.
2. Scarce Expertise – Doing it yourself
doesn’t automatically mean
inexpensive. Many companies hire
expensive code quality consultants in
order to complete the platform.
3. Inability to Scale – Many
aggregators work very well at an
individual developer or small team
scale. However, an organizational-
wide adoption of software analysis
requires sharing of information, and
easily accessible visibility.
4. Opportunity Cost – We estimate that
it takes five resources over two years
to build an adequate software quality
analysis platform (10 man-years).
That means these resources have to
deprioritize tasks that may have better
returns.
5. Waiting for Return – Typically the
value of software quality analysis and
the process that implement to address
it begin to materialize six to 12
months after initial implementation.
Building your own means that the
return is much further away.
6. Licensing Legality – Although the
initial license had no cost, many
companies face legal action due to
mismanagement of code analyzer
components—a serious drain on
resources, money, and talent.
7. Component Updates – New releases
of code analyzers must be
researched, before upgrading. You
may face several new releases within
a very short time frame, misaligned
with your release schedule.
8. Version Control – Some components
in your application may be written
using the same technology, (e.g.
JDK), but in different versions (e.g.
JDK 5 and JDK 6). When code
analyzers are updated they may
change language version support,
requiring you to have more than one
instance of the analyzer.
9
The Hidden Costs of DIY
Building and supporting a software quality measurement platform using single-technology analyzers is not a small task. The length and hidden costs of the projects are alarming.
10
Enterprise Software Quality Measurement Solutions • Advantages of an Enterprise Solution
• Managed Services – A Prudent Approach
CAST Copyright © 2014 - 2016 11
Advantages of an Enterprise Solution
Investing in a balanced and refined software quality measurement platform produced by a company with sound expertise and business understanding is the wiser option.
Enterprise software quality
measurement solution
Software Quality Expertise
A provider who has dedicated all of its time
and energy in software quality solutions will
likely deliver a more refined and valuable
product, than a solution that is built in-house
with individual code analyzers. By buying
an enterprise solution you can accelerate
toward returns through their specialization in
the space.
Customer Driven Road map
You have a bigger voice in your provider’s
roadmap decisions than with single-
technology components where thousands of
voices are competing to be heard. With a
larger share-of-voice you have more control
overall how future versions of a product fit
your needs.
Dedicated Support
When you do business with a solution
provider, you are the boss. Not only do you
have access to expert user support, you can
also enlist help for implementation,
adoption, and operation of the platform from
people who really know it well.
Lower Cost of Ownership
With no need for development resources
and simple update packages, enterprise
solutions will cost you less to own and
operate in the longer term. This also allows
your company to focus on its core
competency, selling the products and
services you know best.
CAST Copyright © 2014 - 2016 12
Managed Services – A Prudent Approach
Risk has become a centerpiece in enterprise software with the growing need of companies’ to innovate through new solutions.
Managed services allow companies to quickly implement and try new software at low risk, and, for many, has become the only way to buy new software.
Implementing and operating an enterprise
software quality measurement platform by
using the provider’s managed services
can have many benefits.
1. Shift Risk to Provider
Let your provider take on the risks of their
own solution. Traditionally, to onboard a
new software solution, you may need to
dedicate project management resources,
staff to operate and support the new
application, and invest in some additional
hardware.
With managed services, your provider will
provide the man power and hardware to
deploy the software. In the case of
software quality measurement platforms,
this means that they will:
Stand up physical and logical
infrastructure
Define the code gathering and
analysis process
Design the consumption model
Determine channels of
communication
It also means that you do not have to hire
more staff, or worry about what will
happen to them if the project is aborted.
2. Greater Budget Control
Buying software in a managed services
model means that a large portion of the
funds can be sourced from the operational
expenses bucket instead of capital
expenses. This enables companies to
make faster decisions to start or end a
project.
3. Accelerate to Time to Value
Because the providers are experts in how
to deploy and operate their software, they
will enable to you get ramped up and start
measuring your software quality in a
meaningful way in much less time than if
you managed it on your own. This helps
you accelerate toward your payback
period and vastly increases your returns.
4. Capture Opportunity Cost
A managed services model allows you to
focus on your key value-adding activities.
Operational distractions such as bugs,
performance issues, and upgrades are all
handled seamlessly by the provider
without your involvement.
CAST Copyright © 2014 - 2016
About CAST “CAST’s holistic system approach, looking at the
architecture, transactions, control, and data flow across multiple technologies, may be very
beneficial, with numerous
engineering studies showing that bad software engineering practices in the ways components are interrelated and interact…account for only 10% of total defects, but can lead to 90% of production issues.”
CAST is a pioneer and world
leader in software analysis and
measurement, with unique
technology resulting from more
than $100 million in R&D
investment. CAST introduces
fact-based transparency into
application development and
sourcing to transform it into a
management discipline. More
than 250 companies across all
industry sectors and geographies
rely on CAST to prevent business
disruption while reducing hard IT
costs and software risk. CAST is
an integral part of software
delivery and maintenance at the
world's leading IT service
providers. Founded in 1990,
CAST is listed on NYSE-Euronext
(Euronext: CAS) and serves IT
intensive enterprises worldwide
with offices in North America,
Europe and India.
www.castsoftware.com