Hollow Ware Toilet Speciality Ware Novelties Other Ware P39141 M
Exec Guide to Slashing s Ware Dev Costs
description
Transcript of Exec Guide to Slashing s Ware Dev Costs
-
The Executive Guide to Slashing Software Development Costs
How to reduce software development costs by 80%.
Utopia? No just the modernisation of
out-dated software development methods.
An eBook by
Ken Galbraith
A no-holds-barred look at current IT practices and the impact they have on
business
By k.g
-
Copyright 2013 by K M Galbraith. All rights reserved.
This eBook may be downloaded and circulated in its published form.
No extracts may be used without the authors permission.
Disclaimer:
This book is based on the personal experiences of the author.
Outcomes and productivity gains may vary in other organisations.
Experiences relating to Circatec clients or other parties have been changed to the extent
required to avoid identifying the other party.
All research material is from publically available sources.
Trademarks:
CA, CA Plex and Plex are trademarks of CA Technologies.
XSOL InOrder is a trademark of XSOL Limited.
Additional copies:
Copies of this eBook can be obtained from Circatecs web site:
www.circatec.com.au
-
Contents
Introduction 1
Software excellence 2
The current state 3
The traditional software development life cycle (SDLC) 3
The Agile software development life cycle (SDLC) 4
Over reliance on methodology 5
Cant I just send it offshore? 7
The fundamental flaw 7
There is a better way 8
Automated software development tools 8
Application development 8
Rules Engines 10
Benefits of automated software development 11
Managing automated development projects 13
Requirements gathering 13
Architecture and design 14
User interface and patterns 15
Development teams 16
Planning automated software development 17
The benefits are enormous 18
It doesnt stop here 19
The larger the development the greater the savings 19
Compelling case for change 20
How corporate IT resists change 21
Common practices used to maintain control 22
I can do it just as quickly by hand! 24
The CEOs toolkit 24
Bringing about change 25
Structure 25
Existing systems 26
What about work-in-progress? 26
Resistance to change 26
Be prepared to start again 27
What will it take? 28
Change will occur when change is demanded. 28
About the author 29
-
Page 1
Introduction Fundamentally the way software is developed and delivered hasnt changed in 40 years.
Over the last ten to fifteen years tools and methodologies to dramatically streamline software
development have been maturing in function and sophistication.
And yet the software industry has been able to resist change. It is reasonable to say that no
other industry has been able to resist change for so long and get away with it.
The impact on an organisations ability to deliver, adapt and take advantage of the rapidly
changing commercial and legislative environment is severe. In many instances IT dictates
corporate agility and speed to market.
While boards and CEOs require system agility and quick response to changing requirements,
their IT department often remains inflexible and resistant to change.
IT departments are empowered and trusted by their organisation to provide software and
technology solutions for the benefit of the organisation. Far too often they fail this trust by
blocking new ways and technologies, often using dismissive statements that, if challenged,
cannot be justified mostly however, they go unchallenged.
Too often Corporate IT is making key and fundamental decisions that have a significant
impact on the business and there is no mechanism in place to hold them to account. The
business managers most affected are seldom engaged in these decisions and are, therefore,
unaware of the alternative solutions available to them.
In 2005 Gartner Group (a leading technology trend forecaster) forecast that 40% of IT job roles
as they existed then would be lost to automation. They went on to forecast that more
American IT jobs would be lost to automation than to outsourcing. The most significant
forecast was that between 2005 and 2010 the productivity of IT workers would increase by a
factor of 10.
Today, some 8 years later, the uptake of automation tools and agile methodologies has
effectively been ignored (the generous view) or blocked (in my experience) by the broader IT
community.
The impact is much more than just high cost and long delivery cycles for software
development, there is a very significant impact on operating costs when systems are out of
step with organisational needs.
This book looks at the current state of software development and then explores commercially
available and proven alternatives before discussing how to bring about significant change.
Let me clear here. The change I am talking about is a fundamental shift in the way we design,
develop and deliver software applications.
If we keep doing the same things we will continue to get the same results.
If we want to change the outcome we must change how things are done.
-
Page 2
Software excellence We need to understand what constitutes excellent enterprise software in order to objectively
look at the tools and methodologies used to develop an enterprise application.
User experience The software must be easy to use and enhance the users undertaking
of their everyday job functions.
This means the software must be intuitive to use and reflective of the
enterprise business processes.
In high work load environments the software is so well aligned to the
process that user interaction with the application is highly efficient.
Value add The software must add value to the organisation by providing a more
efficient workplace, eliminating error and waste, enhancing the
customer experience, reducing corporate or financial risk, and other
tangible organisational benefits.
Flexible The ability to adapt to changing organisational demands is critical to
the long term value of the software.
Reliable The software must be available and run consistently day after day.
Other than infrastructure problems the software availability should be
100%. Transactional processing must be so consistent that once
audited subsequent transactions are accepted as correct.
Secure Security in this context is related to the validation of transactions to
ensure data and transactional accuracy. Protections around the
maintenance of data to prevent malicious transactions or accidental
data errors. Incoming electronic data files must pass through
validation filters before being posted to the database.
Cost effective The cost of development must pass a business test. Implementation of
the software must be efficient and the ongoing cost of maintenance,
including changes and enhancements, must be relative to the value
the software is delivering to the organisation.
Scalable The ability of the organisation to grow and/or expand its products and
services must not be hampered by limitations in the softwares ability
to process the increased volumes of transactions.
The tools, standards and methodologies adopted by corporate IT and their service providers
must be totally aligned to the delivery software excellence as judged by the organisation
they serve.
The sole measure of successful software is the value it brings to the organisation.
-
Page 3
The current state The traditional software development life cycle (SDLC)
The waterfall approach to software development has been around since the early 1970s and is
still widely used today. It is a sequential design and development methodology as shown in
this flow chart.
The waterfall model assumes each stage is complete before proceeding to the next.
RequirementsAnalysis &
specificationDesign
Architecture & components
ImplementationConstruct
components
VerificationTest & install
MaintenanceModifications &
defects
Because Waterfall is sequential it is impossible to go back. The assumption is that the
requirements gathering is an accurate and complete reflection of the requirement (which it
seldom is) and the design is an accurate reflection of the requirement.
The advantages of Waterfall are:
It enforces discipline, every stage has a start and end and progress can be monitored
through the use of milestones.
There is considerable documentation which minimises the impact of staff turnover
on the project.
The client (external or internal business unit) knows what to expect. The detailed
specifications describe what the software will do on completion.
The disadvantages of Waterfall are:
Change of requirements is difficult to manage due to the large amount of
documentation.
Once a stage has been completed it is difficult to go back.
If a specification error or change is required the project must go back to the
beginning.
Bugs and design flaws arent identified until after the entire application has been
written.
-
Page 4
The disconnect
As waterfall moves from one phase to another it engages different disciplines and skills. Each
with their own language and terms of reference.
This creates a problem as the language moves from end user terminology to technical or
system terminology, compounded by the separation of development from the client.
What often happens is:
Clients domain expert talks to
business analyst
Business analyst develops
requirements specification
Systems analyst interprets
requirements into development specification
Development specification goes
to programmers to write the software
Business analyst talks to systems
analyst
Client doesnt fully accept the software and the loop starts
again!
Developed software released
to the client for acceptance
Returns the specification to the client for approval
The developed software is the Programmers interpretation of the System Analysts
interpretation of the Business Analysts interpretation of the Clients requirements
The client is now out of the loop!
Programmers are now completely isolated from the client
Highly technical document that would be difficult for the client to understand
Programmers interpret the
development spec into program code
The Agile software development life cycle (SDLC)
To overcome the shortcomings of Waterfall and its variants, Agile methodologies started to
emerge in the 1990s.
The objective of Agile is to replace the sequential methodology of Waterfall with an iterative
methodology using small cross functional teams rapidly delivering small, useful pieces of
software.
These deliveries happen at regular intervals enabling user testing and evaluation throughout
the development cycle. By developing small pieces of software, changes can easily be
accommodated.
As delivery is an iterative process client cooperation and feedback is encouraged. Agile
supports the use of prototyping or modelling where appropriate to ensure the delivered
software meets the clients requirements.
A key aspect of Agile is the minimisation of documentation. Requirement specifications are
just enough to communicate the requirement to the development team. This may be as
little as notes and flows on a whiteboard or a spreadsheet demonstrating financial
transactions and postings.
-
Page 5
While this may be disturbing to some, if we accept the only measure of successful software is
the benefit it brings on completion then we must also accept that all documentation used
during development is obsolete the moment the software is accepted and put into
production.
Each iteration
Iteration test & accept
Iteration release
Agile planningRequirements
Define
Develop
Work breakdownPriorities / dependencies
Iteration schedule
Progressive testing enables early acceptance
of the completed
Just enough documentation for the development team to
build the iteration
User engagement in assessing the iteration or
prototype prior to relaease Feedback
Agile will deliver significant productivity gains, however it will only reduce the time and cost of
software design and specification. This may equate to savings in the 10-20% range.
You can be too Agile
The problem with Agile is that it can easily be taken too literally. In a development where a
single team builds a function or even the entire application, Agile can deliver significant
productivity gains.
In a larger development with multiple teams working to produce a single application, an
overriding architectural design and development standard must be in place. Without this the
teams will spend unnecessary time rebuilding prototypes, or reworking early developments,
in order to provide a single cohesive application.
Ensuring the design work has taken place will ensure the development delivers the benefits
available in Agile.
Over reliance on methodology
We cannot discuss software development without a critical discussion on project
management methodologies.
The vast majority of software projects follow a highly structured project management
methodology based on the waterfall approach to software development.
It is worth noting that many highly structured methodologies are designed for, or evolved
from, construction, engineering, mining, etc. where it is entirely appropriate to have rigid
controls around documentation, certification and change management approval. Whether all
these controls are appropriate in software development is questionable.
-
Page 6
Question: Do you need the same approval process to add a field to a screen and underlying
data table as you do to authorise an engineering change on a bridge? The exponents of these
methodologies say yes, whereas reality says no.
I have some specific issues with highly structured project management methodologies in
software development.
They perpetrate the disconnect between the business unit and the development
team that is inherent in Waterfall.
The methodology becomes the purpose, not the outcome the timely delivery of
usable and cost effective software.
They add a huge amount of unnecessary overhead, cost and delay to a software
development project.
Any prescriptive methodology enables low competency operatives to hide behind
the methodology.
Make no mistake no methodology will guarantee a successful outcome. However the more
structured and bureaucratic it is, the more certain you can be that any failure will be
expensive, well documented and a long time in the making.
Flexibility is the key
Very few methodologies are absolutely prescriptive. The vast majority, including Prince2 and
APM BoK are frameworks to be tailored to a specific project.
In the right hands, they provide a valuable and practical framework for managing a project.
I have worked with some extremely capable project managers. They all have common traits,
in that they are people managers and they are focussed on the destination not the journey.
Plus they have domain knowledge either of the industry they are working in or the
technology they are implementing (which implies a degree of industry knowledge as well).
Problems arise with highly structured methodologies when the project manager is a
practitioner of the methodology. Instead of guiding, mentoring and smoothing troubled
waters to get an outcome, they focus on the methodology demanding strict adherence,
regardless of practicality or relevance.
This can also occur when a service provider has been retained to provide project management
resources. They need strict adherence to the methodology because:
They sold the complete project management package to their client.
It enables them to put in methodology practitioners instead of domain experts.
They can forecast their fee revenue.
If you are going to outsource project management, ensure you are getting a project manager
not a methodology practitioner.
-
Page 7
Cant I just send it offshore?
I have yet to meet anyone who has had a good experience with the offshore development of
an enterprise application.
Generally the problems have nothing to do with competence. The problems usually come
down to two issues.
Communication of requirements.
Requirements must be very precise because of the geographical and time zone separation
between the business analysts and the developers.
I have already demonstrated the disconnect and lack of precision inherent in the
conversion of business requirements into programming specifications and then into
delivered software. This disconnect is exacerbated with offshore development.
Lack of domain knowledge.
Inherent in most specifications is an assumption that the programmer understands the
domain. For example, in specifying financial transactions, a system analyst may assume
the programmer knows the transaction must be validated and doesnt even think to
specify it.
The absence of domain knowledge anywhere in the development cycle will lead to
misunderstanding, errors and exclusions.
In a traditional waterfall development there are inevitably disagreements during testing when
the analyst rejects a function and the programmer argues they programmed to the
specification. The frequency and severity of these disagreements and the associated rework
will increase significantly in offshore development.
Unless you are very lucky, any savings in development charge rates will be consumed in travel,
communication, rework and dispute resolution.
The fundamental flaw
Regardless of methodology the fundamental flaw is the lack of automation in the
development of application software.
A typical software development department consists of a team of programmers writing code
just as they did 40 years ago. Sure, the programming languages have changed but its like
giving a carpenter a nail gun. Hes a little quicker but hes still driving in nails.
While Agile may deliver savings in the 10-20% range, it is a fraction of the savings to be made
by automating software development.
-
Page 8
There is a better way We see how the dependence on the manual development of software, regardless of
development methodology, fails to meet the cost restraints and agility demanded by modern
business.
There is a much better way to develop enterprise software applications that will deliver:
Cost reductions of 70 80%.
Extremely flexible software solutions.
A high level of client engagement in the development cycle.
A significant reduction in the elapsed time to specify, develop and implement a
solution.
Bug free software.
Massive reductions in the size of your IT department.
Automated software development tools will deliver this and much more.
Automated software development tools
Typical characteristics of automated software development tools are:
A structured modelling framework that enforces standards and consistency.
Definable patterns that establish consistent user interfaces and behaviours.
Inherent functions to speed up model building.
An extremely high level of component reuse.
The automatic generation of program code.
There are two groups of tools that between them will virtually eliminate the need for any
manual program development:
Application development tools.
Rules and decision engines.
The strength of these tools is the ability of the developer to work directly with a domain
expert, creating a model of the requirement and generating the software code without the
use of manual programming.
The structure, models and patterns that characterise automated software development tools
enforces the standards and consistency essential to deliver software excellence.
Application development
Circatecs requirement was for a toolset that enabled us to build commercial grade enterprise
applications. This means large server based applications delivered via an internal network,
remote desktop or via a web browser without the need for a secondary web based layer.
As a software developer we were also reluctant to limit ourselves to a single platform and
database.
-
Page 9
There are a number of excellent automated application builders on the market but CA Plex
was the only automation tool we found that met our broad criteria.
Plex originated in the 1990s as IBMs Synon product. Computer Associates acquired it some
years ago and it has continued to evolve into what is arguably one of the best tools currently
available. Plex applications have been proven in very large enterprise systems so we know
they are scalable.
CA Plex is multi-platform. It can generate C#, C++, Java or RPG III & IV code to run on Sequel
Server, Oracle or DBII databases.
Plex provides enormous productivity gains, enabling us to build applications in 15 20% of the
time it would take to manually program the application.
Some of the key aspects of Plex are:
Patterns.
There are two classes of patterns in Plex, the user interface (UI) and process patterns.
The UI patterns provide a consistent user interface across the entire application. Typically
we would have three or four patterns for use in specific functions, for example:
High volume data entry may be a single screen so all the fields pertaining to the
function are displayed in one large panel.
The enquiry screen however may reflect the needs of a call centre and display core
information with links to every other related area of the system.
Functional patterns will define a regularly occurring pattern. For example, in a financial
system the standard process may be to enter and save a transaction, have another staff
member confirm the transaction and, on confirmation update the underlying ledgers. By
building this as a pattern we enforce the process and where the confirmation cycle isnt
required it is turned off.
The point of patterns is that they enforce consistency across the entire system but they
also save a huge amount of development time. When a developer is building a function
the first thing they do is select the pattern and the user interface is immediately inherited.
Data base generation.
The database is generated from the model. Developers do not have to build a database
before they build the application.
This is why, in a Plex world, we can generate the application to run on SQL, Oracle or
DBII. We simply tell Plex what database to compile when we run the model.
Models.
To call or initiate another program its as simple as naming the application in a link in the
model and Plex generates the physical links.
Most business environments have situations where exceptions trigger a different process.
In the Plex model the developer simply defines the If / Then statement.
The whole point of these examples is to illustrate how quickly complex or repetitive
programming tasks can be defined in a model without any manual programming.
-
Page 10
Plex generates the program code in the designated language and compiles the designated
database.
Because the code is machine generated there are no bugs, so testing is limited to ensuring
that the model is a true reflection of the requirement. If there are design errors or functional
gaps the changes are made in the model and a fresh set of code is generated. This ensures the
software code is always clean and free of the redundant code that builds up in manually
programmed software.
Rules Engines
A rules engine provides a graphical model in which to define the business rules. Having
defined the rules flow, the formula for each branch or decision node are defined and the
model tested.
All good rules engines will have inbuilt test capabilities and will be self documenting.
The rules engine will automatically generate the model into program code to be deployed into
the host environment.
In an enterprise environment, rules engines are generally deployed to:
Add functionality that is difficult or expensive to develop in the host application. Rules
engines can play a valuable role in extending the life of legacy software by adding
functionality that would be prohibitive to develop on an out-dated platform.
Encapsulate legacy products. Wealth management, superannuation and insurance often
have closed products they are legally bound to maintain until there are no more clients in
the product. A rules engine is ideal to define all the product rules and calculations into a
platform independent rules engine that can be mapped to a host system. For example a
pension scheme may take member data from the host, apply the pension scheme rules
and return the pension payment to the host to process the cheque or EFT. If the fund
migrates to a new administration system, the rules engine remains unchanged and is
mapped to the new system.
Data validation and rectification is a perfect domain for a rules engine. Far too often
organisations rely on scripting languages or spreadsheets both of which are time
consuming to develop, difficult to test and impossible to guarantee their ongoing
integrity. The version control of a good rules engine, combined with the fact that the
generated code is maintained by changing and regenerating the model, provides
certainty over the rules and calculations being applied to the data.
As with application builders such as Plex, a rules engine does not require any programming to
create even very complex applications. Because they generally access and/or update a host
system, there is a data mapping exercise in the deployment of a rules engine. This is a
technical role, which in most cases it can be carried out by one of your development team or a
suitably qualified analyst.
-
Page 11
This is a conceptual model of a Defined Benefit Superannuation Scheme illustrating how
relatively easy it is to graphically define a very complex set of rules.
Benefits of automated software development
The benefits of automated software development are significant.
Model based development tools enforce consistency and structure leading to a quality
outcome.
Testing is substantially reduced in a model/pattern environment.
If the model or pattern works once, it works consistently everywhere its applied.
If the model isnt changed, there is no need to retest the whole application if a
change is made elsewhere.
Productivity gains are enormous. The more complex the development the greater the
productivity gain.
Implementation is simplified as a result of:
The consistent behaviour of model based software.
The user engagement in the requirements gathering.
The solution is based on business processes.
The software is bug free.
The model generates a new set of program code every time its deployed.
The code is machine generated so there are no program errors.
Any software faults will be in the logic of the model.
The model is corrected and a fresh set of code is generated.
-
Page 12
On-going cost of ownership is reduced substantially:
If the business requirement changes simply change the model and redeploy.
Software compiled by the models is well structured, consistent and stable.
Every time a model is changed and deployed the software code is totally recompiled
so there is no build up of old code from previous changes.
-
Page 13
Managing automated development projects Requirements gathering
We are translating business requirements directly into modelling tools, so we arent restrained
by software development considerations when gathering functional requirements.
We are, therefore, free to define business requirements from a process improvement
perspective. This approach has many benefits:
Its an opportunity to completely review existing practices, identifying opportunities
for process improvement.
The revised business processes become the specification for the new application.
Users have been engaged in defining business processes not the system definition.
When the developed software is delivered, it is seen as facilitating the process
change users have already identified not the cause of change.
This last point is very important as user resistance can be a major barrier to a successful
implementation.
Business process modelling
There are many process modelling tools on the market. The problem with many of them is
that they are far too technical for the business user to relate to. A lot of process modelling
standards have emerged from IT and are related to software design, rather than business
process improvement.
I favour tools and modelling techniques that reflect the user environment and that the user
can relate to. There is nothing more powerful than having the process model up on a screen
and working with the users to highlight problems with current practices and opportunities for
improvement.
At Circatec we use two tools Microsoft Visio and XSOL InOrder from XSOL in Auckland, New
Zealand.
XSOL enables us to model business processes in a graphical format, within a structured model
that reflects the organisation not the software or database design. This is important if we
return to primary measure of successful software the value it brings to the organisation.
A big advantage of the XSOL structure is that a process must be completed in the model, it
cant be left as a to be resolved later. I have used this feature many times in workshops,
demonstrating to participants that a process as weve mapped it doesnt work, forcing
continued discussion and resolution of process conflicts.
XSOL also captures notes in user defined categories. As we work through the process
modelling we capture process improvement notes for the implementation, developer notes
for development team and legislative or compliance notes to ensure the developed
applications are technically and legislatively correct.
So when we start building the Plex models or rules engines we know they are based on clearly
defined and agreed business processes and requirements.
-
Page 14
Architecture and design
As with any development, design is critical, however it is very different to the detailed
architectural design required for a hand coded system.
In model based development we know the patterns in the automated software development
tool will provide a consistent architecture and will generate the database from the model.
Therefore we only need to determine the modules or functional groups and their
relationships.
With the model having facilities to call other programs, we only need to consider what
functions are core and what functions will be external modules or rules engines. The following
schematic illustrated the conceptual design of a superannuation administration system. Core
functions in this instance are considered to be 90% stable with incremental enhancement over
the life of the application. External modules are more volatile. They will be fund specific
functions or subject to legislative change.
Database
Core Functionality
Fund specific functions
Fund specific variables
Legislatively sensitive functions
Ele
ctro
nic
File
Man
ager
Document management
Workflow processes & alerts
User Interface
Browser
Workflow
Data validation and rules layer
In this instance there are supporting designs for each element of this high level model. The
graphical designs guide the development of the functional models and subsequently the
entire application. The designs have been developed in consultation with domain experts and
developers. It can be printed on a large sheet of paper and put up on the wall.
This is not just an architectural guide, it is used to structure the work breakdown and work
allocations to support the modular approach.
-
Page 15
User interface and patterns
What is this application to be used for, therefore what are the key requirements for the user
interface?
The user interface determines the user experience and therefore user acceptance of the
system and user efficiency.
I am past letting developers determine the UI design. So before any development begins, we
determine the UI requirements for each user group within the organisation. For example:
The manual entry of large volumes of data from incoming documents calls for a
single screen laid out to reflect the format of the document. It needs to flow from one
field to the next in the same flow as the document and it needs appropriate
validations to prevent data entry errors.
A call centre needs fast access to the callers details to verify the callers identity, then
seamless access to every part of the system needed to answer queries or process
instructions.
Once these requirements are established the UI patterns are designed. Commonality and
reuse is the key to good pattern design, determining how many patterns are required and how
each will behave.
Here is an example of a UI pattern design. Totally graphical and created with a drawing tool in
a matter of hours. In this particular application five patterns were drawn up in a little over a
day.
This screen, its functions and logic will be reused wherever an enquiry screen is required. In
this example it will be used for all member, employer, planner and other external party
enquiries in a superannuation administration system.
-
Page 16
The buttons include enforced checking and verification of data entry and modifications, a link
to the document repository to retrieve documents specific to the subject of the enquiry and
tabs to every system function relating to the enquiry.
Once the pattern is built it is reused in every function requiring this style of pattern. The
developer does not have to develop every screen in the system, they take the pattern and
build the function model within the pattern, suppressing anything in the pattern not required
for a specific enquiry.
Development teams
Small is wonderful when it comes to automated software development teams.
In a Plex environment one business analyst (domain expert) to two Plex developers is
regarded as a good balance. In complex functions it can be one to one for a period.
Whiteboards, pin boards and electronic boards are the key to effective agile development. The
analyst communicates the requirement to the team and other stakeholders in the most
effective manner. This can be as simple as notes and flows captured on the electronic board to
a spreadsheet replicating financial transactions and postings. We seldom replicate source
documents theres no value in that. For example the Australian Taxation Office has a
document for every form of federal taxation. The document contains the tax rules, working
examples and software developer notes. Whether its a Plex model or a rules engine this is all
the information the development team needs.
As the business analyst is part of each development team, questions are answered as they
arise. The BA also does iteration testing so design errors are fixed in the model as they are
found. In well functioning teams the interaction between the BA and the model developers is
so dynamic that potential omissions or misunderstandings are rectified before the first
prototype is generated.
Building a rules engine is even more dynamic. It is often a one to one ratio with the domain
expert talking directly to rules engine builder. The rules and calculations can be tested within
the model so by the time the model is finished it has been tested and validated, ready to
deploy.
A major advantage of these small, dynamic teams is the ease with which the client can be
engaged. A three way meeting of client, analyst and developer can add enormous value and
impetus to the development of a functional model.
Models can be refined and code regenerated as often as required to prove the application
Developer models the requirement
Direct &unambiguous communication between experts
Prototypes are developed and refined until the
model accurately reflects the requirement
The model is finalised and the
software is generated
automatically
Domain expert talks to the developer
Developer models the requirement
Deployed Application
The lines of communication are short and direct, eliminating the ambiguity and disconnects that are inherent in the traditional SDLC.
-
Page 17
Planning automated software development
By now the relative simplicity of the methodologies surrounding automated software
development is apparent. The same applies to the planning and management of the
development. By breaking the development into small components, development is
production, not a project.
Whats the difference?
A project allocates resources to task. It assumes a relatively elastic supply of resources, hence
the tendency to throw large numbers of programmers and contractors at a development, with
the resulting well documented failure rates.
Production, on the other hand, allocates tasks to resources. It assumes relatively fixed
production resources, placing the emphasis on effective use of resources through work
allocation planning and bottleneck management.
If we accept that domain knowledge is vital in the development of enterprise applications this
is the correct view of our software development environment.
As in any production environment there is selective outsourcing or sub-contracting to manage
demand peaks or skill shortages.
In an automated software development environment this approach is extremely effective
when:
i) Development tasks are broken down into the smallest practical blocks of work, ideally 1 5 day blocks.
ii) A block of work is specific to a skill, for example analysis, development, testing, etc.
iii) Work is allocated to each team based on both availability and domain knowledge. A production planning board shows to whom each task is allocated to and how long they have
to complete it. The visual planning has significant benefits:
Who owns the work at any point in the production plan.
What the forward workload is by person and job.
The impact of being late.
The interleaving of jobs for multiple projects or clients is also highly visible and gets away
from the concept of individual projects vying for shared resources.
The team leader or production manager knows exactly the state of every job at a glance and
can take action immediately to alleviate problems. So too does every team member.
The planning board time scale is days, so every day the whole team can see exactly where
every job is up to.
A longer term view is easily set up a maintained in Excel, printed on A3 and posted on the wall.
Now there is a highly visual, easily maintained view of the immediate and long term work load
and work allocations with an absolute minimum of management overhead.
Once again the emphasis is on visibility and simplicity without sacrificing accuracy in any way what-so-ever.
-
Page 18
The benefits are enormous We benchmarked a recent Circatec development of a superannuation and investment fund
administration system using function point analysis, an industry standard measure of
software functionality.
The benchmark we used to cost a traditional SDLC was the costing model the University of
Southern California, Center for Systems and Software Engineering refers to as the
Constructive Cost Modell methodology COCOMOII. (You can access this model using:
http://diana.nps.edu/~madachy/tools/COCOMOII.php)
I have used a monthly cost of $22,200 being 18.5 average working days a month at $1,200 a
day, calculated as the average internal costs of a development team and management
structure plus employment costs and overhead.
The model gave these results:
Work effort in person months
Development cost
Benchmark 598 $13,275,600
Circatec build 97 $2,153,400
Ratio 16%
This costing model shows a cost saving of $11.12 million using automated software
development instead of the traditional SDLC.
Furthermore:
The function point count is 2898 which is at the upper end of size and complexity.
At no stage did we have more than four people working on this development.
This comparison used cost criteria reflective of a well run project.
There is ample literature available to support the view that a development of this size
following a conventional SDLC has an extremely high probability of failure.
So lets change the costing criteria to be more reflective of a large development team with a
high number of contractors, limited domain knowledge and increasing time constraints.
Drivers Benchmark Large team Project under
pressure
Team Cohesion High Low Very low
Analyst capability High Nominal Nominal
Programmer capability High Nominal Nominal
Personnel continuity High Low Very low
Application experience High Low Very low
Platform experience High Low Low
Language & toolset experience High Nominal Nominal
Time constraint High Very High Very High
Use of software tools High Low Low
Person months 598 2,706 3,845
Cost $13.3 million $60.1 million $85.4 million
The result is a staggering blow out in cost and development effort.
-
Page 19
Unbelievable? Not really. Regardless of what industry you are in Im sure you know of at least
one project that has gone this way. If not, look at some high profile public projects, theres at
least one in every city.
Suddenly 16% of benchmark using automated software development is now 2-3% of the cost
of a poorly performing conventional SDLC. The chances of a blow out of this proportion in an
agile, model based, automated software development are practically nil.
It doesnt stop here
The whole structure of the IT department changes in an automated software development
environment.
Project managers Arent required for any aspect of the software development cycle. A
reduced role in requirements gathering and post development
implementation.
Software architects On a large or complex development an architect may add value in the
initial design phase. This is a short term contract position at the most.
Data modellers Not required as a dedicated role, providing the developers have a
good understanding of data structures and database conventions
which they will if the right people have been recruited.
Testers Testing is incremental and an integral function of the BA / developer
team. There is value in having a small pre-release testing function
within the development group.
Systems support Business focussed requirements gathering, client engagement during
the development cycle and ease of implementation reduces the
client support roles.
The larger the development the greater the savings
The larger and more complex the requirement, the greater the productivity gains and the gap
between the traditional SDLC and automated software development.
Waterfall SDLC
Agile SDLC
Automated software development
Size & complexity of development
Co
st
-
Page 20
There is also much greater certainty in automated software development. The teams are
smaller, therefore you can be more selective who you engage.
The agile methodology suggested earlier requires design and work breakdown phases before
commencing any development work. These will provide a reliable estimate of work plus small
development iterations for cost and progress monitoring.
Compelling case for change
The cost savings are compelling on their own.
Plus the added benefits of system agility, speed of change and greater engagement of clients
in the development cycle.
The business case for a major change to corporate IT is compelling.
-
Page 21
How corporate IT resists change Theres a compelling business case for automated software development and agile
methodologies and yet the uptake is minimal. To add insult to injury this is not new
technology. Most automated software development tools have been commercially available
for ten to fifteen years!
Before we consider a significant change to corporate IT we need to identify the barriers to
change. Yes many of the usual barriers to change exist but there are some unique issues to
address in bringing about wholesale change to corporate IT.
Some of the fundamental problems are:
The very reason many middle and senior IT management roles exist is to manage the
existing IT structure and workforce.
The powerbase and kudos of senior developers, system architects, data modellers and
other specialists relies on preserving the importance of their role and discipline.
For many consulting firms their entire fee base is built on large, highly structured projects
and the supply of large number of IT contractors. Theres not a lot of revenue in
automated software development projects!
There is a whole industry offering training and accreditation particularly Prince2, APM
BoK and, to a lesser extent, Agile.
The market is crowded with consultants and contractors needing clients to buy the whole
package to ensure a reasonable period of employment.
So there is a whole industry built around the status quo and they are very effective at blocking
the uptake of automated software development tools and efficient methodologies.
Unfortunately they are often the people business executives turn to for advice and they fail
the trust placed in them.
Let me give you just two examples. I have amended some details to avoid identifying the
organisations, however many readers will identify with these.
1. An organisation has a number of legacy financial products that are largely rules based.
They want to move them off several old platforms that are becoming more difficult and
expensive to maintain. The internal cost to redevelop these products to run on a common
platform was in excess of $2 million dollars, which could not be cost justified.
A rules engine vendor analysed the rules and estimated it would cost between $2-300,000
to develop all the legacy products into three or four rules engines. The vendor was willing
to provide a fixed price quotation and requested access to all the product rules. A systems
architect blocked any further analysis on the basis that we dont want another
technology on site.
So the vendor was not able to speak to the business unit or obtain any further information
to formalise a quotation nor did the business unit know there was a viable option on
offer.
-
Page 22
From a commercial perspective this one person was able to make a totally unreasonable
and unsupportable decision, effectively forcing the organisation to continue maintaining
legacy products on legacy systems, whilst continuing to incur the increasing cost of
maintenance.
2. A company was developing a web based financial planning tool. A rules engine vendor
offered to build all the financial calculators for a little over $100,000 and 3 months
duration. A web developer assured the company they could program the calculators for
$75,000. Two years and many dollars later the product wasnt complete and the window
of opportunity had closed.
These examples illustrate the difficulty vendors of automated software development
tools face. The people they have to sell to are the very people who dont want the tools in
the organisation.
The above examples occurred because corporate IT controlled the information flow into the
respective organisations.
Common practices used to maintain control
Inappropriate use of standards
No one denies the importance of standards but they must be the right standards for the
right reasons.
There are many technology standards that are critical, especially in system to system
communication or the electronic transfer of data between organisations.
However many standards and methodologies have evolved from professional bodies or
special interest groups. Whilst they may add value in some circumstances they are not
commercial imperatives and are certainly not set in stone.
For example Wikipedia lists::
Data modelling 10 techniques and notation standards.
Process modelling 6 process modelling techniques and 5 languages for BPM
(Business Process Modelling).
Software development - 2 methodologies, SDLC and Agile with 5 common SDLC
development models and 4 common Agile development models.
Project management 10 common methodologies.
Why on earth does any organisation want or need to base the selection or development of
business solutions around any one of the myriad of standards and standard
methodologies.
In far too many cases standards are mandated by the IT group because:
Its the sandpit the technologists want to play in.
The standard is used to block more efficient options.
The only justification for mandating a technology or methodology standard in an organisation
is to genuinely protect or add value to the organisations IT systems.
-
Page 23
So it is absolutely critical that standards are vigorously challenged to ensure they are aligned
with the enterprise core business and core enterprise systems. Otherwise they become a tool
for technologists to block unwelcome automated development tools and efficient
methodologies.
Request for Proposal or Tender
I have seen too many tenders where the technology and methodology sections are as large, or
larger, than the business requirements.
These documents are extremely detrimental to the organisation for several reasons:
1. They shift the focus from a business solution to a technology solution.
2. They shift control from the business units to IT, enabling IT to protect the status quo.
3. They often block innovative solutions to business problems.
For many years Circatec was actively engaged in the specification and selection of medium to
large ERP systems. In all that time I refused to specify the underlying technology on the basis
that we were looking for the best possible solution to a business requirement. Having found
the best solution then wed look at the technological issues.
If your organisation is looking for a solution to a business problem then dont let your IT
department write or run the RFP/RFT process. Only allow them to specify technical
requirements that are absolutely essential and justified from an enterprise perspective. When
youve identified the functionality you need for the enterprise then engage IT to make it
happen.
Do not underestimate how important this is. A number of vendors of automated software
development tools and/or services are reluctant to respond to a tender run by an internal IT
department, especially when the technology and methodology sections are deliberately
prescriptive.
Circatec recently declined to bid on a development because the added cost of meeting the
specified implementation and technology standards exceeded the cost to develop the
solution using automation tools and agile methodologies. We know the project didnt proceed
because it couldnt be cost justified on responses from complying vendors. We also know
that we could have provided a solution within a cost justifiable budget if we had been allowed
to submit a non-complying bid.
Size of the service provider
The average application build using automated software development is approximately
15-20% of the work effort of a standard SDLC build. Rules engines are even faster, often
just 10-15% of the build time.
Agile methodologies and business oriented process modelling tools significantly reduce
the requirements gathering and process improvement activities.
Rules engines can cut months out of data conversion and rectification.
It follows that any service provider using or supporting these tools and techniques will be
small. And so they should be, otherwise they are just another firm looking for billable hours to
feed a large team.
-
Page 24
However size is often used against the service provider:
They arent big enough to support our needs.
They are too small to really understand our requirements.
Our preferred supplier has 200 staff so they are better placed to handle our needs.
(Code for they use the same inefficient practices we do).
The last thing your IT group wants is a small, agile, highly effective organisation delivering
more product in a fraction of the time it takes in-house.
I can do it just as quickly by hand!
If I had a dollar for every time Ive heard this Id be a rich man!
It may be true at a micro level but what the programmer is really saying is:
I can knock up a piece of untested, undocumented code which Ill imbed in the existing
code with no notation. No one will know its there or what it does, perpetrating my role as
an indispensable source of knowledge.
The same is true with data validation, rectification or conversion. Programmers will claim they
can quickly write a script to carry out a data management function. What they dont say is that
scripts have about as much integrity as a spreadsheet in that they are difficult to validate and
have no change control.
Micro level comparisons quickly become irrelevant. Other than patching an existing program
there are very few micro developments. If we follow the scripting example a little further what
we will find is that the single script quickly becomes 100 scripts. New scripts are written to
replace early scripts and the library of scripts grows, as does the risk of running an obsolete or
inaccurate script.
The CEOs toolkit
Cutting through the screen of misinformation is relatively easy using what I call the CEOs
toolkit:
Why
So what
Prove it
The conversation is:
In plain business language, and in the context of our business needs, tell me:
Why: are we doing it this way?
Why: is that important?
So what: if what you say does or doesnt happen?
Prove it: with evidence and a business reason to support your opinion.
You may have to keep asking why to get to the real reason. When you do then you can assess
whether there is a business case to support the IT position or not.
Organisations can no longer afford or justify self indulgence by their IT departments and project management offices.
-
Page 25
Bringing about change Changing the structure and culture of your IT department will not be easy. It will have to be
driven from the very top Board, CEO and senior management.
This scale of restructure requires selective redundancies, a potential HR nightmare. The
management team driving the restructures will need to know they have a board mandate and
CEO support for the changes they are to make. They may need expert industrial law advice
and HR guidance.
It is a daunting but commercially imperative task, and its very achievable.
Your CIO may actually be your greatest ally. Many CIOs feel they are caught in the middle
between the pressure from the business units for higher levels of IT service and an inflexible IT
workforce. They know whats ahead of them if they try to implement change without
organisational support. On the other hand if your CIO is part of the problem this is your first
replacement.
Through recruitment I know there are many experienced developers and business analysts
excited by the potential of automated software development. Small teams attract and
stimulate good quality people. Competent people thrive in an environment that values their
knowledge and skills. You may already have your automated software development teams on
your staff.
The key is attitude. A developer or business analyst can lack some specific experience if their
attitude is a willingness to learn, to take on new methods and challenges and add value to a
small, self motivated team. This is one of the key criteria in determining which members of
the existing IT group will transition to the automated software development group.
Structure
An agile automated development environment has a very flat management structure.
A small number of largely self-managed teams (3-5 members) will generate a tremendous
amount of high quality software. Depending on the number of teams a single development
manager with possibly a coordinator to handle work allocations, time recording, etc. will be
the total management structure.
There is no need for a project management office (PMO) in an automated software
development environment. There will, of course, be other corporate technology programs to
manage, however if your PMO sees itself as a control centre, rather than a facilitator,
dismantle it and start again, otherwise they will just be a barrier to change. Hire project
managers not methodology practitioners.
It is questionable whether there is any need for full time software architects and data
modellers. These are roles that can be outsourced at the commencement of a major
development to define the design and structural elements of the application. The automated
software development tools will enable the teams to adhere to these design principles, and
there will be the skills and understanding in the development teams to apply the design
guidelines.
IT infrastructure, network support and end user help desk functions will remain largely
unchanged.
-
Page 26
Existing systems
Existing systems, especially internally developed and/or maintained, will require continuity of
support. Consider setting up a separate product support group, however it will need strong
management to contain the maintenance of existing systems. New functionality should be
developed by the automation teams.
Alternatively, if enough of the old IT group have been transitioned, maintenance can be
provided by the new group. The advantage of this is that once developers have absorbed
themselves in automated software development tools and rules engines they wont want to
do more than necessary in the legacy system.
What about work-in-progress?
If a development is in its early stages, and is more than a few weeks work, then consider
redeveloping it using automated software development. As a general rule any development
that hasnt past the 50% complete point will deliver cost savings by starting again in an
automated environment.
If a development has passed the 50% complete stage, is on time and budget, and the
development team is still intact, then it should be completed by the current team.
If a development is over time and budget and there is no confidence in the expected
completion date and cost then it should be stopped and reassessed. There is every likelihood
an automated software development team will redevelop it faster, more cheaply and to a
higher quality than persevering with it. Quality is often the first casualty of troubled
development projects and must be a factor in the decision to start again.
Resistance to change
Like any form of workplace automation this is a change management project with little room
for compromise.
The resistors must be identified and removed if they wont adopt the new methods. Weve all
met the passive resistors who agree to change in meetings but wont change. There is always
an excuse and in the meantime they are gathering supporters and the whole rollout is
jeopardised. Fortunately with automated software development they are easily identified
they are either using the tools or they arent.
There is a small but vocal group of IT professionals who can be openly scornful and
disrespectful of anyone with a different perspective to theirs. I call it technological arrogance
and these are some of the most difficult people to manage when implementing change on
this scale, especially when the change diminishes their role and importance in the
organisation.
They can be openly hostile and dismissive and their greatest weapon is to resort to techno-
speak, running out a string of indecipherable reasons why whatever youve just suggested
wont work. Often their claims are facetious, exaggerated or just plain wrong. Fortunately
they do tend to make their presence known.
No organisation is obliged to provide technologists or methodology practitioners their own
personal sandpits. If someone wont play in the corporate sandpit for the good of the
organisation there is no place for them in small, agile, self motivated teams.
-
Page 27
Be prepared to start again
I have experienced first hand the total block of model based and automated software
development tools. Its as if the group psyche is total commitment to the status quo, they
genuinely cant see anything wrong with their work practices and methodologies.
When this happens an organisation must accept that they may have to shut down a whole IT
section and/or PMO and start again.
This may seem drastic, but in these circumstances it will be more effective to start from a zero
base, recruiting the roles and skills needed for an agile, automated environment.
Despite a genuine concern around the loss of knowledge of existing systems, losing blockages
to change in a well managed manner is infinitely more productive than keeping these people
because of their perceived knowledge. When theyre gone its often surprising how much
knowledge is retained in the organisation.
The savings and other tangible benefits are too great to compromise or delay the restructure.
-
Page 28
What will it take? Ive been critical of corporate ITs resistance to change, but in all fairness in many cases they
are doing, and delivering, what is asked of them.
There will be no groundswell of change from within the IT industry.
It will have to come from boards, CEOs and senior executives.
As they look at what their organisation has spent and is being asked to spend on IT systems,
particularly the development and maintenance of software applications.
Then, knowing there are alternatives, they are entitled to get frustrated, even angry, at the
impact on the revenue they work so hard to earn and the profit they are expected to deliver,
they will start to demand change.
These are mainstream alternatives
CA Technologies, the developer of CA Plex, is one of the largest independent
software companies in the world. (Wikipedia)
IBM reduced development costs by $300 million a year by adopting Agile
methodologies. (Google How IBM saved $300 million)
Gartner Group is the leading technology analyst and trend forecaster. A positive
Gartner review can have a significant impact on the market acceptance of a software
product or technology.
None of these are small niche players!
Change will occur when change is demanded.
When senior management will not accept multi-million dollar projects blowing out five or
ten fold.
When IT budgets are reduced and greater output is expected.
When business units demand, and expect to receive, rapid delivery of solutions.
When boards demand greater value for their technology spend.
Only then will agile, automated and delivery focussed IT groups emerge.
And emerge they can. There is no practical barriers to the use of model based, automated
software development tools in business applications including browser and smart phone
apps.
The 40% reduction in IT jobs forecast by the Gartner Group in 2005 is now, and was then,
attainable.
It only requires organisations to insist on change and empower their senior executives to
make it happen.
-
Page 29
About the author Ken Galbraith is the founder of Circatec Pty Ltd, a software development and consulting
company based in Melbourne, Australia.
He has been in the software industry for nearly 30 years. During that time he has been an
owner and/or director of three software development and consulting businesses.
For fifteen years during the 1980s/90s Ken was involved in enterprise (ERP) systems for
manufacturing and logistics. This was a period of radical change in manufacturing globally.
ERP systems and associated process change and consulting services were an integral
component of the change.
In 1996 Ken formed the INL Consulting Group, specifying, selecting and implementing ERP
systems for a number of medium to large manufacturing and logistics companies. During this
period he personally started working in superannuation and wealth management.
In 2000 INL Consulting became Circatec. The business focus shifted from consulting to the
provision of technology and software solutions.
By 2005 Circatec was using model based development tools in business process automation
and rules based business solutions. Services expanded into the development of large and
complex application software. Through this Ken gained enormous experience in the power of
these tools.
Circatec was also marketing the toolsets and supporting services. An exercise that provided
first hand experience in the ability of corporate IT to block the adoption of automated
software development tools and efficient development methodologies.
Today Ken, and Circatec, remains committed to changing the software development and
delivery paradigm by working with senior management to bring about a cultural shift in
corporate IT.
You can contact Ken Galbraith through the Circatec website or by email.
Web: www.circatec.com.au
Email: [email protected]
Additional copies of this eBook can be obtained from the Circatec website.