Agile IT - A value driven approach to IT delivery final
-
Upload
hml-ltd -
Category
Technology
-
view
671 -
download
0
Transcript of Agile IT - A value driven approach to IT delivery final
An HML white paper: Agile IT
A value-driven approach to IT delivery
© HML 2012. All rights reserved.2
CONTENTS
2: About this paper
3. About HML
Introduction
4. The need for
change
Lean agile roots
5. The journey to
Agile IT
7. Overcoming
challenges to change
8. The Eight
Principles of Agile IT
13. The benefits of
Agile IT
14. Agile IT at work:
Forbearance Project
15. About the author
ABOUT THIS PAPER
The purpose of this paper is to demonstrate how applying lean and agile
principles can create value in software development.
It will be of interest to people who are responsible for IT projects, budget
control, organisational change or who have an interest in lean and agile
thinking.
The paper describes how HML has put these principles into practice in a
form of software development called ‘Agile IT’, which addresses key
business concerns like time to market, responsiveness to changing
needs, customer satisfaction and cost reduction.
The paper outlines the history of lean and agile approaches and explains
how these have been applied to create Agile IT. It sets out Eight
Principles that underpin HML’s approach, and how these enable the
delivery of the right IT capabilities at the right time to maximise value to
customers.
“Agile IT has allowed HML to meet the strategic
challenges of wasted resource and lengthy change
management processes. What might have taken
months now takes weeks and our IT department is
engaged and able to offer clients tangible results
throughout the process. ”
Andrew Jones, Chief Executive Officer, HML
© HML 2012. All rights reserved.3
HML‟s success in
applying lean and
agile principles has
resulted in a
nomination for the
Best Agile
Newcomer at this
year‟s UK Agile
Awards.
For more
information about
the Awards see
http://www.agileaw
ards.co.uk/Awards2
012.html
ABOUT HML
HML is the UK’s largest specialist mortgage servicer, providing
outsourced mortgage administration services to over 50 leading financial
institutions. HML operates from three UK locations: Skipton, Londonderry
and Glasgow. The company was established in 1988 and manages
approximately £43bn of mortgage assets and 400,000 customer
accounts.
INTRODUCTION
HML’s iConnect platform provides a sophisticated and comprehensive
tool to support a diverse range of client requirements. iConnect is
constantly updated, driven by HML’s desire to continually improve its
offering, but also by the regulatory environment in which HML and its
clients operate.
In 2008, HML began to apply lean 6-sigma improvement techniques
principally within its operational areas. In 2010 it embarked on an
ambitious programme to improve iConnect in support of this drive for
operational excellence.
To meet this challenge, the IT teams needed to develop their ways of
working, so Agile IT was born.
Agile IT is achieved by the application of eight principles:
1. Make it Visible
2. Make Collaboration Easy
3. Assume Change: Experiment, Discover, Learn, Iterate
4. Focus on Delivering Useful Value Today
5. Build a Sustainable Flow of Value
6. Build Quality In
7. Continuously Improve
8. Agility is Everyone’s Responsibility
HEADLINE RESULTS
In just 12 months, Agile IT has enabled HML to increase its rate of
delivery of new IT capabilities by a factor 4, while reducing the time
to deliver a new feature from months to a matter of weeks; all
without impacting cost or quality.
© HML 2012. All rights reserved.4
At the start of 2011,
HML recognised
that its IT teams
needed to evolve
their ways of
working in order to
become more
responsive to
changing needs
and reduce the
„time to value‟ of
changes, whilst
retaining the
improvements in
quality.
THE NEED FOR CHANGE
In common with many financial institutions, HML has in the past relied on
a sequential software development lifecycle, often known as ‘waterfall’,
where projects progress through a series of discrete stages:
• Request
• Design
• Build
• Test
• Implement
Each stage within this sequence was largely undertaken by a separate
specialist functional group and work was handed off from one group to
the next with formal documentation providing the communication between
groups.
This approach worked well for HML’s needs at the time and over five
years had resulted in a steady improvement in the stability of production
systems, with progressively fewer defects escaping into production, and
improved predictability of delivery dates.
However, these improvements tended to come at the expense of
responsiveness to change and long delivery timescales.
At the start of 2011, HML recognised that its IT teams needed to evolve
their ways of working in order to become more responsive to changing
needs and reduce the ‘time to value’ of changes, whilst retaining the
improvements in quality.
These changes were to be guided by the adoption of lean and agile
software development principles and practices – an approach that
complemented HML’s existing use of lean and 6-sigma techniques in its
Operational Service Areas.
LEAN-AGILE ROOTS
The term ‘Agile’ as applied to software development was first coined in
2001 to describe a range of methods that emphasised adaptability to
change. The Manifesto for Agile Software Development and its
associated 12 Principles sets out what being agile means; while a range
of methods, such as Scrum, eXtreme programming and DSDM, provide
ways of developing software in an agile way.
The idea of lean software development began at a similar time. It takes
the core principles of lean thinking, e.g. flow, pull, and continuous
improvement, and applies them to software development.
© HML 2012. All rights reserved.5
The timely and
sustained delivery
of valuable IT
capabilities to
HML‟s internal
customers and
external clients
remains
paramount, and is
at the core of Agile
IT.
Lean and agile can be seen as complementary approaches: lean tends to
address the wider organisational concerns, while agile focuses at the
development team level.
The combination of lean and agile principles and methods, together with
other approaches such as 6-Sigma, Kanban and the Theory of
Constraints provide the foundations for Agile IT at HML.
See http://agilemanifesto.org/
See http://www.scrum.org/ and http://www.scrumalliance.org/
See http://www.extremeprogramming.org/
See http://www.dsdm.org/
See http://www.lean.org/WhatsLean/Principles.cfm
THE JOURNEY TO AGILE IT
Lean and agile software development is not just something that you do;
it’s equally about how you think. Changing the ways of working in IT had
to address both transformation of its culture and the adoption of new
practices and techniques.
And while lean and agile principles and techniques have been at the
heart of the changes, these have always been treated as a means to an
end rather than an end in their own right. The timely and sustained
delivery of valuable IT capabilities to HML’s internal customers and
external clients remains paramount, and is at the core of Agile IT.
At the start of the change transition, HML appointed an experienced agile
development practitioner to provide expertise, leadership and support
through hands-on coaching. Reporting directly to the Head of IT provided
the appropriate executive authority and clearly signalled the importance
of the changes.
Building awareness of lean and agile approaches, principles and
techniques has been a cornerstone of the transition. Alongside formal
training and hands-on coaching, peer-based informal ‘learning lunches’
and ‘community of practice’ forums have been used to regularly share
experiences and learn new techniques.
Task boards - big visible displays that show a team’s flow of work - were
one of the first items to be introduced and they remain one of the most
visible aspect of the changes to visitors. Impediments – things either
slowing down or blocking flow – are quickly identified enabling action to
be swiftly taken to resolve.
© HML 2012. All rights reserved.6
FIGURE 1: IMPEDIMENTS CAN BE ESCALATED TO A DEPARTMENT
LEVEL BOARD
HML has borrowed
from many of the
well known agile
methods, such as
Scrum, eXtreme
Programming (XP)
and DSDM, but has
deliberately chosen
to avoid mandating
a particular method
in order to
encourage
adaptation,
innovation and
ownership by
teams in how they
work.
Symbols used to indicate type of impediment:
Stopping progress
Slowing progress
Significant risk of stop or slow
Two other key changes were also introduced early in the transition: co-
location of teams and incremental development; where business goals
are progressively broken down into a series of features that are
developed in short iterations and delivered over a series of releases.
Fundamental to the long term success of changing the way of working
has been to make continuous improvement and learning a way of life.
Fortnightly ‘retrospectives’ enable a team to reflect on what is working
well, and what is not; and gives them responsibility for changing their
process. Over time, this process has itself been adapted, refined and
improved, with both a team and department wide improvement regime.
Instead, HML has developed eight principles of Agile IT that can be
used by everyone involved, from individual IT practitioners, to teams and
stakeholders to guide how they should work.
The principles help reinforce the view that Agile IT is primarily cultural,
and represents a journey not a destination.
© HML 2012. All rights reserved.7
Iterative
development and
incremental
delivery creates
significant
challenges for
upstream and
downstream
processes.
Creating an IT
capability to build
and deliver new
features on a just-
in-time basis also
requires a business
organisation that
can define and
prioritise a flow of
desired features,
and one that can
absorb and apply
them usefully.
OVERCOMING CHALLENGES TO CHANGE
Change is never easy, and HML faced a number of challenges on its
Agile IT journey.
One of the first challenges HML encountered was the creation of co-
located, cross-functional teams. Unsurprisingly, some individuals felt
unhappy to have to move desks, and breaking up long standing functional
groups risked losing their sense of community. HML worked hard to
involve those who needed to move in the planning, and introduced
changes progressively. The existing functional line-reporting structure
and regular functional team meetings were also retained. Only at the start
of 2012 did formal line management change to reflect the new cross-
functional structure of teams.
Throughout the transition, HML needed to ensure governance was not
compromised. Many of the existing governance controls were organised
around or were dependent on waterfall stages and functional teams, so
the move to cross-functional teams and iterative development potentially
reduce these controls’ effectiveness. Close collaboration with the Risk
and Compliance teams ensured appropriate oversight and HML is
currently introducing Practice Leaders who will be responsible for
governing key practice standards.
Finally, iterative development and incremental delivery creates significant
challenges for upstream and downstream processes. Creating an IT
capability to build and deliver new features on a just-in-time basis also
requires a business organisation that can define and prioritise a flow of
desired features, and one that can absorb and apply them usefully. HML
has made some progress in this area, using agile techniques to define
business cases for increments rather than whole projects, but this
remains an area of development and improvement.
© HML 2012. All rights reserved.8
Teams own their
visual management
boards, and this
fosters innovation
in how boards and
the underlying
process they
represent evolve to
meet the team‟s
needs.
THE EIGHT PRINCIPLES OF AGILE IT
1. Make it Visible
The process of software development is difficult to visualise, and while
Gantt charts have a place, they rarely map well to the needs of IT.
Agile IT makes use of large, physical display boards that show the flow of
work being undertaken by the team, an approach known as Visual
Management. Task boards and similar visual management tools help a
team visualise the ‘state of play’ of the work they are doing and make it
easy to spot and respond to impediments to a team’s development flow,
contributing to shorter delivery times.
Teams own their boards, and this fosters innovation in how boards and
the underlying process they represent evolve to meet the team’s needs.
For example, blockages are highlighted with stop signs; avatars of team
members are used to indicate who’s doing what; and colour is used to
indicate different types of work.
FIGURE 2: SHOWING FLOW OF WORK ITEMS, TYPICALLY USER
STORIES AND ASSOCIATED TASKS
As well as helping the team, visible task and status boards have proved
invaluable for updating stakeholders. They’ve also fostered greater team
spirit and visibility of the status of the project with the wider business.
Avatars
show
assignment
Impediments
highlighted with
Red Flags
© HML 2012. All rights reserved.9
Face to face
communication is
encouraged as the
primary method of
communication
IT projects face two
key challenges:
customers do not
know precisely
what they really
need; and in
today‟s competitive
business
environment those
needs are often a
moving target.
Agile IT assumes
that change is
inevitable and
seeks to exploit this
rather than resist it.
2. Make Collaboration Easy
IT development is a creative process. Customers and IT professionals
have to work together to ensure they deliver what is needed in a
background of imperfect knowledge and change. Making collaboration
easy is therefore essential for successful projects.
With Agile IT, work is undertaken by small co-located, cross-functional
teams, known as ‘pods’ who take responsibility for the end-to-end
delivery of new software features.
As well as bringing together expertise in different IT disciplines, such as
test, engineering and analysis, customer representatives and other key
stakeholders work closely with teams throughout the development and
delivery lifecycle.
Face-to-face communication is encouraged as the primary method of
communication. Documentation remains important, but the emphasis is
on doing just what is necessary and using alternative forms of
documentation, such as wikis and other electronic knowledge bases,
rather than paper documents.
3. Assume Change: Experiment, Discover, Learn, Iterate
IT projects face two key challenges: customers do not know precisely
what they really need; and in today’s competitive business environment
those needs are often a moving target.
The traditional approach to addressing these challenges is to undertake
extensive up-front analysis before locking down scope and then limiting
subsequent amendments through change control regimes.
Unfortunately it’s also common to see these IT projects take a very long
time to deliver what turns out to be the wrong thing.
Agile IT assumes that change is inevitable and seeks to exploit this rather
than resist it. Using an iterative approach to development coupled with
incremental delivery ensures that the most valuable features needed
today are delivered first whilst others are deferred for later increments.
IT projects at HML typically use fortnightly iterations, and are able to
deliver new features into production monthly. Stakeholders are able to
review working versions of iConnect at least every iteration, and this
feedback is used to adjust what features will be developed next.
© HML 2012. All rights reserved.10
Agile IT seeks to
incrementally
deliver value by
delivering in small,
valuable
increments. Each
increment is
focused on
delivering the
minimum needed to
provide benefit to
its customers
today.
4. Focus on Delivering Useful Value Today
Delivering something of value early is generally much more beneficial
than delivering a collection of things much later. As well as increasing
return on investment, focusing on delivering what’s useful today helps
prevent speculative development or ‘gold-plating’.
Research by the Standish Group suggests this is far from uncommon,
reporting that 45 per cent of software applications’ features are never
used. As well as incurring unnecessary cost, building these features
means that other more useful features are either delayed or not built at
all.
Agile IT seeks to incrementally deliver value by delivering in small,
valuable increments. Each increment is focused on delivering the
minimum needed to provide benefit to its customers today.
Stakeholders are actively involved in determining the order of
development of features and the team is encouraged to seek out
opportunities for delivering increments early. Monthly releases enable
new features to be rolled out quickly so it’s possible for a feature to go
from idea to delivery in just a few weeks.
Delivering incrementally places new pressures on repetitive tasks such
as build, deployment and regression testing and HML has steadily
focused on automating and refining this process. Similarly, architecture
and detailed designs need to support change, and technical practices
such as Test Driven Development and Refactoring have been adopted to
support this.
5. Build a Sustainable Flow of Value
Many organisations seek to maximise the utilisation of staff and the
number of projects that are in progress. Unfortunately, this approach
tends to result in long delivery times for any one activity and relatively low
completion rates. Whilst it can give the appearance of cost efficiency it
comes at the expense of time to value and responsiveness to changing
needs – qualities that usually far outweigh the savings made by
maximising utilisation.
Agile IT seeks to take a holistic view of the whole delivery value stream -
not just that part in IT - optimising for a fast, steady stream of deliveries of
valuable features. Instead of maximising utilisation (and minimising unit
cost), Agile IT seeks to minimise cycle time, the elapsed time it takes for
valuable increments to be delivered, and so increase throughput, the rate
of delivery of benefits.
© HML 2012. All rights reserved.11
Simply put, Agile IT
values finishing
over starting; doing
less in order to
achieve more, and
systematically
finding and
removing anything
that disrupts or
impedes flow.
Agile IT recognises
that speed and
agility requires
attention to quality
and technical
excellence; it is
simply not possible
to go fast if the
process or product
is unreliable.
Reducing cycle time enables returns on investments to be achieved
sooner and improves responsiveness to changing needs. Increasing
throughput maximises the opportunity to deliver what is needed.
Simply put, Agile IT values finishing over starting; doing less in order to
achieve more, and systematically finding and removing anything that
disrupts or impedes flow.
Teams at HML use either iterations - short timeboxes of typically two
weeks - or explicit work in progress limits to manage flow. Projects are
decomposed into small increments which are in turn broken down into
small features that can be completed in days rather than weeks. And
teams actively identify and resolve impediments to flow, escalating those
they cannot quickly resolve to a department-wide daily impediments
meeting attended by management to ensure visibility.
6. Build Quality In
Deming famously declared that “You can not inspect quality into the
product; it is already there.”* Most traditional approaches to software
development focus on the (often late) detection of problems, through
inspections and testing, and make extensive use of manual processes
that inevitably introduce variation and errors.
Agile IT recognises that speed and agility requires attention to quality and
technical excellence; it is simply not possible to go fast if the process or
product is unreliable.
Teams at HML employ automation for repetitive tasks, such as
integration, build, check and deploy; enabling them to execute these
tasks frequently – typically multiple times a day. They also use
techniques such as pair programming and test driven development that
help prevent errors or catch them as soon as possible after they are
introduced.
*From Out of the Crisis, by W Edwards Deming (MIT Center for
Advanced Engineering Study, 1986)
© HML 2012. All rights reserved.12
The idea that a
single process will
work equally well
for all time and
across all projects
and teams is not
reasonable, and so
Agile IT promotes a
culture of
continuous
adaptation and
improvement and
advocates that
those who do the
work are best
placed to improve
how the work is
done.
„Doing agile’ is
important, but
‘being agile’ is
essential.
Fundamentally,
agility is a mindset
that everyone in the
organisation needs
to express.
7. Continuously Improve
The need to respond to change applies just as much to the process of
development as to the capabilities of the software. The idea that a single
process will work equally well for all time and across all projects and
teams is not reasonable, so Agile IT promotes a culture of continuous
adaptation and improvement and advocates that those who do the work
are best placed to improve how the work is done.
HML has made knowledge sharing explicit; sponsoring ‘learning lunches’
and the formation of communities of practice, and has recently introduced
Practice Leaders to guide and support practice improvement.
Frequent and regular retrospectives - workshops focusing on
improvements - are undertaken by teams, typically every two weeks,
where teams are encouraged and empowered to make small incremental
changes to their processes to drive improvement.
8. Agility is Everyone’s Responsibility
‘Doing agile’ is important, but ‘being agile’ is essential. Fundamentally,
agility is a mindset that everyone in the organisation needs to express.
Agile IT provides an organisation with the potential for becoming an Agile
Business.
At HML, agility in general and Agile IT specifically, has become central to
its ethos.
© HML 2012. All rights reserved.13
An HML consultant
feeds back after a
recent project to
automatically and
securely retain
customer card
details goes live:
“I have to give you
this feedback
because it is proof
of the success of
card re-use
functionality. We
had over 200 calls
today and not
dropped a single
one, on last
working day!
Normally by now
we would have
expected to drop
calls as it is so
busy. It's obvious
that there are a lot
of card payment
only calls and we
are flying them.
Customer feedback
is positive and
consultant
feedback is very
positive.”
BENEFITS OF AGILE IT
Over the first 18 months of its Agile IT journey, HML has significantly
improved its IT capability:
Increased throughput
HML has been able to move from quarterly releases to monthly releases.
Delivering more frequently has had two benefits: firstly, it is much easier
to release opportunistically – stakeholders, having seen an iteration
demo, may realise that the software would be beneficial if deployed as is,
and monthly releases allow these opportunities to be exploited. Secondly,
more frequent and smaller releases reduce the risks associated with
each deployment.
Reduced time to deliver value
Before the transition to Agile IT, IT projects at HML, even those that used
a phased delivery approach, would typically take between three and nine
months from approval to their first delivery into production. Agile IT has
reduced this ‘time to value’ considerably, with projects typically now
delivering benefits within three months, and often considerably sooner.
Greater flexibility and responsiveness to changing needs
Focusing on achieving a business goal frees teams to explore and adapt
their solution so that it best meets that goal, enabling them to respond to
changing circumstances and new knowledge about what is important.
This flexibility has enabled projects to start and begin to deliver useful
benefits despite imperfect information; while allowing new needs to be
much more easily accommodated.
Improved stakeholder and client satisfaction
By working more closely with its stakeholders, building iteratively and
acting on their feedback, HML has been able build trusted relationships
and a real feeling that they were a part of our product development.
© HML 2012. All rights reserved.14
INDUSTRY
REACTION TO
AGILE IT:
“I got the impression
that [lean-agile
principles] were truly
embedded in the day-
to-day work of the
teams.”
“I was greatly
impressed by the IT
department...to see an
IT team working so
cohesively is something
we can only dream of.”
AGILE IT AT WORK: PROJECT FORBEARANCE
At the start or 2012, responding to a request from one of its clients, HML
began a project to implement improvements to iConnect to support
anticipated regulatory requirements proposed by the FSA.
Speed of response was critical to the client’s needs, but uncertainty
around what was really needed was high – HML was seeking to be first to
market – and operational usability was essential to ensure minimal
impact to customers and call centre consultants.
During the start-up phase of the project, the team, through a series of
workshops with stakeholders, created a high level vision, a prioritised list
of high level requirements and an outline roadmap. Work then proceeded
using fortnightly iterations that planned, built and demonstrated new
functionality. Regular retrospectives generated frequent improvement
‘experiments’ which drove constant adaptation and improvement in how
the team worked.
As the project progressed many items originally in the backlog were
considered unnecessary, and many new items were added in response
to improved understanding of needs identified through the
demonstrations. Throughout the process, both the client and internal
stakeholders were involved, and this has resulted in high levels of
engagement and satisfaction.
What the client said:
“The projects team's approach throughout has been one of collaboration
best demonstrated by their invitation to me, to not only attend, but part-
facilitate the two day forbearance training event that was delivered to our
Credit Management colleagues.” (Compliance Manager)
“It has been a pleasure to work with the Forbearance project team.
Communication has been excellent throughout the project, they have
been responsive to requirements (including late ones) and all signs point
to delivery meeting agreed scope and originally agreed timescale.”
(Operations Manager)
“I was surprised at just
how good the visual
management was in IT
in terms of its variety
and complexity.”
Delegate
responses from a
meeting of the
Lean Service
Forum, hosted at
HML in April 2012.
For more
information on the
Lean Service
Forum see:
http://www.oeeuk.c
om/community-
leanforum.asp
Key metrics
Mobilisation time: 2 weeks
First demo of useable functionality:
2 weeks after mobilisation, and fortnightly thereafter
First delivery of useful features: 6 weeks after mobilisation, and monthly (on demand) thereafter
Number of software defects escaped into production:
0
Internal stakeholder satisfaction:
Consistently rated high
Client satisfaction: Consistently rated high
© HML 2012. All rights reserved.15
ABOUT THE AUTHOR: Andy Lawrence, Lean-Agile Practice Lead
Andy has over 20 years experience in delivering IT based solutions across a
diverse range of industries, and has been leading and coaching software teams in
lean and agile software development for the last decade. He has presented at
conferences and industry events, and regularly contributes to groups such as Agile
Yorkshire and Agile Testing in Finance. Andy joined HML in 2008, initially working
with the Business Intelligence team before joining the IT department to lead the
lean-agile implementation.
Andy has been nominated in the Agile Coach or Mentor of the Year category for UK
Agile Awards 2012.