Cutter · enterprises face today — not issues you were dealing with six months ago, or those that...
Transcript of Cutter · enterprises face today — not issues you were dealing with six months ago, or those that...
The Journal of Information Technology Management
Cutter IT Journal
Vol. 27, No. 2February 2014
Agile Architecture:
Dodo Bird or Differentiator?
Opening Statement
by Sue McKinney . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Coaching Tips for Successful Agile Architecture
by Jason Tice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
How to Agilely Architect an Agile Architecture
by Stephany Bellomo, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya . . . . . . . . . . . . 10
Using an Automation Architecture to Avoid Becoming Less Agile
by John Sweitzer and Christine Draper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Does Agile Architecture = Agile + Architecture?
by Jason Bloomberg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
The Role of Architecture in Agile Development Projects
by John Wooten and Tom Love . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
“With the emergence of best
practices around ‘continuous
deployment’ and ‘continuous
delivery,’ we need architec-
tures to flex, evolve, and
expand to meet business
demands.”
— Sue McKinney,
Guest Editor
NOT FOR DISTRIBUTION
For authorized use, contact
Cutter Consortium:
+1 781 648 8700
Cutter IT Journal®
Cutter Business Technology Council:Rob Austin, Ron Blitstein, Tom DeMarco,Lynne Ellyn, Israel Gat, Vince Kellen,Tim Lister, Lou Mazzucchelli,Ken Orr, and Robert D. Scott
Editor Emeritus: Ed YourdonPublisher: Karen Fine CoburnGroup Publisher: Chris GeneraliManaging Editor: Karen PasleyProduction Editor: Linda M. DiasClient Services: [email protected]
Cutter IT Journal® is published 12 timesa year by Cutter Information LLC,37 Broadway, Suite 1, Arlington, MA02474-5552, USA (Tel: +1 781 6488700; Fax: +1 781 648 8707; Email: [email protected]; Website:www.cutter.com; Twitter: @cuttertweets;Facebook: Cutter Consortium). PrintISSN: 1522-7383; online/electronic ISSN: 1554-5946.
©2014 by Cutter Information LLC. All rights reserved. Cutter IT Journal®is a trademark of Cutter Information LLC.No material in this publication may bereproduced, eaten, or distributed withoutwritten permission from the publisher.Unauthorized reproduction in any form,including photocopying, downloadingelectronic copies, posting on the Internet,image scanning, and faxing is against thelaw. Reprints make an excellent trainingtool. For information about reprints and/or back issues of Cutter Consortiumpublications, call +1 781 648 8700or email [email protected].
Subscription rates are US $485 a yearin North America, US $585 elsewhere,payable to Cutter Information LLC.Reprints, bulk purchases, past issues,and multiple subscription and site licenserates are available on request.
Part of Cutter Consortium’s mission is to
foster debate and dialogue on the business
technology issues challenging enterprises
today, helping organizations leverage IT for
competitive advantage and business success.
Cutter’s philosophy is that most of the issues
that managers face are complex enough to
merit examination that goes beyond simple
pronouncements. Founded in 1987 as
American Programmer by Ed Yourdon,
Cutter IT Journal is one of Cutter’s key
venues for debate.
The monthly Cutter IT Journal and its com-
panion Cutter IT Advisor offer a variety of
perspectives on the issues you’re dealing with
today. Armed with opinion, data, and advice,
you’ll be able to make the best decisions,
employ the best practices, and choose the
right strategies for your organization.
Unlike academic journals, Cutter IT Journal
doesn’t water down or delay its coverage of
timely issues with lengthy peer reviews. Each
month, our expert Guest Editor delivers arti-
cles by internationally known IT practitioners
that include case studies, research findings,
and experience-based opinion on the IT topics
enterprises face today — not issues you were
dealing with six months ago, or those that
are so esoteric you might not ever need to
learn from others’ experiences. No other
journal brings together so many cutting-
edge thinkers or lets them speak so bluntly.
Cutter IT Journal subscribers consider the
Journal a “consultancy in print” and liken
each month’s issue to the impassioned
debates they participate in at the end of
a day at a conference.
Every facet of IT — application integration,
security, portfolio management, and testing,
to name a few — plays a role in the success
or failure of your organization’s IT efforts.
Only Cutter IT Journal and Cutter IT Advisor
deliver a comprehensive treatment of these
critical issues and help you make informed
decisions about the strategies that can
improve IT’s performance.
Cutter IT Journal is unique in that it is written
by IT professionals — people like you who
face the same challenges and are under the
same pressures to get the job done. Cutter
IT Journal brings you frank, honest accounts
of what works, what doesn’t, and why.
Put your IT concerns in a business context.
Discover the best ways to pitch new ideas
to executive management. Ensure the success
of your IT organization in an economy that
encourages outsourcing and intense inter-
national competition. Avoid the common
pitfalls and work smarter while under tighter
constraints. You’ll learn how to do all this and
more when you subscribe to Cutter IT Journal.
About Cutter IT Journal
Cutter IT Journal
Name Title
Company Address
City State/Province ZIP/Postal Code
Email (Be sure to include for weekly Cutter IT Advisor)
Fax to +1 781 648 8707, call +1 781 648 8700, or send email to [email protected]. Mail to Cutter Consortium, 37 Broadway,
Suite 1, Arlington, MA 02474-5552, USA.
SUBSCRIBE TODAY
Request Online LicenseSubscription Rates
For subscription rates for online licenses,
contact us at [email protected] or
+1 781 648 8700.
Start my print subscription to Cutter IT Journal ($485/year; US $585 outside North America)
Within the world of Agile, architecture often seems
misaligned or is a forgotten value-add to a project,
especially in enterprise and large-scale programs.
However, an Agile architecture can enable a business
to deliver features faster and give it a competitive
advantage. Coupled with SaaS and/or cloud, it seems
obvious that Agile architectures are required.
With the emergence of best practices around “continu-
ous deployment” and “continuous delivery,” we need
architectures to flex, evolve, and expand to meet busi-
ness demands. Development organizations must flex to
deliver faster, especially with the increasing adoption
of cloud services. Many questions arise on this topic.
Why haven’t we codified a best practice on Agile archi-
tectures? Do we have a clear definition of what Agile
architecture is? Can value clearly be seen in the product
delivery cycle, and does it ensure higher product qual-
ity? Is an Agile architecture about supporting rapid
scale-out, continuous delivery, and deployment? Or
is it an exercise in futility?
In this edition of Cutter IT Journal, we offer opinions
and perspectives on how an Agile architecture increases
the value teams deliver to their organizations. We also
seek insight on the possible challenges and risks faced
by project teams working within an Agile architecture
framework, as well as specific practices that can ensure
your team’s Agile transformation executes smoothly.
We begin with an article by Agile development coach
Jason Tice, who focuses on mapping architecture prin-
ciples to core Agile best practices. He illustrates key
artifacts required of an “Agile architecture” in a clear
and pragmatic fashion, also highlighting the benefits
aligned with the artifacts. Lastly, Tice articulates the
relationship between Agile architects and product
owners and recommends the use of a kanban board
for architecture governance.
Next up, Stephany Bellomo, Robert Nord, and Ipek
Ozkaya of the Software Engineering Institute (SEI) and
Philippe Kruchten of the University of British Columbia
discuss how to “agilely architect” an Agile architecture.
They characterize Agile architecture as being versatile,
flexible, and easily modifiable, emphasizing loose
coupling and late bindings of objects, creating separations
of concerns, but keeping in mind scalability and deploya-
bility characteristics as “top of mind.” Their comprehen-
sive and pragmatic view of achieving agility within the
constraints of delivery goals is also balanced with an
emphasis on “quality first” as a primary objective.
In our third article, John Sweitzer and Christine
Draper of ThirdWave Insights discuss the critical role
DevOps plays in an Agile architecture. They highlight
the importance of architecture and methods for devel-
oping automation to help teams become more Agile —
and avoid becoming less so. They apply many Lean
principles to eliminate “waste” as teams employ test
automation and also look at how to “optimize the
whole” in the application of continuous test, continuous
delivery, and continuous deployment best practices.
Next, Jason Bloomberg, “chief evangelist” at
EnterpriseWeb, promotes the use of SOA governance
and policies to distribute metadata policy represen-
tations to ESBs, XML, and other policy enforcement
points. He asserts that this level of control actually
creates new levels of flexibility. Once this foundation
is achieved, Bloomberg suggests that cloud governance
can follow the same pattern.
In our last article, John Wooten and Tom Love of
ShouldersCorp discuss methods and practices to
streamline code complexity with the use of abstraction,
loose coupling, and metadata that describes the busi-
ness processes and business rules. According to the
authors, this approach enables teams to reduce “the
number of touchpoints where security and performance
issues can arise.” They take it a step further, asserting
that the use of workflow engines together with a rule
engine will allow a project to create loosely coupled
packages that can be tested separately.
This month’s authors explore a number of approaches
and solutions to Agile architecture. And their vast expe-
rience comes from a variety of areas, from academia to
large corporate development environments to consul-
tancy. We hope their insights will show you how Agile
architecture can increase the value your teams deliver.
Opening Statement
3Get The Cutter Edge free: www.cutter.com Vol. 27, No. 2 CUTTER IT JOURNAL
by Sue McKinney, Guest Editor
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
Throughout the last decade, organizations embracing
the Agile Manifesto1 have achieved significant improve-
ments in streamlining the delivery of software that
provides business value. Although the manifesto calls
developers and IT staff to work together to deliver
“working software” versus “comprehensive documen-
tation” and to “respond to change” versus “following
a plan,” these practices are best supported by a new col-
lection of Agile architecture activities. One of the mani-
festo’s principles declares that “the best architectures,
requirements, and designs emerge from self-organizing
teams,”2 which has prompted some to argue that the
need for IT and software architecture has fallen by the
wayside. While coaching teams in recent years, though,
I have observed firsthand that successful Agile develop-
ment groups have achieved the best results through a
combination of both intentional and emergent architec-
ture activities.
Recently, I was coaching development efforts at an
organization that has sunk costs in specific software
platforms, languages, and systems that constrain its soft-
ware development practices. Due to systems and staff
limitations, this organization supports a limited number
of technology platforms in its production environment.
If Agile development teams build software without
sufficient consideration for these intentional constraints,
they run the risk of building applications that cannot be
deployed to available infrastructure or maintained by
current operations staff. In this case, the organization has
purposely designed the systems architecture to simplify
sustainment by limiting the number of platforms it runs;
however, the software architecture can still emerge on
top of these platform constraints.
When I’m coaching development teams, occasionally
the emergence of the software architecture triggers the
need for architecture change. I once encountered a team
that estimated they could develop custom code for
a new feature in five days, or they could implement
the feature using a third-party library in one day. The
team’s analysis identified an opportunity to change an
“intentional” architecture constraint, and it adopted the
new third-party library. Effective Agile architecture is a
balancing act between establishing technical standards
to promote common patterns for systems/software
development and identifying (via gathered feedback
and metrics) when the adopted standards need to
change.
Agile architecture is required to ensure that software
development between multiple development teams is
well aligned to a shared IT roadmap. Activities to guide
this alignment provide frequent feedback by which the
roadmap can evolve to enable future innovation. Agile
architecture helps organizations optimize their delivery
of working software, allowing development teams to
work smarter (not harder) and ensuring that complex
and costly problems are only solved once.
KEY PRINCIPLES OF AGILE ARCHITECTURE
While all of the principles behind the Agile Manifesto
can be applied to architecture work, Figure 1 depicts
the principles that are most applicable and identifies
practices that support each principle.
Preference for Working Software
To increase their effectiveness, architectural constructs
and guidance should be expressed in mediums that are
both human-readable and machine-executable. From an
Agile architecture perspective, it is preferable to express
the details of how systems integrate using a collection
of application programming interfaces (APIs) accompa-
nied by a suite of automated conformance tests rather
than a written systems integration document. The APIs
capture the information about how systems integrate
while the conformance tests enable automated valida-
tion that each system communicates using its expected
interface.
Preference for Responding to Change
Agile architecture leverages the feedback provided by
ongoing Agile software development to identify future
improvements. Instead of architecture providing one-
way prescriptive guidance and dictating standards for
©2014 Cutter Information LLCCUTTER IT JOURNAL February 20144
Coaching Tips for Successful Agile Architectureby Jason Tice
INTENTIONALLY EMERGENT
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
5Get The Cutter Edge free: www.cutter.com Vol. 27, No. 2 CUTTER IT JOURNAL
software development, teams should assess the ease
of aligning their software development activities to the
architecture’s prescriptive guidance using quantitative
metrics. Observed trends such as increased cycle time
for development of stories, increased build time,
increased code complexity, or increased defects can
indicate that a team is struggling to align to the current
architecture standards/guidance. These trends may
indicate a need for the architecture guidance and/or
standards to be revised, or they may identify another
area where team improvement is possible. Occasionally,
I have worked with teams that are able to capture and
report on these metrics independently, but more often
as a coach, I have helped teams set up measurement
systems to help identify opportunities for architecture
or team improvements.
Agile architecture is characterized by an open and
ongoing dialogue between architects and software
development teams. This dialogue shares information
and generates feedback to enable both architects and
developers to improve their crafts. Effective Agile
architecture employs the use of technical spikes to
perform “experiments” to test out the relevance and
effectiveness of new initiatives. This differs from tradi-
tional architecture, which many times dictates ideas
and best practices without assessing or testing out their
relevance for a particular organization and/or team.
When a development team conducts experiments as
technical spikes, they gain valuable learning about the
best direction in which to move forward based upon
actual feedback versus theory.
Preference for Early and Continuous Delivery
Within the architecture domain, the need for continuous
delivery of working software manifests itself in the
development of systems and enterprise architecture
components in parallel with software components.
Rather than a multi-month-long process to architect
a full system followed by a multi-month-long process
to implement the full system, architecture and imple-
mentation tasks are split into small, releasable chunks.
Just as Agile development teams demonstrate and
deliver working software after each sprint, ideally
an Agile architect will demonstrate and deliver archi-
tecture components at the same cadence. By dividing
large architecture initiatives into small projects and
reusable building blocks, an Agile architect can achieve
a sustainable pace for architecture development, which
results in better-quality work. Furthermore, by working
closely with an Agile architect, development teams are
guided to develop software in such a manner that some
components will align to shared elements of the IT
roadmap. This enables other teams to utilize these
shared components to streamline their development
activities.
Preference for Self-Organization and Ongoing Collaboration
Agile architects should be engaged with development
teams and projects throughout their lifecycle. Effective
architecture work cannot be done by designing every-
thing up front, or by capturing all the lessons learned
at the end of the project. Rather, architects should work
and collaborate with the rest of the team on a daily
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
Figure 1 — Key principles, practices, and benefits of Agile architecture.
©2014 Cutter Information LLCCUTTER IT JOURNAL February 20146
basis. This ongoing and open collaboration provides the
best feedback as to whether the architecture is enabling
the team to move forward or is holding them back.
Despite the importance of Agile development teams
collaborating with an architect, there isn’t a standard
way to include an architect in an Agile team’s work.
I have coached teams that have the architect attend
the team’s daily stand-ups, planning meetings, sprint
demos, and retrospectives and engage in user accep-
tance testing. Other teams have been successful with
having the architect work as an embedded member of
the team, contributing to the team’s development and
test activities while also doing architecture work. Since
the people on each Agile team are unique, and these
people determine the best way for the team to work,
I recommend that teams discuss how they want to inte-
grate an architect, recognizing that what worked for
a prior team may not work on their current team.
The Agile Manifesto reminds us that successful Agile
teams should be “self-organizing.” This is especially
true when it comes to determining the best way for
an Agile team to interface with an architect.
Preference for Simplicity and Reflection
While the Agile Manifesto challenges software develop-
ers to “maximize the amount of work not done,” I like to
apply this idea to architecture by designing the simplest
software or system that can possibly work. The features
and prerequisite architecture of the system are deter-
mined by examining current and near-term business
needs. Rather than complicating the system by attempt-
ing to design for features that may be needed five years
down the road (which usually are never needed), Agile
architecture limits design to immediate and relevant
business needs. This practice constrains the complexity
of the system and results in simplified designs.
Agile architecture gives preference to reflection upon
real experiences versus speculation on what the future
may bring. Just as Agile development teams engage
in frequent retrospectives to discuss challenges and
how to overcome them, Agile architecture applies the
same practice to assessing how development activities
align with the architecture’s guidance and standards.
Observations and data collected regarding alignment
activities provide the best means for identifying archi-
tecture changes that can offer the greatest benefits to
future software development.
ARTIFACTS OF AGILE ARCHITECTURE
Agile architecture encompasses developing and main-
taining a collection of artifacts that promote a shared
understanding of success and guide successful out-
comes among Agile development teams. Figure 2
highlights the key artifacts that I believe provide
the greatest value to Agile development teams. These
artifacts should be clear and concise to promote under-
standing by all. When possible, artifacts should express
information in an automated/executable fashion so as
to augment existing Agile team test capabilities and
promote improved software assurance. Specific artifact
examples are outlined below.
Conformance Test Suites
A key duty of Agile architecture is guiding the design
of system and service interfaces (also called APIs),
seeking to simplify API design to enable greater reuse.
Not only does this design activity produce the API
specification, it should also result in the creation of an
automated conformance test suite for each API. A con-
formance test suite provides a mechanism to confirm
that a system or service is communicating in accordance
with expected parameters. Teams developing services
to support a specific API can use the conformance test
as an additional source of test criteria to confirm their
software meets its requirements. Teams consuming
an API developed by someone else can use the confor-
mance test to confirm that data is being exchanged as
expected and be alerted to any changes in the API that
could impact their software.
Automation of conformance tests is key, as it significantly
reduces the amount of time required to troubleshoot inte-
gration issues. It can also allow components of a large sys-
tem to be built by different teams in parallel and then be
integrated with less risk and rework.
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
Figure 2 — Key artifacts of Agile architecture.
7Get The Cutter Edge free: www.cutter.com Vol. 27, No. 2 CUTTER IT JOURNAL
Service-Level Agreement Tests
Whether acknowledged or not, all systems have service-
level agreements (SLAs), such as page or service response
time when the system is under anticipated high load.
These SLAs must be met for the system to be considered
a success. In fall 2013, the problematic launch of the US
health insurance marketplace (www.healthcare.gov) pro-
vided a refresher on this concept for every Web developer
out there, as having feedback from automated SLA tests
could have produced a different initial outcome.
Agile architecture seeks to define such SLAs, gather-
ing sufficient details from both business and technical
stakeholders and then expressing them in an automated
fashion. The result is an automated performance/stress
test process that includes build, deployment, config-
uration, and test components, whereby a development
team can confirm their software meets required SLAs
at any point in time. By automating this SLA testing,
teams can test at greater frequency (without the need
to slow ongoing development) to ensure their software
meets both functional requirements that stem from user
stories and nonfunctional requirements that originate
in SLAs.
Simple Visual Architecture Models and Common Primitives
In the heat and complexity of ongoing Agile software
development, it’s easy to become focused on the details
of small user stories and lose track of the big picture.
Agile architecture embraces the significant and strategic
value that comes from maintaining a collection of visual
models that depict key components of each system/
service, how key components interact with each other,
and how each system/service relates to the bigger
enterprise picture. Agile architecture models need
not be complicated engineering diagrams, but rather
are communicated at a level of detail that everyone
involved with a project can understand, from software
developers to customers and business stakeholders.
Staff responsible for creating and maintaining these
models calibrate the appropriate level of detail by ask-
ing for feedback from project participants. The visual
models created use primitives (terminology) that also
can be understood by all.
Through my years of coaching, I have worked with
a few Agile teams that questioned the value that
visual models provide and consequently elected to not
develop or maintain them. Not surprisingly, many of
these teams struggled to sustain their motivation and
a strong whole-team ethic. I have observed that teams
and projects that conduct their development activities
while being constantly reminded of “success” through
the use of simple visual models (1) communicate better
with their customers/stakeholders, (2) exhibit greater
motivation toward their work, and (3) consequently are
better equipped to develop high-quality software.
THE “AGILE ARCHITECT” MEETS THE “PRODUCT OWNER”
Many Agile development teams have adopted the
Scrum framework, which prescribes a product owner
role to support each team. In a recent coaching engage-
ment, I was working with a team that was developing
their own product but then was asked to integrate
that product to be part of a larger product line after the
first release. While some enhancements were needed to
the initial release, the majority of the work to be done
for this second release entailed aligning the existing
product to the architecture of the product line. My
stakeholders inquired whether the team still needed
support from their original product owner or could
take direction from an architect instead. Taking the
latter approach would allow the product owner to tran-
sition to another project. This query led me to explore
the different factors that drive the way product owners
and architects make decisions.
The product owner helps to define project success for
the development team, its customers, and business
stakeholders by capturing functional requirements in the
form of user stories, which are then prioritized for devel-
opment by assessing their business value. A top priority
of an Agile architect, on the other hand, is determining
the nonfunctional requirements that apply to the project.
These nonfunctional requirements constitute additional
acceptance criteria for all user stories and are frequently
established by factors other than a team’s direct cus-
tomers, such as domain-specific compliance mandates
(e.g., HIPAA, SEC regulations) or business-driven per-
formance, reliability, and scalability needs. These non-
functional requirements constrain an Agile development
team’s backlog, as such requirements must be met for
user stories to be considered “done.”
Another difference between the product owner and
Agile architect are the paradigms by which they seek to
maximize value. The product owner guides the devel-
opment team to write software that provides the most
business value for the team’s customers. The Agile
architect looks to maximize cost savings and streamline
development. Working together, the product owner and
the Agile architect can prioritize development tasks by
identifying the components that provide the most busi-
ness value with the least amount of effort, complexity,
and risk. This strategy, called Weighted Shortest Job
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
©2014 Cutter Information LLCCUTTER IT JOURNAL February 20148
First (WSJF), streamlines delivery as it accounts for both
business value and complexity/size as software devel-
opment activities are prioritized. Consider two features
that provide equally high business value: one can be
achieved by simply integrating with an existing enter-
prise service, while the other needs to be developed
from the ground up. WSJF would prioritize the simple
integration with the existing service over the more
complex new development.
The duties of the product owner and Agile architect
complement each other. The product owner should
focus on learning the fine details of a team’s business
context in order to write user stories that, when imple-
mented, provide maximum business value. In all envi-
ronments, the Agile architect should help the team
identify common patterns, recommended practices,
and shared code that can streamline software develop-
ment. In a multiteam/enterprise environment, the Agile
architect should look across the organization to identify
building blocks, enterprise services, and common needs
that impact multiple teams. By guiding development
and integration of shared components, the Agile archi-
tect can maximize cost savings and optimize ROI.
While the widespread adoption of the Scrum framework
has demonstrated the value the product owner brings to
an Agile development team, I believe including an Agile
architect as a team member can provide opportunities
for additional cost and time savings to development
teams working in large environments. Ultimately, the
stakeholders who asked me whether they could replace
a team’s product owner with an architect saw the value
of balancing priorities between business needs and non-
functional requirements, as well as streamlining soft-
ware development. In the end, they decided to add a
part-time architect to the development team, while keep-
ing the product owner focused on enhancing the team’s
product.
ARCHITECTURE GOVERNANCE USING KANBAN AS A KEY DIFFERENTIATOR
I suggest that one of the simplest ways to begin a shift
toward effective Agile architecture is to adopt the use
of a kanban board to manage architecture initiatives.
Figure 3 provides an example of a simple architecture
kanban board constructed during a recent coaching
engagement. When using a kanban board, architecture
tasks are summarized on cards, and the position of each
card in one of the lanes on the board represents its sta-
tus. Each card on the architecture kanban board clearly
describes the details and benefits of the architecture
work being completed. A kanban board provides sev-
eral key advantages that support Agile architecture:
n Complete transparency. Effective Agile architecture
cannot be done in a vacuum. By posting all architec-
ture tasks on a kanban board, everyone is made
aware of current and upcoming initiatives. This
enables the entire project team to best coordinate
alignment activities, monitor when architectural
building blocks are available for reuse, and prevent
duplicate efforts.
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
Figure 3 — Agile architecture kanban board.
9Get The Cutter Edge free: www.cutter.com Vol. 27, No. 2 CUTTER IT JOURNAL
n Improved innovation. Staff with relevant experience
and ideas can contribute to architecture development
for initiatives that they learn about by reviewing the
architecture kanban board. Increased opportunities
for collaboration result in improved innovation.
n Collaborative governance. Architecture approval
decisions are rendered when architecture cards
reach well-defined and clearly visible points on the
kanban board. Kanban provides a simple and easy-to-
implement mechanism for advising all project stake-
holders of upcoming architecture decisions as well
as capturing the results of decision-making activities.
Just as kanban promotes increased collaboration
during architecture development, it enables greater
collaboration as decisions are made.
n Flexible staffing for architecture work. Kanban pro-
vides the means by which development to support
architecture initiatives can be shared across teams.
This allows architecture initiatives to be completed
for the benefit of all without having to staff a dedi-
cated team to implement them. Architecture features
are broken into smaller stories and implemented
by development teams alongside their user stories.
The architecture kanban board tracks dependencies
between teams to guide the completion of the com-
ponent stories that comprise the shared architecture
initiative.
Beyond basic implementation, use of a kanban board
to facilitate Agile architecture tasks provides an oppor-
tunity to introduce work-in-progress (WIP) limits for
architecture projects. WIP limits curb excessive ideation
and ensure focus is maintained on completing architec-
ture projects where they can provide benefit to multiple
development teams. WIP limits also provide a mecha-
nism by which an organization can manage how much
investment is set aside for shared components versus
other development activities.
THE SECRET TO SUCCESSFUL AGILE ARCHITECTURE
Agile architecture is a collection of activities all seeking to
reduce the complexity, risk, and cost of software devel-
opment over time. The secret to success involves striking
the right balance between emergent architecture activi-
ties, which are driven by ongoing Agile software devel-
opment and refactoring, and intentional architecture
activities, which reduce fragmentation and simplify sus-
tainment by identifying and guiding the development of
building blocks that result in less new development and
more shared code. When a proper ratio is found, Agile
architecture provides significant benefits to an organiza-
tion by ensuring that complex and costly problems are
only solved once. The ratio of emergent and intentional
architecture activities is unique to every organization
and is largely determined by the personalities and skill
sets of team members in architecture and software
development roles.
In my own coaching experiences, I have found that
every team/program has used a different ratio between
intentional and emergent architecture, and nearly all
these teams have been successful in achieving their
goals. The majority of teams I’ve worked with that
struggled (or even failed) never established an effective
ratio — most focused exclusively on emergent archi-
tecture without ever assessing their code complexity,
which resulted in bloated and difficult-to-maintain
code. Some level of balance between intentional and
emergent architecture activities is critical, as an exces-
sive focus on one type of architecture increases delivery
risks. Too much intentional architecture results in sys-
tems that are too brittle to change, while only focusing
on emergent architecture can increase fragmentation,
complicating integration and reuse.
Agile architecture can help organizations strike this
balance, as it aids all in understanding how each small
development activity contributes to the big picture and
provides shared terminology and visual models that
facilitate effective communication. By providing ongoing
and enduring reminders of the big picture, Agile archi-
tecture enables motivated development teams to support
business agility while minimizing delivery risks.
ENDNOTES
1Agile Manifesto (http://agilemanifesto.org).
2“Principles Behind the Agile Manifesto”
(http://agilemanifesto.org/principles.html).
Jason Tice, MBA, is the VP of Asynchrony, a St. Louis, Missouri-
based Agile software development and enterprise architecture firm.
Mr. Tice has a decade of experience coaching Agile teams and leading
enterprise-level Agile transformations and has pioneered efforts to
leverage the values, principles, and practices of Lean and Agile soft-
ware development to increase the effectiveness of enterprise architec-
ture, cloud computing, Six Sigma, and business process management
initiatives. Mr. Tice is a frequent cohost on the “This Agile Life”
podcast (www.thisAgilelife.com), where effective use of architecture
to scale Agile beyond small teams is a frequent topic of discussion. He
can be reached at [email protected].
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
AGILITY AND ARCHITECTURE
The phrase “Agile architecture” evokes two concepts:
1. A system or software architecture that is versatile,
easy to evolve, and easy to modify, while resilient
enough not to degrade after a few changes.
2. An Agile way to define an architecture, using an
iterative lifecycle, allowing the architectural design
to tactically evolve over time, as the problem and the
constraints are better understood.
The two concepts are not the same: you can have a
non-Agile development process that leads to a flexible,
adaptable architecture, and vice versa, an Agile process
that leads to a rigid and inflexible architecture. One
does not imply the other. In the best of worlds, though,
we’d like to have an Agile process that leads to a flexi-
ble architecture.
The eleventh principle behind the Agile Manifesto1
— “The best architectures, requirements, and designs
emerge from self-organizing teams” — reinforces the
belief among some teams that an architecture will grad-
ually emerge out of applying Agile practices, such as
biweekly refactorings. This thinking was cemented by
mantras such as YAGNI (You Ain’t Gonna Need It) and
No BDUF (No Big Design Up Front), as well as a belief
in deferring decisions to the last responsible moment.
Principle 11, however, is neither prescriptive nor
testable.2
This thinking about the spontaneous emergence of archi-
tecture is reinforced by experiences with IT software
endeavors that do not require a significant amount
of bold new architectural design because (1) the most
important design decisions have been made months ear-
lier, (2) they are fixed by current preexisting conditions,
or (3) they are the result of a de facto architectural setup
in a specific domain. These architectural decisions are
already embodied within the choice of frameworks and
off-the-shelf software packages. The operating system,
servers, programming language, database, middleware,
and other choices are predetermined in the vast majority
of these software development projects, or the project
team has a very narrow range of choices. There is in fact
little significant architectural work left to be done.
Architectural design, when it is really needed because
of a project’s novelty, has an uneasy relationship with
traditional Agile practices. Unlike the functionality of
a system, design cannot easily be decomposed into
small chunks of work, user stories, or “technical
stories.” Most of the difficult aspects of architectural
design are driven by nonfunctional requirements,
or quality attributes: security, high availability, fault
tolerance, interoperability, scalability, and so on. Other
difficulties are driven by quality attributes related to
development itself — such as testability, certification,
and maintainability — which cannot be parceled up
and for which tests are difficult to produce up front.
Key architectural choices cannot be easily retrofitted
on an existing system by means of simple refactorings.
Some of the late decisions may gut out large chunks of
the code, and therefore many of the architectural deci-
sions have to be made early, although not all at once.
Many practitioners have grappled with the issue
of marrying an Agile approach to designing a solid
architecture, such as Cutter Senior Consultant
Alistair Cockburn and his “walking skeleton”3 or
Dean Leffingwell and his colleagues’ Scaled Agile
Framework® (SAFe™).4 Common thinking nowadays
is that architectural design and the gradual building of
the system (that is, its user-visible functionality) must
go hand in hand, but there are several delicate issues:
n How do we pace ourselves?
n How do we make decisions over time in a way
that will lead to a flexible architecture and enable
developers to proceed?
n In which order do we pick the quality attribute
aspects and address them?
The concept of Agile architecture is not new: evolvabil-
ity, software evolution, and reengineering of existing
systems have been studied and understood for a long
time. Indeed, Manny Lehman started this investigation
circa 1980.5 The novel challenge that Agile architecting
©2014 Carnegie Mellon University.CUTTER IT JOURNAL February 201410
How to Agilely Architect an Agile Architectureby Stephany Bellomo, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya
BEYOND YAGNI
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
11Get The Cutter Edge free: www.cutter.com Vol. 27, No. 2 CUTTER IT JOURNAL
brings to system evolvability is the practices that allow
architecting the system in smaller chunks. Successful
teams are those that can take advantage of existing soft-
ware engineering techniques with slight modifications,
in particular those that provide early feedback and
learning, such as prototyping.6
In this article, we present lessons learned about the
characteristics of an Agile architecture that enabled
an organization to develop its architecture in an Agile
manner and continue to rapidly deliver features when
more stringent quality attribute requirements emerged.
To investigate why some teams have fewer delays and
disruptions to continuous delivery of features than oth-
ers, we interviewed project teams from several govern-
ment and commercial organizations. The examples we
present here come from one such project team, which
was challenged when an unexpected performance qual-
ity attribute requirement surfaced in customer feedback
during a user demo. Fortunately, the team was able to
react to the emerging requirements without disrupting
the project. When we examined the team’s practices, we
found they were doing Agile architecting that resulted
in an Agile architecture.
AGILE ARCHITECTING
Agile architecting is not only the process of allocating
architectural work to iterations. There is also a winding
route in which development proceeds as the require-
ments, architecture, and design/implementation are
elaborated iteratively and often concurrently. Early
understanding of requirements and architecture choices
is key to managing large-scale systems and projects;
however, linkage between architecture and implementa-
tion design choices is also crucial.7 While requirements
originating from the problem space inform architecture
and development, explorations originating from archi-
tecture and implementation investigations also assist
in eliciting and detailing requirements. The essence
of Agile architecting is to conduct these activities
concurrently with the right balance.
In Figure 1, we illustrate an example (captured during
our interviews with the team) of such an Agile archi-
tecting process for prototyping with a quality attribute
focus.8 In this example, the team probes for quality
concerns during a user demo, discovers an emerging
performance requirement (the user expected faster
rendering of data-intensive pages than anticipated),
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
Figure 1 — Agile architecting process example.
©2014 Carnegie Mellon University.CUTTER IT JOURNAL February 201412
analyzes the requirement, architects a solution to the
performance problem, implements the solution in a
demonstrable prototype, and gets feedback from the
user at the next user demo. The user accepts the proto-
typed implementation, and the team merges the imple-
mented code into the project baseline for a future
release.
The team operates in three cycles in which develop-
ment is iterated through biweekly sprints; the archi-
tecture and product team manage the releases through
forward-looking architecture-focused investigations;
and a separate R&D cycle investigates long-term strate-
gic technology goals. Figure 1 shows the process steps
as integration points (IPs) between requirements and
architecture or between architecture and design/
implementation:
n IP-1* (Rqmts->Arch): Probe for emerging quality
attribute requirements at user demo.
n IP-2 (Rqmts->Arch): Analyze quality attribute
requirements.
n IP-3 (Rqmts->Arch): Conduct deeper analysis
of emerging quality attribute.
n IP-4 (Rqmts->Arch): Identify architectural
approach/patterns.
n IP-5 (Arch->Design/Imp): Elaborate architectural
patterns for specific system.
n IP-6 (Arch->Design/Imp): Implement portion
of prototyped solution as a spike.
n IP-7 (Rqmts->Arch): Get user feedback on prototype.
n IP-8 (Rqmts->Arch): Analyze feedback on prototype.
n IP-9: Get approval for prototyped changes (merge
with release).
*Note that the team probes for emerging requirements dur-
ing the user demo as part of the requirements elicitation
process, which explains why IP-1 starts there.
Two particularly important practices that helped the
team succeed are prototyping prior to the target sprint
and prototyping in a separate environment. This contin-
uous architecture exploration allowed the team to focus
on making the architecture Agile enough to support
upcoming needs, which helped strike a balance between
too much up-front architecture and not enough.
The example shown in Figure 1 illustrates the proto-
typing process the team uses for a relatively small archi-
tectural change. The team handles these smaller changes
at the sprint/release planning level. For larger-scale
infrastructure improvements targeted at future phases,
the team explained that they often create an R&D proto-
type to begin reasoning about foundational architecture
that may be needed to support emerging functional and
quality attribute requirements (e.g., clustering infra-
structure for the scalability quality attribute require-
ment). The team creates such prototypes in an entirely
separate lab environment, so there is no risk to the
development environment.
AGILE ARCHITECTURE
We found that the team has an Agile architecture that
they described as a “flexible architecture.” This allows
them to explore technical options rapidly with minimal
ripple effect. The architecture can be understood in
terms of patterns and tactics that influence the time
and cost to implement, test, and deploy changes.9
The overarching pattern the team uses to enhance modi-
fiability and control the time and cost of change is the
layered pattern. Other supporting patterns and tactics to
separate interfaces, restrict dependencies, and separate
concerns are employed as well. The team described
their architecture choices as follows:
n Layered architecture, client-server architecture, and
separation of the presentation from business/data
layers. Over time, the architecture and vision have
been implemented as various layers, and components
can be replaced wholesale without significant disrup-
tion elsewhere. The layered architecture supports
adding new client interfaces and replacing presen-
tation components with minimal ripple effect. For
example, the presentation layer was modified to add
a new Web client to an existing smart client imple-
mentation with no effect on the underlying architec-
ture and no down time in operations to introduce the
new interface. The smart client was later replaced
with .NET C# WPF with no impact to the back end.
All three presentation technologies existed simultane-
ously using the same back end.
n Service orientation with separate interfaces and
restricted dependencies. The data layer was built
behind an API service layer to insulate the client
from changes in the data layer. These tactics enable
the team to fence off third-party dependencies by
creating a metadata definition layer (in C#). They
also support replacing the underlying database and
access control without impact to the service-oriented
data layer, thereby allowing the presentation layer to
continue in production without modification.
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
13Get The Cutter Edge free: www.cutter.com Vol. 27, No. 2 CUTTER IT JOURNAL
n Publish-subscribe pattern. This pattern is used to
monitor the data layer schema for changes since the
database is owned and managed by a third party.
The team uses the following patterns and tactics to
improve scalability:
n Clustered architecture with load balancing and
replicated copies. The cluster-based architecture
allows rapid scaling, whether increasing for mission
need or scaling back due to budgetary constraints.
It is possible to scale up by increasing the size of
the virtual servers or to scale out by increasing the
number of clusters, the number of analysts using
the App-V client, or the number of Citrix servers
in the cluster.
n Encapsulation of algorithms. Computationally inten-
sive algorithms, which have a significant impact on
system performance, are encapsulated to allow them
to evolve or be replaced.
n Data caching. A local copy of a subset of data is kept
to optimize performance of data retrieval and data
processing for frequently accessed data.
The team also focuses on flexibility in deployment and
controlling the cost and time for testing by using several
patterns and tactics:
n Virtualization, layering both the infrastructure and
the application. By adopting virtualization at a very
early stage, the system may be hosted in any data
center with the appropriate network connections. For
example, infrastructure (VMware) and the application
(App-V) are virtualized. This enabled the system to
be consolidated from deployment in five regional
centers to two, allowing the customer to reduce costs.
n Standardized and configurable architecture: para-
meterization and static and dynamic binding. Each
cluster has a standard configuration and can be cus-
tomized. This standardization allows installations to
be accomplished at will; for example, the team can
allocate new servers (in the cloud), install the soft-
ware, and replicate joins to scale the system to meet
new needs. Additional processors and memory can
be added to a virtual machine to increase capacity.Similarly, clusters can be removed at will to meet
resource constraints.
n Executable, interface-driven code structure.Executable code structure enables automated testing
of business layer functionality through interfaces.
A solid understanding of the overall architectural struc-
ture helps the team to respond to stakeholder feedback
or learning from spikes (that is, prototyping activities
that are timeboxed10), because they have the architec-
tural knowledge to rapidly analyze the impact of
changes and conduct architecture tradeoff analysis.
INTEGRATING PRACTICES WITHIN INCREMENTAL DEVELOPMENT ENVIRONMENTS
Understanding the desired state of development and
delivery helps tie together Agile architecting and Agile
architecture. A desired software development state is
one that enables Agile teams to quickly deliver releases
that are valuable for stakeholders.11 The teams them-
selves typically define the desired state. It is their vision
of the ideal development infrastructure that they would
like to work with. When product development starts,the desired state does not necessarily exist. Setting up
the initial architecture helps push the team and the
delivery into the desired state and enables Agile
architecting that results in an Agile architecture.
In this example, the team needs both Agile architecting
and architecting for agility to stay within the desired
state. Agile architecting provides the team a regular
cadence for periodically considering whether they are
already out of bounds or about to get out of bounds.
The regular cadence of Agile architecting also helps the
team manage architecture exploration and balance com-
peting concerns of too much or too little architecting.
Agilely architecting an Agile architecture, then, mini-
mally has four key requirements:
1. Focusing on key quality attributes and incorporating
these into technical explorations within prototyping
and spikes
2. Understanding that a successful product is a combi-
nation of not only customer-visible features but also
the underlying infrastructure that enables those;
hence, architectural requirements are incorporated
into sprint planning and demos
3. Recognizing that an Agile architecture that enables
ease of maintainability and evolvability is the result
of ongoing, explicit attention given to the architec-
ture, not a natural byproduct of an Agile — or any —
software development process
4. Continuously managing and synchronizing depen-
dencies between functional and architectural require-
ments and ensuring that the architectural foundation
is put in place in a just-in-time manner
At a high level, the process for Agilely developing an
Agile architecture can be seen as a zipper, as shown in
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
©2014 Carnegie Mellon University.CUTTER IT JOURNAL February 201414
Figure 2. From the requirements, as they evolve, the
designers extract and separate functional requirements
(mostly features seen by the end users) and architec-
tural requirements (mostly derived from key quality
attributes), which are already expressed or anticipated.
Dependencies between these two kinds of requirements
must be managed to ensure that necessary elements of
the architecture are present (or at least “stubbed”) in
upcoming iterations with functional requirements that
depend on them. This skeletal foundation must be
woven into early iterations of architectural and func-
tional increments.
This approach will facilitate a deliberate (not accidental)
emergence of an architecture, constantly validated by
the functionality developed on top of it. The develop-
ment of the architecture occurs over several iterations,
without being stopped, blocked, or slowed down
by developers claiming “YAGNI” or “No BDUF.”However, as when a zipper gets out of alignment, caus-
ing it to get stuck, teams that do not pay close attention
to evolving dependencies can get caught off guard with-
out the architectural foundation they need to support
emerging requirements. Agility and architecture do
support each other very well.
ACKNOWLEDGMENTS
We would like to acknowledge Sphere of Influence
for their technical contributions to this article. This
material is based upon work funded and supported by
the Department of Defense under Contract No. FA8721-
05-C-0003 with Carnegie Mellon University for the
operation of the Software Engineering Institute, a
federally funded research and development center.
ENDNOTES
1“Principles Behind the Agile Manifesto”
(http://agilemanifesto.org/principles.html).
2Séguin, Normand, Guy Tremblay, and Houda Bagane.“Agile Principles as Software Engineering Principles: An
Analysis.” Lecture Notes in Business Information Processing,
Vol. 111, edited by Claes Wohlin. Springer, 2012.
3Cockburn, Alistair. “Walking Skeleton” (http://
alistair.cockburn.us/Walking+skeleton).
4Scaled Agile Framework® (http://scaledAgileframework.com).
5Lehman, Meir M. “Programs, Lifecycles, and Laws of Software
Evolution.” Proceedings of the IEEE (Special Issue on Software
Engineering), Vol. 68, No. 9, September 1980.
6Ozkaya, Ipek, Robert L. Nord, Stephany Bellomo, and Heidi
Brayer. “Beyond Scrum + XP: Agile Architecture Practice.”Cutter IT Journal, Vol. 26, No. 6, 2013.
7Nuseibeh, Bashar. “Weaving the Software Development
Process Between Requirements and Architectures.” IEEE
Computer, Vol. 34, No. 3, March 2001.
8Bellomo, Stephany, Robert L. Nord, and Ipek Ozkaya.
“Elaboration on an Integrated Architecture and Requirement
Practice: Prototyping with Quality Attribute Focus.” Proceedings
of the Second International Workshop on the Twin Peaks of
Requirements and Architecture. IEEE, 2013.
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
Figure 2 — The zipper model.
15Get The Cutter Edge free: www.cutter.com Vol. 27, No. 2 CUTTER IT JOURNAL
9Bass, Len, Paul Clements, and Rick Kazman. Software
Architecture in Practice. 3rd edition. Addison-Wesley
Professional, 2012.
10Leffingwell, Dean. Agile Software Requirements: Lean
Requirements Practices for Teams, Programs, and the Enterprise.
Addison-Wesley Professional, 2011.
11Bachmann, Felix, Robert L. Nord, and Ipek Ozkaya.“Architectural Tactics to Support Rapid and Agile Stability.”CrossTalk, May/June 2012.
Stephany Bellomo is a senior member of the technical staff at
the Carnegie Mellon University (CMU) Software Engineering
Institute (SEI). Ms. Bellomo is a member of the Architecture
Practices group and an active member of the Value-Driven
Incremental Development research team. She teaches SEI courses in
Service-Oriented Architecture Migration of Legacy Components and
Software Architecture Principles and Practice. Ms. Bellomo has over
15 years’ experience in the software field. Prior to joining the SEI, she
worked as a software engineer for several organizations, including
Lockheed Martin, Intuit, and VeriSign Network Solutions. She served
as tutorial chair for SEI’s SATURN Conference and is currently a
member of the organizing committee for the International Workshop
on Release Engineering 2014 (hosted by Google). Ms. Bellomo
received an MS in software engineering from the George Mason
University. She can be reached at [email protected].
Philippe Kruchten is a professor of software engineering at the
University of British Columbia (UBC), in Vancouver, Canada,
where he holds an NSERC Chair in Design Engineering. Dr.
Kruchten joined UBC in 2004 after a 30-plus-year career in industry,
where he worked in large software-intensive systems design in the
domains of telecommunications, defense, aerospace, and transpor-
tation. Some of his experience is embodied in the Rational Unified
Process, whose development he directed from 1995 to 2003. His cur-
rent research interests reside mostly with software architecture, in
particular architectural decisions and the decision process, as well as
software engineering processes, especially the application of Agile
processes in large and globally distributed teams. Dr. Kruchten
teaches courses in entrepreneurship, software project management,
and design. He is a senior member of IEEE Computer Society; an
IEEE Certified Software Development Professional; a member of
ACM, INCOSE, and CEEA; the founder of Agile Vancouver; and a
professional engineer in British Columbia. Dr. Kruchten has a diploma
in mechanical engineering from Ecole Centrale de Lyon and a doctor-
ate degree in information systems from Ecole Nationale Supérieure des
Télécommunications in Paris. He can be reached at [email protected].
Robert L. Nord is a senior member of the technical staff at the SEI.
Dr. Nord is engaged in activities focusing on Agile and architecting
at scale and works to develop and communicate effective methods
and practices for software architecture. His collaboration with
Philippe Kruchten and Ipek Ozkaya is helping shape the research
agenda on technical debt. He is coauthor of the practitioner-oriented
books Applied Software Architecture and Documenting Software
Architectures: Views and Beyond and lectures on architecture-
centric approaches. Dr. Nord is a member of the steering committee of
the WICSA Conference series, in addition to organizing events at
software engineering, Agile, and architecture venues. He earned a
PhD in computer science from CMU and is a distinguished member
of the ACM. He can be reached at [email protected].
Ipek Ozkaya is a senior member of the technical staff at the SEI. With
her team, Dr. Ozkaya works to help organizations improve their soft-
ware development efficiency and system evolution. Her work focuses
on software architecture practices, software economics, and require-
ments management, and her latest publications include articles on
Agile architecting, dependency management, and architectural techni-
cal debt. Dr. Ozkaya also chairs the advisory board of IEEE Software
and serves as an adjunct faculty member for the Master of Software
Engineering Program at CMU. She also organizes different events
(tutorials, workshops, and sessions) and is an invited speaker at
software engineering, Agile, and architecture venues (e.g., ICSE,
OOPSLA, SATURN, and WICSA). She holds a doctorate from
CMU. She can be reached at [email protected].
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
AGILE: LAYING WASTE TO WASTE
A passion for eliminating waste is one of the ingre-
dients that bonds the members of an Agile software
development team. Software developers are inclined
to leverage technology to eliminate any of the seven
wastes of software development identified by Mary
and Tom Poppendieck:1
1. Partially done work
2. Extra process
3. Extra features
4. Task switching
5. Waiting
6. Motion
7. Defects
This inclination manifests itself in automating as much
of what the team does as possible. Automating tasks
frees up programming resources so the team can
deliver user stories that result in value to the business.
These investments make teams more efficient, but —
ironically — they have the potential to make more
mature Agile development teams less Agile! In this
article, we discuss the circumstances under which this
phenomenon occurs and how to avoid it.
ELIMINATING WASTE BY AUTOMATING
Automation encodes the team’s practices so they can
be performed at machine speed, with fewer errors
and with far fewer human resources. Creating automa-
tion is a programming activity, so producing reliable
automation requires designing it (determining what
scripts need to be run and how you will know the
automation works), coding it (writing scripts), and test-
ing it (verifying that the automation results are reliable).
The ROI for automation is a function of the amount of
resources freed up after spending resources to create
and maintain the automation. Resources are freed up
each time the automation is used instead of a human
resource.
For example, let’s say a task performed 12 times each
iteration can be automated so it can be done in four
minutes instead of an hour. The investment required
to deliver reliable automation is 40 hours (five person-
days). This investment frees up about 11 hours per
iteration, so the break-even point will occur in four
iterations. However, if the ongoing maintenance is:
n 4 hours, then it will take 6 iterations [40/(11-4)]
to break even
n 8 hours, then it will take 13 iterations [40/(11-8)]
to break even
n 11 hours or more, then it will never break even
The maintenance cost is a critical part of the investment
equation, since it can quickly erode the ROI, and it con-
sumes a percentage of the programming capacity each
iteration.
“Automation drain” refers to the situation in which
teams with a large set of automation assets are required
to allocate an increasing percentage of their program-
ming resources to updating/maintaining the automa-
tion. By focusing on the effect automation has on the
“programming capacity” of each iteration, one can see
that teams become less Agile when:
n They delay important or needed user stories because
the capacity to update affected automation is not
available
n The amount of programming spent on user stories is
reduced because more programming resources are
spent maintaining (updating) automation
That’s right — the investment made in automation to
free up programming resources can actually become a
significant consumer of programming resources in later
iterations.
Some additional factors that contribute to this
situation are:
©2014 Cutter Information LLCCUTTER IT JOURNAL February 201416
Using an Automation Architecture to Avoid Becoming Less Agileby John Sweitzer and Christine Draper
THE “WHOLE” TRUTH
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
17Get The Cutter Edge free: www.cutter.com Vol. 27, No. 2 CUTTER IT JOURNAL
n Automation dilemma. The team needs to automate
to be efficient. There is more to automate than the
team can afford. How should they choose which
automation to work on?
n Decreasing ROI from automation. Automation proj-
ects start by automating simple tasks, then eliminate
variances that make tasks overly complex, and are
finally left to automate tasks with lots of inherent
variability. The ROI decreases as they move through
this progression.
n Temporary automation gains. Much like perfor-
mance improvements in software, the positive impact
of new automation fades when it becomes the new
norm. Say that automating a task reduces the cycle
time from a human speed of 8 hours to a machine
speed of only 30 minutes. This frees up 7.5 hours of
resources. The team adjusts the way they work to
take advantage of that improvement, and eventually
they are running through the cycle twice a day rather
than once a week (i.e., 10 times more often). At this
point, the 30 minutes is almost as much of a bottle-
neck as the original manual task (5 hours of delay
over a week).
As the amount of automation increases and the software
solution evolves in response to user feedback and busi-
ness requirements, changing these encoded practices
can consume programming resources that would have
been spent on user stories. Each iteration will use pro-
gramming resources to automate new aspects of the
system and to rewrite automation that will no longer
work due to the system’s evolution. Since programming
skills are needed to design, code, and test automation,
spikes in automation rewrites compete for user story
resources.
CREATING WASTE BY AUTOMATING
The business impact of automation drain is more signifi-
cant when efforts to automate are actually creating waste.
Partially Done Work
Automation that does not contribute to or result in a
complete cycle creates waste. In this context, a complete
cycle is something like a continuous test cycle. For a
continuous test cycle to be complete, one needs to auto-
mate the deployment of a system topology with the test
harness, to automate test cases, and to automate verifi-
cation of the test cases. The work is only partially done
when any of the three aspects of automation is not
in place.
Automating the Unknown
When one encodes practices in software, the result
is more reliable and less expensive if the author has
intimate knowledge of how to do the work being auto-
mated. Automating something that is being done for the
first time results in a lot of rework, so the initial invest-
ment and maintenance costs are high.
Automating deployment of a piece of software that
has only been installed once or automating unit tests
for newly committed code is a form of waste if the
automation needs to be rewritten many times before
it is executed repeatedly. To be efficient and effective,
automation should encode something that is well
understood and isn’t expected to change much.
Consider the following scenario. A developer is work-
ing on a user story that is expected to take five days
to design, code, and unit test. The developer mentally
allocates 20% of his or her time to perform unit test
and decides to use that time to automate unit test cases.
Almost 7 of the 8 hours allocated to unit test is spent
automating a relatively small number of unit test cases,
since the code for the automation and the code for the
user story need to be debugged. So only 5% of the code
paths are actually tested instead of the 40% that would
have been covered if 6 hours were spent unit testing
and 2 hours were spent automating a fewer number
of unit test cases for the more challenging paths.2
Scenarios like these create waste, especially if the user
stories in the next iteration change the code some more.
At minimum, the quality of the code is less than it could
have been. The focus needs to be on automating what
will make a difference, not just automating the simple
stuff (which likely isn’t going to fail) in order to have
a large number of automated test cases to run.
Inventory Exceeds Capacity
Having automated tests or automated deployment
scripts that do not get used regularly can be classified
as excess inventory waste. Having 1,000 automated
function test cases and only the capacity to execute
750 of them means the effort put into the other 250 is
wasted unless the team introduces a rotation strategy
(which is new work).
EMPOWERING TEAMS WITH AN AUTOMATION ARCHITECTURE
To empower a team (or teams, in the case of some
DevOps structures) to make good decisions about
what to automate, an architecture is needed to provide
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
©2014 Cutter Information LLCCUTTER IT JOURNAL February 201418
a line of sight into ways to minimize automation drain
on programming resources. An architecture defines the
manner in which parts or components are arranged and
integrated in a system. Effective use of an architecture
focuses the team members on issues that need to be
understood to empower the best contributions.
An automation architecture covers a system in which
tasks normally performed by an individual are dele-
gated to a computer so they can be executed at machine
speed without human errors. This is an architecture for
the software that helps the team deliver its software sys-
tem, rather than the software system itself. The proper
application of an automation architecture adds value
by increasing the agility (measured in terms of pro-
gramming capacity per iteration) of the team.
Automating Whole Subsystems Frees Up Programming Resources
The application lifecycle consists of five continuous
cycles for integration, test, delivery, deployment, and
load balancing (see Figure 1). All these continuous
cycles need to be delivered to realize the full benefits
of automation.
Each continuous cycle requires context-specific build,
deploy, and test processes to be integrated into an
autonomous subsystem, as shown in Figure 2. For
example, the continuous integration subsystem needs to:
1. Execute the process to create the runtime artifacts
(aka “build the code”)
2. Execute the process to deploy those artifacts into a
system so they can run (aka “deploy to build test
server”)
3. Execute the process to test the running system to ver-
ify it meets expectations before declaring it is ready
for developers to use (aka “build verification test”)
The goal of continuous integration is to provide devel-
opers with an updated code base for their work that
doesn’t impact what they are doing in a negative way.
Therefore, continuous integration doesn’t complete until
the verification test process is executed successfully (or
at least successfully enough).
Focusing on delivering one or more autonomous
subsystems that can execute between builds (i.e.,
between executions of the continuous integration sub-
system) increases the likelihood that the automation
investments will be beneficial. Figure 3 illustrates this
point by showing three test subsystems delivering dif-
ferent types of testing and running in a pipeline in a
20-hour period between builds. All these subsystems
need to complete execution before the next build so they
are ready to start again. The interval between builds
creates a timebox that helps the team decide what to
automate next; for example, automating more test cases
versus improving the deployment automation.
Each of the three subsystems shown in Figure 3 takes
hours to deploy the needed topology and hours to run
the actual tests and will only just complete within the
20-hour window. Automating more test cases than can
be run within that window would be waste. Improving
the deployment automation to increase the window
for automated test cases would make more sense.
When that window is increased, automating more
tests becomes a useful thing to do. These are examples
of the tradeoffs the team is able to make within the
framework provided by an automation architecture.
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
OperationTest
ContinuousIntegration
Development
Integrate, test, deliver, deploy, balance as an unbroken whole without interruption
ContinuousTest
ContinuousDelivery
ContinuousDeployment
ContinuousLoad Balance
Automate buildand verification
Automate testenvironment setup
and testing
Automate deploymentof various staging
environments/topologies
Automate deploymentof various production
environments/topologies
Automate startingand stopping resources
Application Management Lifecycle
Figure 1 — Continuous cycles within the application management lifecycle.
19Get The Cutter Edge free: www.cutter.com Vol. 27, No. 2 CUTTER IT JOURNALNOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
Build Process
Built Artifacts
Deployment Process
Test Process
ProposedSystem
ValidateSystem
Build Scripts andInstructions
Deployment Scriptsand Instructions
Test Scripts andInstructions
Source Code
Machines
...
Integrate, test, deliver, deploy, balanceas an unbroken whole without interruption
Continuous...
Figure 2 — The “build-deploy-test” processes of a continuous cycle.
Friday’sBuild
2 hoursto test
4 hoursto deploy
7 hoursto test
3 hoursto deploy
2 hoursto test
Build Acceptance Test
FunctionTest
Build N
2 hours to deploy
SystemTest
RegressionTestContinuous
Test
Automate testenvironment
setup and testing
Monday’sBuild
Tuesday’sBuild
Thursday’sBuild
Wednesday’sBuild
20 hours 20 hours 20 hours 20 hours
Figure 3 — Test automation is only valuable if it completes within a window between builds.
©2014 Cutter Information LLCCUTTER IT JOURNAL February 201420
It is important for automation to deliver something that
is “whole.” When coding, a common root cause of user
stories that are not whole is that they are implemented
“left-to-right.” For example, a five-step wizard is imple-
mented by doing the first two steps in iteration 1, the
next two steps in iteration 2, and the last step in itera-
tion 5. The user story is not whole until the fifth step
is implemented. The same is true in the “build-deploy-
test” processes in the continuous cycles. Until there
is enough automation across the entire cycle, the team
is not delivering a whole. A common occurrence is to
automate the build and test processes while deferring
the deployment process. In this case, the middle step
breaks the continuous flow.
Over time, there can be one or more autonomous sub-
systems for the continuous cycles in the application
lifecycle (see Figure 1) that could be timeboxed in the
interval between builds. So there are two different
approaches for developing subsystems:
1. Perform as many tests as possible to understand
how new user stories are impacting the overall
stability/quality of the code. Figure 3 shows this
approach, with three different test subsystems
pipelined together.
2. Flow through as much of the application lifecycle
(build to production) as possible. Figure 4 shows
the creation of a pipeline involving subsystems
for continuous test, continuous delivery, and
continuous deploy.
As the pipeline grows, it will be more difficult to
make it fit within the build window. An approach
that addresses this is to enable subsystems to be run
in parallel, as shown in Figure 5.
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
Continuous Test
Continuous Delivery
Continuous Deploy
Figure 4 — DevOps creates a pipeline of continuous processes.
Build Process
BuiltArtifacts
Build Scripts and Directions
Source Code
Deployment Process Test Process
Proposed System
Validate System
Deployment Scriptsand Directions
Test Scriptsand Directions
Deployment Scriptsand Directions
Test Scriptsand Directions
Deployment Process Test Process
Deployment Process Test Process
Proposed System
Validate System
Proposed System
Validate System
Deployment Scriptsand Directions
Test Scriptsand Directions
Figure 5 — Executing subsystems in parallel.
21Get The Cutter Edge free: www.cutter.com Vol. 27, No. 2 CUTTER IT JOURNAL
Supporting Multiple Topologies Enables Agility
Agile teams embrace change when dealing with user
feedback. When an Agile team also embraces a DevOps
culture, they have an improved line of sight to how
their code is deployed in production. Under these cir-
cumstances, there is a tendency to want to create a sin-
gle topology across the application lifecycle. There are a
couple of typical motivations for this:
n The same process (automation scripts) can be used
to deploy the test environment and the production
environment. Most automation investment streams
go through a phase in which teams attempt to make
environments the same so the same automation can
be used.
n Testing code in an environment that is equivalent to
the production environment increases the likelihood
that the code will work in production.
Once a team has standardized scripts across topologies,
they can be reluctant to allow variations to occur. This
reluctance can make the team less Agile if they defer
supporting additional topologies. There are many situ-
ations in which different topologies are necessary and
beneficial. This is especially true for multitier applica-
tions delivered in a global market. For multitier appli-
cation systems to scale globally, they must deal with
variances for language, culture, regulation, preferred
practices, and so on. These applications will require
different topologies and/or configurations. Some other
common reasons for multiple topologies are:
n Developers are interested in topologies that put as
much of the software on their development machine
as possible.
n Function test environments may include the mini-
mum required software elements in order to reduce
the time it takes to deploy an environment.
n To handle workload and increased availability, the
production system will deploy tiers across multiple
machines. The broader the context of use (e.g., the
market for the product), the more likely it is that dif-
ferent production topologies will be needed to match
variation in expected workloads.
The desire to standardize topologies and keep them
standardized has the potential to create an inflexible
team, since they will not learn to handle the variability
that arises when they do need to support multiple dif-
ferent topologies.
A better approach is to explicitly identify the desired
variability and use this to architect an appropriate
structure and reuse strategy for the automation. This
approach is illustrated in Figure 6, which shows a sys-
tem with seven different topology variations. Across
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
Development Environment (Topology)
Integration Test Environment (Topology)
Staging Environment(Topology)
Production Environments(Topologies)
Tools: Application Components:
Infrastructure:
8 deployable components used in the deployment topologies:
10 stacks/machines used in the deployment topologies:
Numerous environments/topologies throughout the application lifecycle:
System Test Environment (Topology)
Figure 6 — Desired variability across topologies provides an architecture for the automation structure.
©2014 Cutter Information LLCCUTTER IT JOURNAL February 201422
those topologies, there are eight distinct deployable
components: two representing tool variations, three
representing modules of the application to be deployed
in different configurations, and three representing
infrastructure (e.g., prerequisite, machine, or operating
system) variations.
These eight deployable components define an appro-
priate granularity for architecting the structure and
reusing the automation. Creating automation with a
coarser-grained structure (e.g., per machine) results
in automation that is inflexible and costly to maintain.
In this example, creating automation aligned with the
deployable components results in eight scripts, one for
each deployable component; a change to a deployable
component typically requires a change to only one script.
Contrast this with the per-machine approach, which
would result in 10 automation scripts (one for each
unique combination of deployable components shown
in the topologies in Figure 6), and where a change to a
deployable component would require changes in the
majority of the scripts. Investment in finer-grained
automation than this may result in additional costs due
to supporting a larger number of reusable components
and writing additional coordination logic.
Delaying Refactoring Avoids Waste
Delaying refactoring avoids waste, as it allows the team
to learn what is needed rather than guessing. However,
“little and often” (i.e., incrementally refactoring) is still
a good idea. When a need arises to break apart a mono-
lithic script, automatically jumping to breaking up the
whole thing into atomic tasks can dramatically increase
refactoring cost and destabilize the system. It is also dan-
gerous to delay refactoring to the point where the team
is creating complexity to work around the monolithic
nature of the script (e.g., complex configuration and con-
ditional paths in the script to support needed variability).
Monolithic scripts are perfectly acceptable for sections
of topologies that are the same and that are not chang-
ing much. This observation suggests it is better to start
out with monolithic scripts and then refactor them as
variations in the topologies are discovered.
When authoring deployment scripts, the desire to
“do the simple first” and to “maximize reuse” usually
results in an approach that creates fine-grained auto-
mation scripts. For a robust multitier application, this
can easily result in hundreds of scripts. While this
approach can yield decent reuse numbers for the
scripts, the expected productivity gain is offset by the
need to author additional scripts to properly sequence
the 80-120 tasks required to install and configure a
multitier application. It often makes sense to start
with fine-grained automation of simple repetitive tasks,
but as the amount of automation grows, these need to
be combined into higher-level automation architected
to align with the system structure and variability, as
discussed in the previous section.
Minimizing Pillar-Oriented RolesCreates More Automation Capacity
From iteration to iteration, the amount of different types
of work needed to complete the allocated user stories
varies. Teams are more Agile when they have the capa-
bility to deal with this fluctuation without delaying
desired user stories.
Figure 7 shows an example of variations in work type
across 10 two-week iterations with 900 person-hours of
capacity (approximately 12 headcount). It shows how
the resource allocation by skill can vary significantly
from iteration to iteration.
Accommodating the skill mix variance in Figure 7
requires a team of multiskilled individuals. Figure 8
shows two 12-person teams with different skill mixes.
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
Amount of Work Performed by Type
900 pds
1 2 3 4 5 6 7 98 10Iteration:
Skill Type:User Interface
Database
Business Logic
Testing
Automation
Figure 7 — Example of work type variation across iterations.
23Get The Cutter Edge free: www.cutter.com Vol. 27, No. 2 CUTTER IT JOURNAL
Team A is staffed with specialists, while Team B is
mostly staffed with individuals with multiple skills.
Team A has a maximum capacity of 70 person-hours
for developing automation, but Team B has a maximum
capacity of 420 person-hours. In Team A, some impor-
tant user stories may have to be delayed until the con-
strained capacity can be addressed.
Teams that formalize roles like system administrator or
that consist of specialists from multiple disciplines (e.g.,
a database administrator assigned to work on multiple
projects part-time) tend to have more situations in
which constrained capacity delays user stories. In con-
trast, a team with many multiskilled individuals has
the flexibility to absorb these work type variations and
deliver user stories in the desired order. Such a team
may also gain additional benefits from the continuity
that results when one individual has the skills to com-
plete a “whole” piece of work, rather than having to
hand it off to a specialist.
TAKEAWAYS
Agile teams rally around the theme of “eliminating
waste.” DevOps teams rally around the theme of
“breaking down organizational pillars.” In both,
freeing up resources to do work that delivers value
to users/customers is paramount. The benefits (and lim-
itations) of the practices that result from these themes
need to be understood to execute the practices success-
fully. In this article, we have focused on one aspect:
applying automation. Automation is an obvious way
to free up resources and increase agility, because it
replaces the need for humans to perform certain repeti-
tive tasks, it is not a constrained resource in the way
that headcount is, and it typically executes tasks faster
and more repeatably than humans can.
To apply automation successfully, it is important to
understand its costs as well as its benefits. Creating
automation is a programming activity since the auto-
mation needs to be designed, coded, and tested. After
enjoying the benefits of automating the low-hanging
fruit, the desire to automate more and the reality that
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
Person Skills
AutomationUser Interface
Business Logic
DatabaseSkill Type:
Testing
Skills Maximum Capacityfor Iteration
Person-Hours
210
210
210
210
70
490
420
560
350
420
Team
A
Team
B
Person Skills
Skills Maximum Capacityfor Iteration
Person-Hours
Figure 8 — Skill mix examples.
©2014 Cutter Information LLCCUTTER IT JOURNAL February 201424
existing automation needs to be maintained start
to drain the programming capacity of the team. Auto-
mation can also become a barrier to change and create
bottlenecks due to constrained resources with the
skill to code the automation. These problems can be
mitigated by:
n Prioritizing automation investments using an
automation architecture that integrates build,
deploy, and test processes into a “whole” subsystem
that can continuously run between daily builds
n Identifying the necessary variability in system
topologies and using this to architect appropriate
automation structure and reuse strategies
n Delaying refactoring until it is needed to avoid waste
n Building a team that has the flexibility to absorb
peaks in automation work by ensuring that “auto-
mating” does not become a specialist role (like system
administrator) on the team
The perspectives we have shared here are the results
of our experience in building Agile development teams
and our careers in IT systems management. Of course,
all these recommendations and insights are situational;
their potential benefit is only possible if they are
applied under the right circumstances. This is one of
the real challenges of leading an Agile team — what
is “best” to do (the amount you test, the amount you
automate, the way you staff the team) will change as the
project evolves. But if approached with the particular
needs and situation of the team in mind, the strategies
we’ve outlined here can help ensure that automation is
a gain — not a drain — for your Agile team.
ENDNOTES
1Poppendieck, Mary, and Tom Poppendieck. Lean Software
Development: An Agile Toolkit. Addison-Wesley Professional,
2003.
2Individuals on the team should be encouraged to leverage tech-
nology in their workspace to be more productive. Automating
some unit test cases quickly is OK. It is then important to dis-
tinguish these from the test cases used in the automation sys-
tem for the team as a whole — those that need to be formally
managed, etc. This distinction is important since developers
may not formally test the automated unit test cases they throw
together. However, if these are part of the overall automation
system, they need to be tested more formally to ensure the
results are accurate (no false positives or false negatives).
John Sweitzer is the President and cofounder of ThirdWave Insights,
LLC, a software product business. Mr. Sweitzer was an IBM
Distinguished Engineer and a member of the IBM Academy of
Technology, with broad experience in all aspects of software devel-
opment (including leading Agile development teams) and cross-
organizational design practices. Mr. Sweitzer was the Chief Architect
for IBM’s Autonomic Computing initiative and the Chief Architect for
IBM’s Tivoli division. He coauthored the book Outside-in Software
Development, which describes easy-to-understand principles for
delivering business-relevant software that any software development
team can apply without special training. During his 31-year IBM
career, Mr. Sweitzer received nine technical contribution awards,
filed 20 patents, published two books, and wrote numerous papers.
He can be reached at [email protected].
Christine Draper is Vice President and cofounder of ThirdWave
Insights, LLC. She has 20 years’ experience in the IT industry in a
range of technical leadership roles. Ms. Draper was a Senior Technical
Staff Member at IBM. In this role, she led several software initiatives
in different areas of systems management and software development.
She can be reached at [email protected].
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
There is a growing buzz in the blogosphere that agile
architecture means combining Agile software method-
ologies with software architecture best practices. So,
is that what agile architecture is all about?
Perhaps, but that’s not the whole story. We also need
to reinvent the practice of enterprise architecture (EA)
to achieve business agility in the enterprise. Software
plays an important role, but agile architecture isn’t
really about software. It’s about the people in the
organization.
Agile methodologies like Scrum and XP also focus on
people, as the Agile Manifesto essentially calls for a
more human-focused software development approach.
Still, there’s no arguing that the Agile Manifesto is
about building software. Yet regardless of how Agile
your methodology for building software is, that doesn’t
mean the software you end up with will actually make
your organization more agile.
And there’s the rub: if what you mean by “Agile archi-
tecture” is software architecture compatible with Agile
development methodologies, then you’ll likely end up
with higher-quality software, but it won’t be any better
able to support the organization’s agility requirements
than any other software. For that you need a new archi-
tecture paradigm.
THE ENTERPRISE CONTEXT FOR AGILE METHODOLOGIES
When most people outside the development organi-
zation think of an Agile development team, they envi-
sion a close-knit gaggle of coders frantically sprinting
toward the next release date. Sure, proper Agile incor-
porates stakeholder input, encourages quality, and,
done properly, delivers better software than traditional
waterfall-driven projects. But there’s no question that
this notion of Agile — Agile with a capital A — retains
its skunkworks feel.
Skunkworks-driven bespoke software projects, however,
are rarely the norm in enterprise development shops.
For every greenfield software project, there are a dozen
efforts that leverage existing capabilities in one way or
another, either tweaking functionality on an ongoing
basis or as part of a more strategic consolidation or
modernization effort. Even those few blank-sheet-of-
paper apps must still play nicely in the complicated
distributed environment we call “enterprise IT.”
Further complicating the role Agile plays in the enter-
prise is the fact that once an Agile project is done, it’s
done. Put it into production and move on to the next
project. Stakeholder requirements, though, have an
annoyingly persistent way of continuing to evolve.
Now our skunkworks-built software that eschewed
extraneous capabilities and only focused on the specific
requirements du jour turns out to be annoyingly inflexi-
ble. The bottom line: building software in an Agile way
doesn’t make the organization any more agile.
Enter Agile software architecture, which often goes by
the name Agile architecture, but I’ll include “software”
in the phrase to distinguish it from the agile EA I dis-
cuss in my book The Agile Architecture Revolution. Agile
software architecture takes into account the fact that
enterprise software must participate in a distributed
environment, and distributed environments require
system architecture to set them up properly and keep
them running smoothly. Clearly, if your bit of code
must talk to a database over here and a Web server
over there, then a bit of architecture goes a long way.
Agile software architecture also calls for architects to
consider change cases. While traditional, skunkworks-
style Agile focuses on building precisely to the require-
ments as gathered during iterative conversations with
stakeholders, the notion of change cases allows for the
fact that those same stakeholders typically require some
level of flexibility in their software after deployment,
so it’s important for the developers to build just a bit
beyond today’s requirements to allow for this flexibility,
all while avoiding the dreaded specter of overbuilding.
The challenge with change cases, of course, is knowing
when to stop. Just how much flexibility should you
build into your software, anyway, and how do you
know when you’re done? You can’t really expect the
stakeholders to answer those questions, as even they
25Get The Cutter Edge free: www.cutter.com Vol. 27, No. 2 CUTTER IT JOURNAL
Does Agile Architecture = Agile + Architecture?by Jason Bloomberg
THAT’S SO META
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
©2014 Cutter Information LLCCUTTER IT JOURNAL February 201426
don’t know what they’ll want your software to do in the
future. Therein lies the core limitation of Agile software
architecture.
Agile software architecture helps the heads-down
Agile development team to pop their heads up from
their screens, look around the cube farm for once, and
incorporate a measure of systems and software archi-
tecture into their methodology. From the perspective
of the greater enterprise, however, the software team’s
focus is still far too narrow and inflexible. Software
alone — even well-architected software — can’t make
the organization more agile.
THE ENTERPRISE “POWER TOOL” PROBLEM
The CIO has bigger problems to worry about. Intractable
legacy challenges. Mobile devices in everyone’s pocket.
Then there’s the cloud. On the one hand, new applica-
tions, increasingly powerful tools, and rising stakeholder
expectations — all within the context of constrained
budgets — lead the IT executive to focus on doing more
with less. On the other hand, the complex, brittle mess
of legacy continues to loom like some giant spider in a
Middle Earth lair.
Service-oriented architecture (SOA) was intended
to solve this rat’s nest legacy problem by abstracting
legacy application and data assets as business services
that facilitate flexible processes. It sounds wonderful in
theory, but in practice many IT shops got bogged down
in the middleware, and even for those architecture-
savvy organizations that actually got loosely coupled
services up and running, SOA governance issues came
to the fore, creating a new set of challenges.
Done properly, SOA shifts the enterprise legacy focus
from integration to governance: get the services right,
and the problem areas shift from connecting systems
to how to consume, compose, manage, and evolve those
services to meet ever-changing business needs. I call
this situation the “power tool” problem: the more pow-
erful your tools, the more dangerous they are, and thus
the more important it is that people know how to use
them properly. Just as you wouldn’t give power tools
to children, you don’t want to release sophisticated
business services to an unsuspecting user base.
Today the power tool problem crops up across the IT
portfolio. Bring-your-own-device (BYOD) challenges fall
into this category, as the rank and file bring increasingly
powerful and diverse mobile devices to work. Think cir-
cular saws in the hands of eight-year-olds: the problem
isn’t getting the technology to work properly. The chal-
lenge shifts to creating, communicating, and enforcing
policies for the use and management of all devices in a
consistent, coordinated fashion that empowers users,
instead of limiting their productivity or effectiveness.
In other words, governance.
Perhaps the most salient example of the power tool
problem, however, is the cloud itself. By automating
the operational environment, the cloud pushes the con-
trol over provisioning, management, and consumption
of IT resources to the user, separating control from
responsibility for the operational environment. The
cloud provider is responsible for making sure every-
thing is up and running, but the user maintains control.
Without proper governance, however, even the most
sophisticated cloud-based app can easily go off the rails
— even though the cloud service provider has done its
job properly.
NEXT-GENERATION GOVERNANCE
If you rolled your eyes when I emphasized governance,
you’re not alone. There’s no question that governance
initiatives have historically delivered decidedly mixed
results, especially in IT departments. Governance brings
to mind excessive rules, policies, and procedures, where
some pointy-haired middle manager somewhere creates
arbitrary hoops that everyone must now jump through
to get their jobs done.
This old-fashioned, Big Brother approach to governance
lowers morale and productivity, and in extreme cases
can even lead to a “black market” for IT resources, as
people find ways around excessively onerous policies
and procedures. Clearly, Big Brother doesn’t meet our
needs — and as the power tool problem grows, so too
does our need to reinvent governance.
Fortunately, we can find elements of the solution to
Big Brother governance in SOA and the cloud. If we
can first get the automated SOA and cloud governance
to work properly, and second, extend the core princi-
ples of this governance to the IT organization and the
enterprise at large, then we’ll have the glue that ties the
Agile architecture story together.
At the center of SOA governance (and by extension,
cloud governance) is the ability to represent policies
in a metadata format. In fact, perhaps the strongest
members of the Web services family of standards are
WS-Policy, WS-SecurityPolicy, and a handful of others.
Using these or other standards, it’s possible to represent
a policy — say, a security or content-based routing pol-
icy — in an XML-based format that policy management
tools can create and manage while policy enforcement
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
27Get The Cutter Edge free: www.cutter.com Vol. 27, No. 2 CUTTER IT JOURNAL
tools can tackle enforcement in a fully automated
manner.
The goal of automated SOA governance is to centralize
the creation and management of SOA policies using a
model-driven tool, and then to distribute those meta-
data policy representations to enterprise service buses
(ESBs), XML appliances, management tools, and all the
other policy enforcement points. It’s difficult and expen-
sive, but if you can get this policy management infra-
structure to work properly, you will have immense
control over your SOA deployment as well as previ-
ously unattainable levels of flexibility.
Automated cloud governance follows the same pattern,
although JSON-formatted policy representations are
more the norm than the XML of the SOA world. In
either case, you can control the behavior of the infra-
structure by controlling the policies. Code only the
behavior that doesn’t change often. Shift all dynamic
behavior to the metadata-driven policy layer for auto-
mated communication and enforcement.
In other words, apply the declarative programming
model that separates logic from control flow to
the enterprise distributed computing environment.
Declarative approaches have been with us for years —
SQL and HTML are two of the most familiar declarative
languages — but translating the declarative model to a
fully distributed environment is the key to reinventing
governance for the 21st century, or what I like to call
“next-generation governance.”
THINKING AT THE META LEVEL
So you represent policies as metadata. You create them
in a visual modeling tool then automatically commu-
nicate them to enforcement points. And you control
the behavior of your infrastructure by managing and
updating your policies.
Where in this set of processes is the governance? The
answer: in all of it, of course. Creating policies, manag-
ing policies, communicating them, and enforcing them
— all these processes constitute governance. What,
then, are our policies for doing governance? Or to put it
another way, when the architecture team sits down and
figures out how to set up these automated governance
policies and procedures, what policies do they follow
for doing so?
I call these higher-level policies metapolicies. A metapol-
icy is a policy that applies to policies — that is, a policy
for doing governance. The lower-level policies are the
specific security, routing, management, and other poli-
cies we represent as metadata. But doing governance
means establishing, communicating, and enforcing
the metapolicies.
It’s important to separate our discussions of policies and
metapolicies because the next-generation governance
approach allows us to change our policies. In fact, the
power of next-generation governance lies in how it
enables us to deal with change. The power tool prob-
lem, after all, is really all about change — changing
behavior of our technology over time, as well as sup-
porting different uses of the same technology across
different constituencies.
Thinking at the meta level, in fact, gives us a better
way of dealing with change across our enterprise,
beyond the scope of governance. In particular, the
Agile Manifesto itself isn’t truly a software develop-
ment methodology. It’s a metamethodology; that is, it’s a
methodology for creating and changing methodologies.
The most important of the four tenets that make up
the Agile Manifesto is responding to change over follow-
ing a plan.1 So, what if your plan is to follow an Agile
methodology like Scrum? Then to be truly Agile, you
must we willing to change that plan. Any development
manager who holds up some Scrum book and pontifi-
cates that the only way to do Scrum is to dogmatically
follow the instructions in the book fundamentally
doesn’t understand what Agile is all about.
The meta level comes up again when we consider busi-
ness agility as a requirement for our technology envi-
ronment. Stakeholders don’t simply want software
that meets some particular set of fixed requirements.
Today’s stakeholders want technology that makes them
more agile — they want their organization to be better
able to respond to change in the business environment
as well as to introduce change intentionally in order to
achieve a strategic advantage. And they’re looking to
their technology to help them achieve this agility.
In other words, business agility is a metarequirement:
a requirement that applies to other requirements.
Stakeholders always have specific functional and
nonfunctional requirements for any piece of technology.
But now they are also saying “make us more agile” —
build technology that inherently responds to changing
requirements.
It sounds like a tall order. Tell developers to code some
software that will respond to changing requirements,
without knowing what those requirements will be
— oh, and furthermore, do so without having to go
back and change the code — and they’ll look at you as
though you’re insane. Add Agile software architecture
to the mix, and you still don’t have a clue how to build
software that’s inherently agile.
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
©2014 Cutter Information LLCCUTTER IT JOURNAL February 201428
Fortunately, we know how to deliver such technology,
since after all, that’s the point of next-generation gover-
nance. By shifting the behavior of our technology to the
metadata level, we’re empowering our organizations
to leverage the technology in different ways to meet
diverse and changing needs — without having to mon-
key with the underlying software. In order to get next-
generation governance to work, though, we must think
at the meta level, which requires agile EA.
SLICING ACROSS THE SILOS
Enterprise architecture complements software and
systems architecture in large part because of its added
focus on process and organizational challenges. Gov-
ernance, therefore, is clearly in EA’s wheelhouse, as
next-generation governance involves both process and
organizational change. To truly take the agile EA per-
spective, however, it’s essential to tackle the big picture
of organizational change that moving to next-generation
governance–driven IT necessitates.
Traditional IT shops — and by extension, traditional
enterprises — generally follow siloed organizational
models. This vertical divide-and-conquer approach to
any organization is often at the root of its inflexibility.
Data silos limit the organization’s ability to achieve a
single view of the truth. Application silos lead to rat’s
nests of integration. And when organizational decision
making falls into departmental or divisional silos,
companies end up with complexity, duplication, excess
costs, and all the other woes that characterize today’s
large organizations.
The organizational core of Agile cuts across these silos,
encouraging horizontal organizational models. First,
Agile leaders want to bring quality assurance and
development together into every iteration. Software
improves. Next, they bring the stakeholder onto the
dev/QA team. More improvement. Then they automate
the operational environment, shifting the role of the
ops personnel, and add them to the team as well. Now
we have DevOps.
For DevOps to achieve its goals of continuous delivery
of high-quality software that meets diverse, changing
business needs, however, it’s essential to have both
next-generation governance in place as well as an EA
that focuses on the meta level. Statically scoped projects
are an artifact of siloed organizational models. Agile
EA, in contrast, requires all members of the team to
work at the meta level. Operations personnel work on
the cloud provisioning and management recipes that
support change in the cloud environment. Developers
focus on metadata-handling code in support of a declar-
ative programming model. Quality assurance becomes
a management-centric, full-lifecycle endeavor as the QA
team focuses on guaranteeing sufficient agility.
The stakeholders — the business users, managers, and
the customers themselves — must play their roles as
well. We as technology consumers must understand
and leverage this new paradigm for IT, where we have
many increasingly powerful tools at our disposal; the
automated, policy-driven governance infrastructure
both constrains and empowers our behavior as we do
our jobs. The Web-enabled world of consumer technol-
ogy leads the way, as we navigate through an increas-
ingly interconnected mélange of social media–enabled
apps. Enterprise IT isn’t far behind, as the cloud, mobile
technologies, and agile enterprise architecture finally
break down the silos of legacy technology and legacy
business.
ENDNOTE
1Agile Manifesto (http://agilemanifesto.org).
Jason Bloomberg is Chief Evangelist at EnterpriseWeb, where he
drives the message and the community for EnterpriseWeb’s next-
generation enterprise platform. He is a global thought leader in the
areas of cloud computing, enterprise architecture, and service-oriented
architecture. He is a frequent conference speaker and prolific writer,
and he also serves as blogger for DevX. His latest book, The Agile
Architecture Revolution: How Cloud Computing, REST-based
SOA, and Mobile Computing are Changing Enterprise IT, was
published in 2013.
Prior to joining EnterpriseWeb, Mr. Bloomberg was President of
ZapThink, where he created the Licensed ZapThink Architect (LZA)
SOA course (and associated credential) and taught the LZA course as
well as his Enterprise Cloud Computing course around the world. He
was also the primary contributor to the ZapFlash newsletter and blog
for 12 years. Mr. Bloomberg is one of the original Managing Partners
of ZapThink LLC, the leading SOA advisory and analysis firm, which
was acquired by Dovel Technologies in August 2011.
Mr. Bloomberg’s book Service Orient or Be Doomed! How Service
Orientation Will Change Your Business, which he coauthored with
Ron Schmelzer, is recognized as the leading business book on service
orientation. He also coauthored the books XML and Web Services
Unleashed and Web Page Scripting Techniques.
Mr. Bloomberg has a diverse background in e-business technology
management and industry analysis, including serving as a Senior
Analyst in IDC’s eBusiness Advisory group, as well as holding
e-business management positions at USWeb/CKS (later marchFIRST)
and WaveBend Solutions (now Hitachi Consulting). He can be
reached at [email protected].
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
Developing a successful application or system using
Agile development can be very challenging. The
process mandates that we not create a full-blown set
of specifications, but rather work closely with knowl-
edgeable users in short sprints wherein small incre-
ments of functionality are created, vetted, tested, and
released. While we don’t have detailed specs or com-
plete requirements, we should have a “big animal pic-
ture” of what needs to be developed. The lack of a big
picture will be a major stumbling block in the process.
Similarly, such projects usually begin without good sys-
tems engineering information such as projected number
of simultaneous users, information bandwidth require-
ments, security requirements, and so forth. How, then,
can one successfully use the Agile development para-
digm and expect to handle these unknown require-
ments when they are encountered later in the project?
At ShouldersCorp, we have led 19 large-scale Agile
development projects that had most of the above
unknowns and more. These projects were all success-
ful because we approached the problems as we will
describe in the rest of this article, building upon a flexi-
ble, data-driven architecture that is loosely coupled
and can be easily adapted to handle new requirements,
expanded user numbers, and changes in business
processes, rules, and security requirements. We start by
discussing some design principles that help in defining
how such an architecture can work and then outline
how the pieces fit together.
ABSTRACTION
Abstraction is one of the principle tenets of object-
oriented (OO) design and one that many software
development projects ignore. Much of what is
expressed below is obvious to experienced OO design-
ers but might be overlooked in Agile development. For
instance, if a “3GL” is used to develop an application,
it presents the developer with a screen-oriented view
of the developing system. As each screen is built, it
usually involves grouping some fields into a form.
Once these are defined, the system builds an object
that can be persisted to encompass the fields. This
object becomes a table in the database. Each form tends
to create a new table. Since these tables contain many of
the same fields, many database-stored procedures have
to be created to keep the values “synchronized.” This is
definitely not abstraction! As consultants, we have been
asked to review many software projects where we have
seen thousands of tables and associated stored proce-
dures supporting a system that consists of a relatively
small number of actual “objects.”1
When this is so, you must ask, “What is the ‘object’
I should be representing?” It’s actually pretty simple.
Can you draw a picture of the “objects” that are asso-
ciated with your project? Well-defined objects are
easy to draw. Poorly defined objects are impossible
to visualize. Our all-time favorite from many years
ago was a class in a Smalltalk application named
“ProcessClaimFormEntityManager.” The word
“Manager” itself is often a clue to an area where addi-
tional design or architecture is likely to be required.
An object should be a noun, and it should represent
something concrete. We have seen many instances
in which tables for Customer, Client, Clinician,
Technician, Provider, Insurer, and many others are
represented as objects and associated tables, and it is
true that each of these is a noun and could be repre-
sented as separate objects. However, these all represent
people or an abstracted idea of a Person that might
be called an Associate: something that has a “name,”
a “role,” an “address,” an “email address,” and per-
haps a “phone.” A Person extends this and adds a
”gender,” “birthDate,” and “dependents,” which is a
list of Persons. An Organization extends Associate and
perhaps contains a “taxId” and a “legal representative,”
which would be an Associate (i.e., Person).
By reducing the number of classes or objects through
abstraction, the volume of code is reduced and, conse-
quently, the number of touchpoints where security and
performance issues can arise. Later we discuss how
these abstractions are used to simplify the architecture,
reduce the amount of testing, and improve scalability
and performance. However, you’ll want to watch that
29Get The Cutter Edge free: www.cutter.com Vol. 27, No. 2 CUTTER IT JOURNAL
The Role of Architecture in Agile Development Projectsby John Wooten and Tom Love
OO, BABY
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
©2014 Cutter Information LLCCUTTER IT JOURNAL February 201430
inheritance isn’t being overused. In some cases, these
abstractions can be handled through compositing
instead of inheritance. Generally, inheritance is best
if the object is not being directly persisted, while com-
positing is best for persistence. However, the idea is
to provide for the variety of person-like entities through
a small set of Associates with particular roles such
as “Customer,” “Client,” “Clinician,” “Technician,”
“Provider,” or “Insurer.” Queries can be created to
select from the Person table all that have the role
“Client” to get the clients, for instance. Now we have
reduced the number of objects and associated tables
from six to one or two and the number of stored pro-
cedures greatly. By using the concept of “Actor,”
Associates can have multiple roles (i.e., John acting as
Client, John acting as Technician). Now the attributes
of the John object are in one place (e.g., address, email,
phone), while the person can be grouped together with
other Clients or Technicians.
To create the proper classes and abstractions, create
a list of the nouns associated with the project before
beginning to code. Don’t forget to draw a picture
of each! Then step through the list to determine
which could be subclasses of others or which can be
abstracted. Those become the base classes with their
associated tables. Doing this at the beginning may
reduce the volume of the code by a factor of 5 to 10.
As the base classes are created, look at how refactoring
can reduce the volume of code through reuse of methods
for instantiating, persisting, updating, and managing the
objects. Since industry data shows that there is at least
one “bug” for every 1,000 lines of code, less code is
always better code! Refactor at every opportunity to
reuse the same methods where appropriate. When
using the model-view-control (MVC) design pattern,
use inheritance for the view and control classes because
they are not usually persisted, while the model classes
may be, and thus are candidates for compositing.
LOOSE COUPLING
After the base classes have been designed, coded, and
tested, the control aspect of the usual MVC design is
begun. Here it is important to consider how movements
between forms and their associated persistence in the
data store is accomplished, as well as how operational
business rules and different presentations of the data
will be managed.
Of primary importance in this phase is to remember
that we want our design to be resilient to change, and
there will be change! What if the organization changes
database providers? What about changes in the flow
between screens, or the business logic that determines
whether a form is acceptable, or the way a transition to
another form or screen is done? How do we handle this
without constant changes in coding, with associated
new errors and testing?
It is a useful process to sort the list of classes (or class-
responsibility-collaboration [CRC] cards) based upon
the likelihood of change. Then carefully reconsider the
classes that are most likely to change and see if you can
find a way to “factor out the flux.” An example is if the
workflow, or screen navigation, within an application is
likely to change, then factor it out of the code into a file
or database so that changes to workflow can be accom-
plished without code modifications.
We can factor out much of the flux by loosely coupling
different aspects of the system. Objects should not know
how they are being persisted, or what the rules are for
persistence or presentation. By coupling subsections of
the system or packages through a “notification” system,
messages can be sent notifying the rules subsystem that
objects must be validated, or that those validated objects
need to be persisted. Figure 1 shows an example of such
a system.
WORKFLOW
In order to achieve agility and handle changing require-
ments, the architecture of an Agile project needs to fac-
tor out change. For example, in Figure 1, several items
appear in the top row representing metadata stored
external to the actual code. These can be tables in the
database or XML files available to the program. By cre-
ating, buying, or obtaining a workflow engine from an
open source project, one can achieve a huge reuse of
code. A business process is always started using the
same piece of the workflow engine. It is advanced to
the next state by another piece of the same engine. The
choices, or outcomes, that the user has to make on the
screen is data that can be provided through the work-
flow engine for each screen or “state” of the process, as
can the name of the form and what data or objects are
required to be filled in and saved. Now, hundreds of
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
In order to achieve agility and handle
changing requirements, the architecture of
an Agile project needs to factor out change.
31Get The Cutter Edge free: www.cutter.com Vol. 27, No. 2 CUTTER IT JOURNAL
different business processes are all managed by the
same workflow engine.
Many workflow engines provide mechanisms to create,
edit, and view the business processes by graphical
means. This allows subject matter experts (SMEs) to
actually create the business processes using terms they
understand. The results are stored in database tables
or as XML files for the processes. After the SMEs create
the processes, Web designers or graphical experts can
create the associated presentation layer and insert the
screen names into the workflow XML files as well. The
same XML files can be the basis for the process and
role-based documentation of the system by using simple
XSLT transforms. Now the workflow engine can deliver
the name of the presentation file, the data, and the name
of the business rules to be applied to the data before
transitioning to the next state via a Web service.
BUSINESS RULES
In an Agile project, as a part of factoring out the change,
your architecture should allow business rules to be
determined through discussions with the SMEs. Again,
by obtaining a general-purpose business rule engine,
you achieve flexibility and reduction in code volume.
Some business rule engines provide graphical tools for
writing and testing rules. These rules are again stored
externally to the program in database tables or XML
files. Changes in the rules do not require recompiling
and distributing the code base. They should be simple
rules that are easy to read and easy to combine using
Boolean logic. Don’t write hundreds of specialized
rules; instead write very simple rules and combine
them. These rules are read in from the rule storage as
XML or from a list of rules or a database, and they are
applied by the rule engine when it gets the notification
that an object is being persisted. Rules should be easy
to change and can be edited and created by a non-
programmer. A flexible Agile architecture takes advan-
tage of workflow engines and rule engines as a way
to reduce risk, improve scalability, and reduce code
volume.
SERVICES
When designing an application that is to be distributed
and that you wish to be flexible, scalable, and secure,
then part of the architecture should involve creating
services at the level of your business processes. As the
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
Web Service Application
RegulatoryData
WorkQueues
RouteMap
RuleSets
Work-flow
Layout& XML
MessagingConfig
OtherSystems
Database Adapter
TOPLinkAdapter
Data EJB
DataIOAdapter
Archiver
Router
Coercion
Expeditor
EvaluatorEJBs
Regulator
ActionEJBs
QueuingPolicy
ProcessScheduler
Navigator
HTMLGenerator
XMLParser
Binder
Adapter
XMLProcessor
Notifier
AppServlet
SessionBean
SecurityBean
Directed Graph Analyzer
RegulatorGraph
ExpressionsGraph
Finite StateGraph
Each stack
is an EJB or
Session Bean
WebPage
Application Server
Figure 1 — Architecture showing the role of messaging in achieving loose coupling.The arrows between the stacks are messages creating actions.
©2014 Cutter Information LLCCUTTER IT JOURNAL February 201432
business processes begin to mature, it will be clear what
services are needed. A process to enroll a new employee,
for instance, involves getting the personal information
about the person, saving that information, assigning that
person to a position in the organization, creating that
person’s payroll information, gathering information for
his or her benefits, and so on. At each stage in the
process, certain sets of information must be gathered
and persisted. Certain checks need to be made for
uniqueness and to validate the new employee’s informa-
tion. These each represent examples of useful services.
In addition, the workflow engine and rule engine can
be represented as services. They provide mechanisms
to select a business process, start that process, advance
to the next state using the outcome of the previous one,
enforce the rules during each transition, and determine
what organizational rule should service a particular
state in the process.
TESTING
When designed and used this way in an Agile project,
each of the loosely coupled packages can be separately
tested. This allows the various parts of the architecture
to be designed, developed, and tested in parallel and
concurrently. You do not have to know what the busi-
ness process is in order to create the workflow engine
and make it available as a service. The same is true for
the rule engine, notification layer, persistence layer,
and adapters to other systems. Since they communicate
through messages, these can be “mocked up” for early
testing. Other portions of the application can be tested
to be certain that they produce the proper messages
even in the absence (during testing) of the receivers of
those messages.
CONCLUSION
An Agile development project can benefit from an
architecture such as the one described above since many
parts can be designed and developed in parallel and
tested without dependencies. The use of abstraction,
loose coupling, and externalized services, along with
metadata describing the business processes and busi-
ness rules as well as the communication messages,
allows rapid sprints and more flexibility in the face
of change. Such architectures enforce security at the
exposed portions of the services as well as provide cen-
tralized logging of messages and service notifications.
Combined with the proper level of abstraction in a test-
driven process that utilizes continuous refactoring and
testing, these architectures allow development to pro-
ceed in a truly Agile fashion.
And Agile development should be just that: agile! Thus,
heavy process, requirements gathering, critical design
reviews, and other systems development lifecycle con-
cepts have to be revisited and refined. The same goes
for large team meetings, governance debates, and end-
less discussions of earned value, to say nothing of using
the guesses of team leaders instead of objective metrics
to determine project status. While oversight is neces-
sary, it should be minimized. This becomes possible if
you have addressed the issues associated with architec-
ture agility from the start.
Without such a base architecture, and without proper
abstraction, code volume and complexity can balloon
rapidly, along with way too many tables and associated
stored procedures. Such an ill-planned development
project will appear to be moving very rapidly at first,
but it will become bogged down and intractable as it
approaches the 80% completion phase.
While our approach to architecture and design may
not be the only way, it has worked in 19 successful
Agile development projects ranging from financial risk
management websites to a metals order management
system, a travel planning application, a very large-scale
Internet payroll application, a system for tracking and
managing border-crossing transactions, and a visual
tool for managing complex shared medical terminology.
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
While oversight is necessary, it should be
minimized. This becomes possible if you
have addressed the issues associated with
architecture agility from the start.
33Get The Cutter Edge free: www.cutter.com Vol. 27, No. 2 CUTTER IT JOURNAL
All these projects used the same fundamental principles
described above and the same services and code base
for workflow, rules, screen management, persistence,
and messaging without any stored procedures. Oh, and
by the way, one of the above is allowing over 30 million
users to safely access detailed records they wish to
keep private.
ENDNOTE
1While it has become commonplace to refer to “objects,” of
course what we are really talking about is classes, which define
the characteristics of the “objects of that class.” Classes inherit
from other classes.
In 1997, John Wooten teamed up with Tom Love to found
ShouldersCorp, where he serves as Chief Architect and Lead
Developer for Agile development projects. Prior to forming
ShouldersCorp, Dr. Wooten was a computational physicist at the
Oak Ridge National Laboratory, where he did pioneering work in
several different areas, including 3D neutral beam focusing simula-
tions, 3D ion heating, and 3D and plasma stability simulations. He
also created a programmable HP calculator–like tool that used arrays
and matrices along with visualization tools for data analysis of
Tokamak and Stellarator data channels. Many of these projects
involved large-scale computer code and evolved toward object-oriented
concepts involving teams he managed according to what is now called
Agile development.
Dr. Wooten has a PhD in theoretical physics from the University of
Nebraska. He was also the Principal Investigator for the Department
of Energy Oak Ridge Educational Network (OREN), a program that
led the US state of Tennessee in extending the Internet to regional
schools, teachers, and students. He can be reached at jwooten@
shoulderscorp.com
In 1997, Tom Love teamed up with John Wooten to found
ShouldersCorp. The firm has done major projects for commercial
and government customers, including the US Departments of
Defense, Veterans Affairs (VA), and Human Services. ShouldersCorp
is currently managing the replacement of all hardware and software
systems at all intensive care units in over 100 VA medical centers
throughout the country. Since ShouldersCorp’s founding, Dr. Love
has led numerous Agile development projects, all of which have deliv-
ered on time, on spec, and on budget.
Prior to forming ShouldersCorp, Dr. Love held management positions
at GE, ITT, IBM, and Morgan Stanley. He also was a founder of
PPI/Stepstone, OrgWare, and Worldstreet. Dr. Love has a PhD
in cognitive science from the University of Washington. He is the
author of the 1993 book Object Lessons. He serves on the Board of
Visitors for Carnegie Mellon University’s Software Engineering
Institute. He can be reached at [email protected].
NOT FOR DISTRIBUTION • For authorized use, contact
Cutter Consortium: +1 781 648 8700 • [email protected]
CUTTER CONSORTIUM
In just 4 days (or less!) with Cutter Senior
Consultants Hubert Smits and Lynn Winterboer,
you’ll have not only the skills and confidence
you need to ensure success, but you’ll also
have satisfied the training requirements you
need to become CSM and/or CSPO-certified
by the Scrum Alliance.
CSM Certification:Scrum for Professionals
with Hubert Smits | March 31-April 1 Greenwood Village, Colorado
In the 2 days of this class you will learn
Scrum-by-the-book (the Agile Atlas and the
Scrum Guide) and then explore how the Scrum
concepts fit in real-world projects — in your
projects, with the challenges and limitations
that your organization and your customers
put around a software development project.
What You’ll Learn:
This class is based on the recently published
Agile Atlas (Scrum Alliance), which describes
the concepts of Core Scrum and forms the
basis for the Certified ScrumMaster (CSM) test.
With this class under your belt you will pass
the final CSM test — many have done this
before you!
What You’ll Get:
n Top-notch training from the experts
n CSM certification (after passing online exam)
n Coffee breaks and lunches
n Workbooks and other materials
n Supporting content from the Cutter archives
n Planning Poker card deck
CSPO Certification: AgileAnalytics for Product Owners
with Hubert Smits and Lynn Winterboer 2-3 April | Greenwood Village, Colorado
In this two-day course, Cutter Senior
Consultant Lynn Winterboer, a seasoned data
professional and Agile analytics trainer, and
Cutter Senior Consultant Hubert Smits will
provide you with an understanding of the
Product Owner role as defined by the Scrum
Alliance — specifically as it relates to the chal-
lenges often faced by business intelligence (BI)
Product Owners.
What You’ll Learn:
This class will show you how to shape the
initial vision for a program and establish an
Agile roadmap for program success. You’ll use
ACCESS TO THE EXPERTS
CSM Certification: Scrum Training andAgile Analytics for Product Owners
Cutter Consortium 37 Broadway, Arlington, MA 02474, USAwww.cutter.com
@cuttertweets
Facebook.com/CutterConsortium Register at www.cutter.com/events.html
March 31-April 3
9:00 am - 5:00 pm daily
Marriot Courtyard
Denver Tech Center
6565 South Boston Street
Greenwood Village, CO
Cutter Consortium 37 Broadway, Suite 1, Arlington, MA 02474-5552, USA Tel: +1 781 648 8700; Fax: +1 781 648 8707; www.cutter.com; [email protected]
Innovation Games for ideation, convergence,
and prioritization, and learn how to write
strong DW/BI user stories. You’ll also explore
how to slice BI/DW development into small
chunks of business value; identify and under-
stand users, their roles, and personas; and
leverage use cases to develop strong user
stories. At the end of this course, you’ll be able
to demonstrate working DW/BI features to the
stakeholder community.
Certification
Participants who attend the 2-day Certified
ScrumMaster course fulfill the training require-
ments necessary to take the online CSM test.
After passing the test and agreeing to the
license terms, you’ll have earned your CSM!
The CSM course also satisfies two elements
of the Certified Scrum Developer (CSD) track:
Scrum Introduction and Elective.
Participants who attend the 2-day CSM course
(or have previously earned their CSM), and
also attend the Certified Scrum Product Owner
course will have met the training requirement
for their CSPO certification at the conclusion of
the 4 days!
About Your Instructors
Hubert Smits is a Senior Consultant with
Cutter Consortium’s Agile Product & Project
Management practice. He is an innovative,
assertive, and goal-oriented Agile consultant,
coach, and trainer with a successful track
record of spearheading enterprise-level Agile
and Lean enablement efforts. Mr. Smits is a
Scrum Alliance Certified Scrum Trainer. He
served as the lead consultant at Cisco Voice
Technology, where he designed and imple-
mented the Agile Software Development Life
Cycle, replacing multiyear waterfall processes
with iterative product development cycles.
Lynn Winterboer is a Senior Consultant with
Cutter Consortium’s Agile Product & Project
Management and Data Insight & Social BI prac-
tices. For more than 15 years, Ms. Winterboer
has studied and applied Agile analytics,
serving in numerous roles within the analytics
BI/DW space. She understands the unique set
of challenges faced by BI/DW teams that want
to benefit from the incremental style of Agile
development and leverages her experience and
training to help deliver practical solutions for
her clients and students.
“BI teams can meet top-priority
needs in short time frames
when they adopt Agile
practices.”
— Lynn Winterboer, Senior Consultant
“BI Product Owners Love an Agile PMO!“
18 November 2013
TRAINING | AGILE PRODUCT & PROJECT MANAGEMENT
Register now and SAVE up to $500!
Use Coupon Code CSM+PO500 to register for both
CSM Certification: Scrum for Professionals and CSPO
Certification: Agile Analytics for Product Owners for just
$2490 — You Save $500 off the regular rate of $2990!
Use Coupon Code SCRUM400 to register for either
CSM Certification: Scrum for Professionals or CSPO
Certification: Agile Analytics for Product Owners for just
$1295 — You Save $400 off the regular rate of $1695!
3 Ways to Register:
Online: bookstore.cutter.com
Call: +1 781 648 8700
Email: [email protected]
Cutter IT Journal
About Cutter ConsortiumCutter Consortium is a truly unique IT advisory firm, comprising a group of more than
100 internationally recognized experts who have come together to offer content,
consulting, and training to our clients. These experts are committed to delivering top-
level, critical, and objective advice. They have done, and are doing, groundbreaking
work in organizations worldwide, helping companies deal with issues in the core areas
of software development and Agile project management, enterprise architecture, business
technology trends and strategies, enterprise risk management, metrics, and sourcing.
Cutter offers a different value proposition than other IT research firms: We give you
Access to the Experts. You get practitioners’ points of view, derived from hands-on
experience with the same critical issues you are facing, not the perspective of a desk-
bound analyst who can only make predictions and observations on what’s happening in
the marketplace. With Cutter Consortium, you get the best practices and lessons learned
from the world’s leading experts, experts who are implementing these techniques at
companies like yours right now.
Cutter’s clients are able to tap into its expertise in a variety of formats, including content
via online advisory services and journals, mentoring, workshops, training, and consulting.
And by customizing our information products and training/consulting services, you get
the solutions you need, while staying within your budget.
Cutter Consortium’s philosophy is that there is no single right solution for all enterprises,
or all departments within one enterprise, or even all projects within a department. Cutter
believes that the complexity of the business technology issues confronting corporations
today demands multiple detailed perspectives from which a company can view its
opportunities and risks in order to make the right strategic and tactical decisions. The
simplistic pronouncements other analyst firms make do not take into account the unique
situation of each organization. This is another reason to present the several sides to each
issue: to enable clients to determine the course of action that best fits their unique
situation.
For more information, contact Cutter Consortium at +1 781 648 8700 or
The Cutter Business
Technology CouncilThe Cutter Business Technology Council
was established by Cutter Consortium to
help spot emerging trends in IT, digital
technology, and the marketplace. Its
members are IT specialists whose ideas
have become important building blocks of
today’s wide-band, digitally connected,
global economy. This brain trust includes:
• Rob Austin• Ron Blitstein• Tom DeMarco• Lynne Ellyn• Israel Gat• Vince Kellen• Tim Lister• Lou Mazzucchelli• Ken Orr• Robert D. Scott