Section 1 - The beginning

Introduction

Section 2 - Modelling Chapter 1: Modelling with UML

Section 3 - Delivering Software

Chapter 2: The Waterfall Chapter 3: Initiation Chapter 4: Requirements gathering Chapter 5: Analysis Chapter 6: A Design primer Chapter 7: Design Chapter 8: Visual Design Chapter 9: Writing code Chapter 10: Releasing software Section 4 - Approach and people

Chapter 11: Iterative, incremental methods Chapter 12: Managing the environment Chapter 13: When can I have it? Chapter 14: The way of the SA Chapter 15: Patterns & antipatterns Chapter 15a: Show me your architecture Chapter 16: Enterprise architecture Chapter 17: Where do we go from here?

Section 5 - Aardvark & Aardvark, Solicitors

Chapter 18: The Aardvark & Aardvark assignment

From here to software architecture

There is a tremendous amount of software being developed for an equally huge amount of reasons. Unfortunately, in this high technology, leading edge, paradigm-shifting world of computing, we have forgotten the original promise of technology. It was meant to make our lives easier.

This book is about how individuals can take the lead in realigning our development efforts to that promise. In doing so, we shall in turn take our lead from the practice of Architecture, when Architects formed themselves into

organisations determined to combat the problem of buildings falling on people’s heads.

The change is already underway. The position of Software Architect, or a similar title, has been created by many organisations in the realisation that software development needs to be led by a single mind, or more accurately, by a single vision. This book will guide you to the skills you need to become a software architect and survive as one.

Who needs software architects?

If, for a fleeting moment, you imagine a city built the way we build our software, it would be a city of uniform greyness. It may or may not include a water system, depending on whether or not one was asked for. Ninety percent of the buildings would remain unfinished, or would have toppled to the ground as unsalvageable waste for want of better foundations.

Traffic would flow, but only because someone had found a giant to lift cars from the end of one jam and place them at the foot of another. And as you know, the cars would rarely fit the roads. Worst of all is the unending string and continuous maintenance required to hold the buildings together.

There in the midst of this rubble, picking a way through the web, walks the hastily hired architect with a simple brief: 'Please make our rubble into the glorious living spaces we imagined.'

Before you enter the city, I should warn you, we haven't yet looked at the buildings. The doors are not where we want them, and no-one measured a person to see if they would fit in.

Now you may argue this is not quite the case, but no-one can be sure. IT failures are largely invisible to those outside the city walls.

Enter the Software Architect. The Software Architect teamed with a Business Analyst will extract, define and create a software architecture to deliver your needs. Then matched with a Project Manager, will control the technology and flow of development while the Project Manager controls the timelines and risks.

At the highest level, the Software Architect will work with the sponsor to identify architectural requirements while the Business Analyst identifies the business requirements. Together they will model the process the software must fulfil, and the means by which it will do so. The sponsor will then identify those who have input into the project, and the model grows as it moves down through the ranks. Detail is added as appropriate, and when the model is presented to development, the architectural process moves from one of vision and design to one of management.

In organisational structures, a Software Architect will begin to layer development efforts. A team of a hundred will have a Senior Software Architect, and further Software Architects who lead individual projects or manage specific areas of

expertise. Senior developers will take the Software Architects' models and convert them to skeleton code for junior developers and specialists to complete.

Doing so will make development run more smoothly, be more controlled and more measurable. It is the job of the Software Architect to ensure standards are being met and that development plans are adhered to.

At present, software development teams have little hierarchical structure. Putting in place an architecturally led process will give the development teams a clearer career path and career choice. Instead of the traditional Developer, Team Leader, Development Manager, they will see Developer, Senior Developer, Software Architect, Senior Software Architect. They may choose to become Technical Specialists, or follow the more traditional path of Team Leader then Development Manager.

As more of us become Software Architects, we will begin to conform to the standards of the Software Architect community. This, in turn, will mean businesses begin to manage the processes and techniques of software development as they would manage any other discipline, from accounting to product development.

For business leaders, Software Architects will be looking over the horizon when modelling systems. We will be looking at new technologies, and the business milestones you wish to cross and hurdles you must jump to get there. It all depends what you want in your software, more hurled up skylines or more beautifully crafted cathedrals.

What does a software architect do?

Figure 1.1. Joe

Meet Joe. Joe is the product of long years at the forefront of software development. One day, perhaps we will have a Joe in every organisation. For now, he is a mythical being, a shining light on the horizon towards which we might travel.

If the description of Joe overwhelms you, fear not. You will be a Software Architect long before you are Joe.

Joe is the perfect Software Architect. We can define him by who he is and what he does.

Who Joe is

Joe is experienced. He knows the software architectures of many systems, and the design patterns used in their creation. He has used various software tools to enable him to understand and communicate his experience to others in and beyond his business. He has used many languages and is quite familiar with the techniques of object orientation and frameworks. He has been involved in creating, maintaining and using many software systems and is well aware of solutions to recurring software problems.

Joe works for a stockbroker. He understands stockbroking in mind-numbing detail, is very familiar with the whims, prejudices and needs of the senior decision makers. Indeed, he often golfs and sails with these decision makers and is considered a peer. Together they look to the glowing future of stockbrocking and how they can best grow their business with the strategic use of information technology.

None of Joe's peers understand a word he says when he's on the phone to his software buddies. He talks of singletons, proclaims one day that factories are better than brokers, then changes his mind the next. He talks of the software process, of waterfalls and spirals, of RUP, OOSP, OPEN and RM-ODP. He considers with the development teams how they might best utilise these processes and concepts in their business, and how they can best support the business process with their software process. He talks of leveraging the process to deliver outstanding software within tight timeframes, while using his time estimation skills of COCOMO and Function Point Analysis to protect the developers from overambitious schedules. In discussing process, he is aware that the architectural process is different to that of a project.

He is a technological expert. He knows the alternatives and how to compare and select new technologies. He keeps up to date with the marketplace and prides himself on knowing the emerging ideas and products in the software world.

He is an expert visualiser and expounder. He can abstract solutions to explain them to his peers, clients, other architects and development teams. He favours UML as a modelling environment, but rarely uses it for conversing with those outside development.

Joe is a nice fellow. He presents and communicates with aplomb. He is empathic, approachable, a good and keen mentor, highly credible and dedicated to his profession. When differences arise between developers, or more usually between development and management, he is a skilled negotiator, able to resolve differences so both sides appear to come off well.

He is insightful, creative, investigative, practical, visionary, and pragmatic. He sits comfortably interdependent between development and management. He is proactive, able to put first things first. He synergises, and seeks to understand before seeking to be understood.

All of this is who Joe is. He's a good egg, easy to get along with and a good oiler of seized cogs. The rest of Joe is what he does.

What Joe does

He delivers on the promise that technology is meant to make our lives easier. He still remembers the (retrospectively poor) predictions that computers would lead to a three day working week because there simply would not be enough work to do that could not be automated.

He is a visionary and a strong leader. He leads the architecture team, the developer community and the technical direction of the organisation. In creating the vision, he creates the architecture to back it up. When doing so, he ensures the architecture fits existing, planned and external architectures, carefully formulating designs through technological feasibility studies and efficient prototyping, while seeking to reduce architectural and project risk.

He ensures each system meets its purpose and fits the envisioned architecture by steering each project (whether architectural or not) through a defined process. He understands requirements and identifies stakeholders. He extracts, analyses and pushes back unrealistic or costly requirements, resolving ambiguities and understanding the business needs to allow him to do so.

He ensures correct implementation by asking if stakeholders, sponsors and developers are happy. He reviews project requirements, analysis, designs, timescales and project plans, code, style and test plans. Even after a project has been released, he looks for user satisfaction and reviews post release change requests.

The structure of this book

Section 1 – Preface and Introduction

From here to software architecture Who needs software architects? What does a software architect do? The purpose of this book Climbing the Software Architects’ mountain With whom does a Software Architect interact?

Section 2 – Modelling

Modelling in UML

Section 3 - Delivering Software

The waterfall Project initiation Requirement gathering Analysis of requirements Design primer Design Visual design Writing code Releasing software

Section 4 - Approach and People

Iterative, incremental methods Management and the environment When can I have it? The daily grind Patterns and antipatterns Enterprise Architecture and views Where do we go from here?

Section 5 - Software Architect

Aardvark & Aardvark

The purpose of this book

This is a primer to help a Software Developer or Software Engineer become a Software Architect. It is intended to introduce the role of the Software Architect and the arena in which the Software Architect will operate.

There is a lot to the role. You will need to study a great deal to be an effective Software Architect. For this purpose, I have provided references for further reading. I hope it is not too daunting. You will study the arts and sciences of software architecture; you will study current and emerging technologies, the software process, organisational politics and self fulfilment.

In comparison with building, just as a draughtsman with sufficient study, skill, personal attributes and motivation can become an Architect, so too can a Software Developer, with the same study, skill, personal attributes and motivation, become a Software Architect.

The change from Software Developer to Software Architect will require you to:

• Be knowledgeable in and beyond your technical domain • Have the personal qualities which allow you to understand, articulate,

innovate and sell your vision and ideas to your peers, superiors and clients • Understand the process which will allow you to create, manage,

communicate and deliver your vision

You may already possess all of these skills, in which case, you can

• Help to define and create the job, not the role, of Software Architect

I have invented no new ideas or methodologies. We have had enough ideas. It is time to begin implementing them. Our ideas must be about the software we are inventing, not the way we are going to invent it.

Are you alone?

Most certainly not. There are practicing Software Architects in many software houses and organisations right now. You may be working with one.

There are also architectural groups you may wish to join in many standards based organisations. I recommend the Worldwide Institute of Software Architects, or WWISA. You can find out more at www.wwisa.org.

So what are these Software Architects doing?

They are applying architectural techniques to the problems and challenges of building software products and systems. They are also upholding best practice for the software development process in a symbiotic relationship with Developers, Business Analysts and Project Managers. Architecture flourishes best in an architecturally led environment.

How do I become a Software Architect?

Software Architecture is the act of applying repetitive, measurable methods to the development of software products, whether they are systems, applications or objects. To be a Software Architect, you must study and apply the practices of Software Architecture, as both a manager and technician.

You can become proficient by following the path laid out in this book. Of course, on your way, you may look up one of the many references provided, and that will give you more ammunition and supplies to help you reach your goal. You may also decide to join us in the Worldwide Institute of Software Architects (WWISA), in creating a profession for Software Architects.

Climbing the Software Architects’ mountain

Imagine you are climbing a mountain, or if you prefer, a ladder. The ladder has eleven rungs, the mountain eleven stages. It does not matter if you think mountain or ladder as they are both models or abstractions of the real climb from developer to architect. Even climb is a metaphor, for you will not be any higher physically. Metaphors and models are what Architects use to materialise the immaterial (or to display their thoughts). Imagine away, I prefer mountains.

Basecamp/ walking up to the ladder

Our first stage will be an overview of the waterfall model, which is a simple metaphor, model or abstraction for a linear software development process. In the basic waterfall model, we go from requirements gathering to development, to test, to release, a four stage process. All of the musings of this book will be based upon a development process derived from the waterfall model, called the Unified Process for software development. We shall also discuss other development processes along the way.

Camp 1 / rung 1

We begin by stretching out our waterfall at the points of initiation and development (writing code, drawing pictures), and introducing requirements analysis and system design, which should precede writing code.

Camp 2 / rung 2

We shall then take the waterfall and make it an iterative or cyclic process.

At camp2 / rung 2, we shall take a look around at our position. Understanding the process is not enough. We must be able to model and communicate it. It is not good enough to make plans in your head, you must also sell your ideas to your peers and your superiors.

Camp 3 / rung 3

Onto the first snow slope we go, or if you’re a ladder climber, the rung you once spilled grease over which has been rather slippery since. We have to learn to model.

There are lots of ways to draw interconnected boxes. There are also a lot of drawing tools you can use to make a really nice job of it. The problem is that all of these methods are your methods. No-one else will do it your way. There is already a standard software modelling method called the Unified Modelling Language (UML). It exists and it is sufficiently detailed to model software and business systems, so why invent your own? Somewhere between camps two and three, you must give up your blob and line diagrams.

Let’s drop the ladder analogy and concentrate on our mountain. It is a better model in that the struggle is greater. Almost anyone can climb a ladder. Less than anyone will become a Software Architect.

Camp 4

Rest day. Let’s brew up a few mugs of tea and stare across the peaks beneath us. Each small peak could be an architectural method; we’ll take a look at a few of them.

Camp 5

Onward and upward. It’s time to look at how to structure your products. We’ll look at design from the point of view of the architect.

Camp 6

The air is getting thin, but the preparation of the earlier stages means you’re tough enough. Patterns and antipatterns, software methods copied from psychology and architecture.

Notice the use of the word copied here. You are familiar with copying and pasting so I used it. I could have written stolen or borrowed. Each term would

have sufficed. But we have been calling a spade a simple entrenching tool for long enough. Let’s call it what it is for a change. People will like us better. Learn to speak from the point of view of the user.

Figure 1.2 A simple entrenching tool

Camp 7

Time for some map reading. But do you know how to read the map? What options do you have for mapping your software world? This is the world of architectural views.

Camp 8

The peak is in sight now. Time to learn how to sell your newly found architectural skills to your peers and superiors. When you’ve climbed the mountain, you will have to help others at least part way up the slope. No Software Architect can work in isolation.

Camp 9

Things are getting tough. It’s time for some dot combat. Life as a Software Architect is never easy. Even people from your own team will undermine you. They will rubbish your ideas. Be inclusive, invite them all in. Mentor them, listen to their ideas, and leave room for them to have some of their own. They will like you more. You will be more successful.

Camp 10

Art. You may be gasping for air, but take a moment to study the marvels around you. Is your development environment helping you to develop beautiful architectural solutions and interfaces? Does it use the most magnificently constructed algorithms, but look like a button and text box? What’s the point of reaching the summit with a broken camera?

Summit

Be proud, you have reached the top. All you have to do is get down and tell everyone. Some won’t believe you made it. Some think man never walked on

the moon. How will you adjust to life on the ground now you’re a Software Architect? How will they know you made it?

This is your imaginary mountain. Keep it in mind as you go. The chapters of this book are arranged to help you up the mountain, but are not numbered as are the camps. If you like, this imaginary climb is one view of the architectural trail. Another view is the chapters of this book. Both look at the same thing, but from different points of view. When you are a skilled architect, you too will provide alternate views of your proposals. You may create multiple sets of views, one per client. After all, viewers are mostly interested in their own (rather limited) point of view.

With whom does a Software Architect interact?

The following diagram represents the interactions between individual roles on a software project. It is an example of a software house developing a bespoke business solution for a client.

Figure 1.2. Software Architect collaboration

In the following discussions, I shall be referring to the roles with whom a Software Architect will interact. Here is a whistle stop tour.

User

Someone who will ultimately use the deliverables of this project

Sponsor

The person who will pay for the project

Business Analyst

The person who analyses the operation of the business and defines the requirements the project will fulfil

Software Architect

Why not you?

Project Manager

The person who is responsible for delivery of the project in the identified time period.

Development Manager

The person managing the development teams, the development environment and process, and assigning people to project teams.

Team Leader

The person managing a single development team on a project

Developer, Software Engineer

The writer of code and drawer of pictures

Technical Specialist

Someone who specialises in a specific area of development

Test Designer

The person designing the tests used by the testers

Tester

The person responsible for ensuring all of the original requirements have been met by the results of the project, and that the deliverables are bug free.

These are the roles involved. One person may fill more than one role, or a role may be shared by a team of people. This book looks at the role of the software architect.

Chapter 1 - Modelling

Architects and software architects

Journals of software architecture haven’t caught on in the same way as architecture journals, for the simple reason that you cannot see the end product. The architect, on completion of his work, can show someone around, pointing at his cleverly designed features and enthusing at his use of materials. The architect and the viewer share the same carpet, walk on similar legs, and look with similar eyes. They can sit, at the end of the day, on a far hillock, and watch the sun sink behind the magnificently realised edifice of their creation.

Far away, or close up, the viewer with no knowledge of architecture can appreciate the architect’s work. In a software system, there is nothing to see except a few dusty servers and some rather quirky looking screens.

We will never have those moments that architects share with their clients on completion of their work, because there is very little to appreciate. If it works, then that’s what was expected. The internal architecture, regardless of its skilful insight and execution, means little.

This lack of joint understanding has created a gaping rift in the fabric of relationship between software developers and their sponsors. Each are apprehensive of communicating with the other, because both are blind in each other’s world. There is no common ground on which we can meet and discuss what we are doing. We do not yet have the wealth of historical communication, nor the similar views, that the architect and his client enjoy.

Consider this: When a bridge disaster occurs, the architectural experts (technical) explain why. When a plane falls out of the sky, expert engineers and designers (technical) explain why. When the spacecraft fails, the NASA scientists (technical) explain what went wrong. When an IT system fails, management (non-technical) explains why. Why?

It is because they are scared to ask us. They won’t understand what we’re talking about. We live in a different world. We are defined by our nerdy stereotype, unable to step beyond it. The unwritten rule is don’t go asking a techie to communicate, at least not in IT.

We need to change. We need to change the way we approach our work. We need to change how we communicate with others. We need to change our personalities, and most of all we need the change to be appreciated.

To begin, we learn how to model. Our models communicate, just as an architect’s models communicate. Perhaps ours are a little more energetic, but at least we will have representations that others might begin to comprehend.

When the architects began modelling, there were lots of different methods available to them. There was sketching, building wooden models, drawing plan and elevation views, carving models from clay, building them from matchsticks, drawing perspectives. I’m sure you could add a few more. Nowadays, a plan, elevation and perspective might look commonplace. They probably didn’t then. Similarly our modelling method may appear a little strange, but it works, and it will improve and gain wider acceptance over time.

Architects’ models let people appreciate their intentions. Software architects, by modelling, will begin to change the landscape upon which we walk.

It is time to begin.

Modelling

As a software architect, you will be required to work at one or more levels of abstraction above code. Your work will be used in communication with others in your development teams, customers or external partners in your project. Blobs and lines are OK for many diagrams, but they do have their limitations. The greatest objection to blob and line drawings is that they are not defined by a standard. A blob and line to you may mean ‘install this here’, but your sponsor will see only a blob, a line and bit of text.

Fortunately, the software industry has compiled its best modelling ideas into the Unified Modelling Language (UML), now standardised by the Object Modelling Group (OMG – www.omg.org). It is a collection of nine standardised modelling techniques and is used throughout this book.

This chapter is a primer for the UML, and introduces the core aspects of its diagrams. UML is a rich modelling environment, and there are features beyond what is mentioned here.

UML 1. Use Case Diagram

If you have not yet come across the term use case, think of it as something a system does. For example, a drinks dispensing machine probably has a single use case: dispense drink. A machine to supply train timetables and provide tickets probably has two use cases:

Use Case 1: Provide timetable information

Use Case 2: Sell train ticket

A software system will typically have somewhere between one and ten use cases.

Ivar Jacobsen, who first coined the term use case defines it as:

A behaviourally related sequence of interactions performed by an actor in a dialogue with the system to provide some measurable value to the actor.[1]

The UML use case diagram presents the use cases of a system and shows the actors with whom they interact. It also displays the boundary of the system within which the use cases operate. No further detail is required.

Figure 1.1 The format of a Use Case Diagram

An actor is an external system. It could be a person who is using an application, or a timer set to regularly perform a task. It could also be a thermometer from which the application is reading the temperature, or a thermostat used to control the temperature. It may seem a little odd representing a thermometer as what is obviously a stick man. Feel free to draw a thermometer if you can. I don’t think it will hurt.

The environment is usually a software application or a software and hardware system such as an ATM or mobile phone. All use cases are contained within the environment, and all actors are external.

Figure 1.2 A Use Case Diagram

Figure 1.2 displays a library member accessing a library’s internet portal. The use cases show that the library member logs in, and is able to view and order books.

Figure 1.3 Use case diagram for a residential heating system

In this rather complex use case diagram, we see a note in the top left corner. Notes are represented by a dog-eared rectangle and can be used on any type of UML diagram to add pertinent information. In the actors we see a janitor actor and a resident actor who inherits all of the janitor’s rights to the heating system. Both the resident and the janitor can use the thermostat, but only the resident can control the automatic timer. The heating system in this diagram is shown as an actor, yet it could also be represented by a collection of use cases in another system. For this project, the heating system is external and we are not concerned with how it works, only that we must provide an interface to it. The relationship between the automatic timer and the heat controller is that the automatic timer uses the heat controller. Had the arrow been on the other end, then the heat controller would use the automatic timer. These are largely academic points at present. A use case diagram is not used for creating code. It is used to communicate a high level view of the nuggets of work a system will do, and the actors it will do it for, or rely on.

From this view, the automatic timer is portrayed as a piece of software which may be controlled. If it was a hardware timer, then it would be presented as an actor rather than a use case. Timers often present this problem as they can be internal use cases or external actors. It is usually fairly obvious how to model other system objects.

The text contained within double chevrons or guillemots, e.g. «uses» is known as a stereotype of generalisation. They are used to define relationships between use cases. The use case diagram allows for «uses», «includes» and «extends». As you can see from the actors in figure 1.3, actors can also «inherit» the behaviour of others, though some claim this is taking the use case diagram too far.

It is unlikely that you will create such complicated use case diagrams, yet the mechanisms are available to fully model complex systems should you wish to do so.

next >>

UML 2. Activity diagrams

Activity diagrams are similar to flowcharts. They are often used to model the paths though a use case.

The activity diagram has few features, unlike a flowchart, and is all the better for it. The beauty of activity and use case diagrams is that anyone can understand them almost at first sight. The rounded rectangle is the unit of work for this diagram. The black dot and black/white dot identify the start and end points of the activity.

Figure 1.4 Activity Diagram

Another feature of the activity diagram is a diamond, used as a conditional branch. Alternate paths through the activity should split at a branching diamond, and merge at another diamond, as shown in figure 1.5.

Figure 1.5 Activity diagram showing alternate paths using branch and merge

Figure 1.6 shows the use of synchronisation bars, and also displays a conditional path, travelled only if the video shop customer is a member of their bonus scheme. In this type of diagram, the conditional paths of a branch, or a conditional activity, are enclosed in square brackets, e.g. [member], [non-member] or a conditional path for [Bonus scheme member]s only.

Figure 1.6 Activity diagram showing synchronisation and a conditional path.

A feature of this diagram is the self referencing activity check out video. In this case, check out video returns to itself, so another video can be checked out before accepting payment. Here is a similar example, this time for keystrokes.

Figure 1.7 Activity diagram section showing a self referencing activity

The activity key stroke contains the processing of the key. Only a selected group of keys cause activity to move on to the next unit of work. Self references are often used in UML as a shorthand notation.

The following diagram shows the activity of dinner. It uses a modelling technique called swim lanes to describe how dinner takes place in both the kitchen and the dining room. Two swim lanes are shown, one for the kitchen and one for the dining room.

Dinner begins in the kitchen with hand washing, then ends up in the dining room with dinner served. One activity - wait for kids to arrive - shows how the self referencing unit of work implies the wait for the kids to arrive.

Figure 1.8 UML Activity diagram with swim lanes

Swim lanes are used often in business applications where certain activities belong to a specific business entity. For example, creating invoices belongs in accounts; taking an order belongs to sales, and filling the order belongs in the warehouse. They are all part of the sales order process, but accounts, sales and the warehouse could each have their own swim lane.

Figure 1.9 Noise sampler

A real time system for sampling noise may have swim lanes for the analogue to digital conversion, the frequency analysis (using a fast fourier transform or FFT), logging of the result, and its display. Sample noise would be a use case, the path through the use case would be sample noise, perform FFT, record results, display results. Each of these would be broken down into substeps, and assigned to a swim lane. In such a system, each swim lane would be a separate electronic device or set of components.

Notice the use of synchronisation bars to show how the sampler continues to sample data, while passing on the latest complete sample to another system for processing.

Part of the activity may include sending or receiving messages from another system, or another part of the system. It is not necessary to show this other system if it is not relevant to the model, as the UML activity toolbox includes symbols for messaging beyond the current model.

Figure 1.10 Activity diagram showing messages

Figure 1.9 shows an alternative to the use of swim lanes when a separate part of the system is not interesting enough to be included in the model. Instead, messages are sent out from the model, and messages are received.

Your models are not meant to be complete renditions of systems. They are meant to detail a specific aspect of the system for a specific viewer. Including external, uninteresting or obvious parts of the system will only confuse the viewer. You decide what to put in, and what to leave out. You are doing the modelling.

next >>

UML 3. State diagram

State diagrams, also known as state charts or state machines, show states and transitions between states for objects in response to internal or external stimuli. They were originally developed for modelling integrated circuits or other electronic devices. They are used to model objects with complex states, such as business objects, user interfaces, or objects used as controllers for other objects within an application.

Imagine placing an order for stationery over the telephone. The initial state of your order while you are creating it is incomplete. Its next state may be complete and awaiting payment. When payment arrives, it is in the ready to be picked state. The warehouse person takes your order, and it is in the picking state while they wander around the warehouse filling your bag, then the picked state. It sits in the picked state a while, then the person responsible for dispatch goes around the warehouse collecting the picked orders. It is now in the state of waiting for dispatch. When collected by the courier, it is with courier, then finally delivered. If this state is recorded and made public, you can track your order.

States may also exist within superstates. For example, the states to awaiting payment may be contained in Order with salesperson. Up to waiting for dispatch, may exist within order with warehouse. The final superstate would be order released.

Figure 1.11 Statechart for sales order showing superstates

Occasionally, the order doesn’t get to the delivered state. In that case, it remains with the courier until it is returned to the warehouse. The process for a returned order may itself have multiple states, such as returned, being reshelved and order cancelled.

The state diagram is visually similar to an activity diagram, except the activity diagram displays units of work, and the statechart models the states of a dynamically changing system. The dynamics of a user interface can be modelled using a state diagram, before the states are coded in a controller class.

The example in figure 1.12 is a simple modal dialog box with a text box and an OK button. Even this simple dialog has many different transitions between its states. For example, a user may click the text box with a pointing device, then press the tab key to get to the OK button and press the space bar to operate the OK button. This is just one of the many routes through the dialog. To model it effectively, all states and transitions between those states should be included.

Figure 1.12 Statechart for a dialog

If this statechart is a complete model of the dialog, you can see that once the button is reached, there is no going back. There is no route, i.e. state transition, away from the button other than activating it. You may decide that your dialog will be kinder to its user, and allow a click or Alt-T to take them back to the text box.

Declaring states and their transitions in a diagram this way can help the programmer to implement your model using a set of switch statements. All events on the dialog are routed through the switch statements, which act as filters for blocking illegal transitions. This makes it quite straightforward to manage complicated user interface rules, such as ‘you must fill in the text box before the button is enabled, or if this is user X on account Y, they must provide a reason of: Product out of stock, or Product unavailable.

Statecharts are often a necessity in our loosely coupled, object oriented world. They can save some severe headaches debugging user interfaces if the states and transitions are controlled by switch statements in a class separate to the user interface.

next >>

UML 4. Class diagrams

The class diagram is the most directly applied when it comes to coding. Most UML tools will generate a set of classes in many programming languages straight from the class diagram. They are not always complete, especially where collections or templates are concerned, but they can save a lot of repetitive typing.

Class diagrams are also the most complex, and certainly the most discussed when modelling object oriented concepts such as composition and aggregation. Hopefully, we can avoid such arguments for now and accept that class diagrams are able to model many object oriented, and even entity relationship particulars.

Class diagrams come in two basic flavours. The simplest uses class stereotypes, and the more complex details the member properties and functions of the classes. We shall begin with the class stereotypes.

Class acts

The class stereotypes are boundary, control and entity classes.

Figure 1.13 Boundary, Control and Entity classes

You may see these stereotypes used in other types of UML diagrams. They are useful for doing initial high level modelling, mapping out what objects a

component might contain, and what relationships exist between them before specifying the class details.

Boundary classes are used to specify interfaces between systems, control classes are used as objects to control other class actions, and entity classes are typically data stores.

Figure 1.14 An example using boundary, control and entity classes

As I have been showing basic objects, I have allowed myself a booklist object, which will hold a collection of book objects. This is not strictly required as in a more detailed diagram, I might have defined the relationship between the objects book selector and book as in figure 1.14.

Figure 1.15 Class relationships

The relationship here is 1 to 0..n. It means one book selector is related to zero or more books. In reality, I know my database will only hold a certain number of books, such as 232, but 1 to 0..4294967296 is not usually required. The numbers defining the relationship are known as cardinality.

Cardinality can contain more than one item. For example, 1..4,6..n means any positive number except 0 and 5. Negative numbers are not used.

Figure 1.15 implies a collection is used, but the UML tools do not take the cardinality and create a collection holding zero or more books. Despite this, the cardinality notation is used as class diagrams begin to get cluttered when they contain collections.

Figure 1.16 Spaceman’s food dispenser

In this example, our spaceman has access to a food dispenser. Actors always connect to a boundary class. A dietician creates his main course from meat and vegetable substitutes, and the spaceman, lucky fellow, is allowed to choose which fruit he wants from his fruit bowl. It’s not real fruit, which would deteriorate over the five years he is in space, but there are three fruit definitions, all derived (inheriting) from a base class of fruit. UML uses quite a few line and arrow types, all with specific definitions. We shall look at other line and arrow types shortly.

These simple class diagrams are fairly easy to produce, and not too difficult to understand. Getting it right at this stage makes finalising object oriented designs far easier, as the high level view without properties and functions means you do not get lost in the detail. Displaying them in this format means you have a good foundation upon which to build the detail, and a good picture to use in early discussions with designers and developers.

I believe all diagrams should be able to fit on one page of printed paper. Any fitting together of pages or searching for a larger printer is a fruitless activity. An A3 diagram (like a Business Analyst’s magnificent flowchart) is usually impossible to understand without some serious study. It is far better to create one skeleton diagram and a series of more detailed diagrams each representing a bone. Unfortunately, many become lost in their works of art, and spend so much time on them that they begin to understand them. What is worse, this creates a false belief that others will have their immediate insight into the odd shaped boxes and decision flows. We should be eternally grateful to the UML creators for their foresight not to use the flowchart symbols.

Class diagrams are very useful design diagrams, representing specific implementations. They can also become works of art. Beware.

Figure 1.17 A class diagram for the Apple class

Figure 1.17 shows a more detailed representation of a class. It is quite straightforward, having three public properties and one private property. The class name is Apple. SkinColour is an RGB value, Juiciness is measured in the Standard Juiciness Number, the Variety is a string, and the Worms collection is private. You won’t know about them until you take a bite. You could say the same about some software.

It’s not a very good class, as classes go. Let’s try a little harder.

Figure 1.18 A better model of an apple

The apple now derives quite a lot from its base class of fruit. Not only does it have properties, but it is able to rot too. Calling the apple.rot() function could decrease its Freshness. It may also decrease the size slightly. The apple class itself has only two properties and one object. The object Worms does not have to exist, as an apple can contain either 0 or 1 Worms collections. The Worms collection has no properties of its own, but contains 0 or more worm objects. Worms can be added to the apple, or removed, or counted. Every property in this set of classes is public, and shown with a +. You will know about the worms before you take a bite.

Figure 1.19 A fruitier fruit model

Figure 1.19 shows a fruit with a private freshness property. However, you can derive its freshness by using its smell and look functions. Anyone can look at the fruit as it is a public function, but you may only smell it if you are a friend.

Here are two examples showing the relationship between a teacher and her students, and a pentagon and its sides.

Figure 1.20 Teacher and student

The teacher teaches between 2 and 25 students. The name of the association is teaches, and direction shows that it is the teacher doing the teaching. An opposite arrow could show the students learning from the teacher, but that is not always the case.

Figure 1.21 A pentagon and its sides

The cardinality shows one pentagon has exactly five sides. Also we see that the sides are ordered, and that the pentagon is composed of its sides. The black diamond is a composition sign, declaring that the pentagon is made of sides, and that the sides are contained within the pentagon.

Figure 1.22 A class qualifier

This shows the relationship between train journeys and ticket sales. The train journey class has a qualifier of seat number and departure time, for which either no ticket or one ticket can be sold. The classifiers can be used as keys in data structures or databases.

You may also define associations between classes when they are not specifically defined in the class relationships. To do so, the normal class relationship is stated, then the associated class is shown connected to the connector which states the relationship between the associated classes.

Figure 1.23 A reservation class shown as an association to the sales relationship of train journey and ticket.

Let’s take a look at the relationships we can show in a class diagram. Most of these connectors or more correctly associations are not confined to the class diagram and are used in most of the diagram types. The last two are not used in class diagrams, but are included here as a reference of all the UML association types.

Figure 1.24 Connector types in UML

These connectors represent object oriented concepts. I expect anyone developing software today uses at least a few object oriented methods, so at least a few of the connections will be familiar. Some of these concepts are discussed further below.

Dependency and interfaces

Figure 1.25 - Interface dependency

Figure 1.25 shows class B is dependent upon the DataOut interface of class A. This means class B uses the DataOut interface of Class A to provide it with some information, and is unlikely to operate properly without it.

Realisation

If a class is a realisation of another class, then it fully implements its behaviour, but does not inherit (unless specifically stated) its structure or interface. The behaviour must be coded in the realised class.

Composition and aggregation

These two terms mean different things to different people. Both terms allow a class to be made up from a number of other classes.

Figure 1.26 A dog class made up of other classes

In both aggregation (shown in figure 1.26) and composition (solid black diamonds rather than the outlined diamonds), the dog class is made up of a head, a body, the legs and a tail. If you kill a dog made by aggregation, the head, body, legs and tail live on and can be attached to another dog. Killing a dog made by composition would be killing the parts too. Frankenstein believed we were made by aggregation and could be reassembled and kick-started with a few good shots of electricity. And so have we proved, that as long as we work on the interface with anti-rejection drugs, we can supplant organs from dead (or live) donors into others.

Class diagrams may contain many other constructs. Some of them are very useful shortcuts for defining relationships. Too often, information discovered about the system being modelled is written up in paragraphs, where a little extra work on the class diagram can state relationships exactly.

In the following class diagram we see a business entity called a group which is made up of practices. The practices themselves are an aggregation of practices, i.e. a practice can contain other practices. As it is an aggregation, the practices do not rely on the parent practice for their existence, and can be moved to another parent practice if desired. Such a practice hierarchy is usually implemented as a linked list where each practice is linked to a parent. Those with no parent will be the top level practices in the group.

Figure 1.27 Class diagram example

The cardinality attached to this aggregation means a practice may or may not contain child practices, and also may or may not have a parent practice.

The relationships between practice and employee shows that a practice has one or more members. It shows how a practice can have no manager (if the last one left and has not been replaced), one manager (as is normal), or two managers, in the rare case of joint heads of practice. Manager is identified as a subset of member, implying that a manager is also a member, and defined as a {subset} of member.

This model states relationships quite succinctly. It states a manager must be a member of the practice, and a member may or may not be a manager. It also shows the relationship between practices and their children are specified, or keyed, by the property PracticeID

That’s quite enough on class diagrams for now. The only way to get the hang of them is to model with them.

Class diagrams, as you have no doubt realised, are extremely rich in the methods available to define their properties and relationships.

next >>

UML 5. Object Diagram

An object is the instantiation of a class, i.e. it is the object created from the class at runtime. Object diagrams are similar to class diagrams, except the object name and class they are instantiated from are underlined.

Each object is given a name, and this name will be used as the object’s definition when writing code. As a simple example, consider this:

Figure 1.28 Deefer Dog

Here is an object instantiated from the class Dog. The name of the Dog object is Deefer. Its properties and functions are defined by the Dog class.

The differences between modelling a class and modelling an object are shown in figure 1.29

figure 1.29 Examples of classes and objects

A class contains a class name, properties and functions. An object shows the class name it is instantiated from preceded by a colon (:), then optionally preceded by the object name. The class in a class diagram displays properties and functions, whereas the object in an object diagram shows only properties, along with their values at the moment of interest to the modeller or viewer.

Figure x.xx shows a snapshot of the data in a system used in vehicle collision detection. There are two objects of the class car, named according to their registration numbers. A road object, named as Route 66, is included in the calculation. The CollisionDetector class is not named, as that piece of information is superfluous to the model.

Figure 1.30 Object diagram for traffic warning system

The CollisionDetector object has knowledge of the two cars. The condition of the road can be ascertained by passing in the coordinates of the likely passing point to the two functions in the road object. The cars are approaching each other with a relative speed of around 200 kph. They are not quite in a head on collision, but are not far off.. The collision detector calculation may ascertain that the two cars have only a 0.01% chance of colliding based on past data, and that only a slow down warning might be given. In the past, the slow down warning has prevented all collisions on this particular stretch of Route 66.

You may be presenting a normal operational examples, or critical information for use in testing the system. Perhaps this is the limit for an amber warning, and any vehicles travelling faster, or travelling on a more dangerous stretch of road or in worse conditions, will require a red warning.

Similarly, object diagrams can be used to demonstrate data which leads to state transitions, and in complex systems a series of examples may be required, specifying the conditions for a state change in the required warning.

Object diagrams can also be used as organisational charts.

Figure 1.31 Organisational chart with data

UML 6. Collaboration Diagram

In the first object diagram, the Dog class was instantiated as an object named Deefer. In the CollisionDetection class, it was not shown instantiated. It is not always necessary to define the name of the object, if only one object of that type will exist, or the context is such that it requires no declared object to fully explain the model. Similarly, collaboration diagrams may contain declared or implied objects.

Figure 1.32 Messages in a collaboration diagram

In collaboration diagrams, messages are shown as a sequence defined by a number and colon. Our singer Guy Greenfield first sings to the audience, then the audience applauds. Another arrow could be added in the vicinity of 1: to show Guy’s response to the audience’s applause - 3: Bow. The Audience class is undefined. Whenever Guy sings, they applaud. When they applaud, he bows. It’s a monotonous life. He’s thinking of opening a café somewhere where there are yachts, where clouds are always sparse and fluffy and the sun shines all day long…

Here is a story about two dogs called Doggay and Dogbee.

Figure 1.33 Dog and bone man

Wow, that was quite a story. I wonder what happened when Doggay caught up to Dogbee after his master telling him off?

Such a brief collaboration diagram containing four objects probably takes up around the same page space as a narrative. More complex diagrams are likely to take up less space than an equivalent narrative, and also be more quickly understood as they are specific. Narratives can be ambiguous. Objects in the story can also be positioned on the diagram to imply some kind of group or relationship, and that is the main reason for using collaboration diagrams. You can reflect the messages passing around an organisation or system by the proximity of those objects on the diagram. Collaboration diagrams are highly related to our next diagram type.

UML 7. Sequence Diagram

The collaboration diagram and sequence diagram are so highly interrelated that software tools can create one from the other. The information presented in each is exactly the same.

In a sequence diagram, the sequence begins at the top and ends at the bottom. It was a sequence diagram I used when presenting the software architect’s interactions in the introduction.

Figure 1.34 The format of a sequence diagram

In this sequence diagram, we see two objects and a message passed from object 1 to object 2.

Figure 1.35 Sequence diagram showing Doogay chasing Dogbee

Here, the class names are also shown, and Doggay is chasing Dogbee. Displaying the class names is optional, as in the object or collaboration diagrams, where they are included only if they are of interest to the viewer. In the next example, we see what did happen when Doggay caught up to Dogbee after his master's chiding.

Figure 1.36 What Doggay and Dogbee did later

Wasn’t that nice of them. You also know where the bone came from in the first place. I skipped a few interactions, but I’m sure you get the point. As: time proceeds from top to bottom in the sequence, there is no need to number the events as you have to in a collaboration diagram.

The large X at the base of the Bill :Buffalo column indicates the object is destroyed at that point. Bill :Buffalo creates (by decomposing) the bone object as it is destroyed.

The classes in your software system need to communicate with each other to fulfil their tasks. Both sequence diagrams and collaboration diagrams are known as interaction diagrams, as they display the interaction between objects. Sequence diagrams are clearer, but it is difficult to get many objects and interactions on the same page, whereas a collaboration diagram is far more densely packed. With them you can model messages between objects in an application, and also model messages between disparate systems, both synchronously and asynchronously.

In the next example, we use the class stereotypes. We see the creation of an apple object from the fruit class, and an asynchronous response where the requester may take his apple when he pleases.

Figure 1.37 Class stereotypes and asynchronous messages in a sequence diagram

Figure 1.38 shows the class stereotypes again, in sequence of events leading up to the death of a mosquito.

Figure 1.38 Sequence diagram with class stereotypes

next >>

UML 8. Component Diagram

Now we have all of the difficult methods out of the way, we are left with two rather simple ones. Both the component and deployment diagrams use similar constructs. The component diagram shows dependencies between components in a software system, and the deployment diagram shows the deployment and configuration of the components on physical nodes of the networked computer system.

Figure 1.39 Component diagram of a sales order processing and accounting system

Each component is drawn as a rectangle with two rectangles in the upper left corner. The various arrows used to define class relationships can also be used in component diagrams. In the diagram above, the database is shown as a component, as are the main applications of the system, i.e. Sales application and Accounting application. Both these and the data access component, are composed of sub components, whereas the database is considered a black box.

UML 9. Deployment Diagram

The deployment diagram is used to show the distribution of components, typically across a multi-tier system or the deployment of components around a heterogeneous environment.

Figure 1.40 Multi-tier deployment

In this diagram, we can see a multitier system comprising of a client with a web browser and a widget. The widget connects to the KoolTool application on the web server. This, in turn, accesses information directly from the document server, and from the Kool database via a tier of components on a component server. Kooltool has been written as a set of business objects which communicate with the database through a common data access component.

There is nothing startling in this deployment diagram, which is how it should be. If there are complex interactions within the layers, they are best presented in another diagram, or more correctly in a component diagram. Deployment diagrams are intended for deployment and should be quite clear.

Figure 1.41 Heterogeneous environment for the survival suite

This deployment across a heterogenous environment simply shows which chunk of information sits on which server, and what the servers’ hardware configurations are.

next >>

Other UML Concepts

UML supports the concept of packages. They are loosely defined structures and look like a folder with a tab. Groups of objects can be shown in the folders, or they can appear on their own to show packages of applications. Most of the UML modelling relationships apply to them, so you can show dependencies etc. You can also use them in collaboration, object and class diagrams to display a higher level view than at the class level.

Figure 1.42 UML Packages showing dependencies

Layer diagrams and UML

Layer diagrams still haunt networking publications. They live on in approaches to business systems, yet there is no layer diagram in UML. Why not?

The weakness of layered diagrams is they are often ideals rather than reality. Some components always sneak across more than one layer, and drawing parts of the system straddling layers, or working across more than one layer soon becomes awkward.

Here is a typical layered design for a business system.

figure 1.43 layered design

Lets take a look at those layers drawn as a UML equivalent.

Figure 1.44 Layered development shown in a class diagram

This approach is far easer to draw and understand, especially where complex layered structures are being displayed. If you need layers and objects, it is simple enough to draw layers as packages or classes grouped together visually.

Figure 1.45 shows the Internet protocols mapped across the seven layer OSI networking model.

Mappings of other networking protocol stacks can be found in [2].

Figure 1.45 Internet prototcols compared to the OSI seven layer model

It is clear the internet does not sit happily across the seven layered OSI model. In fact, no networks do. Not even IBM’s SNA Server, upon which the OSI model was based. Figure 1.46 shows an implied layered design for the internet protocols without using a layered diagram.

Figure 1.46 Internet protocol layers displayed as packages in UML

UML Summary

The UML is made up of nine interrelated diagramming methods. Ideas can be traced between the diagram types, and all are directly applicable to (and have been designed for) modelling software. This does not mean you cannot model other systems with it. In the examples, I have used everyday objects and modelled them, so that the message I am portraying is clearer than if I had modelled a software system of which you had no experience.

The nine diagram types can be subdivided into static and dynamic diagrams where static diagrams model structure, and dynamic diagrams model behaviour.

Static UML diagrams

Use case diagram

Identify the units of work in a system, and the external actors it communicates with.

Class diagram

Displays classes showing their interfaces, static collaborations and relationships.

Object diagram

Shows data structures or snapshots of system data at times of interest.

Component diagram

Shows the components of a system and their relationships at a level of abstraction above the class diagram.

Deployment diagram

Shows physical nodes (computers), their relationships and the components installed on them.

Dynamic UML diagrams

Activity diagram

The flow of control of activities in a system or part of a system.

State diagram

Shows a dynamic view of the state of the system or part of a system.

Collaboration diagram

Shows the structural organisation of components and the interaction between them.

Sequence diagram

Shows messages between classes or components in time order.

next >>

The UML tour guide’s Parthian shot

UML is a rich language and this has been a whistle stop tour. There are other mechanisms and notations available in UML, but this is enough to be able to

model. For more information on UML, see the references below. [3] and [4] were written by the architects of the UML, Grady Booch, James Rumbaugh and Ivar Jacobsen.

The ghost of modelling past

Many modelling notations have been superseded by UML. This includes the Booch method, OOSE and OMT, which were combined into the beginnings of UML. Other ghosts such as Data Flow Diagrams (DFDs) are still floating around, but sequence diagrams are far more useful, and have largely replaced them. It is up to us, as a community, to lay them to rest and unite with UML as our method of choice, and the method of our profession.

The future of UML

UML is a live modelling language and although it is no longer in its infancy, there are still modelling problems to address. Future versions may support business modelling and data modelling. At present, data modelling can be achieved using the class diagram, where each class represents a table in a database. Some UML tools will generate database generation scripts from the class diagrams, complete with primary and foreign keys, indexes, unique and null qualifications. This does not mean you should generate your table scripts from your class diagrams, as a 1:1 mapping between classes and tables will produce a lousy database.

Some believe the goal of UML is to specify software with sufficient detail that it can be compiled into applications. I do not know how that might be achieved, but it has already begun with class diagrams directly generating code. Whether it will ever become fully compilable is a matter for debate and will be for a long while. Modelling, and specifically UML, is a large part of the software architect’s arsenal; you will be better armed with it than without it.

The reality of UML and sponsors

Technically astute buyers are demanding software models in UML. This is likely to become more and more prevalent, and more the norm.

There are also, and will continue to be, buyers who are interested only in the resulting software and not in its internal mechanisms. These buyers when confronted with UML diagrams will switch off. They want, and should be presented with, colourful, high level blob and line pictures. Give them what they want.

A common term in software development is IKIWISI - I’ll know it when I see it. If they expect to see colourful blobs and lines and get UML, you will lose them.

Modelling folklore

In the meantime, you may bear in mind this light hearted comment which has almost grown into modelling folklore, and certainly quoted in every piece of modelling discussion:

All models are wrong Some models are useful [5]

Wrong, they may be, but as Leonardo wrote:

Do not interfere with matters for the eye trying to get them in through the ears[6]

References

1. The Unified Software Development Process, Ivar Jacobsen, Grady Booch, James Rumbaugh. Addison Wesley Object Technology Series.

2. Network Essentials Study Guide, James Chellis, Charles Perkins and Matthew Strabe. Sybex.

3. The Unified Modelling Language User Guide, Grady Booch, James Rumbaugh and Ivar Jacobsen. Addison Wesley Object Technology Series.

4. The Unified Modelling Language Reference Manual, James Rumbaugh Ivar Jacobsen and Grady Booch. Addison Wesley Object Technology Series.

5. George Box 6. Trattato della Pittura, or Treatise on Painting. Leonardo da Vinci.

In this section, we look at a software process which supports architectural work. The software architect cannot deliver software that conforms to a vision and is built upon solid foundations without the support of an architecturally led process.

For the same reason, architects do not design houses with sand for a foundation, nor expect them to be built without building regulations upheld by tradition and law.

Chapter 2 - The waterfall

The Waterfall

One of the earliest attempts at creating a software development process was the waterfall method. The name implies software development begins at the top and somehow reaches the bottom. The top is the project sponsor wanting something, and the bottom is the project sponsor receiving something.

The waterfall itself is not one of those plunging torrents seen in Tarzan films or Yosemite photographs. It is more stepped and serene, with a series of easy falls where the water splashes gracefully from one level to the next.

The four steps of the basic waterfall are requirements gathering, development, testing and release.

Figure 2.1 The waterfall model for software development

As we’re on the architect road, I should point out the nature of the waterfall: it's not really there. We are using something with which people are familiar to make an unrelated point. This technique is called abstraction; we take a common object and place it in another setting. Doing so helps both ourselves and others to understand the unfamiliar by comparing it to something familiar.

Using the waterfall is creating a model, or an overview of the process. This is the basic skill an architect requires. You must be able to model, to abstract. You must be able to build high level models and detailed models, and then models of the same thing from different perspectives. Imagine a bridge designer having two models, one made from cardboard and cotton shows the form, or shape, of the bridge. The other is a mathematical model of nodes, connections, strengths and degrees of freedom used to calculate the bridge's static and dynamic strength. Similarly in software, you may model a view of the user interface and a view of the class structure supporting it. You may create views of the instantiated classes at runtime, and examples of the data contained, then a project view of what will be delivered to the customer, and when.

Figure 2.1 is an example of a high level model. It does not tell you how to gather requirements or test something, but it does tell you it happens.

The waterfall is being used as a metaphor, an abstraction or a model of what we are really discussing. You will use the same method to discuss your intended software products with your sponsors and peers.

The amount of detail you need to put into each model depends on its intended audience. The sponsor may need only the highest overview. A developer will need far more detailed information. The more detail you put in, the less the model is abstracted.

Figure 2.2 Level of abstraction versus intended audience

The waterfall model can be further detailed in a number of ways. Some show a falling then rising waterfall, commonly known as a V model, with testing activities to match each of the development activities. Others show a many stepped linear waterfall or a linear waterfall with feedback for an iterative process. The wealth of different versions of the waterfall show the strength of the analogy to software development. It is very useful as everyone can understand it. We shall return to our waterfall model many times, turning it into an incremental, iterative software development process, rather than this rather worn and slippery linear one.

The waterfall has been around for a long time, and there are some wonderful varieties to be found. Later on we shall look at other attempts to model the development process with spirals and words like inception, elaboration, transition, risk confronting, and other such lengthy words. If you learn them all, you will be able to confuse almost anyone. Unfortunately, many such etyma have been given a place in visions of the software development process. By the end of this book you will hearken back to the simple waterfall with fond memories.

How to apply the waterfall to software development

Although the waterfall has been supplanted in many development environments by methods enabling their proponents to be paid more money, it is still the foundation of all the superior processes.

Simply put, software development begins, travels through a number of phases, and in the final phase a software product is delivered to the project sponsor.

Our rather simple four stage waterfall shows that first we must gather requirements. We must interview our sponsor and find out what the software must do. We then go ahead and write the software. When the software is complete, it is tested. If we followed our model rigorously, the next step would be to release the results of our project.

As we are all painfully aware, this never happens. The testers always find something wrong, or create mountainous disagreements on font colour, size etc. To and fro the product goes between development and test until all of the disagreements have been argued away, and some of the obscure bugs have been found and possibly fixed.

Finally, after much agitation and argument, the product goes out to the users for its real testing. You call it user acceptance testing, a beta release or a release candidate. They think it’s finished, get on with their job, and complain when it doesn’t work properly. They may even say something like: “That’s not what we wanted.”

These are the problems inherent in the waterfall model.

The waterfall should show a little turbulence between test and development, and then slightly larger amounts of turbulence once it hits the users.

Figure 2.3 The reality of the waterfall process

Software development is an inherently complex process. When it is simplified down to the waterfall model, it begins to look easy. When things look easy, people assume that they are. Assumptions of ease lead to disappointment.

To overcome such disappointment, we have created more complex models. The waterfall has been supplanted by more rigorous statements of how we are meant to go about what we do.

Chapter 3 - Project initiation

When does the project begin?

We will continue with our waterfall model for a while as it serves to delineate the tasks of software creation. In the next few chapters, we will look at the steps of the waterfall, concentrating on one step at a time. To begin, we require a new step at the beginning of the waterfall to discuss project initiation.

Figure 2.1 The waterfall model with project initiation

Exactly where a project begins depends very much on the political environment. Here are a few scenarios of what might occur.

Scenario 1. Software development team working for an external client

A salesperson makes a sale. She calls in a Project Manager, and together they begin to scope the work to find out what is involved.

Scenario 2. A new Software Architect is hired

A project is already underway, but is ill defined. The software architect extracts a design from the work already being done, then defines the direction or redirection of the project.

Scenario 3. Software development team working for internal client

The chief of an organisation calls in her favourite analyst and reels off a set of requirements. The analyst is then asked when the software will be ready.

Scenario 4. Internal development

A software architect extracts the need for a common component for addition to the component library. A Project Manager is assigned to the task, and a project initiation document is created

Scenario 5. Internal development

A project, having spent a year on the back burner is suddenly required. The work is assigned to a Project Manager.

These are only five scenarios. There are many more. If you are working for a software house developing solutions for external clients, the interface between you will be well defined, and is probably a Salesperson, Account Manager and/or Project Manager. If you are developing software for use within your organization, the interface will change constantly, often on whim, and be based around individuals rather than roles.

We shall take the more traditional route, in which a project begins with it being assigned to a Project Manager.

Project management may also take place within the realm of a larger idea enclosing all of the projects. This is programme management, led by the Programme Manager, and usually takes place at a higher strata in organisational structures than project management.

Programme management

A busy software development department or organisation will be managing more than one project at a time. Programme management consists of many of the same tasks as project management, but always at a higher level of abstraction from the project. It involves:

• Scheduling projects on the master programme plan • Managing dependencies between the projects • Managing the high level programme as a project in itself, complete with

all project management tasks of cost, risk, resource, communication, quality etc.

The programme is subject to many of the same problems a project is. For example, if a project in the programme slips, i.e. will not be delivered by its

expected delivery date, then the programme must be adjusted to allow for it. If the project has other projects dependent upon it, they will slip too. The difficulties in managing the programme are not dissimilar to managing tasks within a project, but are often more political in nature. As a sponsor, if your project is going wrong, you need the Project Manager. If it is not going, you need the Programme Manager.

Large software projects can also be broken down into a set of smaller projects. Under such circumstances, the large project’s interdependencies and tasks are managed as a programme of work. This is the key to success on larger projects, and is the place where a Software Architect can create true technical masterpieces, or a collection of stovepipes.

A stovepipe is a narrow, vertical software solution. There is a top, a bottom, and a raft of bespoke components and interaction in between, which never break out of their column. They will be discussed later in antipatterns.

It is within the realm of the programme or large project where the software architect will focus, for even when working on small projects, the architect must think at programme level for reuse, and a common look and feel for all software. It is debatable and somewhat arbitrary where the line is drawn between a software architect and a software designer. Some aspects of smaller projects, i.e. extracting common components, choosing technologies and defining the environment in which the project will work could be considered the architect’s job. Similarly, ensuring the components internal to the project work correctly and defining the user interfaces is largely a designer’s job. As long as it is borne in mind that the no-man’s land between the architect and designer includes both rather than excludes either, then we will more likely succeed.

Managing large projects

Great amounts of work are hard to finish. Encyclopaedia are often out of date before they are finished. Maps of the world soon become inaccurate. Similarly, large projects are rarely completed. The biggest problem with large projects is that they are driven into obsolescence by the technology upon which they sit. If a map of the world can become inaccurate, what chance has a large software project with the rate at which our technological foundations change?

To conquer large projects, they must be divided into small projects. This is the logical leap many make, and it is a good start, but it is only a start. The individual smaller projects must be defined in terms of interfaces with other systems, navigation between them, and their interdependence. A programme plan must be built identifying which projects will be developed and when. Furthermore, these projects will be built upon a services framework*, and use common components, methods and tools to fulfill their shared requirements. Development of these tools and components must be built into a programme plan as mini-projects in themselves.

* A services framework is a layer between individual applications and the operating system where shared services are delivered to the applications.

Project Management

Project management, as defined by the Project Management Institute[1] is:

The application of knowledge, skills, tools and techniques to a broad range of activities in order to meet the requirements of the particular project. Project management knowledge and practices are best described in terms of their component processes. These processes can be placed into five Process Groups:

• Initiating • Planning • Executing • Controlling • Closing

and nine Knowledge Areas:

• Integration management • Scope management • Time management • Cost management • Quality management • Human resource management • Communications management • Risk management • Procurement management

To this I shall add two more areas of concern to the typical IT project manager

• Managing expectations • Managing without supervision

And one that usually defeats the traditional project manager

• Managing technical, iterative work in a loose medium

Many of these activities are similar to what a software architect is doing, except the software architect is leading from a technological point of view, and integrating with existing systems, whereas a project manager is looking at it from the point of view of delivering within the time, quality and personnel constraints of the business. Balancing these needs is the function of the (often strained) relationship between a project manager and a software architect.

Project initiation

Suppose a client calls in a salesman or programme manager, and says, ‘I would like X.’ The salesman or programme manager might then say ‘OK, we’ll look into it.’

The process of looking into it may mean placing it on the programme plan (the one that plans the projects), or it may mean studying whether X is possible. This may be an entire project in itself entitled ‘X - A Feasibility Study’

Let’s assume the project is given the OK. The Project Manager at this point creates a Project Initiation Document, or PID. It may have another name such as Project Overview, or Project Charter. In some environments, it may comprise more than one document such as a brief introduction followed by a more rigorous statement of work.

The document should be created from a template so that all interested parties are used to the format (managing expectations) and will not have to learn a new format each time before extracting the information. Depending on the purpose of the project, it may contain:

Purpose and benefits

Why the project exists, and the benefits it will bring.

Sponsor and client contacts

Who is paying for this project, and who do they consider will provide information to aid its execution?

Business environment

The area of business this project’s deliverables will work in.

Initially desired features

An overview of what the product is expected to do.

Success measures

What project success will be perceived as. This should be a list of measurable, key performance indicators.

Delivery schedule, including primary project phases and outputs

What must be delivered and by when? For example, a prototyping phase may output an end to end prototype through a distributed or layered system, or may model an entire set of pages for the user interface before requirements are detailed. Each phase, their order and their outputs are listed.

Risks

What could prevent the project from meeting the critical success factors, and what can be done to avert those risks?

Primary Stakeholders

A who's who of interested and important parties.

Role assignments

Who will do what on this project?

Preliminary architecture

Of the installed environment. What hardware is it expected to run on and what is the current software environment?

Global or geographical scope

Where it will be used. What languages and cultural sensitivities must be included.

Budget

How much is the sponsor willing to spend?

Dependencies

What does this project depend on, and what depends on this project?

In the role assignments, you are identified as the Software Architect and presented with a copy of the PID. It contains just enough information to begin work in the next waterfall step, the gathering of requirements. If this is a business or engineering system, the process will be led by a business or systems analyst. Your contribution as software architect is to review, help identify existing software which could be used to fulfill the requirements, and identify costly requirements to the sponsor, removing them when necessary. You will also add technical requirements to the business requirements, and extract architectural requirements which must be met to deliver the business system efficiently and effectively to allow for integration with existing and future systems, and also allow for future growth.

References

1. A guide to the Project Management Body of Knowledge, aka PMBOK. Project Management Institute.

Chapter 4 - Gathering requirements

Gathering Requirements

Gathering requirements is a sortie into the unknown. Someone wants something from you; they are even prepared to pay you to get it. All you have to do is get into their head and figure out what it is they need. They may not be very good at articulating their vision, but they will no doubt have a strong case to uphold their desires.

You must also begin to find out from them who the other stakeholders in your project are, and what it is that they want. This is a tough job. What your sponsor wants, and what your users want are often opposing ideas. You have to resolve their conflicts, and clarify their ambiguities.

Every project has one or more stakeholders. They are the people who will pay for your project, the people who will use your product, and the people who will be affected by it. If you are unlucky, it also means those who do not want your product and oppose the view of the sponsor.

When gathering requirements, you must identify all external interfaces to your project, people included. You must talk to them all, sometimes together, sometimes apart, depending on the political situation, and somehow end up with a clear message of what their collective need is.

The high level questions to be answered are: what does the product have to do, and who and what must it communicate with to be able to do so?

If you think this is the easy part of the project, think again. Remember, you are partially responsible (fully responsible if you are analyst and software architect). It could be your responsibility when you miss the crucial question that would have identified the antagonistic subgroup who decides to burn down your building because they do not like the company for whom you are writing software. More likely, you will miss a crucial piece of functionality that has to be fudged into the project at a later date, spoiling your beautifully crafted solution.

As Fred Brooks points out:

The hardest single part of building a software system is deciding what to build. No other part of the work so cripples the resulting system if done wrong. No other part is more difficult to rectify later.[1]

Ask the dumb questions, but ask them with purpose. People who ask dumb questions are usually idiots. You are seeking to understand. Never try to guess what is in your sponsor’s mind. Use walkthroughs or scenarios. Keep an open mind. As you explain back to your audience what you think they have said, slow down your explanations into easy steps and use illustrations where possible. Remember, your sponsor is not a software developer. You must avoid words like database and user interface. File, storage and screen or page may be acceptable.

Requirements are written in the language of the user and are implementation free.

Requirements must be free of implementation details. For example, User must be able to identify themselves to the system is a realistic requirement. User must be able to enter their username and password using the keyboard might appear realistic enough, but when it comes to reengineering the system in the future, the same requirements should be reusable. In years to come, software designers may guffaw at the naivety of using a keyboard and the absurdity of a username. It may also be that the solution stated in the requirement is the wrong solution, and based only on the experience of the user. Enshrining solutions at requirements stage can lead to designs which are not necessarily best.

Why gather requirements?

You may wish to gather requirements:

• To define an architecture, application or component • To purchase software, and discover beforehand what it must do • To aid your choice between writing or buying and customizing software • To gather your thoughts • To gather someone else's thoughts • To give the testers something to base their tests on, other than an all out

bug hunt

You may also wish to avoid gathering requirements.

Gathering requirements blindly can limit you to getting what you think you want, or limit your sponsors to getting what they think they want.

When you do not gather requirements, you do not begin to formulate a solution. This may be the best way to go about purchasing a piece of software. If you gather requirements, then fit each potential suitor to your requirements, you may well filter out the best suitor for your enterprise.

You may also not gather requirements initially in the hope of creating an insightful solution to a problem before becoming lost in, or guided by, the detail. In this case, you will still need to gather requirements, and must sell your prototypical ideas to your users to mould their requirements thinking before compilation.

Avoid the logical conclusion trap and create the insightful solution.

You may consider you have no need of gathering requirements.

If you have ever thought 'hey, I can make my software really cool by doing x', then you are creating a solution to an imagined requirement. You may find it is worth writing down the requirement first and testing it out on a potential user:

programmer: "I'm thinking of putting in a friendly popup animated paperclip to help you through difficult tasks." potential user: "Doh!"

You may save a few units of currency, hours and neurons.

If you have ever thought: I don't do requirements, I write software! You are simply gathering requirements less formally as lists in your memory, or scrawls on post-its. A formal statement of requirements can be tested, discussed, agreed and refined.

So how do you go about it?

Have you ever seen a man - a gendered man in this case rather than a homo sapiens of either calling - rush into a train station late, flustered, and jump on any train about to depart? Any train on any platform, whether it is the right one or not? I have never seen a woman fall to such foible. Then, when the man realizes he is heading north, not south, his agitation becomes fierce and vocal.

The train is a software development team, the man a customer. The journey is his solution, and the wrong direction headed, the software he has been landed with. His requirement was first train south. He got the first train, but unfortunately it was north as he did not have sufficient time to tease out the train direction. The train selection happened within a few heartbeats. Software definition is merely the same lack of time, understanding and communications built into a deliverable product. As gatherer of requirements, you are putting yourself on the platform in the right place at the right time to enable your man to get the first train south.

Such a brave stance in the line of withering fire may or may not be part of your job. If the project is for an external client, it is likely they will provide a business or technical analyst with whom you will work to discover and reconcile requirements. If this is the case, and you should hope it is, then the analyst leads the process. It is the analyst who has overall responsibility for discovering and documenting the requirements. Your input should be to cover possibilities, act as a peer whom ideas can be bounced off, and contribute past solutions to problems to drive the requirements to newer highs.

Two good things can come out of gathering requirements. One is the requirements. The other is to give the early project team a sense of identity and brotherhood. You are in this together.

You must also extract requirements rather than writing down long lists of what the product must do. You must create the fine line between requirement and user whim, and document only the former.

Deliver to needs rather than pander to whims.

You must not expect to get all of the requirements in one hit. It is an iterative process and must continue well into the project. Intermingling your thoughts with the users' is essential to guarantee the delivery of a product to satisfy their needs. Even more importantly, if the users have had significant and constant communication with the software creators, they will get ikiwisi – I’ll know it when I see it - software.

Do not rely on the poisonous practice of document sign off to finalize requirements. It is the equivalent of saying: I have no faith in what I have done, and even worse: I do not trust you, and should we have a disagreement later on, I shall use your signature against you.

Some may argue that document sign off is necessary to force the users into the mindset that they must check and agree the requirements, so that later changes can be avoided, and that the requirements, once met, signal the end of a project, and possibly some monetary flow.

The sponsor and user groups are a part of your project team. You and they are collectively responsible. That responsibility must be stated and accepted by all team members at project inception. You must work with them throughout the project. They must be included in risk management when something goes wrong. They must be consulted on every change, and every step of development and delivery if you are to deliver what they need. Sign off on requirements is to accept that you might not deliver, or that if you do deliver, it will be to their requirements at the start of the project. Three months down the line, the business, or system, may have changed so much that your work is useless. Then you have delivered a useless piece of software, have put up with the stresses, strains, late nights and damaged relationships for nothing. You have wasted your time, and that is in short supply for all of us.

You can gather requirements in three ways:

• Model the best user. Create the software for them and let everyone else rise to their standard

• Gather your individual informants, spend time with each asking probing questions and actively following them through how they perform their tasks, asking pertinent questions and taking copious notes.

• Meet with representative user groups and gather, discuss, present and refine requirements

The three methods above are not exclusive-or, you may intermix them freely.

Modelling the best user - one to one - monarchy

When modelling a heavily manual process which is to be automated, modelling the best user is the best way to go about it.

In other environments, using this method alone is a path for the brave and possibly foolhardy, and should be done with care. From a business perspective it appears, and can be, the best way to get the job done in the least possible time. It can also lead to software dead-ends.

The way to do it is this: sit with your user, and ask them to show you what they do. You write down the process flows, asking questions where you need to dig a little deeper, or understand a little better.

The following questions need to be on the tip of your tongue

• What if...? • How do others go about it? • Is this always the case? • How does that help? • Why don't you do x?

Whenever you can, get paper documents and screen prints. Create activity diagrams, and talk your user back through the process as you understand it. You will find them only too willing to help you get it right, and it can make your job a

breeze. Always bear in mind that you are trying to understand them, you are not trying to make them understand you - that will come later.

Interrogating informants - one to many - oligarchy

Understanding how a few people do what they do will give you the clearest picture. You will not fall into the trap of sating your best user and disabling or alienating the rest. Neither will you sit through the fog of user groups awaiting a clarity which seems forever delayed.

Interrogating informants is modelling a representative sample of users one at a time. User groups are representative samples tackled together.

User group meetings - many to many - anarchy

The last exercise is the traditional one for most analysts. It is also the most frustrating and inefficient. Getting lots of people in a room and discussing how they do - or want to do - things will lead to many drawn out conversations. After sitting through enough user group meetings you may conclude that people cannot describe what they do - they just do it.

Good chairmanship is the key to productive user group meetings, but unfortunately at this level of operation, good chair-people are as easy to find as tears in the rain. It is not a quirk of language by which the leaders of organisations are blessed with the title of Chairman.

Here are a few schedule ideas for getting a user group together. Remember that this motley collection of people are part of your project team. If you can create a synergistic atmosphere in these meetings, you will deliver a far better product than if you create an adversarial atmosphere. I would go as far as to say if you create an adversarial atmosphere, you will not deliver at all. Handing a compiled product to them is not delivering!

Day 1. Preliminary user group meeting – one to two hours

Do not expect to get any requirements out of the first user group meeting, especially if they do not know you or each other. Instead, use this meeting to create a team spirit, and identify your expectations and the process you will go through to find out where you and they are headed.

1. Establish your credentials

Tell them who you are, and make it sound good. Let them know they can trust you to help them. Call people by their names. Nothing is sweeter to a person than the sound of their own name from the voice of another (where cultural practices permit).

2. Explain what the purpose of the user group is

A user group is a team of selected users who will provide insight into their activities for the benefit of the software developers.

3. Explain what you want

An open meeting where all ideas are encouraged. Free speech from all.

Tell them how you will allow everyone to have their say by summing up discussions with a round robin approach. You all end up somehow in a logical sequence (circles are best, but not easy with the way we design rooms and furniture) and each person has their say in turn about their ideas, thoughts and doubts.

Watch governments meeting. ‘Western’ governments meet around rectangular tables and the top person sits in the middle of the longest length. They are most important. ‘Eastern’ governments sit in circles. All are considered equal. Think on that when you arrange your seating.

To silence the obstinate ridiculers, here are two short scripts you might use:

Script 1: "I have seen a great and insightful idea ridiculed into non-existence by someone who just didn't get it."

No-one will want to to be seen not getting it.

Script 2: "Sometimes a great idea will come from a member of a user group. For a moment, fate will hang around the great idea and lead or fall on the next word. So if you hear a great idea, pick it up and support it. If you hear what you consider a bad idea, pause a moment, and wait to see if another can recognize and polish the hidden gem."

The first of these, I would liken to a hammer and a nail. It uses a form of subconscious suggestion to keep people quiet. The second is a carrot and stick. A carrot and stick is always better than a hammer and a nail in the long term as you will raise people's expectations of themselves rather than force them into a brief submission. The greatest leaders balance hammer and carrot. The greatest people need no hammers. You must choose for yourself what you use, or how much you will, or can, invest in carrots.

4. Explain how these ideals will create a far better software product

There will be better ideas, and ideas that can be morphed into software solutions. If everyone has input, then nothing will be missed out of the software.

5. Explain what you don’t want

Rubbishing of other people’s ideas. People talking and not listening - get a conch if you have to.

If you do not know what the conch is for, you might like to read Lord of the flies[2]. One of the characters of the novel, Ralph is all for law and order, and one of his rules is the person holding the conch (a large seashell) is the only one allowed to speak; the others must listen. His nemesis is Jack with his warrior band, driven by their animal instincts for hunting, playing and disorder.

There may be a Jack in your user group. The meetings can dissolve into chaos, but that chaos can be good, as people search for ever more radical solutions. You must ensure that out of the chaos and excitement, you end up with enough information to deliver.

Day 2. Identify preliminary requirements – half a day to one day

Scope out the project, and begin to flesh out the bones provided in the Project Initiation Document.

This session will be filled with discussions around statements compiled by the Business Analyst on how the participants in the user group do their work. Each task will be outlined as a scenario or series of steps, and the group will argue their way through it, clarifying options and preferences as they go.

Further meetings

Then meet once a week, or once a month, whatever you believe will be most beneficial. In subsequent meetings, the user group is informed of what is happening in development and what decisions have been made. Should these decisions be wrong, you have lost only a small amount of time. Some projects find they have followed the wrong decisions only upon delivery. That means they deliver a product, but do not deliver to the needs of the users.

In brief, here is what you want to discover during the user interviews:

1. Who are the stakeholders? 2. Who are the actors with which this product must connect (people, other

systems and objects)? 3. What does the project need to accomplish? 4. How does the user group think that these accomplishments will be best

met? 5. How often do all of the things the product must do happen? 6. How can the success of the project and product be measured?

During all of these meetings, you have a dual goal of extracting information and building your reputation. To do the latter, you may find it useful to observe other technical people and learn from their mistakes before you make them yourself. Some, for example, dive at the whiteboard to force their point of view on the discussion, or fall to lecturing. Others talk too fast for people to follow and often lose words somewhere between their throats and mouths. Keep in mind that you're just having a chat to discover what is done or what they would like to do. Don't start defending a point of view or get upset over some irrelevance. If it gets too heavy or boring, leave. Get a drink or a snack, or some fresh air and go back in when you feel you can contribute usefully once more.

Identifying the stakeholders

The most important stakeholder is the project sponsor. This is the person who will ultimately sign the bill. He or she is by no means the only stakeholder though.

There are three varieties of stakeholder

• People who work for your sponsor • People who work for your company • People, animals, objects or businesses which will be affected by your

product

Everyone who works with you on the project is a stakeholder, as is your boss and his boss. Their pay probably depends upon the successful conclusion of your work, and as a Software Architect, you are likely to be held at least partially responsible for non-delivery.

Question: How many stakeholders does it take to change a light bulb?

Let’s ponder on this a while. You will be facing such thoughts. We shall draw some analogies. In the left column of the ensuing discussion is the story of a light bulb which naughtily goes out. Your family are all in on the race to get it fixed. The kids are tearing around the house in the excitement, banging their heads off table corners. In the right column is the owner of a company wanting some software. His thoughts are racing around in an unfamiliar domain, and he has that dangerous bit of knowledge...

Last time the light bulb went out, you had to get an ambulance because your mother fell down the stairs when her candle went out.

Last time you worked for this customer, he wasn’t happy with the solution, even though you gave him exactly what he asked for. This time your boss has told you to deliver not what he asks for, but what he needs.

The light bulb

It is dark. You have to find a light bulb.

The kids are in on your search, your wife is sure she has one in a cupboard somewhere, but it’s dark. No-one can see. You’re all moving around with outstretched arms.

You can see your mother coming down the stairs, with a flickering candle lighting her way.

Is she going to help you find the

The software project

There is not yet a project vision. You have to create one.

Your development team members have differing ideas for the project, as does the sponsor. Everyone’s moving around in the dark, and slowly creating their own lights.

The sponsor has a bright idea. He will lead you to what he perceives is a good starting point.

He’s heard that files are far quicker

light bulb, or show you how candles are better? You have more control over them you know. When she was young…

Of course, you know that light bulbs are part of a whole system bringing you better and more convenient lighting. A failing light bulb is a minor inconvenience.

than databases and is sure this is the way forward.

You know in this case, that databases are the best way to storethe data. How can you get your sponsor off implementation specifics and onto his business needs?

In the short story of the light bulb and the candle, the primary stakeholders are obvious. It was you, your wife, your children and your mother.

Who else is a stakeholder?

You know for a fact that when your mother carries anything down the stairs, she doesn’t look where she is going. You have warned her many times that carrying things down the stairs is dangerous for her and she should avoid it. She never listens though, and has fallen down the stairs three times in the past year while carrying something. And here she is, carrying something in less than ideal conditions.

If she never fell, then there are no more stakeholders. If she does fall, then who is a stakeholder? Certainly the ambulance driver and nurses and doctors who check her over. If you need petrol to go and pick her up afterwards, then your local garage is a stakeholder, as is the manufacturer of your car and petrol refiner. All that burning petrol is also bad for the Earth, and there are groups who oppose the use of cars now petrol is getting scarce. Suddenly there are millions of stakeholders just because stairs are dangerous. You’d think we would have invented something better by now.

What if you decided to go the candle route? Would the light bulb maker go out of business? Would the national grid be better off? Would the candlestick maker be happier? Would the air be less healthy? Would it be more romantic?

Everyone affected by your project, either directly or indirectly, is a stakeholder. You must decide whether their stakeholding qualifies them for input into the project.

Standing in other people's shoes

In To Kill a Mockingbird[3], I was told when I studied English, the most important lesson to learn is how to stand in another person’s shoes. That is, to be able to see things from another person’s point of view.

If you can see something from another’s point of view, you will be able to understand them far better, and that means you will be able to empathize and communicate with them far more easily.

I do not believe there is a lesson more valuable. I wish I could do it better, and more often.

Make a quick list of the stakeholders in the tiniest project you have worked on. It may look something like this:

Me (top programmer) Bill (my mate who likes writing SQL) Ben the tester (chap who likes finding obscure problems) My boss (old fool, why doesn’t he retire? I could easily do his job) The owner of the company (he always ends up talking to irate customers when things go wrong – I think he enjoys it) The salesman (he always promises too much) The customer (clueless!) The users (brains the size of plants) Their customers (who cares? – I’ll never have to talk to them)

Now let’s stand in their shoes a while, each one in turn. What do they think:

Of you? Of your company? Of the product they want to buy or have bought from you?

Answers:

Me

(Of me) I’m great. I’m the best programmer we’ve got. (Of our company) It’s OK I guess, we get cakes now and then. I don’t get paid enough though. (Of the product) I’ll end up writing most of it, but then I always do. No-one appreciates my efforts

Bill

(Of me) He’s OK, doesn’t know anything about databases. (Of our company) It’s OK I guess. I don’t get paid enough though. (Of the product) I’ll end up writing most of it, but then I always do. No-one appreciates my efforts

Ben the tester

(Of me) I could write better stuff than him. Nothing he gives me ever works properly (Of our company) It’s OK I guess. I don’t get paid enough though. If it wasn’t for me we would certainly put out a load of rubbish. (Of the product) I’ll end up having to tell them how to write most of it, but then I always do. No-one appreciates my efforts.

My boss

(Of me) I wish he’d be more professional, and try to understand the business a little better. I’ll be up for retirement soon, and he may take over my job, perish the thought. (Of our company) It’s OK I guess. I don’t get paid enough though. (Of the product) We must do it better this time. I wish they would listen when I tell them to spend less time thumping out code, and more time thinking about what they’re going to write. I should make them use punched cards like I had to do when I started. It’s a good job I’m here or the boss would spend all day screaming at them.

The owner of the company

(Of me) As little as possible (Of our company) Why don’t they get on and do a proper job. I’m tired of bailing them out. I don’t get paid enough for all of this, and my ulcer is playing up. Oh no, I’ve got a lunch date with Jimmy Wilkins. I don’t know if I can put up with him today. Still, he’s the customer. Why does no-one understand what I have to go through to keep them in a job? (Of the product) Why doesn’t it work properly? I click here, and press this button, and the whole thing falls over. We can’t release this!

The salesman

(Of me) I tell him exactly what the customer wants, and he still can’t do it. I’m not looking forward to that meeting later this week. It’s all his fault. Why doesn’t he do what I tell him? (Of our company) I don’t get paid enough for all of this. I should have a better car. (Of the product) Mon Dieu!

The customer

(Of me) Anorak (Of our company) I don’t know why I keep using them. I don’t know who else to go to though. (Of the product) That’s not what I asked for.

The users

(Of me) Anorak (Of our company) I don’t know why we keep using them. (Of the product) Why don’t they write better programs? They always crash and they never do what we want. I wish they’d ask us when they’re writing this stuff. We’re the users after all.

Their customer

(Of me) They don’t (Of our company) Again, they don’t (Of the product) Look, all I want is a list of all the orders we made for this financial year. The software should be able do that. It’s a simple enough request.

Standing in other people’s shoes is a scary thing to do, but probably the most valuable lesson you can learn. You won’t always get it right, but you can at least try.

Try it next time you’re out shopping with your loved one. Are you the one wanting to visit the twenty-third clothes or gadget shop, or the one expected to show some enthusiasm?

Try writing down all of the stakeholders for the project you are currently working on. Next to each, write down what you think they will expect from the system. If you don’t know, how can you expect to deliver? Go and talk to them. It may be your first journey as a software architect. Share your ideas with them, get some information. Ask them for their ideas.

It is entirely possible that you are shielded from all possible contact with the customer, and that requirements always come through the salesman or analyst. Be bold. Speak directly to the project sponsor or user. You can not afford to play Chinese Whispers with requirements.

Chinese Whispers begins with a sentence given to one child at the end of a row. In turn, each whispers the sentence to the next in line, and the final child shouts out what she heard. The result shows just how different we are to computers.

Identifying the actors

While you are building up the list of requirements, you should keep in the back of your mind that when you begin analysing them, you will be looking for the actors in the system. These are the people and objects with which the system will interact. To better identify these actors, you should always try to write the requirements in the form of subject, verb, object when you can. We shall see in analysis how this will help.

Discovering what the product might do

In the user interviews, you will discover many wonderful things that the product might do. Many people will happily list requirement after requirement, which will merely be their own personal whim. One person might say “All warning messages must be in red”. Another might add “Less severe warnings could be in orange” Yet another may say “I disagree. All warning messages must be in bold black twelve point Helvetica.”

These are preferences, not requirements. Agree to let the setup of the system define typefaces and colours and state the requirement as The system settings will define all colours used in this application. If you do end up with All warning messages will be in dark red bold twelve point Helvetica, except when they do not prevent the current item being executed, then they will be orange, you are losing the battle.

It is even more likely for someone to ask for something logically sensible (at least to them) yet technologically absurd. This is when negotiation by your good

technically astute self will shave weeks off future argument. You might say something like…

"… so instead of the eSandwichRequest automatically beaming you to a sandwich shop in Barbados when you fancy some spicy prawns, I think we can agree that the canteen will deliver to your desk just as effectively."

Digging in

Don’t be afraid of digging in through the layers of need. It could save you a lot of time in the future. Consider the following conversation:

User: I need it by Friday because we run invoicing on Friday and that means we get paid earlier by the customer. Analyst: How does getting paid earlier help? User: It means our profitability is better. Analyst: Does good profitability affect you in any way? User: Yes, I get a better bonus. It is linked to how I control the age of payments. Analyst: And why is your bonus important? User: My wife wants us to get a new car this year. Analyst: Why? User: Smoke comes out of the exhaust on our current one, and she’s into environmental concerns.

The benefit to the business is that aged debt is less. The benefit to the user is that his wife is happier, and therefore he is happier.

Needs are usually very delicate things. Imagine if our user went home with an amazing new exhaust invention which filtered out everything from the exhaust fumes. You might imagine his wife would be happy. He might discover that she really wanted a new car and was using the environmental impact of the current fumarole as a means to getting one.

Deliver the business needs, and your system will be acceptable. Deliver the users’ needs and they will be happy.

What is a requirement?

According to the IEEE:

(1) A condition or capability needed by a user to solve a problem or achieve and objective. (2) A condition or capability that must be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed document. (3) A documented representation of a condition or capability as in (1) or (2). [5]

Here are a few examples of a requirement

• The customer must be entered into the system before orders can be made

• All access to the system must be secure and encrypted to our standard operating procedures

• The system must respond within 1/10th of a second 90% of the time • A salesperson must be able to check the stock level of a product while

placing an order

Some people prefer to think of requirements as being a certain type. Functional, non-functional, technical, inverse and constraint requirements may be considered as requirement types.

Functional requirements

These are the fundamental building block requirements. Each is a statement of exactly what the software must do

• A salesperson will create, change and cancel orders • A salesperson will be able to check the stock level of a product at all of

our warehouses during order entry

Non-functional requirements

Some refer to these as global or system requirements, or call them the ~ilities. They are concerned with security, performance, portability (across operating systems), testability, understandability, etc.

• The user must have identified themselves before gaining access to the system

• The system must respond within 12 seconds to any user request more than 95% of the time at any global location and within 3 seconds 95% of the time at head office

• The system must support partially sighted users

Technical requirements

Some requirements documents will be composed entirely of technical requirements. If you (as architect) have extracted the need for a common component from a set of business requirements, then the component can be spun off as a separate project, and delivered against a set of technical requirements such as:

• The component will use a set of pre-defined transforms to convert the incoming message to an outgoing message

• It will use X.400 or secure sockets to communicate between the vendor and the service provider

• The components will sit inside our operational web framework and be written in Java

Inverse requirements

What the system must not do.

• A salesperson must not be able to change the credit status of a customer • Employees must not be able to see other employees’ salaries in the

people browser

Constraints

Often, software systems are expected to run on hardware already in place, or meet previously defined look and feel specifications. It is possible that it must run on specified hardware under security already in place, or be delivered according to a certain methodology. All of these requirements are constraints.

• The web application must be fully usable within an 800x600 pixel window • All screen designs will conform to our corporate branding • It will be delivered by February to fall into our 4 monthly integration

testing cycle • The application will run on the mainframe

Architectural requirements

Architectural requirements are added as a means of moulding a software project to an encompassing architecture, or extracted from one project to provide a foundation for others.

• All dropdown lists must be initially limited to 50 items to minimize network traffic and latency.

• Reordering of lists must be done on the client without a server round trip • The PayBuddy component framework will provide all payment processing

It is not necessary to group requirements according to these headings in a requirement document. The value of being able to identify a requirement's type will help you to understand how and where to deliver it.

Discovering system throughput

During requirements gathering, you must identify the system throughput. The questions to answer are:

• how many users will be using the system at any one time? • how many potential users will it have? • how many transactions, requests, posts or messages does a user

create in a given time? • what is the time distribution of that use?

Consider the following conversation about a sales ordering system:

Sales Person: Jill Bird ‘We take a hundred orders a day each, and there are twelve of us.’ Sales Person: Gill Warden ‘Yes, but most of them come in between 9am and 10am or 3pm and 4pm, before the order cutoffs. Yesterday I took about twenty orders before the morning cutoff.’

Business Analyst: T. Leaf ‘Are there any days or months where more orders come through than normal.’ Managing Director: Lettie Hout ‘Here is our monthly distribution’ – presents chart marked Evidence 1 – ‘We also get more orders on a Monday, probably twice as many as on other days, and of course, there are always the post holiday explosions.’ Software Architect: S. Capee ‘How many order items are there on each order.’ Operations Manager: Cliff Gard ‘Our average is 1.6. We’re trying to get that up to 2. It depends on the customer. At the moment we take a maximum of fifty lines per order, which occasionally means placing two orders for the larger customers who compile their orders and place only one a day. What we really want from the system is to be able to get in the order details as quickly as possible.’

From this discussion, you can see an order most often contains only one or two items and you have enough information to write a detailed order distribution graph by day, week, month and year. You have no excuse now for an overloaded ordering system. The last ‘quickly as possible’ needs assessing. Does it mean they want to be able to type quickly? Do they want a digital telephone system to pop up order details based on the caller’s telephone number?

Discovering how the project and product success can be measured

Software products must save time, money or effort. As more effort takes more time, and time costs money, we usually address all three.

There may be a requirement of

• Reduce the processing time from 30 minutes to 10 minutes

Such as requirement can be broken down into a benefits case. The cost saving benefit in pure processor time will save the company $20 per process. At 100,000 processes per day, that will save in the region of $2M per day. If other effects come into play such as a faster time to market, less stock inventory etc, then they too can be built into the benefits case.

Benefits cases must always be balanced against development phases. Hit the biggest, easiest benefit first, and measure the benefits case against each phase as you go. There will be diminishing returns beyond the first phases, and you must ensure the benefit is always bigger than the cost of development and delivery.

One expression for this approach is the 80/20 rule, where you hit 80% of the development by doing 20% of the work. I prefer the term gather the low hanging fruit. It is for people to grasp.

How to specify requirements

Requirements are usually displayed in a requirements shell. The shells are used to ensure information is complete, and can be traced back to its originator. Here is an example of a shell:

Requirement No. 1 Use Case or Event No.

1

Description We must monitor the following environmental states: temperature (internal and external), light intensity, humidity, external wind speed, pollen count.

Rationale Although all of these measurable states must be available, not all will be enabled, nor connected to an external sensor or group of sensors. Yet these are the conditions we display in the presidio product

Source Engineering (Pete Weaver) Test All controller inputs must be calibrated and tested

over one week to monitor drift. The output of the unit must be mapped against the calibration unit.

Supporting Materials

\\DocStore\Reports\Drift

History These are the environmental factors we monitor, and have built our business on

Must have X Should have

Dependencies Marketing material is in preparation

Could have Would like

Conflicts None

Figure 4.1 Requirements shell

Requirements are supported by walkthroughs, or scenarios. In these brief plays, the stage is set, the actors appear, they fulfill their obligations, then exit. For example:

A customer calls on the telephone. A salesperson, already connected to the system, identifies the caller’s account and enters the account’s initial screen. The display represents the status of the customer, the options available to them, and any special offers they might like to take advantage of. The customer opts for the free see-through stapler, requests the current balance of her account and decides to place an order. She asks for three packets of pencils, a pencil sharpener and a stapler, requesting the price of the total order before confirming that the order is complete. She then hangs up the telephone.

Scenarios such as this are very useful when identifying the detail of the requirements, and the conditions by which they may be fulfilled.

The document or system for recording the requirements must be formatted in a standard way, ideally from a predefined template. It is confusing for developers or anyone else receiving many different formats of requirements document.

Presenting requirements to software developers

Requirements are presented in the shell of figure 4.1 and supported by other pertinent information. They should not be presented in any hierarchy. Relationships are defined by the dependencies and conflicts of each requirement. Number them 1, 2, 3 and don’t get carried away with alternate numbering schemes.

If you feel the need to try a numbering scheme, cast your mind back to the days when computer programs had line numbers. We used to make a real mess of that, and there are no renumber routines in requirements gathering.

The other information in a requirements document sets the scene for the requirements, and builds upon the knowledge of the PID. For example, you may include:

The current situation: How the work is carried out at present

Stakeholders: Who they are. What their involvement is.

Roles/Goals: What are the roles involved in these requirements, and what goals or activities are associated with the roles.

Assumptions: What is assumed by these requirements.

Constraints: What environment do these requirements fit in.

Dependencies: What are these requirements dependent upon, or what is dependent upon these requirements? Also, what is this project dependent upon, and what is dependent upon this project?

OK, I’ve got my requirements, what now?

When you have what you consider is most of the requirements, you must analyze them to check that they are stated clearly enough for you to build some software. This is the process of analyzing the requirements.

We need to stretch our waterfall a little more at this point, and introduce the falls of analysis and design, which precede writing code.

Figure 4.2 The waterfall model showing analysis and design

Analysis is very much a part of the requirements process, and should continually feed back into the requirements themselves.

Requirements beget requirements

Think about the Apollo Program. It started with one requirement: Get a man to the moon. That probably triggered another: and get him back safely. From there, requirements grew like leaves on a tree.

The reality of requirements

If you have ever tried to draw anything in a word processor, or write anything in a graphics application, you will have seen the limits of the requirements process. I am sure that the word processor users wanted to draw. I am equally sure the graphics users wanted to write, but most of this software is not written by gathering requirements. It was written by programmers with hives of bees in their bonnets. All of the original spreadsheets, word processors, photomanipulators, vector graphics packages, databases and games were produced with only very high level requirements.

And in this software, the reality is that until it was programmed, it was unlikely that it would have been specified as a requirement. Can you imagine Joe Public specifying that he would like to be able to use Phong or Gouraud shading before it had been thought of by the development team? The benefit of this approach is that groovy new software can be driven out by talented individuals. The problem is that by the time requirements are specified - I want to draw in the word processor; I want to write in the graphics package - they become too difficult to do.

In business and technical software, the situation changes. The experts are the potential users. Here, the development team will begin with the least understanding of what the software must do, but still the best understanding of what it can do. This is where synergy, cooperation and collaboration can make superb software; it is also where the greatest political challenges lie.

Where can I find out more?

Requirements gathering is discussed as a subject in its own right by many authors. Perhaps the high regard in which these books are held is an indication that the people who can best gather requirements are those who can most clearly understand how to articulate them in written form [5], [6], [7].

References

1. No Silver Bullet: Essence and Accidents of Software Engineering, later reprinted in Mythical Man Month, 20th Anniversary Edition. Frederick P Brooks. Addison Wesley

2. The Lord of the Flies. William Golding. There are many imprints of this novel.

3. To Kill a Mockingbird. Harper Lee. There are many imprints of this novel. 4. IEEE standard of Software Engineering Terminology. IEEE Std 610. 12-

1990 5. Mastering the Requirements Process. Suzanne and James Robertson.

Addison Wesley. 6. Exploring Requirements: Quality Before Design. Donald Gause, Gerald

Weinberg. 7. Software Requirements: Styles and Techniques. Soren Lauesen.

Chapter 5 - Requirements Analysis

Introduction to requirements analysis

Often, the act of analysis is a no man’s land, lost between requirements and design. Many are unsure where gathering requirements ends and analysis begins, and also where analysis ends and design begins. This confusion has occurred because of our natural delineation of time and actions into comfortable compartments. All three of these processes revolve around one another. Rather than try to delineate the separate actions, it is better to consider the work products. Production of the documents and models then helps to put time and action into logical compartments.

Another confusion is the overuse of the term analysis. Many analytical techniques are required in the creation of software. Foremost among them is domain analysis, and the most prevalent domain is business, where a business analyst gathers requirements by analyzing the business. Other domains requiring analysis are electronic or robotic systems, or areas of science or health.

Martin Fowler's Analysis Patterns[1] discusses domain analysis. He looks at observation and measurement (primarily in the health service), inventory, accounting and trading. In such large and complex environments a lot of analysis is required before, during and after gathering requirements.

Another broadly used term is systems analysis, from the structured systems analysis and design method (SSADM). Systems analysis is a conglomeration of analytical methods. Feasibility analysis, cost/benefit and risk analyses, and

objective and constraint analyses may be required along with hardware analysis to determine machine build specification.

If you are building a newer version of an existing product, you will have to analyze the previous version. If it shares libraries with other products, you may have to carry out reuse analysis and impact analysis to discover what might happen if you break another application.

All of the ways the word analysis is understood makes it important to identify the type of analysis being undertaken and clearly communicate the scope of analysis.

Requirements analysis gives us:

1. Requests for more domain analysis 2. A decision on whether we need a new project or activity to look at cost,

benefit or feasibility 3. Additional requirements, deleted requirements, clearer requirements 4. A foundation for our design comprising actors, use cases and process

flows

Requirements analysis will be considered using three examples of potential software systems. Two are straightforward, one a clock and the other an ATM. The third is slightly more complex, and is a sales order processing system.

Requirements Analysis

When requirements seem almost gathered. When user workshops are beginning to uncover less. When the phoning, cajoling and confusion over what the product will do are reducing. When scenarios are written, and when the frequency and timing of the actions represented by the requirements are identified, analysis will become the dominant task. During analysis you are testing the requirements to see that they are complete, that you fully understand them, that they are possible, and that you have enough information from which to create a software design.

This work will take place during and between the user workshops. The reason it is separated from gathering requirements is to clarify the separate goals of gathering and analyzing. However, separate goals do not always mean separate activities. I would go as far as to say that until you have analyzed the requirements, they are not fully gathered.

During the analysis phase, at the project level, you will:

• Identify all the use cases of the system • Identify all of the actors, and their relevance to your work • Identify the primary, secondary and exception pathways through each use

case • Identify the main system classes, objects and relationships (or

interactions) • List the technological options available for this project

• Identify the navigation required within and between use cases from the point of view of the actors

• Create rough mockups of any user interfaces • Identify the intended deliverables of the project • Create a list of the terms used, and the definition of any data items so far

discovered. In a large project, or an ongoing one, these definitions will be recorded in a data dictionary.

• Divide up requirements into delivery iterations

At the architectural level, you will:

• Extract common components • Identify additions to frameworks and common tools • Add requirements to make the project conform to the architecture

Use Cases

Use cases were presented in the section on modelling. To better understand what a use case is, let’s identify the rather well worn use cases of an ATM, or hole in the wall money machine.

When identifying use cases, simply write down a high level list of all the things that the system does, or will do. In the case of the ATM, depending on its complexity, you may be able to withdraw money, pay in a cheque or cash to your account, or view your account's balance.

Here is the Use Case diagram for the ATM.

Figure 5.1 ATM Use Case diagram

The ATM is shown in an enclosed square as a set of use cases. The ATM customer is external to the ATM, so is shown outside the box.

For more on modelling an ATM, see [2]

Use case models can be very useful in reverse engineering existing applications as well as modelling new ones. The use case model of an application is probably the highest level of abstraction you will use. It briefly states what the application does and for whom it does it.

It must be easily understood. If you end up identifying a hundred use cases, either your system should be broken down into a number of smaller projects, or you are modelling use case pathways rather than use cases. If you end up with one use case, then you need to move down a level of abstraction or get a bigger project.

Mapping Use Cases to Requirements

Once you have a use case diagram, you can then map use cases to requirements. If possible, testers can be brought in at this point to begin to get a feel for the product, and to run this first test.

Figure 5.2 Use case to requirement mapping

Some requirements will belong to no specific use case, but will apply to all of them. These are requirements for response speed, or user interface look and feel. In some cases, it is advantageous to create a separate use case for unmatched requirements. In other cases, the requirement may apply to one or more of the use cases you have already identified. In the example of figure 5.2, you may find a loose requirement for security, and yet another for look and feel. You decide to add a security use case for the security requirement, and apply the look and feel requirement to all use cases which have user interfaces.

Figure 5.3 Use case and requirement relationships showing look and feel and system security

The following use case diagram shows how a security use case might be shown in a use case diagram. Notice that the requirement for look and feel has no mention in the use case model. This shows why the above type of grid is useful to test for complete coverage of requirements by the use cases.

Figure 5.4 A use case diagram modelling security

Modelling security in a separate use case does not imply it will be implemented separately. All use cases could have been shown as «inherits» security, implying an implementation of security as a base class. Some modellers claim such decoration in a use case diagram is unnecessary. Analysis is only the beginning of a process in which you understand and design your killer software. Analysis models are often thrown away, as the analysis is more a voyage of experimentation and discovery. Implementation details are defined in the design stage.

Detailing Use Cases

Consider the following narrative:

‘Harry always fetches a battery when the current one has run down. He can’t put it in though, that’s Sally’s job. Sally sets the hands too. Unfortunately, she has a bit of a problem with which number the hands are pointing at, so Harry always has to do the reading. He’s just told us it’s half past two. I can read the clock, but as he always announces it, I don’t need to.’

You have been called in as a Software Architect, and you are alone. The sponsor is the leader of a department comprising the above people. He is tired of the constant time calling out towards home time, as that is when he is most busy. He wants some software to solve his time-calling problem.

To begin the analysis, the conversation is broken down into subject, verb and object tasks. In doing so, it is important to identify the role rather than the person, assuming that the operation would carry on if one person left the team and was replaced by another equally skilled soul.

Subject Verb Object Frequency Occurrence pattern

Result

1 Battery fetcher

Fetch Battery 4 times a year

Periodic A new battery is procured

2 Battery fitter

Fit Battery As above As above New battery is fitted

3 Hand Set Time Once a Periodic Time is

setter week correct 4 Time

reader Read Time Ten times

a day Aperiodic, exponential increase near home time

Time is known to Time reader

5 Time announcer

Announce Time As 4 As 4 Everyone knows the time

In summary, the battery needs fetching and fitting, and the time needs setting, reading and announcing.

We have two objects, time and battery. We have five user roles of battery fetcher, battery fitter, time setter, time reader and time announcer. Although Harry fetches the battery, reads and announces the time, these are separate roles, and should not be combined into one just because they are done by the same person. At implementation time, we may have a time announcer ability, a time reader ability and a battery fetcher ability, all of which Harry, the person, can do.

We also know how often these activities happen, the pattern of occurrence, and the result of the activity. That is quite a lot of useful information about the system.

The frequency and occurrence patterns are used to determine what hardware is required, and where most optimization effort should be applied to the software.

From this, we can identify and number the use cases of the current system. At first glance, we might try this...

1. Procure battery 2. Insert battery 3. Adjust time 4. Read time 5. Distribute time

...only to realize read time and distribute time are hardly separate use cases, but separate activities in the same use case. It is also unlikely the battery will be part of the software system. Once this is identified, its continued discussion will be fruitless. Thus, we end up with a choice of having a use case with read and distribute time, and another use case of adjust time, or a single use case containing all three.

1. Adjust time 2. Read and distribute time

or

1. Use clock

Figure 5.5 the activities of time

Maybe there are prerequisites and post-requisites for these use cases. For example, as this is a work clock, the employees should all have arrived at work before the time machine can function. Knowing the time requires some sophisticated teamwork. If one member of the team is missing, there will be no time, so they won’t know when to go home. Maybe your software will need to take this into account.

Each use case should be given a brief description. In the case of the ATM, you may write something like this:

All of the following use cases require the ATM customer to insert their card and enter their PIN (Personal Identification Number) before proceeding.

Use case 1. Withdraw money

The ATM user specifies the amount of cash and receives their money.

Use case 2. Get account balance.

The ATM user requests and receives their account balance

Use Case 3. Post cash or cheque

The ATM user takes an envelope, puts in their cheque or cash, and posts it into a secure receiver.

Following, or during each use case, the card is returned to the user at an opportune moment.

While the example of an ATM might not strain you too much, I am sure the use cases you define for your business or system will be more difficult.

You may decide to present use cases in a shell much as you did with requirements. It depends on the complexity of the system. Here is a more complex use case presented in a shell:

Use Case Number

1

Title The system allows the user to search for goods in stock

Actors Salesperson, stock controller, warehouse picker

Preconditions The inventory of the warehouse is held in the system and is maintained up to date

Description The use case starts when the user enters the ‘search for goods in stock’ screen. The user enters a product code, supplier or partial description, then the system does an appropriate search, displaying the results of the search in a list. The user then selects from the list, and the details of the selection are displayed. Only one item may be displayed in detail at once, and the user may return to the list or where they were when they initiated the search

Alternative flow

If no stocked item matches the search, then the user may request a search of goods from their suppliers. See use case 3

Exceptions If the system fails at any time during the search, then the user must know why. Reasons for failure may be licensing issues, known or unknown exceptions such as system crashes, data locks etc.

Assumptions The users are aware of the terms used in their stock system. See glossary.

Post conditions

As this use case cannot edit data, nor order stock, the only post condition is that the user is returned to the screen from which they initiated the search.

User interface

{rough picture}

Business rules

Only the above mentioned actors will have access to this system. When showing product details, cost prices must be omitted for salespeople.

Associations Constraints Risks Issues

I: Before this use case is implemented, we need another user workshop to agree the information titles

Reference 1, 2

Revision log

Modified by Date Version Description

IB Curley 01.01.22 0.1 Draft

The shell can be useful to ensure information is not misplaced when it may be required later in the project.

The primary, secondary and exception use case pathways

Each use case will have a number of possible courses of action through it. Each course of action is documented separately, and identified as a primary, secondary or exception path through the use case. Depending on the complexity of these paths, they may be modelled separately or together.

The primary path

This is the most obvious, and most often travelled path through the use case. It is referred to as the happy path. For example, in our sales order system it might be:

1. Customer telephones salesperson 2. Customer and/or salesperson identify account 3. Customer identifies whom the order is for 4. Customer lists all required items 5. Customer states payment method 6. Salesperson confirms order

Secondary paths

There may be no secondary paths for your use case, although this is unlikely. Here are some secondary paths for the order processing use case.

1. Customer telephones salesperson 2. Customer and/or salesperson identifies account 3. Customer selects a ‘quick’ code from a series of pre-defined order

templates 4. Salesperson confirms order

1. Order is emailed in 2. Salesperson identifies account 3. Salesperson enters order items 4. Salesperson emails customer to confirm order

There may be a great many of them, and it may take some effort to document them all. Not documenting them is a sure way of delivering software that doesn’t meet the recipients’ expectations and fails to help them do their job.

Exception paths

Usually considered last are the exception paths. This does not mean they are any less important than the primary or secondary paths. Here you model error traps, or unexpected exits from primary or secondary paths.

1. Customer telephones salesperson

2. Customer/Salesperson identify customer’s account 3. Salesperson enters order items 4. The call is interrupted 5. The salesperson saves the order as a non-completed order

1. User enters invoicing 2. User begins invoice run 3. System crashes 4. System is rebooted 5. User rolls back previous attempt at invoicing, and begins again

Presenting the paths

Use case pathways can be presented in a numbered sequence as shown above, or by using activity diagrams. Figure 5.6 shows an activity diagram for a typical sales order processing system.

Figure 5.6 Order processing activity diagrams - primary path(left) and all paths(right)

Using Scenarios to detail and agree use case pathways

Use cases and requirements on their own can leave a few gaps in the transfer of knowledge between customer and developer. The most convenient way of plugging these gaps is to use scenarios.

An example of a scenario is this:

Scenario 1. Use Case 1. Withdraw money Related Requirements 1, 2

An ATM user enters their banker’s card in the card slot. The machine reads their identity from it, and links to the bank’s main system. The user then enters their PIN number - a 4 digit numeric code - which is verified by the bank’s system, selects an amount of cash from a menu (e.g. £50) and selects to have a receipt printed. The customer then receives back their card, their cash and a receipt, in that order, and leaves the ATM.

Scenarios in this form are better from the customer’s point of view as narration is more familiar to them, and easier to agree with, or change.

Whether you are using use case pathways, scenarios, activity diagrams or a mixture of all three, you must fully detail all use cases and how they fulfill the requirements. Enough information must be included for the user group to fully qualify them, and for the designers and developers to be able to implement them.

Actors

We have already defined the actors in the clock system as the battery procurer, battery fitter, time adjuster, time reader and time announcer. As already stated, the battery requirements are not applicable, so you are left with three actors:

Actor 1. Time reader Actor 2. Time adjuster Actor 3. Time announcer

Remember the sponsor who is paying for this work. He is not an actor, but he is certainly a stakeholder. He wants actor 3 removed from the process. He wants a single time adjuster so they won’t argue over what the correct time is, and a number of time readers, all reading their own time. He doesn’t want use case 3, time announcer, to ever happen again.

Therefore, the system will have two actors. That’s what you are being paid for:

Actor 1. Time adjuster Actor 2. Time reader

Here are the two actors shown with two use cases of read and adjust time.

Figure 5.7 Use case for a clock

One of the actors for the ATM is slightly less obvious than the other.

ATM Actor 1: ATM Customer ATM Actor 2: Bank’s main system

Both of these are actors as they are external to the ATM. An actor doesn’t necessarily mean a person. It can be an object too.

The sales order processing system will have only one actor. The sales order clerk takes the order from the customer and enters it into the system. There is no direct link between the customer and the system.

Sales Order System Actor 1: Sales order clerk

System classes, objects and relationships

Let’s work on our time delivery software. During the interval between two user group meetings, you throw together a few ideas for the visuals from which they will read the time. You already know that they are most interested in time toward the end of the day, and how the countdown to home time has the utmost importance

Figure 5.8 Time and countdown displays

You present your user group with your ideas, and explain how time can be displayed graphically as a clock, or graphically as a shrinking segment showing the remaining work time disappearing. Shrinking time can also be shown as a bar, or finally, the time can be displayed digitally.

Of course, they look at you as if you are mad. All they ever wanted all along was a digital clock, but their boss would never buy them one. Neither was there any way for them to suggest to him that they wanted one as when they remembered

it, he had his door locked. He always did this in the later part of the day so he could get some work done.

Being a conscientious Software Architect, you bill your customer for the few hours you have spent with his team, buy them a digital clock out of the proceeds, and explain to your customer how he can get hold of you again when his next software need crops up. If you do not know why you would follow this course of action when you could easily develop a digital clock and charge a king’s ransom, it will be discussed later.

So much for the clock. Let’s look inside the ATM; it has cogs and things. It may be fairly simple from the point of view of how it is used, but behind that innocuous small screen lurks a world of software control.

Consider what happens.

A user walks up to the machine and inserts a card. The card is drawn in without the user offering too much assistance, so there is a sensor to recognize when a card has been inserted, and a means of pulling in the card at a constant speed while the magnetic strip is read. There is also a sensor to signal when the card is fully inserted. The motor drawing in the card must stop otherwise it will overheat, wear out and possibly damage the card. When the card is ready, the smart card chip is read and verified. At this point, the screen changes, requesting the ATM user’s PIN.

Software records the button presses, and when it has a complete PIN, verifies it against the bank system. In the meantime, the user is informed of a potential wait while the bank system responds. Following a positive response, the ATM user can now demand money. More user interface work is followed by counting out the money. Sensors, actuators and software combine to count out the money. There’s no room for error here. Should it be counted twice for verification? How would you design it?

Figure 5.9 Analysis of the classes in the ATM

The counted money is sent for collection, but first the card is fed back out of the slot. Your card must be removed before the cash door opened. A cunning safety feature. How many times have you stood there waiting for the cash while your card is waiting for you? You take your card, take your money, and the screen informs you your receipt will be with you shortly. More software, more hardware controlling a dot matrix printer. More software, more hardware feeds the receipt out of the slot. After a pre-determined time, even if you have forgotten your ticket, the machine becomes ready for the next customer.

That’s a lot of software required. Figure 5.9 shows an analysis model of the ATM from the point of view of someone other than the user. It shows all of the control,

interface and entity classes of the ATM which have been discovered so far, and uses UML’s package notation to group objects within the card reader and the cash dispenser.

System objects

Perhaps at this point it is necessary to detail system objects, which are created from the system classes. In the aborted clock application, while it was still going to be a bit of software, you might have had a single class of timeRecipient, i.e. one who heard the time announcer, yet multiple objects would be created from that class.

Similarly with the ATM. Maybe it is a new design supporting a row of five users, five card slots and screens, yet only a single money counter, and an additional class of money router to make sure the correct recipient received the correct bundle of money. Why are ATMs always designed as stand alone machines, helping one user each at a time?

While pondering that, ponder over other products too. Do you put as much effort into your software products as product designers and engineers do into theirs?

The road to Moscow, the triage, the traffic lights, eighty-twenty and the low hanging fruit

It is no accident of nature that few ripe cherries hang under the lower branches of cherry trees. They are picked and eaten because they are the easiest to get to. They are the low hanging fruit.

Some requirements provide a huge benefit to the users and are easy to deliver. They are the low hanging fruit of the software development world, and should be plucked first in any project.

If the cherry tree is mature and large, a picker may be able to gather 80% of the cherries by standing in a few places and reaching out. The outlying cherries at the ends of branches or in awkward and dangerous to get to places, are hardly worth picking. Specialist equipment such as ladders and carefully prepared ground may be required, and more agile pickers. Compared to the 80% which were easy to pick, the final 20% take a huge amount of effort for their relatively small return. As a rule 80% of the cherries take only 20% of the total picking effort; and is known as the 80/20 rule. It is also referred to as the law of diminishing returns as each successive batch of cherries picked takes more and more effort.

In software terms, a project should deliver 80% of the benefits for 20% of the effort. It may even be worth moving on to the next project and reaping 80% of those benefits before delivering any more on the current project.

Once upon a time on a battlefield, triages were performed to filter out which wounded might be helped most expediently. Triages are still used in disasters and emergency situations when medical resources are limited. In a triage, four

classes of people are identified: the dead, immediate, delayed and minor. The dead are left where they fell. Thos with immediate needs are seen to first. The delayed are aided once all of those with immediate needs have been processed. Finally the minor, or the walking wounded, are given aid.

The three classes of immediate delayed and minor are comparable to requirements. Some need doing straight away, others can wait, and yet others are only minor requests. Putting this into the traffic light metaphor of green for immediate, amber for delayed and red for minor will provide a hypnotic suggestion to go on the immediate, wait for the delayed and to stop worrying about the minor problems.

Combining the three divisions with the requirements' Must have, Should have, Could have, Would like (MoSCoW) ratings and the difficulty of doing them, will provide a beneficial and more comfortably defined road along which to travel.

Technological options

When listing technological options, don’t be too eager to display your vast knowledge. If the client already has a platform in use, you should do your best to use it.

For example.

Widget X or Y, which is already in use, could be used for doing Z.

Our platform is either server cluster Everest or K2.

The application should fit in with the previous three we have developed for this customer, and share as many components as possible, including the framework.

No decisions are made at this point whether X or Y is better. They are choices made during design.

If some of these choices are going to be difficult, some discussion of their merits would be wise this early on, so those outside the technical team can understand the choices to be made.

Navigation

Most projects will involve some navigation. To enhance the user experience (remember those words) you should identify how the roles traverse the software system, and also how the people fit into the roles. You may find someone on the sales desk is also responsible for managing purchase ordering. In this case, it would be helpful for them to be able to navigate to the purchase ordering function from within the sales order function.

If you consider how we, as humans, interact with each other, we use our interfaces. We call them senses: sight, hearing, touch, taste and smell. We also use our experience, our knowledge of each other, our joint experiences, our

compassion, empathy etc. We have little need for navigation around those interfaces. It happens all at once, we use them all together, all the time.

The concept of navigation can be compared to altering our frame of reference. We may pick up a paper based ledger and think of it in terms of accounting, then put it aside and pick up the telephone to make a purchase order. Our frames of reference are in our minds. If we can capture that flow and present it on a screen, then our user will be more content with the flow of the system. Navigation means detailing how users see the system, and how it helps them move between different frames of reference.

Navigating items represented on a computer screen is limited to point, click and type at present, and to a limited extent, speech. One day you may be able to smell an overdue document or hear a problem coming. One day turning your head a little will allow you to change your point of view.

Deliverables

At this point, the deliverables may be no better defined than at project initiation. If they are taking shape, a best guess at a deployment model is in order. A simple UML deployment diagram is quite sufficient, or if you must choose between alternatives, then a diagram of each of the alternatives, annotated with risks and benefits, will help others understand the choice that will be made.

Breaking down large pieces of work into a number of interdependent projects.

Suppose a project has been defined to deliver a vehicle insurance claims system. The main high-level activities are as follows:

• Divide work between teams and claims handlers (also known as adjusters)

• Establish who is responsible for the accident, and who is liable • Allow the claimant to claim for injuries, damages and loss of earnings • Make payments to claimant • Make recoveries from a liable party or their insurer • Manage initial contact to assign physiotherapy and reduce the need for a

solicitor • Organize the repair of a damaged vehicle • Assign a replacement vehicle while the claimant's vehicle is being repaired • Assign an investigator for fraud identification or investigation into the

accident • Determine the liable party's insurer and validate vehicle ownership

Figure 5.10 Work breakdown of vehicle insurance claim showing dependencies

Early treatment of injuries by physiotherapy has been shown to significantly reduce the number of sessions required. Similarly, careful early management of a claim can lead to resolution between the insured and the insurer, and thus avoid the additional costs once a solicitor becomes involved. These two items will provide greater cost savings than the rest of the activities combined.

Diagram X shows how the initial case management is dependent upon the set up claim activity. Setting up the claim is dependent upon Allocating the work to someone. However, there is a possibility that this can be a manual process carried out by the team leaders in the initial project stages.

The decision of what to do is a balance of three forces:

1. What delivers some measurable benefit? 2. What are we capable of doing? 3. How much will it cost?

These three forces can multiply into a multi-dimensional decision, based on further environmental conditions. As an example, the above three forces could become ten dimensions in a project or programme, considering:

1. What are the benefits? 2. What are the estimated costs? 3. What can we reuse? 4. How will it fit with our rhythmic delivery cycle of four releases a year? 5. What business process re-engineering is required? 6. What are we collectively (people and tools) capable of? 7. How do we deliver iteratively, focusing on benefits? 8. How do we do integrate with legacy systems? 9. What legacy systems do we link to?

10. How do we minimize the pain will we cause our handlers during an iterative rollout?

All of these dimensions affect how the decision is made of what to do.

The initial work is rather obvious, and must focus on managing the initial contact. To do so, setting up claims must also be included. The second phase of work will have little benefit, but will lay the ground for the rest of the work, and will be work allocation.

If the benefits of the remaining work can be calculated, deliver the highest benefit first. If not, pick any. The last deliveries will be vehicle replacements, vehicle repair and investigator assignments. Recovering money is likely to be closer to the heart of those pulling the strings, and recoveries has a dependency chain before it, all of which must be delivered beforehand.

The project will cost 130k, and at the end of the work will deliver 26.6k benefit per month. Adding in operating costs of 5k per month, the break even point will be in September.

Figure 5.11 High level programme plan against costs and benefits

List of terms and data dictionary

For our clock, should the project have gone ahead, we may have identified:

Time - that elusive snatcher of our lives. The wave on which we travel and cannot control.

Hour, minute and second – increasingly detailed divisions of time.

Home time – the hour at which we are released from our work obligations.

Lock – the thing the boss uses in his door when he wants quiet.

For a sales order processing, accounting and management system, we might have a data dictionary defining every field discovered, and the characteristics of each item.

Item Description Object property name

Data name Type Size Accuracy

Employee Someone who works for the company

.employee EMPLOYEE_NAME String Variable up to 50 characters

N/A

Salary The annual or hourly wage paid to the employee

.salaryYear

.salaryHour SALARY (held by year only)

Money 999,999,999.99 2 decimal places

Date of Birth

Date an employee was born

.dateOfBirth DATE_OF_BIRTH Date N/A To one day

Age Number of years since Employee’s birth

.age Derived from DATE_OF_BIRTH

Integer 16 bit To one year

During analysis, this can become fleshed out by adding more detailed information. For example, you might define a person as having a grade. The grades the company uses are listed, as are the various terms they use for their clients, assignments, etc. This list of data alone is referred to as a taxonomy. Taking it a step further and defining relationships between the various pieces of information makes it into an ontology.

User group review

When the information gathered and formalized is sufficient enough, it is presented for user review. In the final stages of requirements analysis, the entire team agrees differences, completes outstanding work, and gets a feel of where their efforts are headed.

The final analysis

The findings of analysis will feed back to requirements, and back to the project plan. After analysis, you may decide that the project is actually a programme of work and must be broken down into a series of projects. In the case of the ATM, we had a package for the card reader, and one for the cash dispenser. Both communicated with the rest of the machine via the master controller. In this case,

we could define the master controller, then spin off three separate (although interdependent) projects to deliver the system.

It is in cases such as this where a Software Architect can flourish, by laying the foundation and rules on how the component projects will work, and how they will communicate with each other.

Summary

A process has been followed whereby use cases are identified and drilled into using use case pathways and activity diagrams. Event frequencies and periodicities have been ascertained, and supporting data identified. This approach is referred to as use case led software development.

The activities of analysis are those of understanding and documenting. How to reach understanding is the subject of [3].

References

1. Analysis Patterns. Martin Fowler. Addison Wesley. 2. The Unified Software Development Process. Ivar Jacobsen, Grady Booch

and James Rumbaugh. Addison Wesley 3. Problem Frames: Analyzing and structuring software development

problems. Michael Jackson. Addison Wesley.

Questions

1. Redraw the ATM showing its two use cases of customer and bank's main system

2. Identify the use cases of a website where you can order books. 3. Write down the primary, secondary and exception pathways of an ATM,

and draw a single activity diagram containing all of the pathways. 4. A simple word processor can load, edit and save files. How many use

cases does it have? 5. An application requires 4 developers at a cost of 5k each per month, and

will save the company 20k per month once it is complete. The development estimate is 4 months, test one month and deployment phased in 10 equal steps over the next month. After the start of development, when is the break even point for the project?

Chapter 6 - Design primer

Introduction

Analysis is all about requirements. You are analysing the requirements to gain an understanding of them, to improve their quality and clarity, and to begin to formulate an appropriate response to the stated needs.

Design is the creation of a technical statement to fulfil the requirements. It will be understood only partially by those who stated the requirements, yet will be sufficiently detailed to lead the developers in construction.

It is a drawing together of the people's needs and the technological and resource limitations and capabilities, to create an artistic and innovative solution. Requirements and analysis are logical endeavours. Execution of the code and release are purely technical affairs. Design is the most creative act as you come to a technical and aesthetic understanding of the problem.

As a designer, you have the same set of artistic and technical tools that everyone else has. What will differentiate your product from their product is your design.

As a rule of thumb, design will take five times as much effort and time as analysis.

The reason for design

Many developers believe design is discovered during coding. Code is a fairly workable medium, and a change of plan can be reflected by moving code around, or rewriting bits of it. Huge changes of plan require quite a bit of fumbling, and often developers find themselves rewriting bits over and over as requirements change. This is a result of failure in the requirements process. It must not happen. The reason it must not happen is that it is a waste. No-one would buy a car whose design was discovered during construction. Why tolerate software created that way?

Figure 6.1 The exponentially increasing cost of change

How would you feel if builders used construction to discover the architecture of your house or workplace? You would be horrified, and the resulting building would be a mess. You would not want to live or work there because it might fall down around you.

In software, we have not always followed the precedent of design before build, and the reason is that code is far more malleable than steel, bricks and mortar. The hero programmer, working long days, nights and weekends, cracks the problem and delivers a solution. Happiness, or relief, abounds on delivery in difficult circumstances, therefore the glory of the hard working programmer lives on.

But hard work in difficult conditions can lead to less structure, less careful coding, less careful error prevention and handling. The resulting software is highly likely to have high defect rates compounded by insufficient testing, and less easily understood code.

The hero programmer creates a lifelong dependency (and possibly employment) between themselves and their code, or creates a nightmare of rework for the next team or generation.

The reason for design is to:

• minimize the dependency between coder and code • create better structured code • allow a view into the solution for agreement and understanding by all

stakeholders before coding begins • reduce rework during coding • communicate to future developers the structure and reasons for

choices made in the software • provide time for investigating and trading off approaches • minimize the coding effort.

Project size

The best unit of work, by common consensus, is a project with three or four developers taking around six months from requirements to delivery. The focus enforced by using few people in a time-boxed arena will provide the impetus for a successful team. A smaller project might not require so formal a methodology, and a larger one, further technical, process and personnel complexities.

Often, the approximate size of project is clear from the outset, particularly with new and replacement work. System updates and rework may require some analysis before the effort can be estimated, but even then, a system expert should be able to provide estimates without a detailed work breakdown structure.

Large scale projects, typically ten developers for a year, must be broken down into a set of subprojects built upon a common foundation. Projects falling inbetween are a matter of judgment.

Figure 6.2 Small and large scale projects

Huge projects, those requiring developers by the hundred, or timescales of years, must be broken down in stages of ever decreasing size and increasing definition. Huge projects are certainly the most risky to undertake. They have a habit of never getting finished, or becoming obsolete before delivery. This is a sad state

of affairs when some organisations excel at huge projects, and with skilful design and management, are more efficient at huge projects than they might be on smaller projects. Efficiency comes with reuse across systems. Reuse comes from good design, and good design on this scale takes a lot of effort. If you are to act as software architect on a large or huge project, you must not have any other role.

This success on larger projects is discussed in Peopleware[1]

Huge projects can also be referred to as programmes of work. A programme manager will take the helm, and coordinate the work of the project managers of the projects into which the programme has been divided. The lead architect will work at the programme level, coordinating the work of architects or designers on the projects.

Tiny projects, where a developer may take a few weeks, can often be shaped rather than designed. This is design based on the ancient art of winging it. Have the developer sit next to a future user, and design it with them rather than for them. Such development is perfectly acceptable, and should be used for such pieces of work. Too many small pieces of development are clogging up analyst and test teams when the best tester and requirement documenter is the user. Involving users to this extent is a fundamental tenet of agile methodologies.

Reducing the project scale

The first key to success on a larger project is to break it down into ever smaller pieces. This division of labour and complexity may occur at any point in the lifecycle, from initial project conception right through to design. If it occurs before design, then some reassembly may be required at this point to appreciate the true scale of the work, and what might be shared or enforced across the projects.

It may seem a thankless task to assemble already divided work, but division from the point of view of shared functionality and technical limitations may yield a different set of projects than division from a purely functional point of view. It will certainly place a different emphasis on the importance and risk of developing shared code.

Figure 6.3 Reducing project scale

At the beginning of the decomposition, a set of subprojects is created. Each project will sit within a common framework and use a common set of tools. At this stage, all internal and external interfaces are defined.

If the subprojects are still large, then break them down again. Add to the toolkit, refine the framework, or create another layer of framework for each successive set of subprojects if this is realistic.

An example of a framework is a layered or multi-tiered set of objects which abstract away operating systems, networks, storage methods, business logic and context to provide more meaningful and helpful operations for the applications and developers. An example of a toolkit is a math library, or set of business calculations, information repository calls, object builders, gui tools and graphical displays.

Each time this subdivision occurs, the subprojects become smaller and smaller, until they can be assigned productively to teams.

To effectively deliver interdependent projects requires time spent defining where those dependencies are, and how and when they will be developed. It is a detailed and demanding task, and must be carried out by, or with the full cooperation of the project managers. Timing and slippage on these shared development are of the utmost importance. If one team is ready to use a component or function call that will not be ready for another month, the most likely course of action will be to write their own, thus circumventing your architecture, reducing reuse and introducing complexity to the code, to testing and future support.

Good architectural design in the structure of the system is more important than in the successive subprojects. It is here, in framework, toolkit and interfaces that the (lead) architect must focus.

Design is an extension of analysis

Many design tasks are extensions of the work carried out during analysis. Technological options are investigated and resolved to technological choices and the reasons for these choices recorded in the design documentation. User interfaces are given detail and look and feel. Object models are defined further with structural and dynamic diagrams.

In some cases, design will simply be a more technical and detailed definition than is achieved during analysis.

Designing large systems

To be able to deliver large systems most effectively, it is vital that there is a single vision for the project. This is most easily achieved by a lead software architect, but could also be achieved by a team of specialist architects working together.

The considerations of large system design are this:

• What is the operating environment and deployment platform? • What technologies are used and why were they chosen?

• What external interfaces will be used/supported and what protocols do they use?

• What internal interfaces are required, using what protocols, exchanging what data with what security/conflict/timeout resolution?

• What framework do the subprojects sit in? • What are the subprojects? • How do they interact with the framework and each other, and how are

timing or data conflicts resolved? • What is the common look and feel? • How is the system accessed and secured? • How is it deployed? • How is information passed and understood between the applications? Do

you need a common data model? • How is information persisted? • What is the subproject development schedule, and how are

interdependencies controlled? • What guidance should be given for development? • How will the entire set of projects be tested and supported?

Package diagrams are probably sufficient at this point for detailing system structure, with state, interaction and sequence diagrams of interacting and collaborating packages.

Designing small systems

Some of the design issues for small systems are the same as for large systems. Where systems and interfaces are defined in large projects, components and interfaces are defined in smaller systems. Within those components, classes (or code structures) and interfaces at an even lower level are defined.

Small system design will consider the following items:

• What is the operating environment and framework? • What technologies are used and how are they used? • What is available to be reused? • How does the user navigate between views/pages? • What are the component parts of the project, and how do they

collaborate? • User interface design and field definition. • The development tasks and schedule. • What will be deployed and where.

Class diagrams will define the static structure within the projects, and sequence, interaction, timing diagrams etc, will define the dynamics. UML's strength is in design, and must be employed to the full within the projects.

All design tasks must take into account the architectural concerns discussed further in chapter 16: enterprise architecture.

Design for politics

There will be occasions when your user group or sponsor demand a solution which you consider will be a worse solution than you have proposed. This blinkered demand is often a result of something going wrong in the past. If you have a better solution and your sponsor is demanding something else, one of you is being pig headed. It is not always your sponsor.

Resolve the problem by refactoring your sponsor’s design with ideas from your own. It will then be their design with slight modifications from yourself in the name of technological feasibility. If you cannot do that, then offer parameterized alternatives or user choices. If someone must have view X, make it the default view, with another (your design) available in a menu.

Without doubt, you will one day hear: ‘I don’t like the colour’. You must avoid all temptations to change it, because if you make this particular user happy by changing the colour, your next viewer will say: ‘I don’t like the colour.’

To avoid colour problems you can ignore people who don’t like it, or use blue. Most people are happy with blue because there are less blue cones in the eye than there are red and green, so it causes less of a reaction. People who have a particular dislike of blue tend to stay indoors to avoid the sky. You are less likely to meet them.

Design tools

There are many computer based design tools, case tools, user interface designers etc. Your development environment provides you with prototyping tools to test out new ideas or components. Yet the best design tools are non-technical. Here are four you might like to use. They are all variations on a theme, being used to simplify complex structures.

Method 1: The consultants’ favourite. Post it notes.

This is probably the consultants’ favourite because it is quite easy to master. It can be explained to groups of people in about a minute, and everyone will understand it and be able to contribute. It is the technique I used to design the structure of this book.

It works by writing down lots of ideas on post it notes. The only rule is one item per note. You write them down, then you stick them on a wall and begin to group them. By doing this for this book, I ended up dividing all my ideas into 18 chapters in 5 sections. It is known as a bottom up method. You start with the pieces and end up with the lumps, and finally, the design. Top down methods would start with the high level design, figure out the lumps and then detail the lumps.

Method 2: CRC Cards.

CRC stands for classes, responsibilities and collaborators.

Figure 6.4 Simple CRC card

Here is the simplest form of the CRC card. The class name is obvious enough. The collaborators are other classes which help this class to do its work, and which other classes are helped by this one. The responsibilities are what the class does and what it knows.

Figure 6.5 CRC Card for a carburettor

This example shows the Carburettor class created by composing it of other classes. This is perfectly acceptable, even when multiple inheritance is available.

The CRC and post it note methods can be used together. Write out some cards and then start sticking post it notes on them. When you’ve finished shuffling the post it notes from card to card, copy their contents to the cards and throw them away.

The Carburettor

I use a carburettor quite often as an example component, so I should explain how it works.

In a car engine, a fuel pump pumps fuel to the carburettor. The carburettor then holds fuel in its fuel chamber, where a float in the fuel rises up and closes a needle valve. The needle valve prevents any more fuel entering the fuel chamber.

The fuel is mixed with air and the carburettor delivers a vapourized fuel and air mix to the combustion chambers of the engine.

Inside the carburettor, fuel is drawn from the fuel chamber by the venturi effect, and when the throttle valve is closed, (accelerator is not pressed) then just enough air and fuel go through a side door to keep the engine ticking over.

As an example component, it is quite perfect. It has about the right level of complexity. It is created from a set of smaller components, and uses a pre-defined technical method - in this case the Venturi effect.

Figure 6.6 A carburettor schematic

It has interfaces to the external world, i.e. the choke lever, the accelerator or throttle, and the line to the fuel pump. Air comes in through the air filter, is mixed with fuel and drawn into the combustion chamber where it burns. The exhaust gasses are then pumped out of the exhaust.

The carburettor is a component composed of other components (by aggregation rather than composition if you are an oo purist), and provides a meaningful piece of work in the system. It has interfaces and takes part in the flow of air and fuel through the vehicle.

When designing classes, you should think about its equivalent in engineering or product design. Here’s what Steve McConnel has to say about design[2], and this can apply to systems, frameworks and classes:

• Use a descriptive name (without abbreviations) • Check the prerequisites • Design the interface • Think about efficiency • Think about the data • Research the algorithms • Define the problem to be solved • Demand peer review • Define tasks and data limits • Change parameters to be as useful as possible • Think about error trapping and reporting

Method 3: Mind maps, also known as spray or thought diagrams

Figure 6.7 Mind map of a car from a driver’s point of view

Here we see a mind map of a car. It is divided into four types of object: pedals, hand controls, dials and comfort. The dials are subdivided into lights, clock dials and numeric styles.

Mind maps are useful for grouping objects from different points of view.

Method 4: Linking the unlinked objects

Finally, a useful method for identifying links between seemingly unrelated objects. It is based upon a system designed for the Dutch police to identify relationships between criminals.

In the police version, it works like this:

Figure 6.8 Tracing links between criminals

Here, we see a suspected international drugs ring. Contacts in different countries have been identified by the local police. Now Interpol have been given the information, and have added their own international knowledge. Geezer Jim in New York has been in contact with French Franc. Franc is in the same group as Duc Url, who works with Fishy Fred, also in New York. A link up between the two New York gangs is imminent, so they are being watched, and at the same time, Interpol officers are watching the ring members carefully in each country. They will build their web, and swoop. If you’re on the list, be careful. Hans CC is the suspected godfather, but so far he's as clean as a whistle. Will they get him? Tune in for next week’s exciting installment...

You can use this method when identifying related routines or objects in existing code. Write down the objects or functions, and begin drawing in the relationships. The more lines, the bigger the rat’s nest. Improving, or refactoring, the code could begin by trying to reduce the number of lines linking routines and objects.

The same method can be used during design to clarify links between components. Once the links have been made, a more logical grouping of components is possible. Imagine our car:

Figure 6.9 Car object collaborations

The relationships between the car’s objects are rather obvious, but this is because of our familiarity with a car. From this technique, it is easy to get to a collaboration diagram, and it also uses very little space to show an overall picture of object relations.

A diagram like this can be transformed into a perfectly valid UML class or collaboration diagram. To make it a class diagram, draw a rectangle around each word. To make it a collaboration diagram, draw the rectangle, add a colon before each word, and draw arrows, labels and sequence numbers along the joining lines.

The Software Architect’s last gasp

The foundations are laid; the building blocks identified. Their interfaces are detailed and the grand scheme in which it all works in terms of hardware and

software technologies is documented for all to see. Is this where architecture ends, and management and development take over?

Let us look at the building architect and software architect, and compare the environments we each operate in.

Comparing Software Architecture to Architecture

The Royal Institute of British Architects (or RIBA) was founded in 1839. In the UK, they define the technical, personal and ethical qualities that an architect must live by. It is a formidable professional organisation which defines, supports, judges and dishonours, when necessary, its membership. It is extremely difficult to practice architecture or to claim the title of Architect without being a member and having passed through a degree following their syllabus, and an internship in an architectural practice once again on a career path defined by them.

In juxtaposition, almost anyone can claim to be a software architect. IT architecture is still in its infancy, and many organisations are being created in the hope of becoming the professional organisation for IT architects. We have our role models to follow, but absorbing the learning and change of over a century in the building world will take some time. The required growth of our field and our professional status is compounded by the problem that IT architecture is a global rather than a national phenomenon. For now, we can only look and compare.

Figure 6.10 The Architect’s environment

Figure 6.11 Comparing the Architect’s and the Software Architect’s environments

We work in a young industry which has grown rapidly. Already our way of creating, constructing and supporting software is as complicated as it is for our living environment. In comparison to the architect, our lines of responsibility are far murkier. Our different organisations approach the software problem with different ideas, and different business structures. This will continue for some time yet, possibly decades, before we begin to settle into more readily understood environments. Other disciplines such as engineering, accounting and HR are managed similarly across organisations because of the maturity of their professions.

A few thousand years ago, builders began building. Functional buildings required design, so architecture was born (many times in many different civilisations) and suddenly there were great temples and monuments. Some were so great that they outlived the gods to whom they were built.

Now we have discovered software architecture, what monuments will we leave behind? Maybe we shall see Abdullah 2038, the classic foundation of all human compatible robots, or Broad 2042, the cybernetic interface framework.

So far, all we have managed is the automation of paper based systems. We may have added a little colour and better search facilities, but in many cases our software still achieves second place. Try shuffling a few hundred electronic post it notes around on a computer screen - it cannot be done. Can you count all of the letter 'A's in the Bible without any programming knowledge? Try drawing this:

Figure 6.12 This

This rather pathetic figure took around ten seconds to draw on paper. The software version took two minutes and I had to export, then import it into the word processor. Is that an improvement? If I want to change it, I have to go back to my drawing application, re-export and then re-import it.

Of course, the computer method does have many advantages. Electronic pictures are more easily saved, catalogued and duplicated, electronic words are more easily moved around and edited. The great benefits are changeability, ease of storage and print output.

So the floor is open to you who wish to take up the software design challenge. It is where you get to be brilliant. In their vision of the space race, the Americans designed a beautiful pen. It wrote upside down underwater on many different materials and lasted forever, all for the mere cost of one million dollars research. I own one. The Russians used pencils. I own some of those too.

The true objects of nature

There are a few good examples of the use of classes in software design, mostly preserved as design patterns (see chapter 15). There are many more in nature, evolved over millennia, and are worth considering.

We have (at last count) eight classes of quark making up the three atomic building blocks: protons, neutrons and electrons. These three classes combine together in mathematical sequences to make up just over one hundred elements. The elements combine to make molecules, and molecules lump together to make proteins, carbohydrates and fats, which make us.

Now that’s not quite the case. We are, in fact, composed of cells. Each cell we have contains the blueprint of us, i.e. our full genetic code. Imagine if every object we defined in software held the properties of every other object. It would be absurd. Instantiating an object would take longer than a user might live.

Maybe our approach to classes is far better. It uses less space and is more specific to the task each class will be used for. We can aggregate and inherit and compose our higher level classes from lower level ones, and these are very useful techniques for keeping our code thin, precise. Yet nature does it differently, at least in animate cases.

Our classes are mostly not animate objects. We are more interested in what an employee’s name is, and that cannot be read from DNA. Neither can address or postcode. Our classes are limited representations. Perhaps this is because of our limited computing power. When a computer can play twenty questions with us, I will be impressed. I wonder if it will contain three base classes of animal, vegetable or mineral?

Approaches to architectural design

As an architect, it is possible that you will have all of the good ideas, and be able to create then pass on your superb design to a team of willing developers. It is possible if you live in cloud-cookoo land. Thinking one person can have all the good ideas is ludicrous. Japanese engineering grew fast, large and better by allowing anyone to have good ideas, while the rigid organizational structure controlled behemoths of the west crumbled.

Measuring and integrating these ideas is your job as a software architect. You should also be allowing them to occur and be appreciated. The creation of common components, frameworks and tools must go through a validation process, and this is the work of a committee (perish the thought).

A camel is a horse designed by a committee <![endif]>

is what my father used to say about committees. To a point, I agree, which is why the single vision of a software architect is so vital in software development. However, I wouldn’t set out across the Empty Quarter[3] on a horse. Camels have their uses. Use a committee to validate and extend shared components. Give your committee an impressive name, like the Integrated Common Component Committee (ICCC). Use it to share ideas which go across systems. Ask everyone involved in software development to put forward components, algorithms and libraries for inclusion in the shared code. Include everyone. That will mean they are less likely to oppose it.

Here is how an ICCC (not the I Can’t Cope Club) might work:

1. A software developer or designer puts forward a component they are working on for inclusion in the common library.

2. The committee reviews the component and either accepts it, recommends modifications, or rejects it.

3. Committee members ensure it has a standard style of interface, and thoroughly review the interfaces, functionality and algorithms.

4. The creator makes the component conform to the committee’s wishes. 5. The committee accepts the common component, and ensures it is fully

documented in a common information repository for shared knowledge. 6. Each new component is sold to the development teams at the subsequent

team meeting. 7. All developers and designers use the common component.

If I had a penny for every bit of 'common code' used only by the developer who wrote it, I would be a very rich man.

Environmental expectations

Users are people. They cannot proceed for 2.7 kilometers in the direction 330°N without technological help. They can turn left at the big tree then right at the petrol station. They cannot then turn first left, second right, on past Macdonald's to the fourth traffic lights, left then the second exit from the roundabout. It is too much information.

If you are writing an application for a windows based operating system, you are communicating with people who are in a psychological environment of

expectation. They expect to be navigated through familiar landmarks. They expect 3D objects to show up as if the light comes from the top left of the screen. They expect scroll bars, they expect to click on things and see something happen that they are used to seeing. As soon as something unexpected happens, like a page of text moving in the direction they are scrolling, they become confused and annoyed.

They expect something to behave the way everything else in the environment does.

They are looking for clarity, harmony, balance, simplicity, refinement, restraint, unity, modularity, sophistication, elegance and economy. They build mental models of what will happen for each of their actions, and have a predefined model for the operating system you are writing for. Write something different and they have to create a whole new mental model for your stupid application.

If they are used to seeing the top of figure 6.13, don’t give them the bottom, thinking: ‘Mine will be different; mine will be cleverer. I wont make them move the mouse so far when they accidentally go past a page, I’ll put the buttons next to each other. I’ll put those ones they never use on the right, out of harm’s way.’

Figure 6.13 The improved interface from geekspeek inc.

They will hate you. They will hate your product. It forces them to create a new mental model, and their grey matter is filling up.

Much has been published on usability, Don’t make me think[4] is a good reference.

The cold call

I cannot stress enough how design is a collective, inclusive exercise. Being an architect doesn’t mean being strange and shutting yourself in a box (or ivory tower - depending on your persuasion) while creating your architectures. Emerging to spread your vision will be a hard sale.

A roomful of people excluded from input into the architecture is the equivalent of a cold call. Selling your design or architecture is far easier if your viewer has something to hook them. If a viewer recognises their input, and is recognised for it, then you are no longer cold calling.

A cold call is a sales term when a salesperson turns up at a potential buyer without introduction and tries to make a sale. It is hard work. Success is rare and fragile. Sales people do it, but don’t necessarily like it. It needs a thick skin. Software people try to do it, but they can’t. They don’t have the skills. The

architect ends up presenting. The development team listens, leaves and forgets. The result is this:

Developer: What was that architect on about? Now I have to write system X. Time to start coding.

Architect: I’m glad that’s over. I hate doing those presentations. Now, on to project Y.

Design wash-up

Sometimes you get designs wrong; sometimes you get them right. Most often, many different designs will deliver the stated requirements. But the measure of success in design, and as a communicator, is not always how the design delivers the requirements. Think on it as how often the code delivered matches the design.

References

1. Peopleware: Productive Projects and Teams. Tom de Marco and T Lister. Dorset House

2. Code Complete. Steve McConnell. Microsoft Press. 3. Arabian Sands. Wilfred Thesiger. Many imprints. 4. Don't make me think. Steve Krug. New Riders.

Chapter 7 - Design

The action and responsibility of the software architect in design

Our software systems are like our cities. We have a few design masterpieces surrounded by suburbia and slum. As a software architect, your responsibility is to remove the suburbia and slum and create new masterpieces.

Be original. Be creative. Be demanding. We are not an eternity of echoes, patiently awaiting the dawn of indifference. We want to create excellence, and we must both demand and earn[1] the freedom to be able to do so.

Freedom in this case is not about personal liberty, but about freedom from assumed directions and structures. As Brooks[2] states, when discussing the role of the organisation chart in software projects:

This [the organisation chart] becomes intertwined with the interface specification. As Conway’s law predicts: ‘Organisations which design systems are constrained to produce systems which are copies of the communication structure of these organisations.’ Conway goes on to point out that the organisation chart will initially reflect the first system design, which is almost surely not the right one. If system design is to be free to change, the organisation must be prepared to change.

If a business leader defines your software, he or she is in all likelihood defining the wrong software. You must work your way into their trust, then use that trust to communicate a better software and business vision.

The design process

To clarify the design process, we shall look at three separate projects and select moments of interest in each.

Project 1 - A Stationer’s system

It comprises purchase and sales orders, accounting, reporting, stock control, system administration, upgrade management and electronic order processing. The system accepts orders from external email or via a web service. It can also query suppliers’ systems to enquire on stock levels and place orders.

Project 2 - Robbie the robot

Robbie is much like the rest of us, except he is hormone and emotion free, so a little more reliable.

Project 3 - A SuperSport X9i virtual car

working in a virtual world provided by the ground breaking True2Life real world environment.

All of these are fairly large projects. Some believe the role of software architect exists only on such large projects as there is little architectural work to do on smaller projects. This belief means software architecture exists within a larger idea of software creation. An alternative view is that software architecture is the creational model, and everything exists within it. Therefore, a software architect defines the architecture of both large and small projects. It is only the level of abstraction, or detail, which changes.

The project approach proposed in chapter 6, of framework and toolset, will only ever exist in architecturally led development. In business led development, it becomes ever more difficult to achieve. There is little visible output when developing these items, and business focused project managers have nothing visible to sell as progress. If you are working in this environment, the toolset and framework are best developed as part of each subproject. Do so while selling the benefits of architecturally led development to the business.

Design Thoughts

Steve McConnel[3], states the aims of design:

• Minimal defects • High correctedness • Maximum user satisfaction • Minimal response time • Good maintainability • Good extensibility • High robustness

Upon these, he builds project objectives:

• Short schedule • Predictable delivery date • Low cost • Small team size • Flexibility to make mid-project feature-set changes

Upon these, I would add, once more:

• Clarity – the message is obvious: biggest = most important, no clutter • Harmony – No sudden changes • Balance – A careful weighing of issues • Simplicity – Easy on the eye • Refinement - Unambiguous • Restraint – No announcement trumpets, no painful colour clashes • Unity – Each part is clearly related, each action produces an expected

outcome • Modularity – Each decision, each information point is separate • Sophistication – You understand and are helping • Elegance – Simple flow • Economy – Least is most • Beauty - Always forgotten, but never unnoticed

Project 1. The Stationer’s system

A stationer has agreed to investigations into replacing their aging and difficult to maintain system. The job is out to tender, and three software suppliers have been invited in to estimate the complexity and cost of replacement.

For those unfamiliar with a Stationer’s business, here is a quick overview.

Our Stationer is a business supplying other businesses with everything from paper clips to photocopiers. It also has a high street shop. Sales terminals in the shop are referred to as Point of Sale (PoS) terminals. The shop and the business customers are supplied from the stationer's own warehouse. Selling stationery is also known as office supplies.

The warehouse is managed by keeping stocked items at a stock location, and recorded against each item is a stock level (count of the number of items), a reorder level and a reorder quantity. When the stock level gets down to the reorder level, an order is made to replenish that item. The amount of items ordered is the reorder quantity. The word reorder is used as if someone has ordered it, and in turn, it is being reordered from the supplier.

The stationer orders its stocked items from wholesalers or manufacturers, and stocks a few thousand items. Stocked items are also referred to as stock lines. Their full catalogue of supplies contains twenty thousand items, and those not held in stock are called non-stocked items.

A discussion about the business reveals the following information:

Figure 7.1 Stationer’s system use cases

The use case diagram shows sales and purchase order processing, account management, stock control, accounting, reporting and sales analysis. Purchase order processing is always done as part of the stock control activity. Reporting is either a separate use case for management, or an extension of the other use cases. A system administrator has a system administration and upgrade management use case. Customers and suppliers have accounts with the stationer.

This represents the core of the business. The stationer also has its own printing facilities where they produce letter-headed paper, compliments slips, printed envelopes and business cards. The print business is managed on a separate system designed specifically for the effective management of printing. Running the presses efficiently (or constantly) is core to profitability for the print business. The print system provides daily updates into the core system for accounting purposes.

The stationer uses a number of different pricing techniques depending on the size and complexity of each customer, and on the generosity the customer extracted from the salesperson when agreeing fixed prices. Fixed prices are reviewed every six months, and allow the customer to predict how much will be spent on stationery with a usage model and non-varying prices.

At this point in the design you may have begun to see the enormity of the task ahead, and use the information gained so far to identify a set of likely subprojects. Pricing is obviously a keen issue, where business rules must be

applied differently for each customer. There are core system actions such as recalling, editing, saving and analysing data. There are links to external systems. Add to that all of the normal sales order processing and accounting functions, and it is quite clear this is a large project.

The design must also allow for nationwide deployment as the stationer is intending to expand by opening shops in all major cities. They may also be buying up smaller companies and expediently merging their customers and management information so they become one seamless entity. They would also like the system to allow for expansion internationally as they have been in talks with similar companies in other nations with a view to consolidating. Providing software with these capabilities will be a big win.

The Straw Man

A straw man is a loosely stated first stab at creating a solution. It is used as an opening bid, which can be refined, redefined or shot down in flames of withering despair. It is also a psychological tool, for he who states his straw man first and loudest most often gets to lay the foundations of the design. Once the straw man is made, he is easier to drive home than burn. And if the flames do come, step back, and say: 'It was just a straw man.'

You gather straw: The system will be distributed. Order processing and stock control information will be stored in one or more locations with regular data consolidation to a single central point. A layered set of objects will be provided to extract, convert and deliver information. Browser access will be provided to all major parts of the system to allow expansion to take place without large installs on different platforms, but there will also be a set of fast, efficient, compiled applications so those who process information all day can do so most effectively. Analysis of the business data will be through reports viewed in a browser, and also in highly interactive compiled applications where required.

The throughput of the system is moderately high, without taxing the transaction throughput of any commercial database. A few thousand orders are placed each day from the business customers, and the shop handles around 500 callers. The shop’s customers are fairly evenly spread throughout the day, but the business customers place most of their orders between 10am and 11am, and again between 3pm and 4pm. These times of higher throughput are known as the morning and afternoon rushes.

The most obvious item in the business at this point is a sales order. It is composed of an order header, a body and a pricing summary.

The header contains the customer’s account code, their business name, the date the order was raised, the name of the person who placed the order, its intended recipient, and also the delivery and invoice addresses.

The body of the order is a collection of one or more order lines, where each order line contains a line number, a product code, the product’s description, the quantity ordered, the quantity available for delivery, the unit price for a single item, the discount they got, and the line value, or quantity multiplied by unit price.

The order summary contains the current state of the order. During its rather short lifetime, its state progresses through order being taken, order being picked, order being delivered, and order delivered. It also contains the number of the invoice which the order relates to. You ask for an example, and they manage to find an old one lying around containing no sensitive information.

Account code PRS, Parliamentary reduction services Order date 25th September 1605 Order placed by Fawkes, G Deliver To Cellar Invoice To PRS House of Lords Yorkshire London

Line number Product Description Unit price Quantity Discount Total 1 2286 Barrel of Gunpowder 6d 20 25% 10s/- 2 4224 20 foot fuses 1d 20 25% 1s/8d 3 1093 Box of lucifers 1d 1 0% 1d Next invoice I293 Order Total 11s/9d Order state Order taken

Figure 7.2 Sales order

Figure 7.3 Order classes

Certain nuggets of detail gained during discussion lead to the class model of figure 7.3. For example, reuse of product codes from one catalogue to the next was discovered, meaning product codes are not unique. Therefore the product description must also be stored, despite this apparent inefficiency. Other solutions, such as storing product codes against their descriptions with valid from and valid to dates may require less storage, but they are likely to add complexity.

The orders require storage in a database. One practice to be aware of is that object oriented software should be designed objects out. Many client server

systems were designed data up. Data up may have produced decent results in a procedural language, but will not lead to an efficient set of objects. Procedural languages and their associated systems are on the wane. They are most likely to continue to be so.

If you disagree, I would like you to do something:

Imagine your favourite cup

Are you doing so? Can you see an object floating around in your mind’s eye? You have a vision of a cup. You can see it. You can imagine it has texture, the friction of its surface, how it burns your hand when the drink inside is too hot. How light reflects off it, how heavy it is, whether it sits on a saucer, the rings of stain it leaves behind on your desk...

First you saw the object, then you added data and interactions to it. You did not imagine height, width, depth and thickness. Think on that a while.

Divide the real-life object (in this case a sales order) into parts. Make each part a class, then add attributes and operations. Finally, add data types and persist it to a database. Don’t begin with data and try to create an object from it.

The layers a sales order might flow through in a layered system are displayed in figure 7.4. A salesperson requests an empty order from the user interface. This request reaches the business rules, which in turn requests this information from the database. Upon true returning from the data, the business rule says OK. The object is created, then adorned with the user’s preferences and sent to the UI.

Figure 7.4 A sales order through the layers

As a scenario, imagine a salesperson arriving at work. She logs on to her machine and fires up her business software. During startup, the salesperson is identified with a number of roles, and one or more of these roles is able to place an order. The order entry option is added to the list of actions she may perform, and she is presented with a menu of items in the software's default startup view.

She selects the order processing screen and it is presented with her preferences already set. The telephone rings. Client y wants to place an order. She selects the client, and down through the layers goes a request: can salesperson x create an order for client y? If she cannot, she will get a message back saying so, with

an action provided by the business rules: please direct this call to Mary or Steve. If the customer has reached their credit limit she will get a message saying so. If all conditions are met to be able to place an order for them, she will get an order screen, set up and completed to her own preferences and overlaid with the customer’s preferences and details.

When writing design scenarios (or requirement scenarios) always use the present tense and avoid conditional words like could, should or might.

A framework and toolset were presented in chapter 6, and now layers have been introduced. Both concepts are shown together in Figure 7.5

Figure 7.5 Application using a framework and toolset

The framework (itself consisting of layers) and toolset, are used by all of the applications in the system. In some cases, the toolset converses directly with the operating system, and in others employs the functionality offered by the framework. As a reminder, the framework is deployed sparingly, often across many servers, whereas the toolset is deployed on every client and server.

In reality, another set of layers will reside between your framework/toolset and the operating system. This is the runtime library you have developed for, such as the .net framework, CICS, MFC or the J2EE runtime etc.

We shall begin our design with a typical layered business system. If our stationery software system requires more or less layers than considered for figure 7.4, we shall discover this during design. At this stage, the class diagram is a mere sketch, which cannot be properly detailed without a discussion of design patterns in chapter 15.

Figure 7.6 Layered design showing object relationships

In actuality, all of our objects are dependent upon the runtime library from our development environment. They may also communicate directly with the operating system, which is not shown both for clarity and in the hope that direct calls will not be required. Each layer of the framework also has access to, or can be accessed by, the toolset. The layered framework is composed of the four objects Business syntax & context to Data translation. Data access, in this case, is a predefined library such as JDBC, ODBC or OLEDB.

Direct communication with the operating system is being strongly discouraged as development tools become more effective in delivering our needs without such a measure. This leads to better encapsulation, higher robustness and easier debugging. It is with the Java runtime or the .Net framework layers we converse with.

The benefits of layered design are fourfold:

1. To be able to change an item in any layer, such as a business rule or data access technology with the least change to any other part of the system.

2. To simplify the code at each place of deployment 3. To support different deployment models and server configurations. If the

business rules and business syntax & context functions sit on a server local to the users, and caches local customer data, it will deliver a better perceived performance to the users. Also if a server supporting the entire set of layers becomes overburdened, perhaps the business rules service can be deployed to its own server.

4. To support multiple objects performing the same task in any layer. This may mean supporting two user interfaces, browser and compiled

application or to switch in replacement or alternative components while the previous one continues to serve.

Business objects such as an order are created and completed in different layers. When traversing across layers, particularly if those layers reside on different physical servers, the objects are decomposed, or serialised into streams of data, and rebuilt in the receiving layer. This kind of design is growing due to higher available bandwidth and technologies providing serialization and transportation. In many cases, the layers become generic data extractors, gatherers, and filters for presentation and manipulation of these objects.

Service Oriented Architecture

A service oriented architecture is a loosely coupled collection of functions providing vital operations and knowledge to applications or their users.

Benefits

• Single point of deployment and reference • Simplified mechanism (e.g. web services) for calling across non-

heterogeneous systems (e.g. Windows to CICS). • A method for switching in and out services as required, possibly

providing some fault tolerance, load balancing and higher upgradeability

• A method for separating or consolidating services across servers without discomfort

Drawbacks

• More work required in design to define message structure and technologies

• Management of message routing error handling is required • Possible slower function calls • Careful management of asynchronous calls required

The majority of service oriented architectures use web technologies because of the pervasiveness of the internet. The opportunities afforded by web based services allow information to be delivered globally from a single point of reference. Thus, information need not be replicated, nor out of date. Http provides the transport over TCP/IP. SOAP, XML and DTDs provide the messaging formats, controls and data, and web servers provide the interfaces between servers and operating systems.

How thin is your client?

The language of user interfaces has gone through a few iterations. When we moved away from text based interfaces we were left staring at WIMPs. Wimps became clients, which in turn became thick clients when browsers, namely thin clients, came along. With the advent of thin client, thick client was perceived as no longer suitable, but thick clients are making a comeback, renamed as smart clients.

To add to the confusion, the browser manufacturers have long been trying to deliver thick client functionality in their thin clients, and development tool (thick client) suppliers have equally been at pains to remove the deployment burden of thick clients. In the intervening space it has been equally difficult to deliver browser upgrades as thick client installs.

The solution to both problems comes from neither, but from faster and easier internet access.

To compare client server, SOA, desktop and thin client business applications is a simple task, but creates many difficult ramifications for the architect to resolve when designing a new system.

Thin/thick/smartness Benefit Drawback Desktop application = thick/smart client

Lots of functionality, single point of deployment per user

Large, often complicated installs. Requires data delivered to desktop

Client server with thick/smart client

Lots of functionality, ability to change data and server functions without a reinstall to all clients

Multiple, often complicated deployments. Requires careful change management on the servers

SOA (thin or smart client) Switching in and out of services, single deployment per service

Must manage complex deployment and operation (as above)

Browser, including downloaded components

No manual installs, able to manage multiple client operating systems (if using java applets)

Lowest common denominator across operating systems for functionality. May need specific browser version. Downloading the components may be slow.

Browser using XML/XSL No installs Browsers and XML parsers may operate differently. Need to check code operates on different operating systems.

Browser, HTML No installs, browser independent

Useful only for presenting information and filling in basic forms. Severely limited interaction

Thin clients can have a huge benefit when absorbing other companies. Getting them on your business software is as easy as installing a VPN (Virtual Private Network) and pointing them at an http address. So goes the theory. The actuality is that all businesses are different. They have different leaders, who have

different ideas, and they operate quite differently. All are sold on their existing business model and will argue quite convincingly that they cannot stop operating the way they do without losing business.

All of their ways of doing business must be added to your software. Business leaders have no dilemma at all when answering the question: ‘Do we upset all of the people in the company we’ve just bought, or do we upset our IT department?’ Then they start using slogans from the great wars, words like together and team, and start dribbling Churchill quotes.

The depth of client offers no real solution to functionality nor deployment, but in certain circumstances, one or the other may provide a better option.

Taking orders

The use case model defined a rather coarse grained system. Each of the use cases identified can be further analysed and detailed as use case pathways in activity diagrams.

Figure 7.7 shows order processing broken down into high level tasks. Common functions of product and account lookup have been separated into a separate swim lane, indicating functionality which will be contained within a framework or toolset rather than in each individual application requiring them. This will ease the complexity of the sales ordering application and also provide a method for sharing and changing product and customer lookup without redeployment to the clients.

Figure 7.7 Sales order high level activities

In addition to the account and product lookups, it is highly likely that all lookup, selection and deletion of the major business objects (customer, employee, product, order etc) are candidates for the shared client or server based functions. Mathematical functions like calculateCustomerDiscount(Customer), getProductPrice(Product, Customer) and drawing functions, stylesheets and javascript libraries are equally likely bound for the framework or toolset.

Without trying too hard, we have the structure of a layered framework on which our individual applications sit, a set of subprojects we will be able to divide among our development teams (one project team per use case). We have the beginnings of a framework including product and account lookups. It is starting to take shape.

Suppose we get to this point after a few weeks of analysis. We have the major use cases, a layered definition of our intentions of how the system will hang together, and we also have a list of applications which will sit on these layers. The sponsor then comes with a request:

We have had some rather severe fluctuations in the price of paperclips this year. The paperclips are made of a type of plastic refined from oil, and as you know, oil prices are up and down. We have been keeping our customers free of fluctuating prices, but the situation is such that paperclips are now ridiculously overpriced as the price of oil has fallen so far. We want to be able to manage

our systems based on the true cost, but we want our salespeople to see a distorted cost based on the sales price we have set. We expect oil and paperclip prices will soon increase quite significantly, so passing on the cheap prices now would mean making huge increases later. Such huge increases in the past have led to bad feelings between our customers and salespeople and we have lost accounts. So can you ensure the software will allow us to do this.

In our design so far, it is not likely to make much difference. Had we completed building the application as a typical bunch of applications, then each place where each application read and displayed the cost price would require reworking. We merely note it for later when we are building the component which handles pricing.

This also indicates that our application, initially intended to be client-server, should additionally make use of a number of services. So our architecture is mostly client-server with a bit of SOA mixed in.

To provide a useful environment for the subprojects to be developed and added to easily, as much of their work as possible should be built into the framework. To decide what goes in the framework, and what stays out, each item in the framework should provide services to more than one subproject. If only one subproject is served and it is unlikely that the service will be required for expansion, then it belongs in the subproject.

Tools such as math libraries and common lookups belong in the toolset, whereas business decision and operation belongs in the framework.

Let’s take a look at worldwide set of servers.

Figure 7.8 Worldwide deployment

A system designed this way will scale geographically and in number of users far easier than a non-layered approach. Perhaps we will one day expand to two US business servers for every data server. As you can see, the layered approach is highly configurable.

We decide that all applications will be available from a browser to facilitate rapid deployment. We also need to provide a shell for compiled applications. The shell will provide each component with a basic set of functionality and cached data, such as who the operator is, where components can be found, and where data is stored. Those functions requiring complex compiled user interfaces will then have a set of shared methods.

To begin, we shall take a high level view of the collaborations of the main objects in the stationer’s business.

Figure 7.8 High level collaborations between principal players

Roles, Goals and Business Objects

The principal roles of the stationer are SalesPerson, Buyer, Supplier, Customer, Picker, Carrier, PaymentsReceived and PaymentsMade. Three of these roles, Carrier, Supplier and Customer are external to the stationer. The PaymentsReceived and PaymentsMade roles are both accounting roles, and the other roles of SalesPerson and Buyer, sell and buy stock. We will define each of these roles as a business object, and give each role a collection of one or more goals.

Role Goals Associated objects SalesPerson Communicate with customers, and

take their orders SalesOrder, outgoing Invoice

Buyer Communicate with suppliers and maintain stock levels

PurchaseOrder incoming Invoice

GoodsIn Shelve deliveries and returned items incoming Delivery, outgoing Return

Customer Place orders, return goods, pay outgoing Invoice SalesOrder Delivery

Picker Pick ordered items from stock locations

Outgoing Delivery

PaymentsReceivedSign in payments Check against invoice and reconcile differences

Payment Invoice Ledger

PaymentsMade Make payments incoming Invoice Ledger

For more on accounting terms, see[4]

Some time must be spent early in design defining all business roles and the business objects they interact with. Following that, create the use case pathways, and identify which roles drive each activity.

As an example, consider the create new sales order use case.

Figure 7.9 Create new Sales order

Each use case is detailed as a set of activity diagrams. Each business component is defined using CRC cards, XML schemas etc, including key and base data and optional sets of collections. For example, in the customer object, you may have a container to collect all of their orders. For a salesperson you may have a (usually empty) collection for their customers.

XML and collections save a lot of headaches when building hierarchies of objects. Consider, for example, a SalesPerson object containing her customers, each having a number of orders. <SalesPerson name="A Aspinall"> <Customers> <Customer ID="000014" Name="Goodies Repeats Inc"> <Orders> <Order ID="0216450"> <Customer ID="000014> <SalesPerson Name=... In a pure business object world, the Order would contain either a data item for SalesPerson, or the business object of SalesPerson. In this case, containing the business object would create a never ending hierarchy as the SalesPerson within the order would contain Customers, and

customers would have orders, over and over again. XML makes this problem much easier to avoid, but consider the following two situations: Situation 1: Business object Order contains business object SalesPerson Situation 2: Business object SalesPerson contains business object Order To avoid having a great number of SalesPerson objects, such as SalesPersonFull, SalesPersonNameAndAddressOnly, SalesPersonSummary, choose instead to populate one single business object of SalesPerson with a number of different sets of data. Consider each one a view. When populating SalesPerson within an Order, do not fill the Orders collection or container.

The framework manages the translation between the database and user, and back again to the database, controlling lookups, creations, deletions and changes. Any access to the operating or file system is also handled by classes within the framework or toolset.

An example in chapter 1 builds on the activity diagram to show the state changes of a sales order. By defining the state changes, they can be coded in a controller class. The activities themselves can also be written (as code) in sequence, reflecting the activity diagram.

To continue this design activity would further detail activities and states for all of the applications and business objects, and clarify the layers in terms of how they operate on a business object. Finally in the design, we need to create and test the various layers so that the objects and management of those objects can be fitted together on predefined layers in their most efficient configuration.

When writing code, the activity diagrams are translated into controller classes and object classes. The user interfaces are designed to lead people through the activities just as they detailed them when describing their business function.

To get a better idea of how to get from idea to design to code, try Visual Basic and UML [5] or UML Components [6]

To keep all of the data definitions tidy, and free from attachment to any object or document, it is good practice is to create a data dictionary.

Data dictionary and terms for stationer

Item Description Data type Data size / key Data name Object name Product An item the stationer sells String 16 chars PRODUCT .Product Composite A product aggregated from other

products PRODUCT .Product

Sales order

A collation of one or more order lines Class ID = integer SALES_ORDER .SalesOrder

Order line Product code, quantity, description, item price, total price, tax

Class ID = OrderNumber/OrderLine

ORDER_LINE .OrderLine

Quantity The number of items integer 4 QUANTITY .Quantity ROL The stock quantity trigger at which a

reorder (order to their suppliers) should be made

integer 4 ROL .ROL

ROQ The quantity of a stocked item integer 4 ROQ .ROQ

ordered when the stock level reachesthe ROL

Employee Anyone employed by the stationer class ID=integer EMPLOYEE .Employee

The development schedule

Once again, the key to large project success is division into small labours, and the extraction from the main functions (use cases) of a framework and a toolset. The framework will contain the major functionality of the system and abstracts away data, operating system and storage. If the framework is defined as a set of layered components which can be developed as part of the subprojects rather than as a separate project, some resource allocation problems may be avoided, but careful inter-team communication is required to ensure interfaces suitable for all. Also, the tools used in the project such as common lookup and selection dialogs, common methods and common script and style components, should also be wrapped up as part of the subprojects where possible. Doing so will modularise your development effort into a set of discrete projects.

We will leave the stationer’s system at this point, as to discuss it further would not benefit our architectural journey. There are plenty of books on object oriented design, and vast areas of publication dedicated to layers, components and business systems in general.

Designing a car compared to designing software

We can compare large software projects to many other large projects such as buildings, vehicles, infrastructures for transport or delivery of utilities such as water or gas. They can also be compared to almost any consumer product such as cameras, computers, music centres and televisions. All of these are based on components. They have principles of science, engineering and aesthetics which must be understood and addressed to make the product successful. We shall briefly compare software systems to cars.

Cars are built of large (chassis, body, engine) and increasingly small, collaborating components. They are designed by dividing up the car into smaller systems and allowing specialist teams to work on their own subsystems. The leaders of the smaller teams will work together, and with senior people who are responsible for the whole vehicle, to ensure the parts work together and fit together.

All of the bits of a car are held together by the chassis. The body sits on it, the engine is contained within it and the wheels hold it up off the road. The design of the engine requires all of its sub components to fit within its space, and that they act as a crumple zone, slowing the car during a crash and absorbing as much of the impact as possible to protect the passengers.

Car designers approach a new design in the following ways:

• Sketches are made to get an idea of the shape of the car • A full size or scaled clay model is built to show exactly what the car will

look like

• Computer aided design (CAD) is used to specify the dimensions and location of component parts

• CAD is also used for strength, noise and vibration analysis • More computer and physical models are used for airflow around the body

and also for the airflow within the engine • Prototypes are built, often using computer aided manufacturing for 3D

parts such as deposition 3D modellers or CNC machinery. • Wind tunnels allow the designers to observe the airflow around, under

and through their vehicle, and allow them to tweak the design to make the body shape more aerodynamic.

The car is ultimately a collection of smaller components. They either pre-exist, are bought in, or are built in house.

Software is ultimately a collection of smaller components. They either pre-exist (databases, user interface controls, components built for other systems), are bought in (user interface controls, widgets) or are built in house (everything else).

Figure 7.10 A car made up of components

This brief list of components grows as more knowledge is gained of the car. Every time the diagram is revisited, more information and more detail is added, possibly continuing right through the design process, and possible into production. This should not be seen as a problem. Design is an iterative process, i.e. it is revisited repeatedly as more is discovered.

It is likely that each component will need to be further detailed in a separate diagram:

Figure 7.11 A carburettor with interfaces and subcomponents

At class level, the carburettor might look like this:

Figure 7.12 Carburettor classes

A car for all seasons

A car is an excellent example of an object based system. It's chassis is a kind of framework. On the chassis sits an engine, which has interfaces with other parts of the car, like the accelerator and gearbox. Each of these objects is a whole system in itself, joined using components such as wires, connecting rods, a clutch, an axle etc.

Each component is itself composed of ever decreasing components. For example, a car has an engine, an engine has a carburettor, a carburettor has a float, the float has a hinge. Some of these items are externally supplied and others are

created internally for one or more different components or cars. Careful trade-off between buying and creating, and even the method of creating can create a more economical product.

Many attempts have been made in software development to allow us to buy in components, but it has stalled at the carburettor level. If we could get down to the washer level, creating software might become a less time consuming and less problematic affair.

The code equivalent of a washer is:

Open database Open recordset Locate first record Read record Is this the last one? No - move to next record Yes - jump to Close records Loop back to Read Record Close records Close database

A washer:

Material: steel manganese (10%) alloy, hardened martinite Thickness: 0.5mm Outer diameter: 12mm Inner diameter: 5mm Metric size: 5M Shape: flat

Rarely do we pick up and place the equivalent of a washer. We get the metal, cut it, drill it and harden it. Only a fool would do that for a car. Perhaps one day we will be able to click on an activity in an activity diagram, and select the equivalent of a washer to provide a partially finished code template.

Let’s compare a car and its environment to a software system. The match is not perfect, yet there are many parallels.

Car Software

Road (infrastructure) Vehicle support (fuel, repair) Car Chassis Engine Carburettor Float Washer

Network of Computers Operating system Software system Framework Application Class Function Bit of code

At the design level, they are not too different. They look equally complicated, and after release they both have their problems. Cars are recalled for faulty tires, fuel injectors etc, and software soon has its service packs and quick fixes. Cars can crash, come off the road, be driven by fools. Cars can kill people, as can software.

If you are designing software to monitor hospital patients, or for air traffic control, your design must have safety redundancy built in. You will also be involved in the design of the hardware so that when failure occurs, emergency routines can kick in or raise an emergency warning effectively.

Design requires different disciplines for different arenas. You must become familiar with your arena. How would you feel if your heart monitor stopped working because the software developer couldn’t be bothered or some fool project manager said I need it by Friday without considering the consequence to you, dead because no software architect had the strength to raise his voice, nor the skill to persuade others away from their course of action?

We can learn a lot about design by looking at everyday products. We can learn by studying them, by dismantling them and seeing how the pieces fit and work together. The Design of Everyday Things[7] is a good place to start.

Project 2. Robbie the Robot

Let's take a look at designing a software system for a human-like robot, named Robbie.

Robbie has senses, but not feelings. It is a genderless robot, but male in appearance. He has a means of motivation, called muscles, and a way to know what load is on those muscles, using nerves. Unlike us, his day is filled with clear, unchanging objectives. He cannot manage complex commands, and instead manages by breaking down each command into a series of tasks, then working with the person commanding to agree the individual tasks which might collectively meet the command.

Figure 7.13 Robbie the robot’s high level classes

Robbie is not perfect. He’s a little like you and I. He has his flaws, but he is doing his best to overcome them. Unlike He who made us, Robbie has been designed by a less than perfect creator, and so he has that floating dictionary class shared between hearing and sight. If there is a fault there, he will be immediately unable to speak, to understand speech, and to read.

That movement controller looks a little overloaded also. It has too many links into some packages. For example, there are three separate links with the nervous system. That’s not too good. I’m sure you would make a better job of it,

yet this is my best solution acting as software architect. I know another software architect who put in a superior design, but I was the best salesman. He was a bit tongue tied and shy. Served him right. What’s the point of being an excellent technician if you aren’t prepared to work on your presentation? Harsh, I know, and as true as the fact that there is water at the bottom of the Atlantic.

At this point we should define the underlying structure upon which our robot software will be built, and define the interactions between the packages. If we succeed at this, then the individual packages may be worked on as separate projects. Smaller pieces of work mean a project is more likely to succeed.

Robbie would benefit from a few things. First, we want him to be excruciatingly polite. Should we find ourselves in an awkward position involving him, he should melt into the background. The root of all of his decisions will therefore be based on ethics. He will have a timer for scheduling tasks and generally keeping him going. In biological or neurological terms, this will be his endogenous circadian pacemaker. I may use that as a class name. Somehow he must be able to prioritise over conflicting schedules, should his pacemaker demand two things at once.

To solve his daily dilemmas, he shall have a project management module. This will manage initiating, planning, executing, controlling and closing of his daily tasks. Each decision will be based on scope management, time management, cost management, quality management, communications management, risk management and procurement management

It will also manage action decisions based on all of the above. Robbie is a whole project manager in himself, excluding human resource and integration management. There is none of the latter to do, and we wouldn’t want him bossing us around. We already have enough people doing that.

Figure 7.14 Robbie's internal use case diagram

Figure 7.15 Robbie’s external use case diagram

Of course, Robbie will also have his real user type use cases, depending on the levels of excitement he has been programmed to perform. He may wash up, tidy up, pick up the kids from school, or hurl himself in front of an oncoming vehicle to save a small dog. He’s quite complicated enough for us to abstract away these real use cases in the sure knowledge that if we build him right, then he will be able to perform many tasks. If we don’t, there is little point expecting him to perform anything. Instead, he will sit in the corner of a room twitching uncontrollably. This is a big success or big failure project. It is extremely high risk as everything we are trying to do has not yet been done, despite our clever toys and Honda’s success in teaching their robot to dance.

Software architecture is all about maximising the amount of work not done. The point of a common toolset is to avoid writing everything twice. The point of a framework is to provide applications with most of their functionality. Additional applications are created more easily, and require less work. It is also about reworking or pushing back costly requirements, so developers do not spend half of their time on a project meeting an asinine requirement added without the appreciation of its cost or complexity.

Design is creativity. We have certain tools at our disposal, such as patterns and a software process slowly leading in to design to help us, but it is the creative aspects that will most affect the result. Good creativity can maximise the amount of work not done. It can make coding easier. It can make code small, tight, integrated and reusable. Bad design can contribute to never ending projects and poor quality code from apathetic, confused developers.

Hopefully, Robbie will be neither confused nor apathetic, as we have built our design based (rather sketchily) on ourselves.

Robbie will have an operating system, a framework for coordinating and controlling his movement, and a number of applications for which he can be used (vacuuming, washing…) He will also be layered in that his action decision will sit atop self preservation, which in turn sits above ethics, his action controller and his operating system.

Figure 7.16 Robbie’s layers

We shall also include a short cut, but this will be used only for the blinking of an eye or dodging of a head when projectiles are incoming.

We might manage a little better if we have a link between memory and avoidance - that fine line in our hippocampus where we make a sudden movement and then recoil. Something like here’s an incoming object, I’d better blink, or last time I was whooshing out of the way of this car, I went down an enormous ditch. Robbie will blink, but is likely to go down the ditch twice. That is good enough; Robbie is a robot. Although he is rather expensive, he is also non-unique, and therefore replaceable.

In the layer diagram, it is difficult to link memory to avoidance while it is also linked to ethics and self preservation. Instead, despite the increased area, a package diagram will aid accuracy.

Figure 7.17 Robbie’s layers as packages

Showing the layers as packages makes it a little clearer as I can show all relationships. It also gets away from having to draw small things in crannies on the layers. Anyone looking at a layer diagram sees that things have relative sizes. Smaller means less code, less important. Drawing all of the layers and objects the same size implies little.

Before we develop our software vision any further, we shall take a look at the pieces of carbon fibre, steel, slips of oil and kevlar which together will fashion Robbie’s skeleton.

Robbie, armed and rather inoffensive

Figure 7.18 Robbie’s arm showing it’s degrees of freedom and his hand’s pressure sensors

Altogether there are six degrees of freedom in our current understanding of spacetime. I do not consider time a degree of freedom, as we are trapped forever in the now. Therefore, we have translation (movement) in the x, y and z planes (directions), and rotation around the x, y and z axes. Thus, the complete degrees of freedom of a joint can be written in terms of x y z θx θy θz. 100100, for example, allows movement in the x direction and around the x axis. Its a bit like a ring on a finger. You can rotate it around your finger, and move it along your finger to take it off or put it on, but it won't shift any other way. The limit of movement can also be listed this way: 70 0 0 360 0 0 allows 70mm of movement along your finger, and 360 degrees of freedom around your finger.

Robbie’s main purpose in his unwaking dream of electric sheep, is to keep the place clean and tidy. He will be programmed with chore motivation and little else. We must take care programming his visual imaging system as he may start seeing things when the light levels are low.

If you are driven, at this point, to begin questioning whether my design is right or not, you may as well ask if the design of a particular tree is correct. All trees work as trees. Some can survive in particular places, some have a more satisfying shape. They are all correct. Similarly my design is correct. I could argue with you over its perfection, but it will still work. Making it happen will only prove my correctness. If I can finish version one, then I can improve version two. Imagine a Robbie Mark III, a Matilda Mark IV. My goal in design is to reach the nth degree of perfection in the first iteration, and then ensure its delivery. Many

good designs never get finished, or are hacked away at by others until the designer gives up and moves on to another project.

Furthermore, no piece of software, however brilliant the design, has yet stood the test of time. We live in a changing world, and software is a pale reflection of it. Most software available today has been through many iterations. We are beginning to see signs of maturity in some simpler software tools such as business applications, or methods such as sorting algorithms. By maturity, I mean that people are not as anxious for the next upgrade to give them functionality X. In a newer wave, the world of 3D modelling has had more products than we ever had word processors. On a yet-to-be wave, we don’t have a common operating system upon which I can build Robbie. I have had to invent one, ROX (Robot operating xystem – please try) for Robbie. Buy ROX, it rocks. I can see the colourful billboards now.

We have had so many office tools that we exhausted version numbers long ago, and switched to calling them by year. Years are passé too, and we are in the chirpy names arena: Omega Office PowerQuest. Buy it now. This year’s model has more superlatives than ever.

Business software abhors variety. Why is that? How can that thought help you make your next million?

Maybe one day our software will be as numerous as the trees, and in as many varieties. Perhaps even your plum tree will be different to mine.

Architecture and music have had their movements. We live in cities of many architectural styles, and musical styles go on changing. Software is still in its early stages. We had our character screen and glorified oscilloscopes. We are now into window and mouse, and fledgeling electronic representations of reality. Maybe our software movements are defined by their technologies. Maybe they will be its gothic or medieval. I think it unlikely though. Perhaps the next generation will provide the dissatisfied youngsters of change, our pre-Raphaelites. And will we be remembered as the Raphaels, the traditionalists, the unblinking robots of the megacorps?

The plan of attack for Robbie

We have a rough idea. Quite obviously, Robbie is state of the art, so we shall need some significant effort up front in research, iterative improvement and proof of concept. To begin, we need an idea of where we are going, and the various models presented here will be a coarse foundation which will need to be improved. We should move next into mock-up and prototype.

Mockups and Prototypes

Mock up

A design technique which creates an artificial, visually complete but technologically challenged model.

Prototype

A full working, complete in itself, but partial representation, of the final system.

Mock ups are used for showing to people so they will understand and be able to contribute without having to read boring documents. A prototype is used to prove that something will work, and to be able to measure its response.

The programme plan

If Robbie is ever to be complete, he will need a programme plan. This will identify the problems to be overcome, the methods we shall use to measure feasibility and risk. It will contain the definition of the interfaces between the communicating parts of the system, and the deliverables which together will make Robbie work. Each piece of work will have its own project plan, and be developed by different teams. The programme manager will look after the interfaces between projects, and the project managers will meet the targets set in the programme. Either that, or a lone scientist can tinker on in his garage until his work is complete.

I would be extremely interested in the outcome of these competing methods. I have the feeling that the loner would create a far better Robbie, but is unlikely ever to deliver him.

The programme plan needs something else. We have a central processor which we shall refer to as Robbie’s driver. It is the driver which ensures tasks happen on time, marshals communication, and divides out time slices of the thought process. We shall drive the thought process at 100%, allocating a slice of this to each project team. We will not rely on an event driven model. Event driven wastes quite a chunk of processor time, and we will need to squeeze out every cycle we can. During the design, the programme manager will reallocate time slices and manage the requests of the individual projects for a larger slice of the time. I have no doubt that all projects will absolutely need an ever increasing slice of the thought and decision process.

We shall leave Robbie at this point. I have used him to demonstrate how huge projects are approached and argue a few points on the design activity. The design of huge projects is a significant task, and is best approached iteratively. The chief architect will spend quite a bit of time getting to know what the system must be capable of, and where the many cruxes of the problems are. He will then begin with a rough, partial design, and with the help of specialists and other architects, begin to flesh out the solution.

Many huge projects fail, embarrassing governments and suppliers alike. Newspapers leap on the bandwagon of failed IT projects and are filled with glee at another superlative flop and the many millions, if not billions it has cost. Failure can be attributed to many things, but the real failure can usually be traced to poor communication compounded on a lack of real process.

There are many arguments I could have made about The internal design of Robbie and his real time systems, but they have been done before and better in Doing Hard Time[8]

Project 3. The Virtual Car

The SuperSport X9i is a software model of a car, designed specifically for the annual virtual race from Paris to Dhaka, on the ground breaking True2Life world. True2Life is a complete model of the world. It supports every country, road and tree which exists in reality, and uses reality physics to measure the effect of wind resistance, fuel consumption and wear on a vehicle during the race.

The Paris to Dakar race was abandoned early in the True2Life world as there were too many cheats building land bridges across the Straits of Gibraltar. As soon as cost was not an issue it became easy. It is the consideration of cost which has stifled everything since the Moon race. Cost was hardly an issue then, it was the pride of two great powers in competition. Competition brings out the best in us. Considering monetary cost can destroy anything. Perhaps the open source movement has heralded a new age of cooperation and collaboration. Maybe we can do away with the competition and cost arguments and raise ourselves as a society beyond the murk of commerce. Or maybe we need the commercial straitjacket around us to make us achieve at all.

Anyone can enter the Paris to Dhaka race, just as anyone with an internet connection can enter the True2Life world. Huge companies use it for predictive modelling and searching for good locations as offices for their next big push. Small companies use it for research for their aircraft and vehicle designs. Innovators use it to test their products. It contains the physical world, the geographic, demographic, populated world. All you need to do to get an opinion is to send your completed 3d model to it, anywhere you like.

All you need to do to race is to put your 3D car model on the starting grid (now an area of three square miles) and start when the green light comes. Winners from previous years get to start at the front, but you are likely to end up with the pack, stuck with a stopped engine until those in front of you get out of the way. If you’re lucky, you may get going around an hour after the green light.

The race is now so popular, it has become a parody of Wacky Races. Contestants hire others to destroy their competitors with rocket launchers, booby traps, and holes drilled in virtual petrol tanks, leaving drivers stranded in vast deserts.

The poor True2Life world got in a bit of a state in the last war. There was a sudden uprising between the game players and the multinationals. It was a True2Life third world war, played out in streams of light and electrons. Of course, it was nuclear. Getting access to such weapons was too easy, as those killed trying simply logged on again and tried once more. By then the True2Life world was self-sufficient. The whole thing was open source and maintained by armies of theodolite waving fanatics, virtual inventors and opinion givers.

It was shut down and rebuilt. Weapons of mass destruction were banned as they were obviously bad for all. These days the True2Life is rather better than the real world, and is known throughout its user community simply as Eden.

The design of our virtual car will take in the rigour of race. It will be lean, mean and low, and look like nothing on our roads at present.

Figure 7.19 Supersport X9i

When stationary or moving slowly, our X9i lowers its two stabilizer wheels. When on the move, it can sneak through traffic, and overtake on the narrow mountain roads, slipping past where no other car can fit. It is built on motorcycle technology, and uses lightweight materials for its body shell and inner structure. It has a large fuel tank, which is a failing of many motorcycles, and can go for a thousand miles without refueling. The engine is straight from a race motorbike, and with the highly efficient streamlined shape, the X9i can manage 0-100 in 1.6 seconds, and a 400km/h top speed.

As the engine is a production model, developed and tested in Eden, I don’t have to do any work on its design. Some of the racers have designed their own engines, but after three years’ work, they are still not as good as the virtual production models.

Bodies are all different, as they are far more malleable with 3D programs, and far more open to style than mechanical bits. The vehicles are created using components from their real counterparts. Every shop in Eden sells real and virtual objects. It does make things easier, especially when you want to bring your vehicle back to Earth and have it made. Unfortunately manufacture is still not quite up with Eden. Even the best 100 element deposition modellers still can’t quite match the quality of virtual or real manufactured parts.

Design

We have a car to design. Design is creativity. Good design creates a product which works better, is easier to use, and performs well in the non-functional requirement arena of updateability, understandability, etc.

Good design in this case will result in a vehicle which is fast, efficient, easy to control, and of course, cool. Everyone else can see your car. They’re all in the race too.

I have my body shape. I tested it in a wind tunnel in Eden created by NASA. No-one was using it so I put in my body shell, wound up the flow to 200kph and tweaked away at the shape until I had all of the microflows between the body and the wheels as linear and thin as possible. I even played about with different materials and polishes on the surface to get the wind resistance down to a minimum. The drag coefficient is 0.23. Most vehicles in the race are higher, but they have bigger engines. Some people enter with novelty cars where drag isn’t

important, but I’m out to win. I’ve even been dieting, as you can only enter Eden as yourself now, and the lighter I am, the faster and further my car will go.

I tested the rubber on my wheels, and have built a cunning interface with Eden, where I project my car into the virtual future on a predictive path always half a kilometre ahead, and measure how the tires perform on that particular bit of road. It’s cheating, but cheating’s OK. I’ve even built in a heatball to counter targeting rocket launchers. I only managed to avoid one last year through shear luck.

The race

Paris to Dhaka is over eight thousand kilometres. If you die en route, then you can only begin again from the start, so once you’re dead, you’re out. That’s one of Eden’s rules.

Eden

When Eden was first designed, it was a simple layered model. All objects in Eden could persist themselves, and interacted within an interactional layer. This was built on the existential layer, which was built on the object-coordinate layer, which was built on the world layer, the physical layer and the operating system.

Figure 7.20 The layers of Eden

Eden has evolved. It now contains alternatives to the original core layers, which provide enhancements. The first enhancement was to record history. Then those dissatisfied with reality created their own object-coordinate layer which supported morphing and wormhole jumps. And those unhappy with the world simply created a better one. Once the international space consortium had modelled the stars, the great cloud nebulas and the dark matter, daily trips into space began.

To hold together these diverse ideas, Eden evolved into a 2D layer design, and uses superstrings to hold together the layer alternatives. Superstrings are set up when an object enters Eden. The object’s path through the layers is set up in the master superstring, which maps the object’s path to its chosen layers. Superstrings are also useful as they get around the inefficiencies of layers and classes by creating a direct memory addressed path through the layers to pass messages faster.

Layer security is taken care of at object creation time. When an object is added to Eden, each of the class properties and relationships is replaced by a live

pointer model, providing instant direct access to any property or function in any layer. Each object on entering Eden, must state its aims, e.g. the superstring ‘In, Ex, OCm, AR, Phy, Op’ is shown on the right. The object would then use the object coordinate (magic) and alternative reality (AR) layers to give some magical properties, but would otherwise exist in the normal world. Perhaps a game character with a little out of the ordinary powers, but no magic from the magic layer.

Figure 7.21 Eden 2D layers and superstring notation

All Eden objects are held in a data dump in the operating system. They are only instantiated if an object in the object/coordinate layer gets close to them. Objects in the data dump are known as ghosts, and are stored in holographic memory on a 10mm3 piece of quartz. Live memory is currently running at one trilobyte. This is not much to be playing with and Eden is a little overloaded. Some are claiming the memory system is a fossil and needs renewing.

The software car's software

The car needs software to work. In Eden, software can run on virtual microprocessors. Occasionally, manufacturers release new chips in Eden for testing before they go out into the real world. There is no shortage of power for this, as Eden shares out its huge calculation burden between all participants in a grid, and of course the original Eden lab built from the discarded microprocessors of twenty years. Between its ever growing distributed lab model, contributed to around the world by armies of techies wiring up and sharing their old machines, and the shared participation of its high bandwidth connected users, Eden outperforms the latest greatest processors and supercomputers by quite a margin.

You have an idea of the car and its host. We shall concentrate on the software. First, a mind map of what is required.

Figure 7.22 Car mind map

I have named the central controller the car brain. It responds to the driver’s commands, controls the car’s suspension and lowers the stabilizers when the speed has dropped to less than 8 km/h. It monitors everything, including the alpha and theta waves from the driver’s brain. When they get too high, the car pulls over and lets him sleep. When he wakes, the engine purrs into life, and they are off again on the race.

All of the information under monitor and display will be shown on a small flat screen in the driver’s dashboard. It can also be used as a heads up display, reflecting off the inner windscreen. This will not be used much, as it obscures the driver’s vision of the road, but may come in handy during the race.

Dashboard

As this is my car, the dashboard does not have to be designed to help others instantly recognize its functions. It will not require usability testing as I am the only user. I can make it bare, functional and able to give me an instant overview of the condition of my car at a glance.

Figure 7.23 Dashboard

A quick glance will show my speed, engine revs, gear, fuel remaining and how far it is likely to carry me. I am conditioned to look top left for the beginning, or most important things, so that is where I have placed these things. The vehicle is reading speed signs, so if I exceed the stated speed, I get a bright hand symbol flashing me. I can see the fuel efficiency I’m getting out of my current speed right next to the fuel level. I’m currently in 5th gear at 6000rpm, slightly down on my engine’s most efficient, which is marked on the rev counter with a dot. The current efficiency and the amount of fuel left in my tank will get me another 620 kilometres. Also important in these races is being able to keep a clear vision. Therefore, the volume of water left in my window washers is shown almost as importantly as the volume of fuel I have.

I can see the state of the battery, whether my tires are under or over pressure. I can see whether the volume, pressure and temperature of the oil (and therefore the engine) are off normal. The temperature’s a little high at the moment.

The trip computer shows some stats about my current journey; there is a picture of the car bottom left to pinpoint problems, and on the right is a map. The black dot in the map is me, and I’m coming up to a roundabout. The various buttons around the map can be used to expand it to full screen, to show contours, scale, zoom in and out. It can also superimpose a grid on the map, align the map to the magnetic grid or my direction, or show the distance between the car and other objects of interest within range.

Keyboard

If the road noise is high or the music is high and the noise cancellation system can’t handle it too well so I am not being understood, I can type in commands on my non-conventional keyboard. I can also use it to program the car while I am in Eden. If things aren’t working too well, I would have to come out, reprogram and re-enter, so I would be back at the starting grid.

A conventional keyboard would take up too much room, and I wouldn’t be able to type while driving. It is mounted on the rear of the steering wheel and uses a set of three strips of pressure sensitive rubber. It is operated by pressing down on the strip to generate a sound. Behind the scenes, the keyboard input uses an allophone based dictionary to decipher the sounds. Each separate sound is represented by a single 8 bit integer for each allophone, and drawn shorthand on the international allophone exchange as a rendering of its 8 bit number by a square within a square.

Figure 7.24 Pressure sensitive keyboard strip

Allophones are pieces of speech. The letter I in English contains two allophones, an A sound and a EE sound. The word tin has three allophones, T then I then N. Related allophones can be grouped into phonemes for a more logical presentation. A (sounded as in at) E (flat E as in bet) E (short e as in beat) EE (long E as in eel) could be considered a partial set of allophones, or a phoneme. All languages are made up of phoneme sets which are slightly different.

Here are most of our sounds classified by the shape and part of the mouth, nose and throat which are creating the sound. I have taken the liberty of a few foreign sounds to make it a little more complete. I have used the Arabic غ (ghain) which sounds a little like it is pronounced and rolled from the throat. I have also used the nasal baNGla where the NG is pushed out through the nose. I have also use the German ich in both its pronunciations, iCH from the roof of the mouth and iCH from the throat. Finally, I have used the Viking derived Geordie nA (no), with its snatched A, and Gannin (going) with its glottal G. The rest you will have to guess at.

These sounds are mapped to a 16x16 matrix of sounds, and represented in an 8 bit format.

Figure 7.25 Eden’s 8 bit allophone format

Here is hello in Eden allophone language. The 8 bit mechanism is easily printed, transported, read from paper and converted into any script. Eden came before Babel

Figure 7.26 'Hello' (or helow) in Eden's allophone exchange format

Each strip of the keyboard represents multiple sets of phonemes, in the most logical order I could think of.

Figure 7.27 Complete keyboard

Some sounds simply cannot be made or cannot be made with enough distinction to be needed. The last 5 columns (not shown in the table above) are for programming sound effects, word or phrase shortcuts, or the arrival of interstellars.

Classes

The display is provided by the car’s brain. It also monitors things without any display, such as the active suspension, and controls yet others.

It’s time for the post-it notes. I shall go back to my mind map and write down all of the objects in the car that are of interest to the car’s brain, then divide them up with a logic which will help me create a set of classes for them.

We obviously have many monitor and controller functions. It makes sense to model those to get warmed up.

Figure 7.28 Monitor and control classes

The monitor objects will use a bitmap, and a screen location and size to allow them to display their parts of the dashboard. The control objects will additionally use their i/o functions to exercise control.

Conceptually, the code layer for providing all monitoring and warning is quite simple. When the car starts, it provides a wake-up call to the car’s brain. In turn, the brain runs through its initialization process, recalling data that was saved on shutdown, and starting the threads for each separate monitoring and control system.

The operating system provides and supports the threads, and each thread has its own memory space. The brain and operating system exist in a microcontroller. The input and output for measurement and control are a set of 8 bit channels, each with an 8 bit analogue-to-digital converter. Each channel is connected to an 8 way multiplexer, so 8 analogue lines can be measured using a single port. As an example, let’s look at how a single 8 bit input port will work.

Figure 7.29 Multiplexed input to microcontroller

Multiplexers can work in a number of different ways. The method we use is the simplest, called time division multiplexing, in that the lowest three bits of each port are used to drive the multiplexer. Together the three lines provide 3 bits of information to the address port of the multiplexer. The three lines count from 0 to 7, and the multiplexer output is connected from the 0th to the 7th analogue input line in sequence. The cycle repeats continuously. The speed of the count is eight times the maximum frequency. That means, to sample each of those lines at 50Hz, we must drive the multiplexer at 8 x 50 = 400Hz.

Each of the voltages Vin0 to Vin7 are in the range 0 to 5 volts, and are converted to digital numbers 0 to 255. Each digital increment is 5/256 or approximately 0.02 volts.

The circuit in figure 7.29 provides monitoring, but we must also control. To do so, we use a similar circuit comprising the same microcontroller, a digital to analogue (D/A) converter, and a multiplexer. This time we write to the port rather than reading from it. The D/A converts the digital number to a voltage between 0 and 5 volts, which is divided out by a multiplexer, this time to eight output channels. Those channels are used to control the car.

Any book on digital electronics will explain multiplexers in more detail.

We shall use four 8 bit input ports for monitoring, and another three 8 bit output ports for controlling. We shall also use 3 bits of our 8 bit output port for controlling the selection of channels from both multiplexers. We shall need a microcontroller with eight i/o ports.

The software to manage this will be based on an operating system which abstracts away I/O ports, memory management for loading bitmaps, timers and counters and other electronic devices.

When creating such systems, there is no fine line between hardware and software. Both drive each other. For example, I could maintain the clock with a software counting device, or a hardware device. Whichever I choose to do affects the other, in that it would be foolish to do neither, and wasteful to do both. Designing hardware is part of the act of the software architect. Similarly, I have to design the operating system. Admittedly it will not be overly complex as the microcontroller itself does a lot of the work. My operating system will be just another layer in my design.

Let's take a look at the software for monitoring and display. I have two separate threads, one for monitoring and display, and one for monitoring and control. By doing so, I can alter the speed at which each thread will run. It may be that the monitoring and control where no display is required, only needs to run at 10Hz, whereas the monitoring and display is best run at 25Hz to avoid screen flicker or values appearing to jump rather than change smoothly.

Figure 7.30 The monitoring and display thread

The class structure for this application will have a base class for monitoring, display and control. All other classes will inherit from the base class as shown in figure 7.28.

The control object will manage engine tuning, tyre pressure, suspension displacement and hardness, and I shall have three threads. Thread 1 will manage measurement and display. Thread 2 will manage measurement and control. Thread 3 will manage diagnostics. They will be driven in the order 121312131213… That will make my task of thread management easier in the operating system.

If I run the sampling clock at 400Hz, then the circuit will be sampling, displaying and controlling at one eighth of that due to the multiplexer. Internally, my processor must manage to run the complete loop of each thread in 5 milliseconds to keep up with the sampling frequency of 50Hz.

Figure 7.31 Monitor, control and display classes

The operating system is now little more than a thread provider, object manager and i/o controller. I shall store the operating system, program and bitmaps in an EPROM (erasable, programmable, read only memory). I shall use a limited amount of memory, probably a single 1Mbit chip will be enough, and have no disks. The program will be written in C using a compiler created for the microcontroller, and written to the EPROM. Should I wish to revise my program, I can remove the EPROM from its plug in base, stick it in the EPROM programmer, and rewrite it.

All of the design work will happen in Eden. A full set of electronic components, circuit constructors and programmers is available, as is a fully operational physical environment.

Our car will exist, fashioned only of light and electrons and delivered to us down a cable of barely a hair’s breadth. Yet we can drive it across the face of Eden, and experience the race as well as if it was a dream.

Design is part invention, part imagination, and part engineering. It begins in the head, and migrates to paper, being improved by an iterative think, draw, think, re-draw, build, tweak, think kind of a process.

Figure 7.31 Car software structure

From here to code is one long, arduous step. Each item above must be built, tested, measured and improved. During coding some weakness in the design will be found, and the design will be revisited. Therefore it will improve. This revisit to design should be continuous within the coding process, by constant review and communication. People writing code will have ideas about the design. They may be better ideas than those they were presented with. If so, thank them profusely. Show what a difference their contribution has made and let everyone else know it too. They would be nowhere without your architectural statement in the beginning. You would be nowhere without them questioning your design and writing the code. You are locked in an interdependent game, but interdependence is greater than dependence.

Invent!

Invention is the pinnacle of achievement of the mind. We create what we believe we can create, however many years it might take, however much must be endured before we can spin our dreams into reality.

Such is the way of the inventor.

As software architect, you are inventor, chief designer and chief engineer. Or, you can claim:

I sculpt patterns of electromagnetic waves in concert with larger light sources to produce realities [9]

References

1. Influence Secrets of Successful Architects. Richard J Black. Fawcette. 2. The Mythical Man Month. Fred Brooks. Addison Wesley. 3. After the Goldrush. Steve McConnell. Microsoft Press. 4. Accounting for Non-Accounting Students. John Dyson. 5. Visual Basic and UML. Paul R Reed. Addison Wesley. 6. UML Components. 7. The Design of Everyday Things. Norman 8. Doing Hard Time. Bruce Powel Douglass. Addison Wesley 9. Posted on a WWISA discussion board by Shawn Bellina

Questions

1. How would you design Eden? 2. What design would you replace figure 7.13 with? 3. Design your own visual display for the virtual car. 4. Why is your design in question 3 better for you than mine is? 5. How much memory would you allocate for Eden's topological model? 6. Which processor would you select for Robbie's Controller? 7. Would you use an event driven or round-robin controller for Robbie, and

why? 8. How would you program Robbie's tactile system, and how would you use

that information to control his movements? 9. How would you let Robbie balance on his own two feet? 10. What would you do as programme manager if a team came to you

demanding half of Robbie's processor time for their own activity. Would you (a) communicate your needs to the processor producer, even though you are using their most powerful processor already? (b) reduce the other teams' allocations? (c) other? - if you answer other, please specify your solution.

11. If you created a being in Eden, and gave him responsibilities, drive, learning, ageing and self-awareness, how could you prove that he is a simulation and you are not?

Chapter 8 - Visual Design

The aims of Art

Once again I warn you against supposing, you who may specially love art, that you will do any good by attempting to revive art by dealing with its dead exterior. I say it is the aims of art that you must seek rather than the art itself; and in that search we may find ourselves in a world blank and bare, as a result of our caring at least for this much art, that we will not endure the shams of it.[1]

Visual design

We are not here to study technology for its own sake. We are to use technology to brighten the dark places of people’s lives. The artistic endeavour in software systems provides the greatest single measure of success or failure from the point of view of the user. And we may as well face up to the fact that no-one will thank us for perfectly designed and executed systems. They are simply invisible, and the more perfect they are, the more invisible they are. The greatest measure of the technical success of a software project it how often it is not thought about.

So what are you left with after designing the perfect system, to let people know how superb it is? The answer is its visual artistry.

A software system is living art when it is pleasing to look at, and flows from task to task easily. Display and navigation is all you have. If you can’t get from one task to another, and the screen is crammed with dialogs, you’re missing the point.

This part of design is sadly lacking in so many systems. Yet it is the most striking example of software failure or success. Designing your object system and database well is pointless if all you are going to do is hurl data onto the screen.

Some terrible software has looked good. It has won awards. Few awards are won for well architected systems, for software architecture cannot be seen, nor picked up, nor heard. Too many in the software field are rewarded for managing crises. The server has gone down. The system crashed. Those heroes who spend all weekend fixing them are rewarded. The person who prevents the system from going down, through careful design, never gets a second thought. They are ghosts. Once a project is finished, the software architect should vanish like a ghost.

In an old parable:

In ancient China there was a family of healers, one of whom was known throughout the land and employed as physician to a great lord. He was asked who in his family was the greatest healer. He replied:

I tend to the sick and dying with drastic and dramatic treatments. On occasion someone is cured, and my name is shouted from the rooftops. My elder brother cures sickness when it begins to take root, and his skills are known only here among the local peasants and our neighbours. My eldest brother is able to sense the mere spirit of sickness, and eradicate it before it takes hold. He is unknown outside our home.

Visual design is all about why this:

Figure 8.1 joe kane, landscapes

Is a better example for us than this:

Figure 8.2 joe kane, gardener

The first card offers the services of joe kane, landscapes. There isn’t much on the card, just a grey shape with a hole in the shape of a leaf. It is a business card, not an advert, so can afford to be slightly ambiguous. Landscapes are implied by the shape of a hill. All in all, it doesn’t strain the eye, and it is moderately subtle.

The second example is as blunt as a charcoal pencil. Joe Kane is a gardener, here’s his address and phone number, and he mows lawns and does something with trees. He’s a what you see is what you get kind of a person. Honest, straightforward, hard working. He's probably a good bit cheaper than joe kane the landscaper too.

So why is the first one best for us?

Firstly, we are creating software, and in your face software soon becomes dull. We need more subtlety, and the gentle arc of joe kane landscapes neatly contains the information. It follows the angle of the 4 as it rises from bottom left, and the leaf, almost invisible, seems well fitted to its container. Joe kane, gardener will soon tire its viewer with its chirpy lawnmower and apple tree. You could even get away with tearing the card down its right hand side and keeping only the informative bit. You would probably want to do the same with your computer screen after a while, but may find it tricky. That, of course, would frustrate you even more. Being inoffensive is often key.

Secondly, here are the thoughts of Frank Lloyd Wright[2]:

Challenge of the Machine. In this day and generation we must recognise that this transforming force whose outward sign and symbol is the thing of brass and steel we call a machine, is now grown to the point that the artist must take it up,

no longer to protest. Genius must dominate the work of the contrivance it has created. This plain duty is relentlessly marked out for the artist in this, the Machine Age. He cannot set it aside, although there is involved an adjustment to cherished gods, perplexing and painful in the extreme, and though the fires of long honoured ideals shall go down to ashes. They will reappear, phoenix like, with new life and purposes.

Either we take on the challenge of software art, or we get someone on our team who is artistic.

The dialog box

Have you ever had a monologue with a dialog box? The word dialogue comes from the Greek dialogos: a conversation. Does it feel like a conversation? I doubt it. It’s more like selecting a few entries from lists, typing in a bit here and there to guttural prompts like ‘User:’ Hardly a conversation.

Wizards are more conversation-like. They tell you about what you are doing, ask you to do one bit of it, then move on to the next bit. Finally, you see a summary of your conversation and you submit it for the record.

Figure 8.3 Keen user interface design

Let's list everything wrong with this dialog

1. It doesn’t look very nice. 2. The message is not very timely. It’s not telling me I’m about to run out,

just that I’m out. I’ve probably noticed anyway. 3. Where is the dialogue? My only possible conversational input is to say OK. 4. It affects my state of mind. It is a taunt. I want to respond with violence,

but violence would damage my screen, and possibly myself. Instead I bottle it up and rage at the air. In my annoyance, I dive out of the car, momentarily without my usual concern for safety, and get run over by a truck.

Form, colour and ornament: design ingredients.

Although rather lacking as an example dialog box, it does contain the three visual design ingredients of form, ornament and colour.

Form defines what something is. We know this is a dialog box because it looks like one. There is some kind of bar across the top, a bit of text in the middle, and a button at the bottom allowing at least some user input. It is something we have come to recognise through our use of computers. I doubt it would mean much to

someone unfamiliar with computers, though they could probably guess what was happening.

Colour is a visual delineator and highlighter. Colours can be used to subdivide visual elements, and also allow some artistic expression to the user who wants all buttons to be green, and backgrounds to be pink.

Ornament or decoration is taking a plain form, and embellishing it using textures, patterns or symbols. We ornament input elements with bevels, or 3d borders. We ornament our windows with close, expand and hide buttons. We ornament many items with small symbols, or icons.

Beyond these fundamental design ingredients, we can also add what could be termed dynamic embellishment. When you click a button, it looks like it clicks down. When you hover over an item of interest, a small note pops up explaining what its function is, or a hyperlink underlines itself.

Mixing these four ingredients and applying them to your user interfaces is what visual design is all about.

You must do the work of a graphic designer and a technologist in bringing your designs to life. A non-responsive screen is unfinished software.

The creative use of white space

In the first example of a business card, there is a lot of free space. Some find this sacrilegious waste of paper offensive, but only when they are doing the designing. They attempt to fill the white space with something of meaning, and in doing so, destroy the message. Leaving lots of space in graphics terminology is the creative use of white space.

It implies that the space left between items is every bit as important as the items themselves. Indeed, the whiteness itself can hold a message, or add to the ambience of the view.

Now

This one word defines us. We live in the era of the quick fix. ‘I want it, and I want it yesterday.’ ‘Daddy, I want to go to the park now.’ You can rely on (any carrier) to get you there on time.’ Our advertising artistry emphasises lateness is losing. We are under pressure to deliver software immediately. We want fast food, fast cars, trains on time, instant gratification. Microprocessors have to go faster. Gigahertz equals sales. We want everything and everybody out of our way so we can race to the grave right now.

Us and the essence of now are interlinked, and we have to provide it. Now means not making your audience think. They have to get the piece of information they want immediately.

User interfaces are designed to lead us down the garden path, but rarely do we get to see the garden. Maybe it is the very words user interface. We think of interfaces (and design them) to conform to logic, a defined process. Unfortunately users are people. They are all different, and they want a conversation, not a presentation. User: Grunt. Password: Grunt. User interface is probably one of the worst clichés to come out of computing. Users interact, they do not interface. We are not cyborgs (yet).

What do you think of your user? Are they worth a little more effort?

When analyzing requirements, we create use cases, and create stories and walkthroughs, imagining a conversation between the user and the use case. Then we kill them stone dead with our grunts of request: our dialogs.

One day we will be able to walk up to our world portal. We shall look into it and be instantly recognized. ‘I’d like a book on graphic design please’, you will say, ‘and a cheese salad sandwich.’

The sandwich will arrive, and you will begin your conversation with your portal so it can get the book which best matches your wishes. What kind of conversation will it be?

My guess is you will be presented with four pictures. Your eyes instantly move toward the one most pleasing to them, and you begin drilling down through an inconceivably clever indexing system, offering input along the way. You might simply say: ‘No, How about some mathematical definition of layout? OK, a little less formulaic, and more example based. Your eyes flit to your final selection. ‘Let’s have a look at that one.’ You read through a few sample pages and give it the nod. The e-book reference is added to your personal library, and you begin browsing. After a while, in your relaxed way, long past the time of the worship of Now, you have a look through your communications. One of them is your daily business summary. It shows you a vision of your business from precisely your point of view. It took a day or two to design with your information stylist, but it was well worth the effort. Now you know exactly how all aspects of your business are doing with a single glance. What did you see?

Was it a page of option buttons and links to data packed reports? Was it a forest of scroll bars, drop down selections and cheesy icons? I bet it wasn’t.

Of course, now is relative. Business people generally want their business applications now, but the same people wouldn't want a their cardiac pacemaker software to be created the same way.

The difference between data and information

Information is an abstraction of data. Just as we abstract away from code to objects to applications to systems as software architects, so data can be abstracted into information.

Figure 8.4 Data

Here is some data. We can present this information in a graph.

Figure 8.5 Data abstraction leads to information

The graph is a common and familiar form to most people. We have different types, some of which are more common than others, such as a line graph, bar graph, pie chart etc. We’re a little carried away on 3D pie charts and histograms at the moment, but that is only because we worship now and want to be cool. It will fade after a while and we shall see once again in 2D clarity. Then we can create a proper 3D pie chart. Whether our third dimension is of the xyz type, or compressed time, is up to us.

The graph is only one single step away in abstraction of the data, as the data can be taken from the lines on the graph. Here are some ways of abstracting the same data to appear in an instantly readable way, abstracted two or three steps away from the data, and delivering information.

Figure 8.6 The data abstracted for instant viewing

The smiley face on the left says your business is ahead of budget. You don’t know how far ahead of budget you are, but you may not care in this momentary glance. The second example shows a bar where the centre line represents the budget (targeted sales perhaps), and the black area represents the ten percent above and below budget. You can see instantly that you are more than ten percent above budget. This is good news. The final offering is merely a graph reduced in size, where the grey area represents the budget, and the line represents the actual. Again, you can see quite clearly you are above budget, and have been since the beginning of the year.

When designing sets of reports, the most common (if not the only) top level display is a list of reports. The user clicks one, enters some parameters and gets a pile of data. Data does not equal information. We need to be more creative.

Data leads to information. Information leads to understanding. Understanding leads to control. If you present data, then your viewer has to extract the

information and then gain the understanding. Our job is to deliver that understanding, then let the viewer drill down into the information, and if necessary into the data if that is their wish.

Reality or understanding?

Do you want to present reality or understanding?

In 1933, H.C. Beck created a superb example of information graphics, which remains today in its original implementation, even if subsequently updated, and is copied throughout the world.

Here is what existed before:

Figure 8.7 The London Underground map before 1933

Beck’s map of the London Underground is a landmark of excellent graphic design. The format has not changed, yet it has proven easy to update as new stations and lines have been added to the underground system.

Despite its simplicity and extensibility, and the fact that it has been copied ever since for rail and underground systems almost everywhere, it was initially criticized for falsifying the geographical locations of the stations.

Figure 8.8 The London Underground map of today

This is often the way of excellence. Around the same time that Beck produced his work, another designer, Alfred John Butts, a New York Architect, was working on a game. In 1938 he began a cottage industry manufacturing and selling his game. It never really caught on, and after a few years he gave up. Nevertheless, his dream survived. Nine years later, in 1947 he tried again, and for five years continued with little success. Then, in 1952, at a time when television was attracting the masses, and promising a dark world of illiterate couch potatoes, his game suddenly took off. Scrabble is the same game now as it was then.

For more on this and other design classics, see 20th Century Design Classics[3]

So we are in a Catch 22 situation. If you design a classic, no-one will accept it. They will need to enforce their point of view on it, ergo (if you will forgive my Latin), you will not create a classic.

Catch 22[4] is the title of a novel which examines the problem that you have to be insane to get out of the army, but if you are insane, you won’t want to.

To learn from Matisse, who declared ‘I do not paint things, I paint only the difference between things.’ How would you display the summary of business information with only a small area in which to do so?

It might look like this:

Figure 8.9 Monthly business summary

I doubt anyone would respond positively to a suggestion that this is a sufficient summary of one month’s worth of business information, yet it is perfectly valid.

Nothing is shown, which means there is nothing of interest for a business viewer to get worried about. They are on target on all their specified measures, so they can go home, put up their feet, and not worry about anything. But worry they do. It is always better to show something which means there is nothing to show, than to show nothing.

What do you have at your disposal for displaying information? According to The Artist’s guide to composition[5], there are seven media for expression.

Figure 8.10 Diagram composition methods

All of these forms are available to us on our computer screens. Some of them, designed for the days of monochrome are used rarely now, and even then only in reproductions of the past. Certainly texture, and possibly tone are not as useful as colour, and do not render very well on our screens. Colour is the most definite modern winner. With colour you can create 3D effects, light and shade, and contrast colours to represent differences. Colours can give us contours, imply relationships and even have meaning and emotion: Red equals danger; blue equals sad; a yellow zigzag means electricity etc.

The relationships of colour

The choice of colours available to us is vast. It is here that many software products fail because designers choose colours that just happen to appeal to their warped vision of the world. Using colours is interacting with your user, and you do not want to silence them before the conversation starts. Reassure them. Use the system colours they have chosen to colour your palettes of interaction.

Colours have properties which are part of our human psyche. A colour can be warm or cool, light or dark.

Figure 8.11 A colour wheel showing warmth and lightness

Colour Terms

Hue is the everyday name a colour is known by. Red, Brown and Turquoise are all hues. No-one can agree how turquoise is pronounced, let alone what it looks like. There is no definition of what these names mean, though there are colour tables and definitions which use them.

Tone is the lightness or darkness of a colour when it is converted to greyscale. A tone is a colour’s position on the scale between black and white.

A tint is a lightening of the tone of a colour, whereas a shade is a darkening of the tone. Note the colour does not change, only its lightness or darkness.

Colours on computer screens are often defined as 8, 16, 24 or 32 bit numbers, where the grouped binary digits of the number represent a colour according to a standard colour model.

The RGB model is most easily understood. Suppose a number (in hex) is FF8000. This contains FF red, 80 green and 00 blue. Mixed together these give orange. The ARGB model also supports an extra byte for a transparency. It is known as the alpha value, where FF means full colour, 80 means semi transparent and 00 means fully transparent.

Similar to RGB is the CMYK (cyan, magenta, yellow, black) model. RGB is additive, and more relevant to a screen based on light, and CMYK is more in tune with printing as it contains the base colours used in many colour printers.

The HSV model divides the number into hue, saturation and value parts. The hue is defined as the angle around the colour wheel clockwise from red. Saturation, also known as chroma or intensity, is the point between the pure colour and its equivalent tone (or greyscale value). The more grey of the same tone is added to the colour, the less the saturation. Adding white to lighten it increases the value, and black to darken it reduces the value.

Colours in printing are often known by Pantone® values. They are partially applicable to colours on screen, but only for colour matching print work.

Colour relationships

Here is where many user interfaces become ugly.

Complementary colours come from opposite points on the colour wheel. When mixed together, they produce grey. When positioned side by side, they intensify one another and are good for showing comparison. A complementary colour can be found by looking at a colour, then looking at a blank piece of white paper. The afterimage is the complementary colour. On a clock, the complement is at 6 o clock when the colour is at 12. I shall use a 12 point clock to describe all other colour relationships.

Split complements match 5 and 7 to a colour at 12. There is some neutrality between the 5 and 7 colours, yet both together are complementary to the original. The mixes between 5 and 12, and 7 and 12 are known as near complements. If you wish to show the differences between two related values and one other, display the related ones in the split complements, say blue and green, and the comparator in the complement, red.

Analogous colours are three or four adjacent colours. They are harmonious and have little contrast so do not jar the eye. Contrast can be added by intensity. This is used often in reports, where similar colours of varying intensity are used to display alternate lines in a report or grid without making it too offensive. Columns or rows of totals may be a shade of an adjacent colour.

Equal triads are the 12, 4 and 8 positions. A mix of any two will produce a complement of the third. Using equal triads means the display may lack vitality. Yellow text on a blue background was popular for a while in text editors. Adding red would create an equal triad. Imagine yellow on blue with red text for warning messages. It would look rather demanding.

Unequal triads give the widest variety. They are any three colours which do not fall into the previous categories, and have unequal adjacent intervals.

Warm colours advance and cool colours recede

This is what artists learn early on in their career. With it, they can make skies and mountains shrink to the backs of their canvases, and make bright flowers stand out in front. It is what makes a room painted blue look larger than one painted orange. The blue recedes, so the walls look further away.

Have you ever noticed how mountains and forests in the distance always look a pale blue, almost matching the colour of the sky? This effect is caused by the air between you and there containing water. Water leaches out the colour and detail. In painting, simply adding white can push something back, to make it more distant.

So now when you are creating a view in a table with headings and subheadings, if you want the subheadings to stand forward of the heading, use a warmer colour. Be subtle; do not have a red heading and a pale dark blue subtitle, instead tint one colour with a warmer or cooler colour to produce the other.

So how can we use colour?

When creating system objects, use system colours. When creating help files, use the default system colour setup. When creating illustrations, or visuals which represent business or technical entities, you have free range. Experiment with your complements and triads.

All works of art (the good ones anyway) are composed. Do you think Mona Lisa just happened to sit down that way and Leonardo painted a snapshot? So think about what is being displayed, how it should be displayed, and what colour combination would best bring out the information.

Great photographs are composed just the same as great art. Perhaps even more so, for the artist can choose simply to omit what is not visually pleasing: the telegraph wires, electric pylons, ugly shaped clouds. The photographer is not so lucky. Poor composition is why your holiday snaps never manage to capture the place as it looked in the holiday brochure.

All software and printed presentation must be composed. It cannot be hurled together. Artists and photographers are lucky. They can choose their own image size, and if others do not like it, they do not look at it. You are stuck with a rather small screen and limited resolution, and a user who likes to resize their view, or change the base font size. And if the operating system supports it, so must our applications.

Fonts

We call them fonts. Printers and graphic artists call them typefaces. Nevertheless, we must approach our use of them in the same way. We are generally worse off than our brothers and sisters in graphic design, because our working resolution is so much poorer. No graphic designer has to worry about how the size of a pixel will affect their art, and our glorious selection of fonts leads us to the assumption that we can use all of them.

The bog-eyed monster font may be your favourite. You may like the lovely florid parchment font. Because you like them does not mean you should use them to present information. At best, you can use the font selected by the system, at worst, stick to very plain fonts. Arial or Helvetica, Times New Roman, or fonts devised specifically for computer screen clarity such as Verdana. They are the items in your font palette. Do not get artistic.

A rule of thumb for graphic designers is never to use more than three fonts in any one piece of work. In software, I recommend never using more than one if you can help it, especially on the same screen, and avoid italics wherever possible. Printed italics are rare, and used only for brief highlights. They are not as clear to read, and reading italics takes longer than reading normal text. On computer screens, italics are more difficult again. Be frugal.

Information graphics

The Visual Display of Quantitative Information[6] discusses the ups and downs of centuries of information display. It refers to graphical embellishments collectively as chartjunk, dividing this into vibrations, grids and ducks.

Vibrations such as cross hatching, or regularly repeated small items, can cause the eye to see them as moving. Grids are the overpowering lines behind chart information, which are so detailed that the information takes second place when attracting the viewer. Ducks are embellishment for embellishment’s sake. You will have come across 3D histograms, 3D pie charts. They are ducks.

Excellence in graphical display is

• Show the information • Induce the viewer to think about the information, not the way it is

presented • Avoid data distortion • Present many numbers in a small space • Encourage the eye to compare different pieces of data • Reveal the data at several different levels of detail from a broad overview

to fine detail • Serve a clear purpose • Be closely integrated with verbal and statistical descriptions of the

underlying data • Present the data with the absolute minimum of embellishment

Figure 8.12 Overstated graph

There are many things wrong with this graph. First, the grid overpowers the data. Text travels in two directions on both axes, making it difficult to read. Finally, the numbers up the left hand side increase in size. There is an implication in this increasing size that 20 is more important than 5. This may be true, but the importance is stated enough in our understanding of the numbers. Increasing its size turns it into a lie. The viewer’s attention is drawn to the larger number. Already implied in the viewer’s mind is that 20 is 4 times (more significant than) 5. It is now overstated by being four times as large. The mind of the viewer resolves both of these aspects and decides 20 is now 4 x 4 larger than 5, once for the number, and once for the size.

A much cleaner solution would be:

Figure 8.13 Understated graph

There is no grid, no lie. The text all travels in the same direction, and there isn’t much of it. Yet the meaning is just as obvious.

Data density and information clarity

This is the true measure of displayed information: how easy it is to understand, and how clear the data behind it is without having to see it.

When deciding on creating graphics to present a point of interest, try out the following questions:

• Is a graphic better than a sentence? • Is 3D really clearer in this instance than 2D? • Is a rendered 3D model necessary, or is a wire frame acceptable? • If this is to be printed, is colour an improvement on black and white? • Do I really need animation, or is a static image just as clear? • Do I need to interact, or is non-interactive sufficient? • Can I present this in simple animation instead of using virtual reality?

The golden section

When specifying a preference for one particular rectangle over another, on average, people will increasingly prefer a rectangle which approaches the long to short side ratio of 1.618:1 or approximately 34:21. This ratio is the golden section. A rectangle where the sides are in the ratio of the golden section is called a golden rectangle. It is the people's favourite.

Figure 8.14 The golden rectangle, or golden section

The golden section is discussed in many graphic design books, for example: The shapes we need[7]

1.618 is a bit of a magic number. If you divide 1 by 1.618, you get 0.618. 1.6182 = 2.618.

Before you rush back to your design and start hacking in golden rectangles, I should warn you: although 1.618:1 is the favourite of the masses, it does not mean that everyone prefers this shape, it is merely the rather small majority of 35% of us. The other 65% prefer other dimensions, albeit to a lesser degree (around 20% preferred 2:3, and 3:4 is favoured by less than 5%. There is also a slight difference between men and women, though they both peak at around 35% for the golden section. Interestingly, only around 3% of males and females preferred a square over a rectangle, but this does not mean you should stretch out your square toolbar buttons a little more. You would get less in. We are limited by our technological resources.

The golden section was much used in Greek Architecture, and has been used in many paintings, in an attempt to get good proportion. It also exists in mathematics, and in nature in the way our features are divided up.

In addition to the golden rectangle, we have a golden spiral, where the spiral is marked out, traversing successive edges of golden section rectangles.

Figure 8.15 The golden section spiral

This spiral is claimed to exist in nature’s spirals, for example, on seashells.

Figure 8.16 The golden section in a pentagon

The Fibonacci series is representative of the golden section

1 1 2 3 5 8 13 21 34 55 89 144 …

Each number in the series is the addition of the two preceding numbers. Dividing one number in the series by its predecessor, we get

1 2 1.5 1.6 1.625 1.615 1.619 1.618 1.618 …

That is, a gradual approach on the golden section.

Given a choice, you might like to use the golden section to size up elements of a dialog. Here is an example:

Figure 8.17 Golden Dialog

The golden section is used for the shape of the main white window, and for the dialog itself. The dialog is divided horizontally by the golden section between the white window and the pale window with the text (large picture) in it.

Not everything uses the golden section; that would be taking it too far. You would be losing functionality for the sake of similarly shaped visual objects.

Figure 8.18 Golden section cats

Expecting the golden section to immediately solve all of your display problems is a mistake. It is merely a useful measure of preference. As an example, look at the example above. Fonts are usually designed to aid clarity and enable you to read quickly. This is why most people still prefer paper over video screens. The printed font is far clearer than its video equivalent.

Cats, as shown above, has had the a and s squashed, and the t cropped at the top to fit into golden rectangles. In addition the whole word has been stretched upwards, destroying its original aspect ratio. It looks ok, but not quite as right as in the unaltered version on the right.

You have an eye. Use it.

The Windows Interface

Visual Interface Design for Windows[8] discusses the design of the Microsoft Windows interface. It shows some fine examples of good design across the whole range of software from interactive children’s education packages to icons.

In the early days, Windows was a clunky 3D in your face kind of operating system. When the 32 bit operating systems came along, the 3D aspects had been softened, and many dialogs which had white backgrounds were given grey backgrounds. It was something of an improvement, and there was a style guide for applications, and a sticker confirming that the software you were about to buy conformed to the new ‘ideal’.

This was about the time when the internet was changing from a place to share information to a place where people could make money. The internet was flat. It was groovy, coloured, and ruled by graphic artists. Suddenly graphic art became the driving part of company X’s web design. It matched their current printed graphic image. There were still the clunky 3D controls for forms, which looked pretty sad when compared to the beautiful designs they intruded into.

We have gone from flat paper, to clunky 3D to smooth 3D, and now we’re back to flat.

It seems that encyclopedia software, maps, games, children’s information, web sites, video and audio players can be groovy, exciting, and artistic, yet business software has to conform to the ideals of the grey, slightly 3D world of the operating system basics. Why is that?

Visual Interface Design rounds off with a collection of great interface work, and the seven pitfalls that user interface designers make. Briefly, they are:

1. Visual inconsistency. You should establish a design strategy before you begin. With web/XML, this is easy, as you can reuse master style sheets. On compiled apps, it is a case of using your eye to line up items on the interface, to make sure they are well proportioned, and not there just because that's where it was first placed.

2. Lack of restraint. It is better to be consistent, than to reach for the odd superb (in your own very important opinion) output.

3. Overbearing metaphors. We have talked enough about metaphors. Some designers take them too literally in their translation of reality to a computer screen. For example, the Lotus Organizer/Diary which had a huge ring binder down the middle, rendering a great proportion of a rather limited area of screen unusable.

4. 3D overkill. So many web sites and applications go overboard to show 3D buttons with shadows that change as keys are pressed and mouse devices are hovered or clicked. Be subtle.

5. Too many bright colours. Use as few colours as possible, and make them softer and more neutral, unless you are trying to get your viewer’s attention. A big red ‘YOUR MACHINE IS ABOUT TO CRASH HORRIBLY’ across a screen usually gets someone’s attention, and it may be quite relevant. Otherwise, refrain.

6. Bad icons. Drawing icons is quite difficult, especially so for non-artist techies. Icons should be consistent, use consistent colours, and be quite clear as to their meaning. If you can’t find a suitable icon, use text.

7. Bad typography. I reckon this should have been number one. It is the greatest failing of techies. ‘Wow what a cool font. I’ll use it in all my systems from now on.’ Then they realise that no-one can read it and decide they have found a far cooler one, something like letters made up from bullet holes in road signs.

Bad typography often comes from mixing too many inappropriate fonts. On a computer screen, anything but the clearest fonts used in large measure are an instant turn off.

Bad typography is also pages and pages of scrolling Times New Roman 12 point Normal without a break for the eye. There are hundreds of societies and universities with absolutely awful pages, as they were written with the intention of providing text rather than sharing information. Most of them are copyrighted so long ago that the originator has probably forgotten about them.

More on fonts

Fonts are extremely complex. Anyone who has coded how text is displayed on the screen or printed page will have gone through the nightmare already. For those who haven’t, here is a summary of font terms. They may come in handy one day.

TWhat we call a character, a printer calls a glyph. Most of printing’s terminology is derived from long ago when printing characters on the page was a painful manual process. Somewhere between carving wooden blocks, and the modern pixel based devices, printing terminology was born. It is steeped in tradition, such as CAPITALS were organized in the wooden case furthest away so the printer had the most common form of letter nearest, i.e. in the lower case. Hence, upper case and lower case letters.

Figure 8.19 Characters

The height a character rises above the baseline is called the Ascent. Ascent for a given font or typeface is usually measured by the M glyph. Descent is how far a character drops below the baseline.

There are a number of different types of font. In the old days we had bitmapped fonts known as raster fonts. They were quick to draw and designed specifically for fixed size, pixellated displays. Now we have truetype and type1 fonts which are vector fonts, scaled to any size, and suitable for any display medium.

We also have two font styles in terms of fixed or variable pitch fonts. Fixed pitch are descended from typewriters where each character is the same width. Variable pitch fonts are descended from printing where characters are separated by the distance most suitable for reading comfortably.

External leading is the distance between one line of text and the next. Leading is pronounced like heading, and is derived from the amount of lead (plumbum – the metal) placed between the lines of text in printing. Internal leading is the distance from the top of the character cell to the top of the character. This varies from one character to the next, but the internal leading for the font is once again measured to the top of the letter M.

Variable pitch fonts use two methods for font spacing. The first method is to give each character or glyph a set of three horizontal distances, a, b and c.

Figure 8.20 abc dimensions of glyphs

The first glyph ‘M’ has a very small a dimension and a zero c dimension. The second glyph ‘f’ has a negative a and c dimension. When an M is drawn next to another M, the spacing between them is a + c. As c is zero, the distance is a. The second glyph, having negative a and c dimensions means that when the two ‘f’s are drawn together, they overlap.

The second method of spacing is named kerning. Kerning is a further quality measure in readability, where pairs of characters are drawn differently to the a-b-c spacing. For example, a ‘K’ followed by an ‘e’ may be defined as a kerning pair as they will be spaced differently to a Ki or Le combination. Each font carries with it a set of abc dimensions and a kerning pair table.

Fonts are further defined by belonging to families. Roman family fonts, for example, Times, are proportional serif fonts. Swiss fonts, for example, Helvetica or Arial, are proportional sans serif fonts. A serif is the short right angled line at the end of a stroke.

Figure 8.21 Serif and Sans serif fonts

Other font families are script, which resembles handwriting, modern, which is fixed pitch and constant stroke width, and finally decorative, which catches anything not in the other categories.

Enough on theory. I shall not trouble you further with diacritics, ligation and kashida, but if you want to display languages other than English, you will come across them. For this, and more discussion on programming fonts, see the Visual Basic Programmer's Guide to the Win32 API[9].

Faux pas, and finesse with fonts and characters

Finesse…

• Use SMALL CAPITALS to emphasize a word where bold or italic looks ungainly, e.g SOFTWARE ARCHITECTURE can be an improvement over all capitals or mixed upper case and lower case.

• Use upper and lower case numbers: Call me on 01234 987654, or Call me on 01234 987654. Lower case numbers with descenders can avoid the blocky look of upper case numbers if desired. However, few fonts use lower case numbers.

• Typefaces have distinct personalities. You must choose the one which delivers the message most elegantly.

• flbacus - does this say Abacus? Paste it into a word processor and add spaces between the letters. When designing graphic displays or splash

screens, it may be worth investing in fonts with ligatures to avoid clashing in the following combinations: ff, fi, fl, ffi and ffl.

• When using ellipses, use the correct character (char 133 in truetype fonts) rather than entering three periods: ... != …

• When using dashes, use an en-dash (the short one) rather than an em-dash (the long one), unless the word following the dash is hyphenated: Pearly Wilks — chief-of-staff of…

Faux pas…

• Do not type two spaces after a full stop. Proportionally spaced fonts and full justification (aligning a line to both left and right borders) make a double space look odd.

• Do not underline. It is a hang over from the days of typewriters and looks messy. Use bold or italic instead. For even greater printed legibility, use bold and italic versions of typefaces rather than the bold and italic options in the menu bar, although on screen this would make little difference.

• Use ‘single’ or “double” quotation marks rather than the neutral 'single' and "double" ones, traditionally used only to indicate units of measure.

• Sometimes the default leading (line spacing), tracking (letter spacing) and kerning (specific letter pair spacing) do not convey your message adequately. Feel free to tweak.

This part was freely adapted from [10].

A change of view

There is a good example of the dying 3D approach, and its replacement with subtle tones on Microsoft’s Developer Network (MSDN)[11]. It is titled Start Making Sense. Here are the before and after interfaces:

Figure 8.22 Pre web, 3D lovelies

Figure 8.23 Post web, flat, toned rather than coloured

Image types

We use four main image types: vector, bitmap, gif and JPEG. For large diagrams, vector graphics is by far the best method, providing the tools to render it are available.

Bitmaps should be kept to an absolute minimum as they are often huge, especially when compared to compressed file formats for all but the smallest icons.

Gifs are better than JPEGs for diagrams, where there is a limited colour range and high contrast between the colours. JPEGs are better for photographs and pictures with a lot of colour blending.

Top Left

Light, according to the designers of our operating systems, comes from the top left of the page. It is a part of our ‘western’ culture. We begin top left when reading a page. If you are defining a 3D look and feel to a particular piece of software, you should keep this in mind while illustrating. If light comes from top left, and you draw all of your interfaces as if the light came from bottom right, all your highlights will look like indents, and vice-versa. Your viewer will be confused. It will strain their eyes and brain, trying to realign their expectations. Simply, it will not work for them.

Other cultures read from the top right or in columns, but that is rarely reflected in our operating systems.

The rhyme and the reason of graphic design

Good design draws the eye and engages the mind. That is the measure of it. Unfortunately for graphic designers, that is not the limit of it in software.

Software also requires usability, and poor usability can destroy your beautiful creations.

Usability is a subject in its own right, and dealt with superbly in Don’t make me think[12].

As good design catches the eye, usability guides it. In a modern windows style system, people know where to click. They look for the 3D buttons with OK or Cancel on. On the internet, they look for buttons, or the underlined or uniformly coloured hyperlinks. If you do not lead your users around by the eye, you will lose them.

And finally

The Speaker, Betty Boothroyd, rebuked a [Member of Parliament] for using a cardboard diagram in the [House of] Commons to explain overseas aid figures. She said “I have always believed that all Members of this House should be sufficiently articulate to express what they want to say without diagrams.” [13]

References

1. The Aims of Art. William Morris 2. The art and craft of the machine (1904). Frank Lloyd Wright 3. 20th Century Design Classics. Chris Pierce. H.C. Blossom 4. Catch 22. Joseph Heller 5. The artist's guide to composition. Frank Webb. David & Charles 6. The visual display of information graphics. Edward R Tufte. Graphics Press 7. The shapes we need. Kurt Roland. Ginn and Company. 8. Visual Interface Design for Windows. Virginia Howlett. John Wiley 9. Visual Basic Programmer's guide to the Win32 API. Dan Appleman. Ziff-

Davis Press 10. Font & Function No12 Autumn 1993. Adobe Systems 11. MSDN article: Start Making Sense. Mike Pietraszak. Microsoft. 12. Don't make me think. Steve Krug. New Riders 13. The Guardian. 7th December 1994

Chapter 9 - Writing Code

Implementation, Construction, Programming …

There are so many words for writing code that are confused with other activities. I have called it writing code. I shall not argue the point, call it what you will.

We are too hung up on the clever (and confusing) names we give to things in software. I even saw a list of comparisons between the terminology of different software processes. It was frightening. Process A calls this an artifact, process B calls it an object, process C ...

As Bruce Lee said[1]:

Learning Jeet Kune Do is not a matter of seeking knowledge or accumulating stylised pattern, but is discovering the cause of ignorance.

If people say Jeet Kune Do is different from ’this’ or from ‘that’, then let the name of Jeet Kune Do be wiped out, for that is what it is, just a name. Please don’t fuss over it.

The software process so far

The software process this book is based on is the Unified Process for Software Development. It is a fairly detailed process, and may not be particularly suitable for your environment. Even so, if you can understand the detailed processes, you can decide how much of them you need to include in your world, and how much you can leave out.

In this chapter, we shall look at some of the processes where writing code far outweighs the other practices. I shall refer to them as agile processes.

Writing code

You know all about coding, right? You’re a developer. You’ve read Code Complete[2] and do your best to implement all of those good coding ideas. It’s a little tight on time sometimes though…

Maybe this is true, but to become a software architect you must approach coding as part of a process. It should be done in a structured, predefined way and follow an architecture and then a design. Writing code should be as effortless as a mechanic knowing where just the right tool for the job is. That is, the coder knows the correct method or algorithm to use, and uses it effectively. Design leads code. Code must not lead design.

As a software architect, you will probably not be writing any code other than test cases or prototypes, but you will be involved in leading how the code is developed, ensuring it follows the vision that requires it. You will also be reviewing code, ensuring standards are followed and correct algorithms and methods have been used.

You will also be involved in defining the environment for code creation. Whether you use a detailed or lightweight process is for you to decide. The lightweights I shall discuss are Rapid Application Development (RAD), Extreme Programming(XP) and Agile Modelling(AM). RAD has its flavours, known as Joint Application Development (JAD), Rapid System Development (RSD) and the Dynamic System Design Method (DSDM). DSDM has it’s own organisation[3]. XP has its own books[4] and websites, and a growing following.

This discussion will be only a light breeze over the surface of each. Whole books exist on each subject, and in XP, whole books exist on a single practice.

RAD, JAD, RSD, DSDM

Not all projects are suitable for RAD, but all system and application development can benefit from its ideas.

Briefly, they are:

RAD 1. Output

The project should be visible to its intended users as soon as possible, and regularly updated. The stakeholders will then be able to have more input to the system, to contribute to it, and see that its growth is continuing in the right direction. When the final release is made, there will be no surprises. They will be able to see their contributions, and also receive something that is immediately recognisable. User input is vital for project success and the users are considered a part of the development team.

Some software houses have a daily build of their product. If the build fails, or fails to work properly, then the developer fixes what has gone wrong, and learns a painful lesson in the process. Daily builds, backed up with configuration management, ensure that problems are never larger than the effort one developer can put into a day.

Daily builds are also an excellent way of controlling flights of fancy. When coupled with frequent reviews, they ensure the software system is a cohesive whole, rather than a collection of unrelated applications.

RAD 2. Business involvement

Get business people involved throughout the development cycle, from your side and your customer’s side. If you actively involve them, they will be part of your team. The less you involve them, the more work you will have to do to keep them happy later. Development decisions are made more easily with input from both sides, and as the two sides work together, they begin to understand each other’s ways of working and their personalities. People factors are critical to project success.

RAD 3. Seating

Have the developers sit with the end users during development. If there is a problem or misunderstanding, it can be resolved with immediate access to the users. It will build a good relationship, smoothing the difficult transition to live. Often going live forces two contrasting sets of people together, i.e. the developers and the users, and that is only the beginning of many problems.

Sitting together removes the communication overhead associated with telephone, email, time zones etc. People communicate far better face to face. Millions of years of evolution has taught us the meaning of the slightest facial or bodily nuance, which is lost immediately when technology comes between us. Even videoconferencing is third rate compared with face to face meetings. Sitting together also enforces overhearing. Overhearing binds teams together. You cannot overhear an email unless you’re snooping.

Check peopleware[5] for a discussion on the communication overheads of telephone, videoconference etc.

RAD 4. Testing and release

Testing and release should be integrated into the development cycle rather than performed as a separate activity at the end of development. If testing is coupled with a daily build which is then released to a live preview system, further strengthened by face to face communication, then the project will run far more smoothly.

Many of the RAD practices are reflected in eXtreme Programming(XP), which has taken RAD through its next evolutionary step. XP has also produced the agile movements.

eXtreme Programming

XP1. The planning process

During project planning, the customer defines the business value of each requirement. This is matched with the cost estimate for development, and the two measures together decide what is to be included or deferred in each small release. The planning process is horse trading. Imagine the sponsor wants your application to have its own wallpaper. It’s not likely to help the project succeed, and along with the extra time it would take to write and test the code, it is not something you want to include in the project until the very end (when your customer may have forgotten he asked for it). So you trade…

Customer: I really need the project to have it’s own distinctive wallpaper based on our company logo.

You: OK. That’s quite a bit of development effort, so for the first release, I won’t be able to include it if you also want me to include the email facility and deliver it by the end of the year.

Customer: Oh no, I must have the email link. The first phase must be in before year end so we can transfer all our accounts. If we don't transfer the accounts, we will have to keep the old system running until year end the following year and we want to phase it out next year because it will not be supported after that.

The customer is then led down the path of not saying what goes in a release and what stays out, but rather makes the choice between options.

It helps both you and your client, as the client is involved (making the choice) and you get to do one instead of both. Project success or failure is often based on the leanness of the offering.

Allowing your customer to choose between items is a sales technique. The trick is to wait until you have some interest from your customer, and then instead of blurting out: “So you want it then?”, you offer them a choice.

Would you like twenty or fifty in the first batch?

Do you prefer the green or the red?

Answering no to this type of question is very difficult, and once you have a stated preference, you’re on the Cresta Run to a sale.

Would you like delivery by year end, or the wallpaper?

For more on selling techniques, see Who Dares Sells[6].

This is much easier with an external customer. When developing software for internal use, the customer is often your superior. You do not have the luxury of saying: ‘Yes you can have A and B. I shall have to raise my original estimate by £100,000. Sign here.’ Instead, your boss/customer says I shall have A and B and you will do it. No interface, no separation, no way out.

Or rather, the way out is your relationship with your internal customer. If it is one built on mutual trust and respect, then you have something to work with. Building that respect is like climbing a snow slope. Steady progress is better than a lunge. A lunge is the beginning of a long slide back down to the bottom.

XP2. Small Releases

Teams put a simple system into production early, and update it frequently on a very short cycle. Again, if your project is visible to all, they will grow with it and contribute to it. It will be seen as their project rather than yours. Another side benefit of this is that all developers must continually update their own set of code, keeping everybody in synch. Having different developers on different versions can be irritating and wastes time as you try to get them up to base. Updating everyone’s code daily improves team interaction. And don’t do the builds at home time!

XP3. Metaphor

The teams use a common system of names and a common system description that guides development and communication. This may seem a rather obvious thing to do, to start a project by defining the business or system objects, and terms that your customer uses. It could be as simple as this:

Employee A person employed by the company. Applies to permanent and contract staff.

Personnel Number

A unique identifier for each employee. Assigned to all permanent and contract staff on joining. Its format is AAAANNNNA.

Assignment A unit of work undertaken by one or more employees

Assignment Code

A unique and unchanging identifier

etc

It’s such an obvious idea that it is frightening how often it doesn’t get done. Let’s do one for this book:

Author Me

Reader You (yes, I know its silly, but let’s not forget.)

Developer Someone who writes code

Requirement Something the product must do

Scenario A brief play to help explain a requirement

Use Case Something that something does, usually at a high level

Implementation Whatever you want it to mean

Constructing and maintaining a data dictionary, or system of names, needs a champion to manage it, and buy in from the various interested parties.

XP4. Simple Design

A system or application should be the simplest program that meets the current requirements. There is not much building for the future. Instead, the focus is on providing business value, and delivering it as soon as possible to bring quick benefits to the business. In these times of rapidly changing business processes, the biggest profits are often made by those embracing the change most rapidly.

This does not mean the programmers should thrash out code. Good design is an absolute necessity for quick delivery of reliable business systems. What it does mean is that adding weight to a system by designing for expansion or adding alternatives to cope with changing requirements, is not done. The code is as lightweight as possible. If the business changes in a year, then write the code again.

Changing platforms, technologies and programming languages have often made our old software obsolete, where no amount of rework will save the ageing product. From this point of view it is better to write thin, then rewrite thin when required. In many cases, the design is also obsolete, so an argument to design for the future and code for the present would be folly.

XP5. Integrated Testing

Teams focus on validation of the software at all times. Programmers develop software by writing tests first, then software that fulfils the requirements reflected in the tests. Customers provide acceptance tests that enable them to be certain that the features they need are provided.

Writing tests before writing code is not a bad idea. What it does mean is that there must be a design upon which the tests can be written before coding starts.

On the negative side, developers writing tests before coding begins leads to procrastination. It is better to have testers writing tests in conjunction with the

developers. Both development and test must be based on a clearly understood design, and clearly stated requirements.

XP6. Refactoring

Teams improve the design of the system throughout the entire development. This is done by keeping the software clean: without duplication, and with high communication. It should be simple, yet complete.

Refactoring is an excellent technique which can be used to continuously improve existing code, to rewrite parts of a system, and to save projects in trouble.

Take a scenario of adding a single function to an existing piece of code. When the function is added, you realise there is another function which does almost the same thing. You refactor by isolating the common functionality into a new function, then call it from both the new and the existing location. A small amount of code will call the new parameterized function from two places, but most of the work will be handled by the new function. The non-refactored code would have two similar functions, leading to test, debug and update confusion in the future.

There are four steps to refactoring on a large scale, they are isolation, re-creation, test and replacement. The first step is to identify all interfaces to other systems or objects and ensure that there is clean isolation by rewriting the interfaces where necessary. The second step is to rewrite part that is obsolete, is no longer performing adequately or requires a more modern algorithm. When the new part is adequately tested, it replaces the old one.

Refactoring is a fairly large subject and discussed in Refactoring: Saving projects in trouble[7].

There are many methods of refactoring, and even refactoring tools which plug into development environments. Refactors have also been classified at the code level into method, field and class refactors. Such classification can detract from larger refactors where entire chunks of systems are refactored as one. Here are some of the more obvious methods of refactoring.

Method refactors

Replace a magic number with a symbolic constant Rename a method to something more logical, more up to date, or more in keeping with related methods

Field refactors

Pull up field (move to a class higher in a hierarchy) Push down field (move to a class lower in a hierarchy) Encapsulate field (make a field private, and wrap it up in a public function or property)

Class refactors

Extract a superclass Extract an interface

Larger scale refactors

In terms of our framework and toolset discussed during design, when a piece of functionality is extracted from an application, then made more general in the framework or toolset, then it is a refactor.

Other refactors

If during design, pieces are extracted from applications and merged into a shared class or common tool, then the design and code are both being refactored. Extracting common themes from requirements is refactoring requirements. Merging test cases is refactoring tests.

XP7. Pair Programming

Developers pair up and select a particular piece of a system they will tackle together. The pair should choose each other rather than being assigned.

Pair programming has been shown by many experiments to produce better software at similar or lower cost than with programmers working alone. There can be problems with a dominant programmer, but the technique works very well with a programming expert and a domain expert.

Of course, there may be occasion when you might suggest some pairs. For example: you have designed an FTP application. You have a user interface designer, an FTP specialist, and Hotin Ruuf, a new programmer. Having the UI man working with the FTP specialist will get the look and feel on the go pretty soon. Then the UI person can work with Hotin to get the underlying functionality of the user interface going. Finally, Hotin can work with the FTP specialist, being brought up to speed on FTP, the politics of the organisation, the best local sandwich shop etc, while the FTP specialist learns the cool new language C2 litespeed, at which Hotin is a whiz.

The idea of the pairs being self assigned means people select those they are comfortable working with, rather than being made to work with those they are not.

XP8. Collective Ownership

All of the code belongs to all of the programmers. This lets the team go at full speed, because when something needs changing, it can be changed by anyone without the delay of waiting for a response from its author.

Working this way requires a full buy in to the change mechanism, and historic recording of change. It does not mean you have a heap of shared code and

everyone hacks away together at it. A strong, well understood process is needed, and it must not be restrictive.

XP9. Continuous Integration

Teams integrate and build the software system often. Depending on the nature of the project, this could mean many times a day.

Integration problems can plague development, especially when it is least needed, i.e. Friday evening, or the end of the project.

Continuous integration keeps all the programmers on the same page, and enables rapid progress. Perhaps surprisingly, integrating more frequently tends to eliminate integration problems that plague teams who integrate less often.

XP10. 40-hour Week

Tired workers make more mistakes. XP teams do not work excessive overtime. They keep themselves fresh, healthy, and effective. Whether you, as a software architect, will have the luxury of 40 hour weeks depends very much on you and your environment. Even if you do work 40 hours a week at software architecture, you must still find the time to keep your technical knowledge up to date.

A 40 hour week requires some personal strength. Some developers seem to relish the family feeling of an all-nighter, or the collection in the morning of the early risers. We should accept that some people perform better in the morning, some late in the day. People working long hours under heavy pressure will end up taking back their hours from the business one way or another. They may leave, die, or grow disillusioned, spending all day surfing the net and avoiding the boss. They might continue to do sixty hours of good work each week for the rest of their working life, just as we might use delicately carved chocolate as a fireguard.

XP11. On-site Customer

The project team has a dedicated individual who is empowered to determine requirements, set priorities, and answer questions as they are discovered. This empowered user is a real, connected and present member of the team. The direct contact the developers have with empowered users means that they do not create their own conclusions about business process or best methods for presenting their ideas. There are many knock on effects, such as there is always a second person to bounce ideas off, so poor ideas are soon quashed. The paper trail, or email trail is also much reduced.

This does require significant business buy in.

XP12. Coding Standards

For a team to work effectively in pairs, and to share ownership of all the code, all the programmers need to write the code in the same way, with rules that make sure the code communicates clearly.

As all code is shared, reviews are continuous although ad-hoc. As architect, you must ensure the coding standards and code produced from them reflect your desires and contribute to the overall architecture.

Peer code review ensures that developers conform to common standards, as those not doing so are creating more work for others. Coding techniques, algorithms and the use of hardware and software technologies comes under intense scrutiny, particularly when those experienced in certain technologies can review where others are going.

If there are no coding standards, the development team must be made responsible for creating them. They may address the following:

• Development and test server environments • Live environment • Layout:

o common elements, such as header details, variable declarations o Comments o Indentation o Line spacing

• Filenames for code, documentation and help, and their folder structure • Variable naming • Data access • Source control, or configuration management • Code reviews • Error handling • Preferred methods • Style, visual presentation

Coding standards may also contain the software development process.

These twelve points are the foundation Extreme Programming. For more information, look at eXtreme Programming Explained[4].

Those having most success with XP tend to be small, tight teams who work closely together all the time. Even then, they pick and mix with the XP practices and build their own version of a software development process around it.

How you, as a software architect, use these practices is a matter for your own judgement. Even if you decide against them, you should define your point of view, and be able to articulate it.

Agility

The agile movement has grown out of XP, not necessarily as a replacement, but to support it and make it more complete from a process point of view.

The principles and values of the Agile Alliance[8] sum up their approach to delivering software excellently. I quote them here without embellishment.

Principles

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. Business people and developers must work together daily throughout the project. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. Working software is the primary measure of progress. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. Continuous attention to technical excellence and good design enhances agility. Simplicity--the art of maximizing the amount of work not done--is essential. The best architectures, requirements, and designs emerge from self-organizing teams. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Values

Individuals and interactions over processes and tools.

Working software over comprehensive documentation.

Customer collaboration over contract negotiation.

Responding to change over following a plan.

The arguments the agile alliance put forward are indisputable, especially their values. Yes, I use a process, but that process is tailored for each project. Many software houses work this way. If you can absorb any of the agile ideas into your process, you will certainly do better. Improving the quality and vision of software is paramount. If you need a process to do it, use one. If you can be agile, be it.

I shall repeat one of their principles, which I believe should be the principal goal of any software development process or agile team:

Maximising the amount of work not done is essential.

If you only learn one lesson about developing software from this book, let that be the one. It is certainly the most important.

References

1. The Tao of Jeet Kune Do. Bruce Lee. Ohara 2. Code Complete. Steve McConnell. Microsoft Press 3. www.dsdm.org 4. eXtreme Programming Explained. Kent Beck 5. Peopleware. Tom de Marco, T Lister. 6. Who Dares Sells. Patrick Ellis. Thorsons 7. Refactoring: Saving projects in trouble. Martin Fowler. Addison Wesley 8. www.agilealliance.com

Chapter 10 - Releasing software

Testing

Testers are usually involved at the end of failing projects.

Or, to put it another way, if you leave testing until the end of a project, it is too late.

In an architecturally led process, testing is part and parcel of development as the architectural integrity is dependent on things working well, and continuously. If you have stated a throughput of 10,000 transactions per second, you must identify the pieces of software and hardware which will have an effect on the throughput, and test that the requirement is met.

And long before the code is being assembled, tests must take place. For example:

• Is our architecture sufficiently defined for this project? • Do we have the capability to deliver this project? • Are the requirements mutually explicit and collectively exhaustive? • Can we build development and test environments for this project?

Testing must be enmeshed with all other activities of development. If you build and release every day, then you must test every day. A tester will typically run an automated regression test to see if everything working yesterday is still working today.

It is the quality of testing which most defines the quality of the released product. Read about IBM's black team in the Mythical Man Month[1].

As we are modelling, and building analysis and design models, then it is reasonable to assume that we can also have a test model. Usually, this model is

represented by a documented test plan and both manual or automated test scripts.

Figure 10.1 The test plan, the documented test model

The test plan must explain how each part of testing will be carried out. It may be a generic template, tweaked for each project, stating the following:

• The use cases will be tested against the requirements • Analysis and design models will be peer reviewed and tested for

adherence to the requirements • User interface designs will be mocked up and usability tests run • Each phase of implementation will be regression tested • All identified bugs will be handled through our bug tracking software

Lots of tests are carried out, either with or without a formal statement of them occurring.

• Requirements are tested for completeness and understanding • Analysis is tested for adherence to the requirements • Design is tested against the requirements and the architecture, and

against the analysis • Code is tested and regression tested for bugs • Compiled lumps of application are tested for integration with the other

compiled lumps, and also for integration with the entire system

• The release procedure is tested on a mirror of live to see if it will work or if it affects anything else

• The mirror of live is used to conduct stress, load and soak tests • Finally, the software process is tested to see if it was realistic for the

project

Requirements must be given a MECE test. This means your requirements must be mutually exclusive, i.e. no requirement is the same or overlaps with another - if they overlap, refactor them. They must also be collectively exhaustive, i.e. they are complete; no gaps can be found. Doing so will provide a clean foundation to build the system upon. Unclear or incomplete requirements can lead to confusion within development, and ultimately within the product.

Analysis is tested as far as it can be. If use case models are produced, then each requirement is attached to a use case. Any hanging use cases or requirements mean either the analysis or use case model is incomplete.

Following design, testing will verify that the design model is a true reflection of the analysis, and where differences occur, the deviation is acceptable. Architectural review will ensure the design conforms to the envisioned or actual architecture, and testing then verifies that the design model is likely to fulfil the requirements.

During coding, two types of test are carried out. The first is to review the code, to ensure it follows coding standards, and that algorithms and methods identified during design are carried through into the code and penned effectively. The second type of test is to extract deliverables for testing. Each deliverable is tested in a prescribed manner, for example, a style sheet is tested one way, while a compiled library file another.

The major role of testing during analysis and design is to extract test information from the models and documentation. The output from this activity is test plans, specifying which tests will be carried out when, and the test scripts or automations to test the objects being created. Test scripts are lists of inputs versus expected state changes or outputs. Automating test scripts can save testers a lot of time retesting after small changes have been made.

Any bugs or problems discovered during testing are worked into the test scripts, so regression testing does not become a chore. Regression testing is re-testing items for all bugs found so far as the item in question is debugged and re-developed.

Use case led testing

When development is led by first identifying the use cases of a system, and then mapping the paths through the use cases, the test scripts are derived from those use cases and pathways. If the use case structure is reflected in the test plan structure, then a casual observer will immediately see the test coverage from the user's point of view.

Design led testing

A design will identify, at many levels of granularity, the parts which are to be constructed and compiled to form the complete system. In a generic design, properties and code are compiled into applications.

Figure 10.2 A Software system is compiled from a granular level

As the granularity decreases from code to application, the focus of testing moves from detail to abstraction, from conformance to timings. Many arguments have been put forth for testing in this granular way to avoid problems later on. Others argue equally passionately that testing at this level can waste resources who would be better employed elsewhere as in many cases, average performance is often more than adequate and testing small granules is not effective resource usage. As always, a healthy balance between the two extremes must be identified, followed and periodically reviewed.

Testing can, and must, take place on each part of the software puzzle. The test team must see themselves as an integral part of the process of developing excellent software systems. In many cases, when the tests are peer reviews, the testers are the code writers themselves, and should consider themselves subcontracted and responsible to the test team while reviewing.

Software Test

Application UI, response, understandability, usability, security

Framework Components and interaction, efficiency of calls through layers

Toolkit Components and interaction

Component Interfaces, object creation, limits, speed of execution

Class Property limits, initialisation and destruction (time and ease)

Function Black box input output testing for efficiency and throughput

Property Test limits

Code Peer review coding style and algorithms

Each set of tests builds upon the results of the lower level test. It requires a defined and articulated process to be able to do testing in this manner.

Furthermore, if testing is not carried out bottom up, then bugs will possibly be fixed higher in the code chain than where they really occur. Immediately, spaghetti code begins to form; arteries that were running clear and smooth are blocked up, and eventually the code has a heart attack. Call in the code surgeons.

The V model

The V model is a waterfall model, with a mirror image created for testing.

Figure 10.3 The V Model for software testing

The V model can imply that testing follows on from development if the horizontal axis is perceived as time. What the V model demonstrates clearly, is how tests are built up in layers, and that the test stages have different objectives.

Types of test, and words used by testers

Testing jargon is every bit as bad as development jargon. You may hear about strange things called black box testing, white box testing, and testing in the small, usability, integration testing, component tests etc. Here’s what a few of them mean.

Occasionally, test are divided into functional and non-functional tests, targeting functional and non-functional requirements. Although tests in a test plan may be divided this way, the testing methods cannot be similarly divided except in a few cases, so are all listed together.

Alpha test

The product is nearing completion, and testers run all test scripts against the product, then feed it out to a small user group for simulated live testing.

Benchmarking test

To determine how a new version of software or hardware compares to previous versions, or how competing products measure up against each other.

Beta test

The product is released to a small group of users to identify and fix the last bugs

In an environment where the users are working with the developers, alpha and beta tests are far less formal, although still required as they are part of a formal hand over process from development to live.

Black and white box testing

The terms black box and white box testing come from electronics. You cannot see inside a black box, you can only see its external interfaces. You can see into a white box and examine the code.

Black box testing

A missing function is the most obvious black box test failure. It’s there in the design, but missing in the component. Interface errors, errors in data structures or external access to a database are typical black box problems, as are initialisation and termination errors.

Black box testing can cover many areas, and depends on the use the box is intended for. Boundary analysis and syntax testing are the more obvious, where function calls are tested against the design. State transitions can be mapped, as can cause and effect.

Others problems are not quite so obvious. Maybe the performance is down, or pushing the component too hard begins to introduce errors. This is when white box testing can help.

White box or glass box testing

When testing goes beyond the interface and into the code, it becomes white box testing. The differentiation may be simply that testers are only responsible for the black box test, and internal problems must be tested and resolved by the developer.

White box tests must be derived from a detailed design, and knowledge of the internal structure of the component.

A comprehensive white box test must

• Guarantee all independent paths within a module have been executed • Exercise all logical decisions in both true and false states • Execute all loops within their boundary conditions and check their

operational bounds • Build and destroy all internal data structures

It should test all statements, equations, branch decisions, data flow, and that the pathways identified in the use cases are reflected in the code.

Compatibility test

Tests how a software component performs under different operating systems, or on different hardware.

Component, unit or module test

To test a component, it needs a test cradle. This cradle can be a test script, or a dummy program built specifically to test a component's exposed interfaces.

Component integration, link or testing in the small

Tests if a compiled component works as expected with other components

End to end test

A system is tested from the inception to the destruction of its objects. For example, a job for an external client goes through a sequence of tasks: idea – proposal – sale – live – closed. The tester goes through the job, creating and completing each stage of the job from end to end.

Another meaning is used in integration testing where a call goes through an integration layer from one system to another, and information is returned to the caller.

Install/uninstall test or rollout/rollback test

In distributed systems, across a diverse set of operating systems and configurations, a rollout test ensures that installation will work on all configurations of each machine used by an enterprise.

Even for desktop applications, they are painstaking tests to perform, as machines vary in age, operating system version, patches, and installed software. This test alone highlights the need for corporate standards in hardware and software as rollout problems can sometimes be difficult to overcome if different versions of dependent software behave differently.

Large scale integration, or enterprise testing

Tests how a distributed system will respond to its users. This is load testing in its largest scale where multiple concurrent test processes run together. It is very difficult to do manually, and test applications or automations will be written to perform and measure the test.

Mutation testing

These test are best automated. A set of data is fired into a software product and the output is measured. The data set is then mutated by changing a few fields, and the test is run again.

Regression testing

Once a test has been carried out on a particular component in a particular way, it should be retested to confirm that behaviour has not changed unexpectedly during subsequent development or bug fixes. Automating these tests can take some effort up front, but pay huge dividends during retest.

Each time a new internal function or component is added, or an interface changes, a regression test should be carried out to see if anything unforeseen has happened.

Sanity testing

A quick once over to see if a product is ready for testing.

Stress, load, soak, volume and performance testing

These are a set of related tests performed prior to deployment, on a mirror of the live system. Each of these tests is performed using automated testing tools, which record the response time and load (typically number of users) placed on a system. Stress, load and soak tests typically use the same automation scripts, with different levels of users and interactions.

Load or performance test

A system or component is tested within its design limits to prove it it capable of supporting the expected load. Stress and Soak tests are intensive and prolonged load tests.

Stress test

A system is considered to be stressed when it is operating beyond its design limits. A stress test will gradually increase the number of users and/or level of transactions until the system fails. A failure may be a catastrophic crash, or merely a response time deemed to long for effective use of the system. The information gathered from a stress test can help the designers to pinpoint where and how failures occur, identifying areas for rework should throughput be inadequate.

Soak test

A soak test is usually run overnight, or longer if time and systems are available. The load exceeds what is expected on a busy day, so problems with high, continuous data and communication rates can be identified.

Volume test

Databases, numeric record counts, backups, archive files, audit and error logs typically get bigger over time and overload file space or data types allocated to them. Volume tests run a simulated high load and monitor data values and file space to predict how long a system will operate effectively before counter resets and file archives are required.

System test, or testing in the large

Tests how a complete application performs within an existing system

The terms testing in the large and testing in the small come from the Object Oriented Software Process (OOSP).

Usability tests

In a usability test, a proposed user is sat down in front of the application and told to ‘do it’. Each fumble or bottleneck, each contorted face, withered curse and and occasional scream is recorded by a silent and unprompting observer. The interface or flow is reworked based on the results of the test.

Usability is a huge subject, and one which is dealt with Don’t Make Me Think[2], and User Interface Design[3]. Both are colourful, thin books, and something of a relief in these days of thousand page tomes.

The reality of usability is that the tests are carried out in a false environment. If you are the viewer you must do everything you can to help the viewer and gauge their feelings. You must not say things like click that button there to start the search; you should ask questions like what are you thinking? What is your first impression of this screen? Can you talk me through your perception of this view?

User acceptance testing, also known as factory or contract testing

The users verify that the system works by performing predefined tests, usually recorded as scripts for manual interaction.

User interface testing

The slant on UI testing is different to usability testing. Usability testing focuses on getting instant understanding from the user when seeing your screen for the first time. User interface testing focuses on interaction, presentation and conformance to operating system expectations.

• Does the window open properly? • Can it be moved, resized, scrolled and closed as expected, and with ease? • Do all buttons, links and controllers work as expected and can they be

operated by both mouse and keyboard? • Does the window regenerate itself after an overlapping window is

removed? • Are all menus, scroll bars, dialogs, icons etc visible and working properly?

• Is the window properly displayed when active? • Does random clicking and key pressing generate unexpected errors? • Are all fields properly operational, e.g. can I put a letter in a box intended

for a number? • Are there any spelling mistakes? • Do all menu commands, buttons and links have shortcut key strokes? • Is the menu text clear? Does each command fully represent its

functionality? • Does the cursor change in response to scroll bars, resize areas, and busy

states? • Are all inputs by the user echoed to the screen efficiently and effectively? • Is invalid input properly displayed? • Is anything visible which may be offensive to any region, group, or

individual?

Architectural testing

It is quite difficult to test an architecture. The two methods of testing are peer review, and time. Peer review will help point out some flaws in an architecture, mostly based on experience. Only time will test it properly.

Is testing important?

News reports which surfaced in 1999 claimed software bugs in a Soviet early warning system nearly brought on nuclear war in 1983. The software was supposed to filter out false missile detections caused by Soviet satellites picking up reflections of sunlight from the tops of clouds. It failed to do so. Disaster was averted when a Soviet commander, based on what he described as a ‘funny feeling in my gut’, decided the apparent missile strike was a false alarm.

Test management

The management of testing is as important as the actual testing itself. It is not enough to find bugs. Getting them fixed, and monitoring that they do not return in subsequent development, is key.

A bug tracking mechanism is essential.

The test plan and test scripts

Test plan

The Test plan is the highest level test document. If there is a programme plan, there will be a test plan for that programme. If the programme has a set of subprojects, there will be a test plan for each project within the programme, all derived from the programme test plan.

The test plan is a document outlining the tests required, and the environment required to carry out those tests. If there are legal stipulations or contract

agreements which need verification, they must be listed or referenced in the test plan.

It will also state how problems are to be detailed and resolved.

Test script or test case

For each test plan, a number of test scripts, also known as test cases, are produced. Each test script focuses on one particular test, stating its preconditions, the actions required to complete the test, and the expected and actual responses of the system.

Figure 10.4 Test script example

The art of testing

Testing has two dimensions, one the planning and execution of the test, and the other the granularity of the test

Figure 10.5 The two dimensions of testing

Testing is also a detailed, designed process. Figure 10.6 shows the structure of test being derived from the IT strategy.

Figure 10.6 The structure of testing

The act of test is also an integral part of the software development process. In a way, reviews themselves are tests, designed to test the whole software development process, and the various facets of it. Reviews improve the delivery of software. Tests improve the quality of a product.

Formal testing is a bit of a dry subject, so hearken Pooh when organising to have the results of the test heard.

One day Rabbit and Piglet were sitting outside Pooh’s front door listening to Rabbit, and Pooh was sitting with them. It was a drowsy summer afternoon, so Pooh got into a comfortable position for not listening to Rabbit, and from time to time he opened his eyes to say “Ah!” and then closed them again to say “True,” and from time to time Rabbit said, “You see what I mean, Piglet,” very earnestly, and Piglet nodded earnestly to show that he did.[4]

I have been Pooh once or twice myself, I am sorry to say.

Release or deployment

Imagine a world where you could write the tiniest piece of code, and have testers and users test it on a continually maintained mirror of the live system. When the tests are complete, it is confirmed for release to live, and the person responsible for the integrity of the live system rolls it out worldwide.

This is the nirvana of the configuration management world.

With a configuration management system, release is not be focused on the action of installing something, but on managing the environment in which it is coded, tested and released. All releases can be traced back through the test and development environments, and from there to their source. Furthermore, all source is traceable through its versions, and to the tasks defined in project management plans which requested the change or creation in the first place.

However, a significant investment is required to put configuration management in place, and a structured environment required in which to enforce its processes. The reality is that most development environments use source code control software, which allow a certain agility, over the more demanding configuration management practices.

The release of any software product, whether it resides within configuration management, or is being deployed manually, must be accompanied by notification of the release to all stakeholders. A typical release will go from development to test, then from test to members of the user group for initial user acceptance or alpha testing. Successful alpha testing will be followed by a release to a limited number of users for beta testing, possibly in the live environment. For a large project, there may be a number of phases of beta testing to ever greater numbers of people. Finally, the product will be released to the full user population.

Release can be a delicate time. Your hard work is being spread among the users, or sold to joe public. If you have been foolish or unfortunate enough to exclude them from development, this will be the point where they say: "that's not what we wanted", or even have the temerity to not buy it. Hopefully, you will have been persuaded away from this path by now.

Even so, release can still be a trying time, particularly where large rollouts of sets of applications are necessary. Such releases can be the culmination of highly fraught work when creating new business practices, or the merging together of two companies where different working practices must be supported by your software.

Rollouts must be planned carefully. If there are many people involved then there must be a timetable of events, with someone responsible for managing the rollout. That management is to see that everything happens when it is meant to happen. If something goes wrong during the rollout, then the rest of the tasks may have to be suspended until the problem is resolved, and the completed tasks rolled back. Running a release rehearsal is a good idea, particularly on complex platforms, even if it is only a walkthrough rather than a deployment to a mirror of the live environment.

User education

If you follow the architecturally led process laid out so far, user education will be a lot easier. If members of the design team are part of the user group, and can see their ideas reflected in the software product, they will be your user education evangelists.

Training in the use of your product, or creating user manuals and training courses, must be addressed long before release. Documentation and training material can begin to be created from a sufficiently detailed design, and training itself can begin once deliverables have first been created and tested, although it may be prudent to defer training for later in the test phase.

You may also want to train the second line support, who may in turn train first line support. You may create a help file or a set of web pages for general consumption.

Reviews

Code reviews

Code reviews have already been mentioned at the lowest level of the testing hierarchy. Such reviews are best carried out by line managers and peers. The reasons for code review are:

• Test that the coding standards are being adhered to • Test coding is carried out in a structured way, following the abilities of the

languages, and also conforming to coding best practice, even if it is not covered specifically in the coding standards

• Ensure that code has the same look and feel throughout the development teams. Picking up someone else’s work is far easier if it looks familiar

• Ensure the best, and most efficient, algorithms and methods are used • Ensure the code is backed up, version controlled, and that changes made

are in accordance with the development vision and the aims of the project

Design reviews or walkthroughs

A walkthrough is when a user group is taken, step by step, through a specific piece of functionality, or element of design. It can be a verbal or graphical presentation or a mock up, intended to explain one particular facet or flow, and gain some input from the user group.

User reviews

User reviews must run throughout the life of the project to ensure the work being done accurately reflects the users’ needs. They can be as informal as emails, or as formal as structured meetings where users are presented with the progress so far and where accepted plans might have to be deviated from. Their purpose it not solely one of information dissemination, but must allow the users to steer changes to benefit them most.

Post release wash up

Each release should be accompanied by a wash up meeting, the purpose of which is twofold. First, it ensures the objectives of the release have been met, and that all tasks have been completed, and second it provides a forum for the promotion of practices to help improve the process next time around. Wash up should take place within a week of release.

Post release review

A released product is not necessarily a finished product. When the product has been in the field for some time, a review of its operation, in terms of reliability,

changing requirements and as a reflection for the project team. The big questions are:

1. Is it working? 2. Did we get it right? 3. Has the underlying need for this product changed?

If an architecturally led, user inclusive, process has been followed, then the answer to question 2 will be a resounding yes. Your success as an architect can be measured upon the response.

The timing of post release reviews is dependent upon the project, but three months and one year after release are probably adequate.

The linear software development process

In all, we have followed a software development process based on the rather linear waterfall. It has been geared heavily toward business systems, but this is where politics is most concentrated and in need of a process. Those working in engineering systems and consider themselves software engineers seem to manage more easily when software teams are devoid of business people. I could have chosen a more technical slant, but I have more experience of business systems. Other software architecture books are more technically biased.

Also, I have spent half a book on the software development process. Some will argue that architecture has nothing to do with the software development process. It has to do with creating software architectures around which systems are built, not the daily grind of creating them.

Well, yes and no. If you do not have a software development process which will allow you to define architectures, then you will be superfluous. You will spend all day creating architectures for nothing. No-one will use them, and no-one will be interested in your output. You will be one of whom questions are asked: What exactly is it that you do? when the hand of redundancy begins its sweep of extinction. You are nothing more than a seller of sand by the seashore. If you have such time to waste, spend it on something better. Your own sand is running out. You are unlikely to gain the rewards of paradise from the endless cycle of sitting at work and achieving nothing.

If there is no process, then you as software architect may as well define it, sell it and implement it. If you are fond of asking ‘Why me?’ slip a not in the middle. Without a process, you cannot architect.

Where can I find out more?

Testing Computer Software[5] gives a good introduction of the testing process, and explains in detail how to go about it.

References

1. The Mythical Man-month. Fred Brooks. Addison Wesley 2. Don't make me think. Steve Krug. New Riders 3. User Interface Design for Programmers. Joel Spolsky. Apress 4. Tigger is unbounced, from House at Pooh Corner. A A Milne. Hunnypot

press 5. Testing Computer Software. Cem Kaner, Hung Quoc Nguyen, Jack Falk.

Wiley

In this section, we look at us, how we approach the software problem, and how we interact with our business and development peers.

Chapter 11 - Iterative and Incremental

Introduction

There are many approaches to developing software within a defined process. Some are skeletal methods around which you build your bright ideas. Others are quite prescriptive. You can build your software development methods around these approaches, or gradually fit their approach to methods already in place. Either way, they will help you define a repeatable process. Repeatability means consistency in approach and delivery. Delivering on time and to expectations will gain you respect, and respect will give you the latitude to create architecture.

I mentioned earlier in this book that the software development process presented so far is based upon the Unified Process, also known as the Rational Unified Process, or RUP. Many software houses use this or similar methods, and tailor them to their own organisational structure and beliefs.

The history and detail of the Unified Process is explained in The Unified Software Development Process[1], and also The Rational Unified Process[2]. It is a collation of best practice software development methodologies, fully integrated with the modelling notation of the Unified Modelling Language.

To some extent, the modelling language is separate from the process of software development, but in other ways they are intrinsically tied. For example, the models created with UML are supported by the process, and the process itself is modelled using UML. This is not to say that other process methodologies do not support UML, for they do.

UML is a modelling language software architects can use to model the architecture and design of software systems. The Unified Process is an architecturally led method to ensure your wonderful architectures get delivered.

It is quite possible for them to be used separately, but the likelihood of success with either is diminished without the other.

The waterfall method

We should revisit the waterfall in all its glory. It is a superb metaphor for software development, and we must understand it better.

The traditional software development waterfall is an anachronism. It arrived before its time in a simplified and inaccurate rendering. It is this trifling, downward stepping waterfall we must eradicate from our memories, and instead embrace the waterfall of the future.

A real waterfall is an exciting, churning mass of water. It generates spray and static electricity. It mixes air into the water so that it might better support the life that exists downstream. The true waterfall: the churning, variable, flow and contraflow is the perfect model for software development.

Iterative and incremental

The Unified Process claims to be iterative and incremental. Other words in other processes imply similar methods. In the spiral model, we home in on the software output through repeating requirements, analysis, development and testing phases.

Figure 11.1 The spiral model

The basis of the spiral and the iterative, incremental approach, mean that a bit of requirements gathering is followed by a bit of analysis, design, coding, test and release. This small iteration is used as the foundation for the second iteration of requirements, analysis, design etc.

Hitting goal sooner

The intent of iterative development is to finish a project quicker.

In a non-iterative, or linear approach, the requirements are gathered, the analysis and design done and the coders get to work. When they finish, the testers give it a good thrash and hurl it out into the dark world of live software. Usually when the testers finish their first test, a huge amount of rework is discovered. Some requirements are missing entirely, some were wrongly

interpreted, and the whole thing just doesn't work; it crashes, hangs, behaves peculiarly during certain keystroke sequences, and generally behaves like an uncontrollable schoolboy.

What leaves development is often thought to be a finished product. What leaves test is sometimes a disaster of rework. Suddenly a project goes from an estimated 100% complete to only 50% complete; timeframes stretch. The developers get back to work, and release their trials to test, who return it again, this time 75% complete. Gradually through development/test iterations, the product homes in on completeness and is released.

Figure 11.2 Development/test iterations and project completeness

To improve this situation, we design the iterations and the increments into the process. An iteration is a cycle through the steps of requirements, analysis, design, code and test, and an increment is the piece of the product being focused on. That is, a part of the project is selected, and worked through the development cycle. Following test, another bit of the project is added in, while bugs are fixed and deviations resolved in the part already developed. On completion of the first iteration, it is easier to correct small mistakes which if left longer would grow into huge mistakes. It is no different than weeding the garden. If you do it often, you have a beautiful garden. The plants are healthy and the flower and vegetable beds free of weeds. However, leave those weeds for a few weeks, and they are as big as your plants. Some weeds even strangle the plants and take over their area. Leave it too long, and weeds are all you have.

Correcting problems before they grow, means there are no huge tumbles in perceived completeness following testing, and less time is spent developing the wrong thing.

Figure 11.3 Iterative incremental development finishes the job sooner

It's as simple as that. Running projects under an architectural umbrella can make move that finishing line left again by reusing components and delivering functionality within the architecture. Of course, there is still the upfront work to create the architecture.

Figure 11.4 Architecturally led iterative development finishes even sooner

The problem with iterative, incremental development

The drawback of a small amount of requirements is that when you get your second set of requirements, you find that you did something wrong, or asked the wrong question when getting the first set of requirements. By the final incremental iteration, your product can end up such a mess that no-one understands how it came to be so.

Figure 11.5 The iterative, incremental waterfall

The best way to approach software creation is to gather requirements as fully as possible, and then select a subset for the first iteration of analysis, design, development and test. A birds-eye view of the entire product must be kept in mind during the design sessions to maintain a product view rather than an increment view. Without the birds-eye view, the final product may look disconnected, or functionally divided. While it is true that iterations of development are most easily defined upon a completed design, a carefully considered set of iterations is a better approach for two reasons. First, the feedback loop from the output of one iteration to the beginning of the next will provide valuable insights, and second, the completed design may not be as complete as believed, and later reflection may uncover time wasted due to an earlier misunderstanding.

The Waterfall model revisited

The new improved waterfall model goes like this:

Figure 11.6 The born again Waterfall

A project begins as a twinkle in the eye of someone who can afford to sponsor it. It is then laid out in conversation as a set of documented requirements. During this requirements gathering process, analysis of the requirements begins. In analysis, the requirements are tested for logic, realism and completeness. The final stage of requirements is led by analysis as the requirements become ever more likely to form the foundation of a software project. Some time during analysis, the design phase begins. Along with ever reducing analysis of the requirements, the design now takes the lead in realising and improving the requirements.

By the time analysis of the requirements is all but over and design of the software is in its later stages, development has begun, either prototyping, or laying down the foundations of the software system.

Design continues on through the early iterations of coding as prototyping brings life to the project, and the sponsor (or the sponsor’s rapid response team) can see, clarify, and improve the design.

Code a bit, test a bit, code a bit, test a bit then becomes the norm, perhaps for three or four iterations before release.

Of course, all through this slide down the waterfall, we have turbulence and frictions to be resolved. We have spray leaping back, carried by the wind into the face of requirements, even after release.

This then becomes the larger iteration on top of the code/test iterations. We do try to keep spray to a minimum, yet accept it exists.

We release. We take the feelings of those the product is released to and feed their emotion, knowledge and logic into the next phase of release, so

requirements to release is the larger iteration moulded by spray from the last large iteration.

Now we see the Waterfall in its true glory. From a distance, its journey is a simple top to bottom flow, but closer in we see many other flows, and points of great turbulence within the main flow. Occasionally, there are flows against the flow, pools of eddies, waterspouts and huge churning masses which indicate energy being expended yet not contributing to the flow. We shall come in a moment to the waterfall's width, and what that might represent, but for a moment, we shall take a look at the terms used within RUP.

Unified Process Terms

Figure 11.7 Phases and activities in the Unified Process lifecycle

The unified process uses the terms inception, elaboration, construction and transition as the phases of the lifecycle. These phases contain the same activities in varying degrees.

The first phase is inception, which covers the early work of domain modelling or domain analysis, gathering requirements, analysing them and beginning the design. Elaboration is the improvement of requirements through analysis, and into design. Construction is finalising the design, writing code and setting up and running tests. Transition is the movement of the software product from development, through test and on to the live system.

Also included in this representation of the Unified Process are the supporting workflows of configuration management, project management and setting up the development, test and release environments.

These are the words, diagrams and ideas used when discussing an architecturally led, iterative and incremental software process. Along the bottom of figure 11.7, the iterations lead step by step across the phases.

Looking down on the waterfall

As shown in figure 11.7, there are other streams of work being carried out in parallel to the main effort of creating a product. There are possibly also other products being developed in other projects by other teams, all impinging upon the work within the project we are focussed upon.

These other projects, and other workflows, are represented by the width of the waterfall. The width is the entire set of concurrent projects being run by an enterprise. Work occurring in one project will have implications on another, indicated by water heaving sideways around a boulder. Occasionally the water is split into two channels, rejoining on the far side of another boulder. There at the joining point is more turbulence and frothing as the parted waters, or parted projects, rejoin and energy is spent as they realign themselves.

Our waterfall can only support a certain flow of water. Our software development departments can only support a certain number of projects.

Figure 11.8 The waterfall from above

What could the boulders in the water represent in your work? The rightmost boulder looks like a difficult technological problem and the development team are held up, using a lot of energy to break past it. Projects A and B share a piece of work, i.e. the oval boulder at the bottom. There will be turbulence at the downstream end where projects A and B merge again. And what of the leftmost boulder: People off sick? Old team members leaving, or new ones joining, or developers being reassigned from one project to the other? Both projects are affected bringing replacements up to speed.

Each individual project has a beginning and an end, but the torrent continues. Another related project or later iteration of an earlier release may follow on, and we begin to talk about streams of work. Projects are constrained in a stream, and together the streams of sequential projects make up a programme of work.

Figure 11.9 Streams of work

The Waterfall is quite a superb model of software development. It indicates the overall process starts and ends, it represents more than one project and the associated workflows. It shows energy being lost, diverted, there are pools of swirling peace, areas of torment, but mostly, the water gets to the bottom one way or another unless splashed onto the rocks of reality, or evaporated into clouds of dreams. Everything that happens in software creation is reflected in the waterfall.

An iterative project plan

Figure 11.10 An idealised iterative project Gantt chart

Figure 11.10 shows an iterative project plan. A team is assigned, requirements gathering begins, shortly followed by analysis. All of the activities overlap, as indicated in figure 11.7, so that implementation is not complete until testing is also complete. Implementation 1.2 also overlaps 1.1, requiring careful configuration management or source code control.

However, this level of detail is not necessary, nor even helpful. In many projects there are so many tasks that defining them down to this level of detail may cloud the picture rather than clarify it. It depends upon the project team, how comfortable they are working iteratively within the linear norm of a project plan, and just how much detailed control and understanding the project manager wishes to have over the team's activities. A non-iterative uncluttered project plan is probably the best structure to create within. This leaves the project manager clear of detailed development, and the development team free from measuring and marking every increment. Progress in a project plan is based on milestones; in iterative development, milestones are gradual changes rather than a concrete post to tick off when passing. These two views can only be reconciled by abstracting the detail for the plan, and using a percentage complete marker rather than a binary done or not done.

How do I divide up the time?

Here is a rough picture of how long might be spent cumulatively on each activity.

Figure 11.11 Dividing up project time

The easiest answer is it all depends. If you know the business, you will spend little or no time analysing it. If you are working on a real time, multi-threaded, military operation support system with built in redundancy across three geographical locations in case of attack, you will spend relatively more time on design and testing. The key is not how much time you spend, but how you avoid spending more time later in support and fixes, and also later within the cycles of development. There are no hard and fast rules, but more complex systems need relatively more analysis and design.

The activities of a project

Projects have large phases. Phases are broken down into iterations. Iterations have development, test and review activities. Somehow all of these linear steps and iterative, incremental ideas must end up in a project plan.

What is important, is to declare key points in the process as if it was a smooth linear process with concrete deliverables, while knowing that really it is not. To manage the project, you might come up with a set of milestones for the project plan like this:

Figure 11.12 The key steps of a typical software project

As a software architect, you may form your own plan for the architecture you are creating or supporting and extending. Your own activities on a single project may be seen only during design and review. Nevertheless, you will have your own vision to head for and you must steer projects in long meetings and painful conversations, driving, explaining and reviewing, seeking to understand what is required, and to be understood when you have requirements of your own. You will have your own increments and deliverables, based on programme planning rather than project planning. Only your level in the business hierarchy defines how you are involved within the programme and its projects.

Other key ideas in the Unified Process

Rational[3] is a software house providing software to help develop your software. They have a product called RUP (The Rational Unified Process) which is a management tool for the software delivery process. They also offer requirements

gathering, automated testing, and one of the best known UML modelling tools in Rational Rose.

We are not discussing the software tool here, but the actual process as applied to architecturally led software development. For the rest of this discussion, RUP and the Unified Process refer to the software process rather than the software tool.

RUP is claimed to:

• Capture and present best practice as an ongoing improvement model • Reduce risk • Increase predictability • Promote common vision and culture

It is based on the following premise:

Planning precedes doing

But isn’t that obvious?

RUP 1. The guiding statements

• All processes are iterative and incremental - there are no big bang deliveries

• Analysis, development and testing are part of an ongoing, cyclic process • It is architecture centric - all software development from request to

release is a continuous process using common, integrated tools and methods

• It is use case driven - development is always focused on the users and uses for the product

RUP 2. The influences

Figure 11.13 RUP's 4 influences

RUP has 4 influences, which are: requirements, tools, technology and skills. Each of these influences addresses a different area of need or ability, and attempts to separate out the areas of interest in any development process.

RUP 3. The roles

RUP identifies the roles in the software development process as breadth and depth roles. They are:

Discipline Breadth role Depth role

Business Modeling Business Process Analyst Discovers all business use cases.

Business Designer Details a single set of business use cases.

Requirements Systems Analyst Discovers all requirement use cases.

Requirements Specifier Details a single set of requirement use cases.

Analysis and Design

Software Architect Decides on technologies for the whole solution.

Designer Details the analysis and design for a single set of use cases.

Implementation Integrator Owns the build plan that shows what classes will integrate with one another.

Implementer Codes a single set of classes or a single set of class operations.

Test Test Manager Ensures that testing is complete and conducted for the right motivators. Test Analyst Selects what to test based on the motivators. Test Designer Decides what tests should be automated vs. manual and creates automations.

Test Designer Implements automated portions of the test design for the iteration. Tester Runs a specific test.

Deployment Deployment Manager Oversees deployment for all deployment units.

Tech Writer, Course Developer, Graphic Artist Create detailed materials to ensure a successful deployment.

Project Management

Project Manager Creates the business case and a coarse-grained plan; makes go / no go decisions.

Project Manager Plans, tracks, and manages risk for a single iteration. (Note that this discipline has only one role. Assigning the depth view to a project coordinator can provide relief for

overburdened project managers.)

Environment Process Engineer Owns the process for the project.

Tool Specialist Creates guidelines for using a specific tool.

Configuration and Change Management

Configuration Manager Sets up the CM environment, policies, and plan. Change Control Manager Establishes a change control process.

Configuration Manager Creates a deployment unit, reports on configuration status, performs audits, and so forth. Change Control Manager Reviews and manages change requests. (Again, note that breadth and depth roles are assigned to the same people in this discipline; assistant or associate managers in the depth roles would be helpful.)

RUP roundup

RUP has its evangelists and critics. Most of the criticism is aimed squarely at its project focus and its view of development as the whole world.

Scott Ambler[4] puts some of this criticism into a white paper. He suggests RUP has another phase following inception, elaboration, construction and transition, which is production. The activities of production are in operating and supporting the software after it has been released.

The process workflows are joined by operations and support. The supporting workflows are joined by Infrastructure management.

There are many other software processes, such as the Open Process, the Object Oriented Software Process

Other software processes

There are many software processes. They are all presented differently, but essentially similar to RUP in the way they are used. Here are a few of the alternative processes in Scott Ambler's paper[4], selected for the ease with which they are pictorially displayed and how they have contributed to the software lexicon.

The Open Process

This process has been developed by the Open Consortium, as a comprehensive software process which enhances the use of object oriented methods.

Although the OPEN process is displayed as interconnected boxes, the coding part of the process is still incremental, as shown by the multiple boxes and the n iterations between development and user review.

Unbounded activities are displayed as rounded rectangles, and tightly bound activities as normal rectangles.

Figure 11.14 The OPEN Process

The left hand side represents the activities for a single project and the right hand side represents cross project activities.

Beyond the single project, which is the sole focus of the Unified Process, are cross project, or programme issues. I have used the term programme management previously. Programme management is a definitive term in OPEN.

Other terms used are OOA, OOD and OOP, which are object oriented analysis, design and programming. V&V means validation and verification, or testing.

The OPEN process from the Open Consortium should not be confused with The Open Group's Architectural Framework[5].

The Object Oriented Software Process – OOSP

OOSP is a similar, yet a semantically different process yet again. What OOSP has in its favour is that it recognises the project management, quality and personal improvement that are lacking in the other processes.

Figure 11.15 The Object Oriented Software Process

None of these processes define how a software architect fits into them other than rather loosely. So which one should you use? They all have their favoured terms, their not quite linear approach to process, and their multitude of boxes, lines and arrows.

The software architect in the software processes

The problem is that software architecture is only part process. The rest is made up of things that do not fit comfortably in a process model. The big arrow along the bottom of the OOSP diagram looks like a rather faint attempt at spreading lots of tasks across the whole project, but as you can see, the others miss them out entirely, or bundle them into a single task, project manage.

As a software process, OOSP is more complete, probably because it is based on the Capability Maturity Model (more later) of the Software Engineering Institute.

All are rather lacking in direct help for a software development house who wants an off the shelf process covering everything from hiring good developers to getting the most out of them. Most software organisations hire on personality and superficial testing. We have no software qualifications which are respected as much as other areas such as accountancy. People who work in accounting study for one exam after another. Studying lays a foundation upon which excellence is built. In software we are lucky if we can keep up with the changing technologies.

And how do you get the best out of people? You want the best interaction, communication, standard of code, use of algorithms, speed of development, effort against unconquerable odds etc. Such things can only be measured within the review points of a software process. Or does slamming a software process down on developer's heads destroy their creativity, their skills and their will to develop outstanding software?

As software architect, you need to exist somewhere in a process. You need to be around during the initial ideas stage. It may be you who is generating the ideas. You also need to be around at the macro design stage, when architecture is defined. Somehow the information you create needs transferred into developers' minds, and you need to ensure it is happening as you planned by having a look along the way.

Figure 11.16 A linear process for a software architect

That’s it. This figure is your path to becoming a software architect. Your reality may be slightly different, but the essence will remain. Some points may come easy, others more difficult. I should imagine the step Identify yourself as a software architect might not be as easy as it looks. Patience alone will help you conquer that one, providing that you have laid the foundations with previous steps.

So where does that leave you with a software process. You have your architectural one, and we have three other examples in the Unified Process, OOSP and OPEN.

That answer is easy. You need to create a process which will fit your organisation, and you need to make it happen. You need to identify your role as software architect and make that happen too. Furthermore, you must tailor your process to each programme or project you undertake, for they will not all be the same.

Tailoring your process for each piece of work will allow you to improve your process incrementally. Stating the process at the beginning of each piece of work will give you both the freedom and control you need to be a software architect.

The Capability Maturity Model (CMM)

CMM measures the quality and depth of your software process. Within it, are the initiatives you can take to improve your delivery of software.

Most people in business are familiar with the quality initiative ISO 9001. Some software houses have used ISO 9001 to structure their software delivery. It can help you structure the way you approach development, and ensure it is a repeatable process, but ISO 9001 does not have any advice to help with the specifics of software development. For that, you need to look to CMM from the Software Engineering Institute (SEI).

The SEI has developed a quality based system for software development, named the Capability Maturity Model (CMM). They have also developed a Team Software Process (TSP) and Personal Software Process (PSP) to augment CMM.

The Capability Maturity Model has been taken up by some industry heavyweights such as NASA and Lockheed Martin. It is very popular in the huge software industry of India, where companies like Infosys have reached the highest maturity, level 5, as have NASA and Lockheed Martin. This is a considerable achievement as the SEI recommends the improvement to a continuously optimising process will take ten years.

CMM is a five step process, in which organisations can climb through an improving structure for managing their software development process.

Figure 11.17 The five levels of the Capability Maturity Model

The CMM level definitions

Level 1. Initial

At level 1, the software process is characterised as ad hoc, or even chaotic. Few processes are defined, and success depends on individual effort and heroics.

Level 2. Repeatable

Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications.

Level 3. Defined

The software process for both management and engineering activities is documented, standardised, and integrated into a standard software process for the organisation. All projects use an approved, tailored version of the organisation's standard software process for developing and maintaining software.

Level 4. Managed

Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled.

Level 5. Optimising

Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies.

Each of these levels is further defined by a set of measurable organisational processes:

The CMM processes

Level 1

No defined process

Level 2

Requirements Management

Software Project Planning

Software Project Tracking and Oversight

Software Subcontract Management

Software Quality Assurance

Software Configuration Management

Level 3

Organisation Process Focus

Organisation Process Definition

Training Program

Integrated Software Management

Software Product Engineering

Intergroup Coordination

Peer Reviews

Level 4

Quantitative Process Management Software Quality Management

Level 5

Defect Prevention

Technology Change Management

Process Change Management

Achieving Level 1

Your organisation is already at a minimum of level 1. If you have no process at all, or a few defined processes such as coding standards and best practice guides, you are likely to be level 1.

Achieving Level 2

To acheive level 2, which is likely to be your organisation’s first target level, you will have to have the following processes in place.

Requirements Management Establish a common understanding between the customer and the software project team of the customer's requirements that will be met by the software project

Software Project Planning Establish reasonable plans for carrying out the project.

Software Project Tracking and Oversight Establish adequate visibility into actual progress so that management can take effective actions when the project's performance deviates significantly from the project plans

Software Subcontract Management Select qualified software subcontractors and manage them effectively

Quality Assurance Provide management with appropriate visibility into the process being used by the project team and of the product(s) being built

Software Configuration Management Establish and maintain the integrity of the products of the project throughout its lifecycle.

Achieving Level 3

Level 3 is built upon the sound implementation of level 2.

Organisation Process Focus Establish the oranisational responsibility for software process activities that improve the organisation's overall software capability

Organisation Process Definition Develop and maintain a usable set of software process assets that improve process performance across projects and provide a basis for cumulative, long term benefits for the organisation

Training Program Develop the skills and knowledge of individuals so they can perform their roles effectively and efficiently

Integrated Software Management Integrate the software engineering and management activities into a coherent, defined software process that is tailored from the organisation's standard software process and related process assets

Software Product Engineering Consistently perform a well defined engineering process that integrates all the software engineering activities to produce correct, consistent software products efficiently and effectively

Intergroup Coordination Establish a means for the software engineering group to participate actively with the other engineering groups so the project is better able to satisfy the customer's needs.

Peer Reviews Remove defects from the software work products early and efficiently. An important effect is to develop a better understanding of the software work products and of the defects that can be prevented.

Achieving Level 4

Level 4 is built upon the sound implementation of level 3.

Quantative Process Management Control the process performance of the software project quantatively

Software Quality Managment Develop a quantitative understanding of the quality of the project's software products and achieve specific quality goals

Achieving Level 5

Again, Level 5 is built upon a sound implementation of level 4.

Defect Prevention Identify the causes of defects and prevent them from reoccurring

Technology Change Management Identify beneficial new technologies (tools, methods and processes) and transfer them into the organisation in an orderly manner

Process Change Management Continually improve the software processes used in the organisation with the intent of improving software quality, increasing productivity and decreasing the cycle time for product development

Read more about CMM in The Capability Maturity Model[6] or visit the Softw

Chapter 12 - Managing the environment

Why should a Software Architect care about management?

You can invent all of the magnificence your intellect, your experience and your insight might conjure together. Invention, and this is what most inventors fail to grasp, is not delivery. All of the good ideas in the world are worth less than a cloud on a planet of water if there is no-one to deliver them.

The world is full of soap box inventors screaming ‘why will no-one listen?’ We do not need any more. We need inventive deliverers. Invention is only the beginning of software architecture. The rest is delivery, and delivery takes management and salesmanship.

You may work with a project manager, or a team of project managers if delivering large systems. If you have created something a project manager cannot grasp, and cannot be sold, then it is unlikely to be delivered. Project managers are under significant pressure to deliver, and if you, as software architect, can help them to do so, they will help you deliver your vision. If you have a good relationship with a project manager, they may even go out of their way to deliver your integrated, multi-magical system if they have faith in you as a person.

It is your job to manage the technology. It is the project manager's job to deliver. If you can help each other you will do far better than if you antagonise each other.

Why should a Software Architect care about selling?

You need to sell your vision to yourself, your business, your managers, your co-workers and your juniors. You cannot simply deliver an architecture. It is an ongoing struggle against the fundamental nature of matter immortalised in the 2nd law of thermodynamics.

The 2nd law states that the entropy of isolated systems always increases. Entropy is energy lost to the internal workings of the system and no longer available to do useful work. In other words, it is a measure of the internal disorder of systems.

Stephen Hawking[1] discusses the entropy of black holes. The parallels in software systems are remarkable.

What is almost unique about it as a physical law is that it is not always true, just in the vast majority of cases. Only you, through selling your vision, process and self, can reduce entropy in your software systems. To do so, you must stand against nature. If you ever wanted a challenge, you will be hard pressed to find a greater one.

What makes projects succeed or fail?

Far more projects succeed or fail because of people factors rather than technical factors. Alistair Cockburn[2] has suggested that people are by far the most important influences on project success. I would agree with this to a point, but believe that a defined and followed software process is key to helping us address the people problem.

As for technologies, I would be surprised if a poorly led team with the latest tools and technologies, produced better software than a well motivated team of amateurs using technology from a decade ago .

Why projects fail

Projects fail because of:

• Unclear or poorly documented requirements

If you can’t agree what you’re doing at the beginning of a project, you will never finish it.

• Not enough user involvement

How many times does this have to be said before people get it?

• Poor change control • Poor management of expectations • Insufficient initial design

My favourite word in the English language is Chaaaarge! Especially when it’s into the valley of the death with the six hundred. All of our greatest moments and greatest follies have been crowned with that glorious hurl into the teeth of fate. Many believe it is suitable for software projects too. It isn’t.

• Trying to build everything at once • Using unproven (or unsupported) technology • Poor management of external dependencies

• Not having the right experience • Poor documentation

You and I both know this means no documentation.

• Inadequate testing • Lack of adequate planning and estimation • Lack of communication • Poor implementation.

If anyone ever says to you ‘but I didn’t understand the design.’ It is your fault, not theirs. Don’t use the old ‘Why didn’t you ask?’ chestnut.

• Overambition • Cultural barriers which were not challenged during analysis • A poor initial vision • Constant requirements change

Why projects succeed

A project will succeed if:

• A programmer wants it bad enough to make it succeed • The project team are driven by clear objectives which remain in focus

throughout the project • There is a good management structure with a Project Manager of high

quality and authority • Users are fully involved • Progress is phased with a clear emphasis on testing • Well known tools and methods are used by a team experienced with them • Requirements changes are kept to a minimum

The CSSA published the following list of success and failure factors[3]:

Success factors Element Failure factors

Short and realistic Split into phases Flexible

Timescale Long Unrealistically short

Tight process Mutual commitment Speedy decisions

Approvals and acceptance

No formal process Procrastination Lack of delegated authority

Well specified Clear acceptance criteria Under management control

Requirements High level Ambiguous and fluid Open to interpretation

Flexible Contingency plans in place Payment by deliverables

Budget Piece-meal Tight No contingency plans

Skilled staff available Project management Skills not available

Trained and experienced Effective structure

Overloaded or inexperienced Structure too layered

Fully committed, accountable Attention to detail Success oriented

Business attitude Lacks interest, delegated Hostile to bad news ‘Must win’ at all costs

Complete Well defined Relevant metrics

Objectives Incomplete Vague No metrics

Applied from start Regular assessment Contingency plans

Risk management Ignored Spasmodic No contingency plans

Firmly applied Documented

Change control Loose Not documented

Close Partnering Good communication

Supplier relationship Adversarial Poor communications

Balanced Success oriented Stable Qualified and experienced PM

Project team Unbalanced Procedure oriented Volatile Diffuse project management

Thorough Fully documented criteria Pre and post integration User involvement

Testing Sketchy No benchmarks No user involvement

Integrated in team Own the requirement Involved throughout project lifecycle

User involvement Not in project team Kept at arm’s length Only involved after delivery

Planned at start Involves users Tracked

Training End of development Done by supplier only Incomplete

Established solutions Realistic expectations Not over complex

Technology Drives the project Lure of the leading edge

Modular Based on standard software Core first, functionality added

Design Monolithic Bespoke ‘Big bang’ implementation

There are further elements of success and failure which depend upon the arena the software is to serve.

Public projects have:

• less clarity in business objectives • less comprehensive scope in specification • greater impact from requirement changes • greater formality with subcontractors

• less full time dedication of staff throughout the project

Finance projects have:

• greater timescale realism • more formal testing strategy

Retail, information and manufacturing have:

• less formal procurement processes • greater use of mature technology

Applications for the control of hospital equipment are far more rigorously tested than business applications, and applications for real time control are developed with poorer tools, and higher quality requirements than business applications.

Should software teams be driven by quality or productivity?

Faster, Cheaper, Better rings the latest sales pitch at NASA. It used to be faster, cheaper, better: choose any two of the above. It was intended to explain that if you chose faster and cheaper, it would not be better.

NASA appears to have succeeded on all three by making spacecraft that meet all three goals simultaneously.

As pointed out in Peopleware[4], when we think of highly productive environments, we also think of highly efficient processes and high quality output. This implies that more of the right kind of thinking is done up front in defining these environment. We must bring that thinking into the general chaos that is software development, and make it faster, cheaper and better. We can do so as software architects by defining the vision, the environment and process, and managing delivery within that environment.

Getting someone to do something

Peopleware claims:

There is nothing more discouraging to a worker than the sense that his own motivation is adequate

and has to be supplanted by that of the boss.

You may be doing entirely the wrong thing by trying to get someone to do something. They may be fully committed to doing it and your foolish impositions on them can remove their will to do so. This is why so many project managers fail to deliver. They view progress as their will imposed on others by means of a Gannt chart and list of project milestones which must be met.

It's something akin to trying to make a hole through a piece of soil with a worm. You have the following options:

• Freeze the worm and bang it through the soil with a hammer

Result: Broken worm, no hole

• Tempt the worm through the soil with a pilot hole and juicy morsel

Result: Tired tempter, apathetic worm

• Let the worm be, it will make the hole itself

Result: Hole

I am not suggesting software development has anything to do with worms, just that forcing something is the wrong thing to do. Do not overlook this fact. It may save you an awful lot of work and an equal amount of pain.

When you must ask for something, it is important to ask in the correct way. Your request must be in tune with the listener, or nothing will happen. Children are the absolute experts at selective listening. You can shout exact orders from the distance of one atom directly into the ear of a child, without the slightest recognition of your efforts. However, whisper that you have chocolate in your pocket, and you will find the deafness magically lifts. Then you will see how to get something. Just ask for it over and over again. No-one can withstand the whining ...but I want... repeated until the said item arrives.

If you ask someone to do something, there can be responses numerous enough to gladden the heart of any statistician. They may do it, they may not.

If they are highly motivated with the ability to do the task, and they are not yet doing it, ask and get out of the way.

Figure 12.1 Motivation and ability

Equally, if someone is not motivated, and has no ability, ask them exactly the same way. After some time this will filter through into at least some motivation. It is far better to raise motivation than to raise ability. Motivation in itself will seek out ability. It is a facet of our nature.

Ability is the result of understanding, training and experience. Motivation is the result of a desire to achieve, confidence, security and possibly incentive.

Your insoluble problems will not be getting an individual to do something, but entire organisations to do something.

Motivation

Abraham Maslow[5] suggested that there are five motivational levels to we humans.

They are:

• physiological • safety • love • esteem • self-fulfillment

Man's most basic needs are physiological. They are the ones which help us stay alive: eat, drink, sleep, keep warm etc. When we are deprived of them, we die. When they are satisfied, we become motivated by safety. Being safe is our desire for protection against danger or deprivation. We seek shelter and security.

When our basic needs are satisfied, and we feel safe, then we yearn for love, or belonging. We become members of groups: family, class 4B tearaways, Kingston chess club. In these groups, we give and receive friendship and understanding.

Then we desire, and are motivated to do things which give us self-esteem and self-respect. We build our reputations and have a need for recognition, appreciation and the approval of those we relate to.

Finally, at level 5, we are on our own. We want self-fulfilment. We are motivated for self-development, creativity and job satisfaction.

How does this relate to software development, you may ask, and the answer is that all of those people you are working with are striving equally for self-fulfilment. It is peddled as the goal of life in all major religions, and by all psychologists. The motivation for self-fulfilment is a huge source of energy, and if you can get out of the way and let someone reach for self-fulfilment while helping you to deliver an architectural system, your life will become and endless sea of ease. And you will be helping people reach what it is they strive for.

Motivation is the key to everything. If we are motivated and do not have the ability, then we will find the ability. That is why it is a mistake to try and take someone from the low ability, low motivation state to the high ability, low motivation state. All the ability in the world will not mean someone will do something.

To change an organisation, you must have a vision to sell, and be able to sell it. The vision will come from your technical and artistic excellence. Selling the vision will depend upon your sales skills, and most importantly you.

You!

This is the trickiest part of the equation. You may have technical, even artistic skills. If you have grown up in the technocentric software world, you are less likely to have selling and personal skills. It is not a criticism. It is simply a fact. I shall deal with neither here, but point you in a good direction. For selling skills, see [6]. To improve yourself, I recommend The Seven Habits of Highly Effective People[7].

In the 7 habits, Covey leads us through seven layers of self improvement, beginning with personal victories, and ending up with maintaining yourself in your improved state, or as he calls it, sharpening the saw. You will have heard all of the habits before in management geek speak, and men of many clichés. Yet Covey’s book is profound. It will change the you in you.

To be a Software Architect you must become a leader. You must take a lead in the development process, the development community and the communication between customer and supplier. To become a leader, you must study other leaders. You must be prepared to take advice from your peers, your juniors, your superiors and your customers.

The world of should

The land of should is a painful environment to inhabit. Some live, forever trapped there by their own inability to escape. They announce with great herald and oyezing that a certain approach is the one that should be taken. The should is seen as an unattainable goal. It is the yearned for perfection that will not be delivered.

You must change your should for the world of will, and state your will with the conviction of one who knows, and will do, better.

We should do more testing, must become: We will do more testing, and here is what we will do, how we will do it, and when we will do it.

Before further consideration, digest the following definitions:

Want-got gap

This is the gap between an ideal and reality. If I have a Jaguar and want a Rolls-Royce, I have a want-got gap. If a business wants its development groups (architects, analysts, developers, testers, project managers) to work together to create superlative software, but instead have intergroup strife, there is a want-got gap between what they want and what they've got.

Causal chain

These want-got gaps create causal chains. If I want the respect of my coworker, but do not have it, we have between us a want-got gap. I can fill that gap by earning my coworkers respect, by helping them with an intractable problem, or by putting in extra effort to help them deliver something. In the meantime, that lack of respect that I want will affect our relationship. It will not be as easy for me to interact with that person, setting off a causal chain.

Contributing problem Source problem Business Problem personal clash creates lack of interaction

->

business does not respect awkward techies

->

projects go less well than they could

The business sees that its projects are not going well and identifies the source problem as techies being awkward. Many smaller individual problems contribute to the source problem. It is a vicious circle as the lack of respect from the business goes to reinforce the personal clash as each believes they are doing their best for the business and that the lack of respect is caused solely by the other person.

Organisation

Figure 12.2 Organisational model

An organisation is made of individuals. An organisation is also hierarchical in that the architecture team is an organisation, as is the IT department, and the business.

The architecture team has its own climate, culture, tools and structure, and this must support the climate, culture and other facets of the next highest hierarchy

in the structure, i.e. the IT department. If it does not, then it creates want-got gaps between the layers (and possibly the siblings) of the organisation.

Goal convergence

Different parts of the organisation may have different goals. If these goals lead to a want-got gap being created anywhere between layers or siblings in an organisational hierarchy, it is known as goal divergence. Filling, or aligning those want-got gaps is the process of goal convergence. Converged goals make for a smoothly run and efficient organisation. Unfilled gaps create entropy.

The world of should is at least a desire to fill those want-got gaps. However, the should is unlikely to be the will without an understanding of the gaps to be filled to make it so. To escape the world of should, start understanding and mitigating the gaps. This is further discussed in [8].

The office environment

Peopleware[4] discusses the development environment, and factors for software success.

Through a development challenge - named code wars - between software teams in professional life and at university, they identified the following.

• There is a 10:1 difference in productivity between developers. No particular methodology, programming language or experience made any difference. It was solely down to people.

• There is never more than a 3:1 difference between people in the same organisation.

This means there is slightly greater than a 3:1 difference between organisations. Research into the causes of difference revealed that the most significant factor was simply the area that each person had around their desks.

A larger area means a less dense population. A less dense population means less noise (background and localised), less interruption, and all the inherent problems that go with being disturbed. Writing code is a task requiring the writer to be immersed in their work. This is why programmers may seem aloof at times. It is not that they non-communicative, it is that they are struggling with mental problems.

I once showed someone around our workplace. I knew her well, and had worked with her previously. She worked as a buyer; that is, she bought in what the company sold. Her day was filled with making sure things got ordered when they needed to, and that once they had been ordered, were delivered. Her office was loud, fast moving, and what I would generally refer to as frantic at certain times of the day.

We entered the development area. It was fairly open plan, containing a dozen people, and was absolutely silent. 'Frantic, isn't it.' she said. I was at a loss to

point out that indeed it was, but only in the developers heads. Frantic to her meant movement and noise. She saw nothing happening.

Immersion is important when writing code. If you cannot get immersed, you will be in an organisation at the failing end of the 3:1 difference in productivity, quality, and output.

The sound of silence

Another astounding piece of information in Peopleware is a piece of research carried out where two groups of developers were asked to develop a piece of software to solve a standard problem.

One group had piped music, the other group had silence. No difference was observed in the solution, either the speed at which it was written or the quality of the output. What was surprising was the fact that the solution took a series of numbers as input, and produced another series of numbers as the output. The two series of numbers were related. This relationship was noted by half of the people who had silence, yet hardly any of those who had piped music.

The recognition of patterns is done by a separate part of the brain to that which performs the logic required to program. The pattern recognition bit also recognises the patterns in music, so is active when listening. The upshot of the listening meant that the pattern between input and output numbers was not seen, because that part of the brain was occupied elsewhere.

We are beginning to see programmers herded together in tighter and tighter offices. To block out the mass of noise, and interruption and allow themselves to get to a point of immersion, they listen to music. I leave you to draw your own conclusion.

Hold on there a moment pardner...

If Peopleware claims that no methodology or programming language makes a difference, why is half of this book dedicated to process?

I have a few answers for that:

• The arguments I have extracted from Peopleware are only two small studies rather than the grander ideas in the book

• Peopleware does not consider repeatability, i.e. being able to deliver consistently time and again

• It does not consider how to get software to version 2 once the version 1 developers have gone

• It works at the level of a project and does not allow for architectures to be created as it is focused on development teams writing and delivering software without the pressure of a business around it

In making these comments, I am not suggesting that Peopleware stopped short of the mark. I consider it one of the most excellent and important books in the

software library. What I am saying is that we, as software architects, exist beyond the project level and require a defined process in which we can work.

If there is no design, it will not be discovered in the code. If there is no approach, then it will not be seen in the execution. If there were no coding standards, then it will appear as if in a foreign language.

Even simple coding standards issues like different tab settings or data prefixes, and management issues such as version control and saving the source code once it is compiled can be hugely important to getting to version 2. And what if version 1 is such a mess, it must be scrapped?

Getting to version 2 is far (I would dare to say 2 to 10 times) more difficult if the version 1 programmer had no coding standards, no methodology, no requirements, no analysis, no design, and no iterated development. If they have no documentation about the final implementation, you will have to discover all of that yourself, and it is difficult, frustrating work. Not only will it be more difficult, but the resulting output will be worse.

I would like to see a study on getting to version 2 from a version 1 missing the above. I have no idea how one might go about it.

References

1. A Brief History of Time. Stephen Hawking. Bantam 2. Characterising People as Non-Linear, First-Order Components in Software

Development. Alistair Cockburn. ACM 2001 3. Getting IT Right for Government. A review of public sector IT projects by

the Computing Services & Software Association. 4. Peopleware: Productive Projects and Teams. Tom de Marco, Timothy

Lister. Dorset House 5. A Theory of Human Motivation. A. H. Maslow. Psychological Review, 50,

370-396 (1943) 6. Who Dares Sells. Patrick Ellis. Thorsons, Harper Collins 7. The Seven Habits of Highly Effective People. Stephen R Covey. Franklin

Covey Co 8. The 10-Day MBA. Steven Silbiger. William Morris/Piatkus.

Chapter 13 - When can I have it?

When can I have it?

Someone may have tried recently to convince you that time doesn’t really exist, that we humans have imposed our collective belief on a timeless universe. They may be correct, but it will not stop people asking you how long something will take. Consider this:

You have just spent a whole afternoon in a meeting with the top honchos in your, or your customer’s, company. It has been a drawn out affair, and rather painful for all concerned. You didn’t begin the meeting very well, because you tried to

sell them the benefits of your new iterative, incremental process. The message came across rather plainly that they weren’t interested in your techniques, just that you deliver on time. You then started waffling on about unchanging requirements, and they didn’t sound too interested in that either. They want software to enable better customer and internal communication and they want it matching their business process. Their process, as they so patiently explained, changes from day to day. They work in a very competitive market.

The meeting has rounded off with the Chief Executive shaking your hand and looking you in the eye. He thanked you for your time, and then, as if it were a mere matter of counting how many blows it would take to clear a dandelion, asked ‘How long will it take?’

It has just happened. It is you he is looking at. It is you, alone, who has to provide an answer. The contract may well depend on it, and on that contract, your business. What do you say?

Option 1: Be honest. Tell him you have no idea. Calculating project times is impossible, some things simply take a long time.

Option 2: Be forthright. Tell him it would be foolish if you mentioned a delivery date as it may be misleading. You say ‘After I have analysed our discussion and identified the salient points, I shall be able to give you an estimate based on our previous analysis and the outcome of this meeting.’ Sharp customers will try to get a deal based on your original estimate if future estimates are longer. You also don’t want to scare off your customer by overlong estimations. Your Sales Director has told you not to say anything about time and price. You have embarrassed him before. He is staring at you with that look on his face.

Option 3: Guess. Tell him anything to close the deal. You can change your mind later.

Option 4: Guess and multiply by your lucky number. It has worked in the past.

Option 5: Based on your experience of similar systems for similar customers, you reckon about eighteen months. Eighteen months, as you know, is far shorter than a year and a half. You answer, ‘Around eighteen months.’ There follows a long, hollow silence while your customer waits for a few foundations upon which to accept your estimate. The Sales Director is glowing. He is frightened.

Option 6: During the meeting you have identified the main use cases of the applications, and carried out a function point analysis. You know the magic numbers for this type of client and your current development team, so you look him back in the eye and say: ‘My initial analysis shows that this assignment will take us in the region of eighteen months. That includes all of the requirements we have identified previously and those added during this meeting. I believe your changing business needs will not impede our progress as we have factored in a 10% requirement change, and will focus in the early stages on your core business where changes are minimal. I expect your core functions of inventory, management information and invoicing will be seventy five percent of the required effort, and our analysts will keep the development in line with your business requirements as we have previously explained with our iterative

approach.’ You smile. You know what you have to do, and how to approach it. You know what management your customer needs. ‘We will remain in daily contact with your people, so you can rest assured that we’ll be right on top of any opportunities that come along during the development.’

Which one do you do?

Which one should you do?

Before we look at some estimation methods, let’s look to the horizon of estimation. What will we end up with?

We will get a number representing the development effort for a project in person hours. But what is a person hour, and how many person hours can you get out of a person in a week?

The average person spends 5 days each week at work. Over those 5 days, they work, they chat and they read Dilbert cartoons.

The structure of their work day also changes with the external ups and downs of their lives, and with their age.

Figure 13.1 Worker activities over a working lifetime

Now, far be it from me to suggest such a model is true. For us, its more like: write code, report, meet and chat. It’s an accurate enough model upon which to begin.

Suppose a worker is contracted to work a 40 hour week. Many work more than this, as is the nature of IT and many other facets of business. Getting things done takes extra time. We shall ignore this heroic and generous behaviour for now.

If, on average, our workers are half way through their careers, they will spend some time prototyping, some time thinking, some coding, some reporting, some in meetings, and some chatting. They will also spend some time on holiday, some time working in support of existing software, going to the drinks machine, the toilet, looking forward to their holidays, and collectively, some time learning, leaving, and joining. All of these activities eat away at a 40 hour week.

A worker may spend 30 days of the year on holiday. If they work Monday to Friday, that will be 30 out of 260, or around half a day a week. Add sick to that,

dreaming of holidays, daydreaming, chatting, and all of the other activities which fill a workers week, then you’re probably down to 3 or 4 days per week. Then one day of support per week, and another few hours of meetings and you’re down to 2 or 3 days, or 16 to 24 hours. Let’s take the mid point and say that, on average, your workers will contribute around half their week to your project, and the other half doing things that are all part of their contribution to the environment.

In estimation, you may calculate the length of time it will take in person weeks, and multiply that by their productivity. The above discussion will mean your productivity factor is 0.5, i.e. you will get half a week out of a person week. Secondly, are you calculating development time, or development, testing and deployment time? Don’t go quoting development time alone; your customer will expect delivery at the culmination of your development time. So you will then have to increase your estimate again by your factor for testing time. You may also have a range of factors for testing times on small to large projects. Larger projects require more time spent on communication between the developers, analysts, project managers etc.

In short, you will need the standard time estimation, multiplied by various correlation coefficients (fiddle factors) that are particular to your development effort.

Figure 13.2 Communication overhead increases with project size

The larger the project you are working on, the more effort must be put into communicating the various aspects of the project. You will also spend more time on the business processes you are programming, and how best to manage their intercommunication.

In a small business, say fifty people, you probably know everyone. You know their opinions, the personalities of the people at the top, and have a fairly good idea about the business products, processes and where it is headed. The business may get together every six months to sell you the vision. You already know it, so it doesn’t take too long.

In a large business, say five thousand people, you will have to answer to layers of people. Project managers, analysts, managers, team leaders, the owner, who has told you his objectives, and all of the layers of management beneath him, who each have their own slightly different version of the truth.

You will spend time involved in organisational politics. You will also spend time finding out who people are, whose chains of command they belong to, and worry

about how to recognise them from their picture on the intranet when you go looking for them.

Figure 13.3 Effect of business size on effort.

The larger the business, the more of your energy is likely to be expended in finding things out, and playing political games.

Productivity is also a function of the person’s hours per week. Most of us are normal, moderately healthy human beings, and we have activities other than work. There may be some who relish all night code-ins and sixty hour weeks, but on average, you might get forty hours in the building. The sixty hour all-nighters are likely to be younger, more virile, more excitable and more likely to spend forty of their sixty hours maximising the work not needed to be done.

Figure 13.4 Worker productivity and their hours per week

When you have conquered the maze of person hours, it’s still not as simple as it might seem. For example, if someone has just returned from a three week holiday touring Kenya, they will have been on safari. They will have ballooned across the savannah chasing herds of migrating wildebeest. They will have dived off the deserted white sandy beaches, and you’ll be lucky to get an hour out of them in the first week. This is exacerbated in larger businesses; there are that many more people to tell.

Have your other workers been overdoing it lately on an urgent job? If so, don’t expect them to dive in with 100% effort on this one. There may be other circumstances that slow them down: cancelled overtime, time off in lieu reduced, personal problems, bad feeling between team members, between management layers or company owners.

The point is that all of these things affect how many person hours come out of a person in a working week. If a worker is stuck on a particularly difficult support problem with a particularly difficult customer, you may even get zero hours. If you ask them why you got zero this week, you will likely get zero next week as well.

In short, here is an equation for the calculation of person hour per week.

Hours on job = Hours per week x Productivity ratio x Project size factor x Business communication factor

You may prefer a single fiddle factor to replace the three factors above, while you improve your accuracy.

Hours on job = Hours per week * Productivity Coefficient

If you have no idea where to start, try a productivity coefficient of 0.5. You can improve on it later.

You may have spotted the obvious implications in the first equation. It belies two important facts

1.You can have an optimal business size 2.You can have an optimal project size

The optimal business size is probably as small as you can get it. Less people means less politics, less communication problems and fewer conflicting ideas on the best solution.

The optimal project size seems to have been agreed at around three to twelve months. Any shorter, and the project team spend too long understanding the problem. Any longer, and you have goal setting problems, developer apathy and high turnover.

Now you have an understanding of how much time you can squeeze out of a developer each week, it is time to estimate the time required for a project.

Time estimation

You can only start estimating time accurately when you have been estimating it inaccurately for a while.

There are two rather different styles of estimation. One style involves estimation based on lines of source code in the resultant application and libraries. The other style involves breaking down the complexity of the project into manageable chunks and allotting time to each chunk. The result is then modified by taking into account technical, team and environmental factors.

You may want to try both methods to find which most accurately measures your development effort.

Time estimation method 1 – Gut feeling

What is your first stab at how long it will take? If you can’t come up with a guess, or your guess is greater than a few months, then you need to divide your work into smaller pieces and estimate for each piece. Gut feeling is often based on

summing up the difficulties, and wondering how long it might take to code it up. Don’t forget to factor in testing and release, and how much developer time you are likely to get in a week

Gut feeling is an OK method. It gives you an order of magnitude estimate, which is better than not having an estimate. It should be measured alongside a more logical method, or at least past history. You might want to write down your estimates as you go, and record against them how long things actually took. After a few projects, you should be able to come up with a fairly good fit between actual and estimated time.

Figure 13.5 Project time: estimates vs actuals

What this example data shows is that things are always taking longer than your gut feeling tells you. It also shows that your estimations for projects that take longer is increasingly unreliable, although not too inaccurate overall.

Gut feeling may be a good enough method for you. If you are fairly accurate, you may promote yourself from gut feeler to expert assessor. Others may want different models to rely on.

Time assessment method 2 – The Putnam/COCOMO Method

I have linked these together as they are both based on the assumption that you can estimate the project size in lines of source code. The Putnam and original Cocomo (Constructive Cost Modelling) equations are the same.

Project Time = KLOC x Multiplier

KLOC = Thousand lines of source code. Take that and multiply it by a number to get your time estimation. Putnam recommends a number of 3 to begin with to give a time in person months. You can further refine this based on your growing experience.

As an estimation method, you won’t have many headaches understanding it. All you need is an estimate of the lines of code likely to be produced, then multiply. The problem you are faced with is coming up with the right number for lines of source code.

If you are a programmer who likes to write source code comments before writing the code, you may be half way to this method already. You might write some pseudo code like this

Public String getPersonnelNumber(String username) {

//this function looks up personnel numbers from database A //from the username //initialise variables //check the database is available, and that the user’s machine is logged in //try a direct match between the name passed in and that held in the database //if the more than one match is returned, allow user to choose between them //if no match, try a more fuzzy approach based on surname + abbreviated first name

}

Before you pass the coding on, or write it yourself, you count the number of comment lines. On average from experience, you know that you end up with between 4 and 5 lines of source code per comment line. Your time estimation would therefore be

lower estimate: 4 * (Comment lines/1000) * 3 person months

upper estimate: 5 * (Comment lines/1000) * 3 person months

If you have 500 comment lines, then your application, according to your previous projects, will take between 6 and 7.5 person months. Bringing in your efficiency factor and multiplying this estimate by 2, gives 12 person months, or three to four months for a team of four. If you are opening your mouth, always go for the higher estimate.

Getting to grips with this estimation method means recording time against source code lines. You may have a source code control or change management system (you’d better have) from which you can check out projects and count the lines of code. If you have a record of the time taken, you may be able to reproduce your own version of the graph in figure 13.5 for use in future estimations.

You can use the Cocomo method to predict defects, or bugs:

Number of bugs = KLOC x multiplier

This will give you an estimate of how many bugs, on average, you can expect to find in your code. In turn, you will be able to spot the less defective programmers, and estimate the likely iterations through test using similar techniques.

Cocomo has been around for a long time, and has grown into a rather formidable set of equations:

where A = Multiplative constant and Size is in KLOC. There are 17 effort multipliers and 5 scale factors. In this, it is similar to function point analysis as it recognises that many other factors influence development time.

As an estimation method, it is OK. However, creating lots of lines of code is a disease of those unable to educate themselves. I have seen many occasions where 1KLOC is 900LOCs too many, and could have been done using a better method. Basing anything on lines of code can imply to developers that lots of lines is a good thing, and can lead, quite logically, to:

Revenue = A x KLOC

Then writing lots of lines of code becomes a goal for project managers to attain.

Time assessment method 3 – Function Point Analysis

Function Point Analysis is claimed to be able to

• estimate a software project's size • establish productivity rates in function points per hour • evaluate support requirements • estimate system change costs • normalise the comparison of software modules

For it to be able to do these things for you, you must first begin to use it to estimate project time. There is an international user group of function points, IFPUG[1] if you want more information. ISO is currently trying to ratify a standard which will be called Functional Size Measurement Standards, based on the work of the IFPUG. We shall be using it as a software development time estimation technique.

The function point metric began at IBM as a means of measuring software size and productivity. It uses functional, logical entities such as inputs, outputs, and inquiries that tend to relate more closely to the functions performed by the software as compared to estimating lines of source code. It has been further developed since to try to pin down its accuracy.

Function point analysis can take your analysis models, and come up with an estimation of time. It works like this:

1. Identify the complexity of your actors (system interfaces) 2. Identify the complexity of your use cases

3. Assign a measurement for technical factors 4. Assign a measurement for environmental factors 5. Add up all of the project influences, and you have your estimation.

In detail, here are the steps you need to take.

1. Calculate your points based purely on functionality. This is known as the unadjusted function point value.

2. Adjust the function points for technical and environmental complexity 3. Multiply your adjusted function points by how long it takes to deliver a

function point's worth of software.

Identify the complexity of your actors

Take each actor in turn, and decide whether their interaction with your system is simple, average or complex.

Actor type

Description Points

Simple Procedural text interface, e.g. ATM 1

Average Interactive text or protocol communication (http, ftp) or interactive text (VT)

2

Complex Graphical Interface, e.g. Windows user 3

Identify the complexity of your use cases

Take each use case in turn, and as with the actors, decide if they are simple, average or complex. This complexity can be estimated from the number of transactions a use case performs, or from the number of analysis classes identified in the analysis model.

Use case type

Transaction based estimate or Class based estimate Points

Simple 3 or less transactions or operations

Fewer than 5 analysis classes

5

Average 4-7 transactions or operations 5-10 analysis classes 10

Complex 7-10 transactions or operations More than 10 analysis classes

15

There are two schools of thought on having reached this point. You can calculate the unadjusted use case points by the following formula:

UFP = UUCP = Sum(Actor points) + Sum(Use Case points)

A simple desktop application such as Notepad will have 3 points for the graphical user interface, and 10 points for an average use case of write document. It is an average use case as it has more than 3 transactions in load, save, copy, paste, type, click. Thus, the unadjusted

use case points for Notepad is 13. In other words, Notepad has 13 function points for this rough level of estimation.

Or, for more complex applications, there may be a need to drill into the transactions to detail your unadjusted function point count.

Figure 13.6 Routes to the unadjusted function point value

What is a transaction?

Transactions are defined as external inputs, external outputs and external inquiries. The word external, in this instance, applies to the use case.

External input

When an actor moves data or control information into the use case, it is an external input. For example, when a user completes a form and submits it to a server, this is an external input.

External output

When the use case delivers derived data or control information to an actor, it is an external output. It may be a form or report, or output file.

External inquiry

A simple input and output, when combined together to request and return basic data held by the use case, is an external inquiry. This does not apply when data must be changed, or derived from the data held by the use case. Then it becomes a separate external input, and external output.

The information being passed around by these inputs and outputs is held either within the use case or outside the use case. If it is held within the use case, for example within an object, it is called an Internal Logical File. If it is held outside, in a database or file, then it is called an External Interface File.

The inputs, outputs and enquiries are given a simple, average or complex rating according to how many file types they reference, and how many data elements

they pass to those file types. For those programming databases, a file type can be thought of synonymous with a database table, and a data element as synonymous with a field.

External inputs

Data elements

file types 1-4 5-15 15+

1 low low average

2 low average high

3+ average high high

External outputs, External enquiries

Data elements

file types 1-5 6-19 20+

1 low low average

2-3 low average high

4+ average high high

Similarly, the Internal logical files and external interface files are given low, average or high ratings based on the number of data elements being passed, and also the number of repositories for those data elements. For database programmers, the Record Element Types is essentially the number of tables.

Internal logical files, External interface files

Data elements

record types 1-19 20-49 50+

1 low low average

2-4 low average high

5+ average high high

We now apply the function point values to each of these low, average or high ratings for the inputs, outputs and files, and add them up to give an unadjusted function point count.

Function point values

Rating External input External output External inquiry Internal file External file

low 3 4 3 7 5

average 4 5 4 10 7

high 6 7 6 15 10

Going back to the Notepad example, we are loading and saving a single data type, a text file which can be read in and saved. That would give us a low External input and external output (3+4) and a single internal file (7). The total unadjusted function points for Notepad is therefore 3+4+7 = 14.

If a single use case has more than ten transactions or 50 function points, try to break it down into two or more use cases. If you have many use cases with more than ten transactions, or many single transaction use cases, it may mean you are not defining them at the correct granularity.

The unadjusted use case points or unadjusted function points are then multiplied by further factors which introduce technical and environmental diffculties.

Identify the technical difficulty

The unadjusted function points are then adjusted by technical and environmental factors. The multiplication factor is known as the value adjustment factor.

The answer to each question is a value from 0 which means irrelevant, to 5 which means essential.

Factor Scores

1 Data communications 0 batch processing or standalone PC 1 batch with remote data entry or printing 2 batch with remote data entry and printing 3 online data entry, batch UI, query system 4 Application with only one comms protocol 5 Application with more than one comms protocol

2 Distributed data processing

0 Application does not distribute data 1 Application prepares data only 2 Data is prepared and transferred 3 Data processing and posting 4 Data processing, posting and retrieval 5 Data processing, dynamic updates

3 Performance 0 No performance requirements 1 Performance requirements reviewed, no action necessary 2 Response critical during peak loads 3 Response critical at all times 4 Performance analysis required 5 Performance tools required

4 Heavily used configuration

0 No operational restrictions 1 No effort required to meet required restrictions 2 Some security or timing considerations 3 Specific processor requirements for part of application 4 Special constraints on application within system 5 Special constraints on application in distributed components

5 Transaction rate 0 no peak transaction period 1 peak transaction period anticipated 2 weekly transaction peak 3 daily transaction peak 4 high rates required, must design for performance 5 high performance rate in SLA, must use performance analysis tools

6 Online data entry 0 All transactions processed in batch mode 1 1-7% interactive 2 8-15% interactive 3 16-23% interactive 4 24-30% interactive 5 30%+ interactive

7 End user efficiency Human factors taken into consideration. A typical windows application gets 5.

8 Online update Applies to the update of Internal Logical Files 0 None 1 Volume updating of a few files is low and recovery easy 2 Online updating of 4 or more ILFs 3 Online updating of many ILFs 4 As 3 with protection from data loss 5 As 4 with highly automated recovery

9 Complex processing A count of the following: Sensitive control or specific security processing

Extensive logical processing Extensive mathematical processing Exception processing for incomplete transactions Complex processing for multiple i/o devices

10 Reusability 0 No code reusable in other applications 1 Little reusable code 2 <10% of application reusable elsewhere 3 10%+ of application can be used elsewhere 4 Specific design for reuse 5 Specific parameterised design for reuse

11 Installation ease 0 No special setup 1 Special setup required, but not specified 2 Conversion and installation requirements, but not considered important 3 Conversion and installation considered important 4 Automated conversion and installation 5 Automated conversion and installation considered important

12 Operational ease 0 nothing beyond normal backup The following all count as 1 _ Effective startup backup and recovery _ As 1, no operator intervention required _ Minimise need for backup _ Minimise need for paper 5 Design for unattended operation

13 Multiple sites 0 One user 1 Multiple users on identical hardware 2 Multiple users similar hardware 3 Multiple users, different hardware and OS 4 Documentation and support plan provided and tested for 2 5 Documentation and support plan provided and tested for 3

14 Facilitate change A count of the following: Flexible query and reporting one one ILF Average reporting from more than one ILF Complex requests, multipe ILFs Business control data online, but written daily Business control data online with immediate update

Add up the influences to give your value adjustment factor (VAF), a value between 0 and 70, then put the number into this formula to give your adjusted function point count.

Function Points = UFP * VAF

That is the IFPUG's version of the story. I mentioned two schools of thought earlier. Here is the other, as recorded in Applying Use Cases[2]. This time there are only thirteen technical factors, and they are not all weighted the same.

Factor Weighting

T1 Distributed system 2

T2 Performance requirements 1

T3 End user efficiency requirements 1

T4 Complex internal processes 1

T5 Reusable code modules or components 1

T6 Easy to install .5

T7 Easy to use .5

T8 Portable (across operating systems) 2

T9 Easy to change 1

T10 Concurrent 1

T11 Special security features 1

T12 Provides direct 3rd party access 1

T13 Special user training facilities required 1

Somewhat similar to before, each T factor is assigned a value of between 1 and 5, and each value is multiplied by its weighting before being added up. The maximum total, as before, is still 70.

Technical difficulty multiplier = 0.6 + (0.01 * sum(T-Factors))

As you may be able to see, the technical difficulty factor will be between 0.6 and 1.3. This means your function point count will be diminished if it is a simple projects, and inflated if it is difficult.

Function points can be calculated by anyone, and it is hoped that two skilled people would come up with roughly the same number of function points for a given piece of work. What it does not give you, is an estimation of how long it will take. To get there, we will use the method described in Applying Use Cases[2].

Identify the environmental factors

Environment is that which makes one team deliver quicker than another.

Environmental Factor Weighting Value

E1 Familiar with development process 1.5 0 No experience 5 Expert

E2 Experienced application developers 0.5 0 No experience 5 Expert

E3 Object oriented experience 1 0 No experience 5 Expert

E4 Lead analyst capability 0.5 0 No experience 5 Expert

E5 Motivation 1 0 None 5 Highly motivated

E6 Stable requirements 2 0 Unstable 5 Fixed

E7 Part time workers or contractors -1 0 No part timers 5 All part timers

E8 Programming language difficulty -1 1 Basic 2 Visual Basic 3 Java 4 C++ 5 Assembler

As with the technical factors, each environmental factor is given a value between 1 and 5. Each value is multiplied by its weighting and added up. The environmental factor is then calculated:

Environmental difficulty multiplier = 1.4 + (-.03 * sum(E-Factors))

The maximum value for the sum of the environmental factors is 32.5. This represents the best team. The minimum summed value is -10. The range of the Environmental difficulty multiplier is therefore 0.425 to 1.7 or 1:4 difference between the best and worst prepared teams. This largely agrees with the Peopleware claim of a 1:3 difference in coding output observed between teams in different environments in the code wars.

To calculate the amount of time it will take, the unadjusted function points have the technical and environmental factors applied, then the whole thing is multiplied by a person hour value. Initially, we shall take 20 hours per point. This includes development and initial testing and should take you up to the point of release.

Time = 20 x UFP x Technical Difficulty x Environmental Difficulty

Our Notepad example of 14 unadjusted use case points can now be multiplied up to see how long it might take. Using the 13 'T' factors, if we give it a point each for performance and end user efficiency, and two points for its internal processes. It must be easy to install and use, so a maximum of five points there gives a total technical difficulty of 0.69. I shall say we are highly skilled and motivated etc, and will program it in C++, so will use an environmental factor of 0.545. Time = 20 x 14 x 0.69 x 0.545 = 105 person hours The result is that I should be able to squeeze Notepad out of a highly skilled MFC programmer in a few weeks if I can keep him solely on the job.

In Applying Use Cases, we are recommended to count up the number of points E1 to E6 which score below 3, and one each if E7 and E8 are below 3. A total of 3 or 4 indicates experience problems and a multiplier of 28 man hours is recommended. If the number is 5 or more, then you have a serious experience problem, and should get some more experienced team members.

I have a spreadsheet based on the above discussion, It was fairly easy to create, removes keying errors, and can be kept for future reference or to recalculate when requirements change.

Figure 13.7 Function point estimation

Does it really work?

The function point technique is very general. I would suggest having your own fiddle factor to apply to the final result. Begin with a fiddle factor (i.e. a multiplier) of 1.5, and as you gain experience with function points you may be able to refine it.

Notice there is nothing in there about business politics, size or communication. If your software is being defined in the US, coded in India and tested in the Ukraine, that would add a few difficulties and certainly extend the project. Yet there is no mechanism for addressing such problems.

The greatest danger with the function point technique occurs when you calculate 20 weeks for four developers for a project. The developers plod on merrily, and lo and behold, you find it takes four of them 20 weeks. When you realise later on that you entered the wrong number somewhere and it should have been 10 weeks for two developers, what will that tell you?

Any estimation technique is useful for predicting approximate times. Developers will live up to your estimations, but it is them meeting the expectation rather than the estimation method being accurate. You are then into monitoring people to see if they are over-stressed trying to meet the demands you have placed on them, or are plodding when you have overestimated.

So, estimation techniques are good. They can give you a good idea of how long something is likely to take. Despite the high precision appearance of the formulae and factors, there is little point in the accuracy. What you are dealing with is people. If a developer is expected to meet the expectation of the company president by getting a spreadsheet macro out by next Tuesday, it will be done. If a developer has to meet the expectation of a Project Manager shouting words like

urgent, under pressure, must have, then the result is entirely dependent on personal factors rather than any estimation by function point analysis.

Good relationships and good communication deliver software. Good estimation is extremely useful, but guarantees nothing.

In Peopleware, the coding wars showed a three to one difference between developers in an organisation, and a ten to one difference in developers overall. That makes rather a mockery of any generic measurement technique.

Even so, if you are not trying to do it, you are not improving your software capability. NASA lists the elements of good estimating practice, and the evidence for maturity in that practice on its website, in a Parametric Cost Estimating Handbook. IFPUG also has a function point counting manual.

And finally, do not be afraid to turn the question on its head. Ask the business how long they would like it to take, then tell them what they will get for their money.

References

1. The International Function Point User Group (IFPUG). www.ifpug.com 2. Applying Use Cases. Geri Schneider and Jason P Winters. Addison Wesley

Chapter 14 - The way of the SA

Survival of the fittest

Long ago, when we used to say ugh and fight off dinosaurs with nothing more than a flaming stick swiftly dragged from the fire, the fittest survived. Not necessarily the ones with the best oxygen conversion rate and lowest at rest heart rate, but those who fitted the survival statistics best. Luck, although hugely important to an individual, meant little in the millions who made up the statistics.

You too can survive as a software architect if you are fit to do so. Survival may mean that you fit the statistical survival pattern so well that it’s a breeze. The less fortunate of us must fight, or at least argue a little, to keep on top of our hill.

We can fight to win, or we can organise to win. We humans have learned that organising and politicing are just as effective as fighting. More so, if you can play the great games of democracy and debate.

Survival of the best organised

The best organised is part of an organisation. To be upheld in your state of software architectness by a professional organisation is likely to help you keep your hill more easily. If the organisation is supported by law, or charter, then you are part of the magnificent structure of civilisation itself.

What do others do?

Professional people are upheld by professional organisations. Architects are chartered by an organisation such as the Royal institute of British Architects (RIBA) or the American Institute of Architects (AIA). The people who build their architectures belong to other organisations, builders to the guild of master craftsmen, gas workers to the council of registered gas installers (CORGI). Whatever the profession, there is a professional organisation which you can support, and it will support you.

If you have a half decent organisation founded when your great-great-grandfather was still around, you get professional recognition by listing your membership in letters after your name. The letters mean you are a recognised professional in a recognised field, and even gives your level of belonging to that field: associate, member, fellow or founder etc.

There are a few software organisations, but as we are still in the formative years of software, it will be some time before these organisations can claim the same membership rewards and rites of initiation.

An architect has the backing of formidable organisation. He has respect, as Architect is an instantly recognisable role with professional standing. The RIBA or AIA will define what you must know to use the title, help you if you have achieved the professional standing required and passed their exams, and hound you if you are not, yet claim to be, an architect.

If you tell someone you are a software architect it is likely they will tell you about their home extension, or the wonders of cathedrals. They won’t hear the software bit. It is, as yet, an unrecognised profession. For us there is no such qualification as Chartered Software Architect. At least not yet.

Yet I propose to you that the software industry itself is in the grips of a paradigm shift. It’s about time. We have collectively been creating substandard software and wasting our efforts for long enough. Software architects are emerging to lead the field of software development the same way that architects emerged as leaders of the building crafts.

We are not all Leonardos or Lloyd-Wrights. For most of us its a day job, just the same as everyone else's. We turn up in the morning, struggle through the day, and go home at night.

The daily grind

There are many different types of software architect. Collectively, they are responsible for defining and defending their architectures. Some specialise in enterprise solutions, others in areas of technological excellence. There are those who create and defend the technological vision for their businesses, and others who are involved in the day to day running of projects.

Some are technologically carefree, some are technology evangelists. Some have a vague idea of process, others nail every action they do to it. We are an eclectic

mix, but the core of our function is the same. That is, we create and uphold software excellence, or at least we do our best to.

Our working day is filled with tasks only loosely related to writing code, instead we operate in the stratosphere where ideas precipitate from vague clouds to rain down on the heads of development. We create architectures and defend them from attack. We argue our points of view and design choices. We prototype. We brainstorm. We communicate. We do our best to keep up to date with the ever growing mountain of technologies, leading people and visions of excellence, creating high level documents, reviewing others, and contributing to the act of ensuring that the code being written supports and delivers our architectures.

Creating architectures

Architectures are derived from many influences. Prominent among those influences is the business or system vision. If the business can state its long term objectives, then it is far easier to create and defend an architecture. Without it, there will be no clear vision, and nothing to hold up as a sword or shield when defending the architecture.

The stages of creating an architecture are the same as those in a project. First, the software architect must extract the architectural requirements from the business or domain. The requirements are then analysed and added to or clarified. Then comes design. The chin scratching part, when long hours pass in thought, and bosses wait nervously for a hint of output. Finally, there is prototyping to test the vision. When architectures are being created, they are continually measured against the architectural requirements. Middleware products are tested against their claims, and against each other. Suppliers are put under the microscope, their products examined and prices negotiated away. Relationships with suppliers are projected into the future to consider how the them and us relationship will progress once the initial purchase has been made. Finally, the architecture is broken down into deliverables and a programme timeline is created.

Putting forward arguments

Architectural arguments can range from the height of architectural style to the depth of technological choices. Whatever the argument, we must be able to clearly elucidate the requirements we are arguing for, and even argue to uphold the vision in the face of the ever chipping financial analysts.

Your arguments will concern software choices, design choices. You must sell your own choices, encourage others, or push back on those you deem incorrect. Incorrect will mean the price is too high, the technology will not deliver what it claims, a project will be the wrong thing to do, or being done at the wrong time. You may consider a supplier/customer relationship will not help you to defend your architecture. If a vendor comes in and sells the business their idea of a solution, you will have do do some mighty fine footwork to avoid it if you believe it will destroy your architecture.

All of this comes through argument. Not the shouting at each other kind of argument (though it may come to that on occasion), but the reasoned, thoughtful argument about what is best for the architecture and what is best for the domain or business. It may be that your architecture is holding up the business. You must accept this possibility, identify it to your leaders and then do something about it. The only way you will be able to manoeuvre around such problems is the respect you have built up previously through your reasoned arguments.

Design

Much of architecture is about careful design. Our knowledge of design is largely embodied in design patterns, which are the subject of the next chapter.

Patterns, however, are only part of the story. You must be able to design visually and technically, taking into account the wishes of the users and the opportunities and constraints of the platform and hardware you have chosen, as well as the constraints of the environment in which you work.

Defending the architecture from attack

As architect, you must review everything being created under the umbrella of your architecture. If you do not, its entropy will increase. All of the quick fixes begin to undermine your creation. It may be a seemingly innocuous change, or a slight addition to solve an operational problem, but it can signal the beginning of the end of the architecture. It may start as component interface changes, the hard coded data extracts in macros, the direct links to databases and assumed rather than looked up server names. Each one is minor on its own, but collectively, they increase the entropy of the system. They make it ever more difficult to upgrade applications, and each slight change is an attack. You must protect against them and fight for your architecture. You must explain why these attacks must not take place, and this is a very difficult thing to do when small attacks create quite an acceptable change for a business person in need of a quick solution. You need to turn the attack into a contribution towards the vision for the future.

If you do not do so, the long term effect is that each small request becomes ever more difficult to deliver. Finally, the architecture is so compromised that new technologies cannot be implemented cleanly, upgrades cannot be rolled out without severe changes, which means an even more compromised architecture. Thus begin the security holes, the ever increasing flow through the helpdesk. Old developers become frustrated, spending all their time helping the helpdesk, and frustrated new developers complain that it was never like this where they used to work.

Entropy is the slayer of architectures, and you are the knight valiant.

Creating documents

The best way to create documentation is from a set of templates. The problem is that architectures are often very different, and it would be foolish to try and squeeze them into a template. Certainly requirements fit well into a template, as

does analysis, test plans, test scripts, release documents and implementation write ups. Designs fit less well, and it is best to structure a document in a top-down fashion so it follows an ever more detailed, but logical argument.

Architecture documents are similar to design documents. They both show hardware topologies, software components and data models. The difference is that architecture documents detail systems and design documents detail single applications or components.

Reviewing documents

We may refer to it as a design review, requirements review etc, but more often than not, we are reviewing a document.

In doing so, we are addressing three issues: one, is this document complete and correct? Two, does it compromise the architecture? Three, can anything be extracted from it to extend the architecture?

Prototyping

Until software architectures are tested and proven, they are mere dreams. During prototyping, the software architect must carefully select the components and aspects which must be proven by a prototype. Too little prototyping will mean an important part of the system has not been proved, and too much means time is wasted and would be better spent during development.

Creating iterative timelines and programme plans

Architectures are usually delivered as programmes of work. Programmes are divided up into projects, and project managers then manage the delivery of those projects. However, there are cross project dependencies, and those dependencies must be a part of the managed timeline.

Leading or attending brainstorming sessions

Requirements and designs can be delivered by a Rodin like thinker, or by a gathering of ideas where the best are extracted out of people's heads, argued about and recorded.

If you have the best architectural solution, and the person who disagrees with you is a better arguer, you will end up with a poorer architecture. You must learn to put forward your arguments. You must argue to preserve them, or accept that your solution was not the best and gratefully accept your opponent's suggestion.

Brainstorming sessions can be highly creative with the right people, or useless with the wrong people. The best sessions are where people largely agree with the way forward, and can decorate and embellish the path. The wrong people are those who dominate the meeting by asking for explanations at every point or have political agendas that have nothing to do with the brainstorming session and suffocate the life out of what could have been.

For an effective brainstorming session, get some like minded people and use an effective chairman to elucidate for the group and to lead the meetings.

Keeping up to date with the news, new methods and product releases

Strengthening the vision is as important as maintaining it.

What may weaken the vision is a change in political situation, a change from outside suppliers, a change in internal personnel with their own ideas of what the vision for the business, domain, or architecture should be.

For these reasons, the software architect with a vision to maintain must keep abreast of the political situation surrounding their architecture. They must also keep up to date with new methods, new technologies and software upgrades.

You must have an opinion on any new technology or paradigm before it hits your inbox. You must know what it is, how it is used, and how to defend an attack by it should you consider it harmful. Each piece of new technology or new thinking promises the earth, and can deliver less than an earthworm.

Displaying the traits of leadership and success

Most of the leaders in our societies are people of unstoppable drive. They are people who relate well to people and can negotiate our myriad complexities to bring out the best in us all. True, some may have reached their pinnacles through intimidation and other unsavoury acts, but they are in the minority.

Leaders have certain traits that have allowed them to surface from the ever deepening layers of society. You must tap into that current if you are to lead as a software architect. Leaders show the traits and habits of success. They are discussed in [2] and [3].

Keeping up with the Joneses

Can you, as a software architect, remain in better contact with the leading edge of software development than your software developers?

Developers spend most of their time coding, buried in technical journals, struggling to make the plethora of fantastic technologies work together, and through the trial, become more skilled, more knowledgeable. Meanwhile, you are stuck in meetings, going over people’s designs, reviewing their output, and worst of all, permanently detached from the path which led you to software architect. You were top banana once; you will not be when you are software architect. You will merely be the oil among the cogs.

To be useful oil, you must lead with vision. It is your vision that will make you stand out as a leader among software developers. Your vision will contain, and promote, the significant decisions made within a software system.

It is you who will define the composition of structural elements and interfaces above the level of individual projects or products. You will define the architectural style, the behaviour and collaboration required to support your organisation. You will be thinking of usability, functionality, performance, resilience and reuse. You will trade off economic against technological constraints, and guide the aesthetic whole of your systems.

You will be concerned with the architectural views which display facets of your architecture to developers, to the business, and to the individual concerns of the personalities within your business.

Your skills will be your proactivity, your experience of software development and the domain in which you are architecting. You will be goal oriented, a leader, an authority and personality, and your judgement in matters of development will be as fair and considerate as in a lawcourt.

If you are the architecture boss, and you have junior architects who answer to you, or designers who create the applications within your architecture, do your bit, then get out of the way and leave them to do theirs. Yes, reviews are good. Guidance is good; coaching in a formal or informal way is good. Interfering is not.

What you bring to architecturally led development is you. A single source of technical and artistic endeavour. A vision. A beacon around which the moths of development will gather, driven into some semblance of logic and reason by your light.

And still, we are discussing how a software architect will be, without a clear understanding of what software architecture is. The best claim for the blueprint of software architecture I have seen so far is from Grady Booch. Here it is:

Figure 14.1 Software Architecture metamodel

He states an architectural charter, which is:

• Defining the architecture of the system • Maintaining the architectural integrity of the software • Assessing technical risks related to software design • Proposing the order and content of successive iterations • Consulting services • Assisting marketing for future product definitions • Facilitating communication between project teams

He also says this:

The life of a software architect is a long (and sometimes painful) succession

of suboptimal decisions made partly in the dark

Both are views into a model software architect.

I used the word blueprint. Booch calls it a metamodel, and uses the term Architectural Blueprint within his metamodel. The software blueprint is the holy grail of software architecture. As yet, no-one has managed to define it in the simple terms of first or third angle projection that an architect or draughtsman might recognise. Instead we struggle to constrain our blueprint by using views,

and struggle again to define what those views are. It is like trying to constrain a sphere of water in zero gravity with only a pair of hands. No-one has managed to do it effectively and agreeably yet; the (software) alternatives are discussed in chapter 16.

Despite the high cost of living, it remains popular

Despite the high cost of technology, it remains popular. New technology, however, is part of the problem in that there is so much of it. Our attempts to integrate immature technologies using terms like early adopter and bleeding edge to pretend we're cool, mean we barely come to understand current technologies before new paradigms are thrust upon us.

What do our compatriots in building and naval architecture do? Do you think they would use a new material on a building or ship without it being fully tested?

Hi, says the electro-salesman (software people actually go out looking for trouble, enthused by a particular magazine article or presales blurb blunderbussed out as a sea of glistening electrons). Click here to download our untested beta. You do so. It will save a month’s programming. You’re hooked.

Hi says the real salesman to a yacht designer. Here’s a material we haven’t fully tested yet, but it’s as strong as steel and only one hundredth of the weight. It's manufactured from silk spun using nano-technology exactly the way a spider weaves. We haven’t finalised the bonding yet and it cracks sometimes, but that shouldn’t be a problem.

Great, says the yacht designer. I’ll use it.

It is time to realise that those in other professions would not dream of delivering excellence in the environment (both controlled and physical) that we do it in.

The gulf between dreams and powers

You have dreams of magnificence. We all do. You have to deliver within a measured time, cost etc. This antagonism of concerns is what dilutes most dreams. It is what drives men to despair, artists to monoearism, and space shuttles to disaster.

We can only reach for excellence within our constraints. That may sound negative, but look at what NASA has achieved on an ever reducing budget. It has been taught to be more, not less, inventive, and gets spacecraft out to asteroids using plasma drives.

If NASA can test software and deliver it to a spacecraft on the edge of our solar system, why can’t you? If they can control a lander on Mars where the round trip between signal and response is measured in hours, what’s to stop you?

The challenge for the software architect

According to Dana Bredemeyer[4], the software architect has five domains of competence.

Competency 1: Technology

What you know What you do What you are

In depth understanding of the domain and pertinent technologies Understanding what technical issues are the key to success Development methods and modelling techniques

Modelling Tradeoff analysis Prototype, experiment, simulate Prepare architectural documents and presentations Technology trend analysis and roadmaps Take a system viewpoint

Creative Insightful Practical and pragmatic Insightful Tolerant of ambiguity, willing to backtrack, seek multiple solutions Good at working at an abstract level

Competency 2: Strategy

What you know What you do What you are

Your organisation’s business strategy and rationale Your competition (products, strategies and processes) Your company’s business practices

Influence business strategy Translate business strategy into technical vision and strategy Understand customer and market trends Capture customer, organisational and business requirements on the architecture

Visionary Entrepreneurial

Competency 3: Organisational Politics

What you know What you do What you are

Who the key players are in the organisation What they want, both personal and business

Communicate, communicate and communicate Listen, network, influence Sell the vision, keep the vision alive Take and retake the pulse of all critical influences on the architecture project

Able to see from and sell to multiple viewpoints Confident and articulate Ambitious and driven Patient and not Resilient Sensitive to where the power is and how it flows in your organisation.

Competency 4: Consulting

What you know What you do What you are

Elicitation techniques Consulting frameworks

Build trusted advisor relationships Understand what the developers want and need from the architecture Help developers see the value of the architecture and understand how to use it successfully

Committed to others’ success Empathic, approachable An effective change agent, process savvy A good mentor and

Mentor junior architects teacher

Competency 5: Leadership

What you know

What you do What you are

Yourself Set team context and vision Make decisions, and make them stick Build teams Motivate

You and others see you as a leader Charismatic and credible You believe it can and should be done, and that you can lead the effort You are committed, dedicated and passionate You see the entire effort in a broader business and personal context

This is quite an exhaustive list of personal and professional traits. It can be used as a guide to where you need to end up, and in the meantime, no-one can stop you calling yourself software architect.

We have no body of knowledge. We are largely confused over our terms and methods. We have many gurus promoting alternatives.

We have the management trap. Senior software people end up doing no software. And we do not have enough software people to go around.

How to make your mark as a software architect?

The road to becoming a successful software architect is a long one. We have no clear statement of what we do, no common view or point of reference we can consult when we are considering one path or another. Thus, the area between software architect and software builder, and between software architect and project manager is difficult to define. The gray areas can be almost as wide as our areas of expertise.

Because of this lack of a central repository of information, or a software architect's book of knowledge, we are largely on our own. To succeed, we must balance many conflicting issues.

Balance

Sometime after birth, as the neurons in our brain begin to join together, and our synapses begin firing in what could be considered something of an order, we learn to balance. Eventually we stand, and our balance allows us to walk. Of course, this is only the beginning of our troubles. We must then learn to balance our food and fluid intake, our relationships, our work and home lives. We must balance our time between rushing forward and pausing for reflection, between earning and spending and between falling and getting up again.

Software architects spend a lot of time balancing.

Sell the vision or sell the process?

You want people to buy into your vision. You also want then to work within a defined process so you have some control over how, or if, your vision will be created. Your selling time and energy is limited, so you must balance the time you have between selling the vision, and selling the method by which you will get to the vision.

Why didn't you sell him the software clock?

If you cast your mind back to chapter 5, the software clock for the rather hopeless department of clock readers and announcers was abandoned. It could have been a tidy earner. The department, and more importantly, the sponsor, would have paid well and been glad to do so, but a different course of action was suggested that didn't earn a penny.

The reason is that it is better to build a relationship which will bring future work, than to fleece someone just because they are willing to pay. You must balance immediate income with building the relationships to bring future income.

Short termism against long termism

One of the recurring problems in software architecture is that is takes more time to deliver something the right way than it does to deliver it the wrong way.

You need stronger arguments than 'if we do it this way, there will be more support calls'. You must have watertight arguments for defending your architecture right down to the long term effects quick fixes will ultimately have on the architecture and on the domain or business.

This is the balance of short term thinking against long term thinking. If you are writing commercial software, the long term solution may mean the business goes bankrupt. You must then balance architectural defense against defense of the business.

Integrate new technology or deliver software?

Six months spent integrating new technology into a software system is six months not delivering anything. You must balance the promise of the new technology against the knowledge of current technology.

If technological advances do not allow us to meet the needs of our clients better, then what is the point? If it takes two months to integrate and costs the equivalent of four months’ salaries, why should it be used when six months work could create its equivalent? You must decide if, and why?

Developing for the integration of future technologies

Similar to the previous example is the question of how much development effort should be put into paving the way for new technologies. I recommend none. Every time I have tried to do so, the new technology has not been quite what I expected, or supersedes the entire piece of software, making it obsolete.

You may have more luck. It is up to you to balance the up and coming promises against the realities of what is available today.

Software fix or social fix?

When I write software systems, I do my best to make them secure. I don't create back door entry points. I demand well formed passwords mixing upper and lower case gobbledegook rather than allowing Username: George Password: George.

When I use software systems, I create security holes. If I need a 4 digit number, I use my date of birth. If I need a character string, I use the name that I am known by to my wife's family. In short, I am sloppy. I am lazy, forgetful, and really believe I cannot remember a new password and login for each software system, website, bike lock and burglar alarm I am faced with. Therefore, they are all the same. If I want to know someone's password for another site, I can create a dummy site on the internet, get them to create a username and password, and, as if by magic, I have their electronic keys to the web.

This is not a technological problem. It is a social problem.

Business often lumps software departments with social problems. You must push back social problems to the business or offer technological solutions.

An example

Emails are taking longer to arrive. Client server and thin client application response is poor. People are screaming. The same people have no conception of the fact that when they do a screen dump and paste it into an office application, it is faithfully rendered as a 24 bit bitmap. Thus, each picture in their presentation or document is around 2.4Mb from a 1024x768 screen grab. Most of the network users paste around ten pictures into a document and twenty into a presentation. The result is that each document is 25 to 50Mb.

There are simple, but extra steps to be taken, to reduce this size to something less significant. For example, a screen shot can be reduced from 2.5Mb to around 50kb by converting the image format to a gif. Compressing a 25Mb document containing such images could make up to 100:1 difference.

You may have recently installed a document management system to alleviate the problems, but have found the users are still emailing documents between one another.

The social problem is one of education and persuasion. Using the document management system would help. Compressing documents or converting graphics to gifs rather than bitmaps would also help. Getting a faster network would help too. Which should you do?

Procrastination or haste?

Sometimes, putting the brakes on is the best course of action. At other times, rushing is the only feasible option. Both extremes will annoy someone. You must walk the thin line between the two to get software out of the door as effectively and efficiently as possible.

Resilient or agreeable?

The waves of wrath will break upon you, as you will be viewed as the sea wall. You will stand between the business and development, between the business and your architecture, and between development and your architecture. They will all rage against you, and you must be resilient if you and your architecture have any chance of survival. At other times, you must accept forces that may corrode your architecture if it is for the greater good, and then you must be agreeable to change.

Risk and return

There are many risks in developing software, most of which you must steer around carefully. Risk management is something the project manager will do for the project, and you will do for the technologies and architecture.

On occasion, a risk will present itself with a return that you consider worthwhile.

Example 1

You need to get someone's attention badly to preserve your architecture. Suddenly, you - mild mannered Clark Kent - become a snorting, raging bull. This is guaranteed through millions of years of evolution to get someone's full attention. It can even work for a room full of people, but I wouldn't suggest trying it more than once in a blue moon.

Of course you may get to practice it only once per job. That is why you must balance its risk against the return you get.

Example 2

SuperWidgets inc has just released a database killing technology. It is based on a search engine and an object database and builds 3D indexes of all the data you hurl at it. There is no need to structure data, you merely pass it your class and it is saved. When it is saved the 3D index updates, and a cube view of it, and all related items is instantly available for searching and drilling. It is called Unstructured, Searchable Object Data.

It is in alpha test. It is expensive. The architecture needs a rejig to utilise it correctly which means a lot of work. The return is huge. Do you ignore it, and stick to your relational, difficult to maintain data model?

Fluidity or immutability

In an early architecture, ideas must be allowed to flow. Offering and criticism must come from all stakeholders in the architecture and it must grow from amorphous lump to crystalline clarity. Once the crystal has formed, you, as defender of the architecture, must match its immutability.

Buy or make?

It is usually cheaper to buy software than to write it yourself. When you can't get what you want out of a box, you must write it yourself. Somewhere between these two extremes is the buy and customise option. You must decide which is better.

Cost vs performance

Performance in this case does not just mean speed. It also means security, stability, fault tolerance etc. Well performing software runs on more expensive hardware and takes longer to write. You must balance acceptable performance against acceptable cost.

How was it for you?

The New Scientist ran an article entitled: Love it or loathe it: Why can't Architects figure out which buildings will be crowd pleasers[5].

Architects are hopeless when it comes to deciding whether the public will view their designs as marvels or monstrosities... [The public] say designers should go back to school to learn about ordinary people's tastes. Many buildings that appeal to architects get the thumbs down from the public. The report cites work from the Journal of Environmental Psychology[6] on a survey of 25 architects and 27 'ordinary' people. Each group was asked to rate a building from 10(excellent) to 0(awful). The architects gave the Disney HQ in Burbank CA a score of 4, the joe publics gave it 7. The Stockley Park Building B-3 got over 6 from the architects and less than 5 from the public, and the Old Bailey got 5 from the architects and 7.5 from the public.

Measuring the success of an architecture can only be done by testing it against its requirements. Peer review could provide some keen insights, but at present, understanding someone else's architecture is a fairly heavy undertaking. People and architects can both judge the architects' output by walking around a building and giving it a score based on their impression of it. We must determine a way to be able to document and consider one another's architectures.

References

1. The Royal Institute of British Architecture (RIBA) www.architecture.com 2. The Seven Habits of Highly Effective People. Stephen Covey. Franklin

Covey Co

3. The Power of Focus. Jack Canfield, Les Hewitt, Mark Victor Hansen. Health Communications

4. The Role of the Architect in Software Development. Dana Bredemeyer, Ruth Malan. www.bredemeyer.com

5. New Scientist, 21st April 2000 6. The journal of Environmental Psychology (Vol21 p93)

Chapter 15 - Patterns and Antipatterns

Patterns

A pattern is a repeatable, documented method that can be applied to a particular problem. There are many types of patterns; we have algorithms, which are essentially code patterns, analysis patterns, design patterns, architectural patterns, organisational patterns and process patterns.

The origin of patterns is open to debate. I would argue that patterns have evolved along with us, and evidence for this is found in every archaeological dig. The fine dividing lines we place between a dinner plate, tea plate, saucer, cup, mug and bowl have evolved over millennia and are all the application of a design pattern to a problem. You can eat your dinner off a saucer, but a plate is better. You can drink off a plate, but a cup is more useful.

The patterns of ancient history are design and process patterns. For example, if I want to hold a lot of water and carry it up from the river, I know of three designs - let's call them design patterns - which will help me to do so:

Design pattern 1 - Yoke, also known as Seesaw

Use two bowls suspended from the ends of a piece of wood. Carve the centrepoint of the wood to sit comfortably on your shoulders.

Design pattern 2 – Head balanced bowl

Use a single bowl, and shape it so it sits easily on a cowl of rope on your head.

Design pattern 3 - Waterskin

Use tanned animal skins to form a flexible water container.

They are referred to as design patterns to make them more specific than just patterns. They are solutions to the problem of bringing up water from the river. Each of these design patterns has associated process patterns.

Process pattern 1 - Wood carving

Select a piece of seasoned wood. Mark out the shape required, and cut it with mallet and chisel. Use an adze to finish the shaping, then sandpaper it smooth. Finish the wood with oil to preserve it.

Process pattern 2 - Skin sewing

Clean the animal hide, stretch and tan. Mark out the shape using a waterskin pattern (see - they're everywhere) and sew. Cut away excess material. Apply resin to all stitching then turn inside out and attach carrying loops to the open end.

In Psychology, we have behavioural patterns, many of them used against the breakers of our laws. The laws themselves are patterns for correct behaviour, and deviation from them may result in action against you, depending on your luck. The processes by which they are applied to you are also patterns. If you do something wrong, your family and friends will stand against you. If you do something very wrong, the law of the land will stand against you. You will be judged to have committed a crime, and subjected to trial and punishment.

Thou shalt not kill. Do not speed. Stop at the red light. Don't drop lollipops on the carpet. Don't sneeze in someone's face. If you hear a siren, pull over; someone is responding to an emergency and should not be impeded. These are all behavioural patterns we are conditioned to follow.

Design patterns give us choices. If I need to get fuel into the engine of my vehicle, I can apply one of two design patterns. I can use a carburettor (carburettor pattern) to premix the fuel and air, or I can inject the fuel straight into the cylinder heads (fuel injection pattern). When vehicles are designed, many design patterns are applied to them. For example, I shall use the pattern of a family car. That means I shall carry four to five passengers and the steering wheel will be placed at the front on the correct side for the country of use. I shall have an engine at the front, luggage space at the back. My passengers will sit either in the front seats or back seat. They will gain entry through doors which open with a handle. All of the doors will lock to prevent unauthorised access. My windows will go up and down to let in air. I shall have a sunroof. I could carry on for quite some time with these design patterns, and indeed should do so.

By following patterns, we conform to a standard set of designs which have been proven to work. I use knowledge already existing in my industry to save time, and to be able to communicate my ideas better to my collaborators. When I deviate from a pattern, I must have a good reason for doing so, for it is less likely to please, less likely to be understood, and more likely to cost more to develop.

When designing a software system, the same arguments apply. I take the cherished patterns of the software community and apply them to my software. The process of design is first to decide on the type of system, then apply design patterns just as I did for my car.

When I build my car, I decide to build it on a production line. First I assemble the components I can buy in. Next I create production lines for the bespoke components. Then I put together the components and create a vehicle.

When I build my software, I decide to build it in accordance with my software development process. I buy in what I can. I set up projects for the common

components. I set up projects for the bespoke components, and another project to manage integration and deployment.

Patterns are already working for us in treeviews, listviews, web pages and applications such as word processors, spreadsheets and 3D modellers. Each of these applications is a solution to a behavioural pattern in itself. If I want to present printed data, I use a word processor or desktop publisher. If I want to present data in grids and graphs, I use a spreadsheet.

The people who designed these applications applied design patterns to them. The applications communicate with the operating system and network using software based on patterns, and the operating system and network are themselves designed using patterns. The patterns were created by invention, iterated through subsequent abstractions and evolutions, and are now in the state where we would be foolish to design our systems without benefiting from the knowledge and experience they encapsulate.

You may be wrestling with an intractable problem right now which someone else has already solved. Not only have they solved it, but their work has been criticised, reworked, improved and immortalised as a pattern. You could pick it up, write it into your software, and you will have used a design pattern and saved yourself a headache.

A pattern example

Patterns in building architecture are closely related to our behavioural patterns. For example, if I enter a cinema or theatre, I expect to get through the door and find where I have to pay immediately on the other side. Architects use this as a design pattern. The pattern might state: In theatres, cinemas and other auditoria, the person entering the building should find where to purchase or collect tickets immediately, even in the event of there being no queue.

Thus the pattern is a solution to a problem in a given context.

Context: Auditoria. Problem: ensuring people can find the ticket booth. Solution: Put them right behind the door, and clearly visible.

Patterns create expectations

Have you ever come across a door with a handle that needed to be pushed to open? If so, you probably pulled it, vented steam, and then pushed it. The reason for this is that there is a general pattern to doors. If you are meant to push it, there is a push panel. If you are meant to pull, then there is a handle.

We are conditioned to push and pull doors according to their panel/handle configuration. Those doors which open both ways, in hospitals, and bat wing saloon doors where cowboys come face to face with the man who shot their pa, have push panels on both sides, or nothing. They do not have two handles. you are not expected to pull.

So when you come across the push door with a handle, the pattern of expectation is that you pull. Annoying isn't it?

Antipatterns

Antipatterns are not like antimatter. They do not annihilate patterns, leaving a tract of residual energy, flash of light, or world ending chain reaction.

Antipatterns are simply patterns which are the wrong way to go.

A cycle with five wheels is an antipattern. There is no point. A kettle with a twin spout might be useful for pouring two cups of tea at once, but it would make it difficult to pour only one cup. Hence, a bad design, a pattern which should not be followed. Antipatterns are bad practice; they are the mistakes we have made and learnt by.

A Pattern Language

Once we learn to speak in pattern language, we are immediately cleverer. We are speaking from a higher level of knowledge, built upon the foundation of those who tried and failed, those who tried and succeeded, and those who took the successes of others and improved them. To speak in pattern language, we must also know that those with whom we converse are also as fluent, and therefore as knowledgeable, as we are.

Alexander's A Pattern Language[1] is where design patterns in building architecture were first formalised. They came to the software world in the late 1980s when Kent Beck and Ward Cunningham wrote up the first software pattern language paper[2].

In software, Design Patterns[3] has gained a pinnacle all its own, and is commonly referred to as the GoF (Gang of Four - after the four authors) book.

Pattern names

Patterns have names so that we might identify and discuss their use. You will see pattern names like pipe and filters, singleton, and antipattern names like bad is better. Occasionally, a pattern will have a number after it, pattern (183). This notation was originally used by Alexander[1] to number and group his patterns. The GoF[3] approach was different, they used the number to refer to the page number on which the pattern was presented. Hence Decorator (175) in the GoF book means the pattern decorator begins on page 175. Alexander's Greenhouse (175) is his 175th of 253 patterns.

The GoF approach has been carried forward into the POSA[4] books. In the context of a book it is a very convenient pattern. Outside that context, it becomes a numbering antipattern.

Alexander's pattern example

As a brief example of Alexander's pattern language, consider a world without windows. It would mean there was nothing to look at on the train or plane. We would have dark rooms and no view.

We are naturally attracted towards light and views. Hence, one of Alexander's patterns is the window place. It is the 180th pattern in his book, and is referred to as Window Place (180). His patterns can be consdered object-oriented as they can be contained in other patterns, and also other patterns may contain them. Thus, they have a hierarchy which is more clear in the building patterns than in software patterns, but perhaps only because of our familiarity with our needs and our surroundings.

180 Window Place

Can be contained by

Entrance Room (130), Zen View (134), Light on two sides of every Room (159), Street Windows (164).

Problem

We crave the light and are drawn towards it. Thus, we are drawn towards places where there is light, and if the place where the light is does not allow us to linger there, it becomes antagonistic. We love window seats for the view and the light they provide. We like bay windows, and big windows with low sills and comfortable chairs where we can sit comfortably and gaze upon the view.

Solution

In every room where you spend any length of time during the day, make at least one window into a window place.

Context

Human habitation

Can contain

Alcoves (179), Low Sill (222), Built-In Seats (202), Natural Doors and Windows (221), Deep Reveals (223).

Does your house or place of work have a window place?

Architectural Patterns

Patterns that can be applied to software systems at their highest level can be considered architectural. They are also known as software architectural styles, although it is far more common in software to mix these architectural styles than it is in building architecture. We shall begin with some familiar patterns:

The Client Server pattern separates presentation from data. Data is persisted in a database, extracted using stored procedures, then passed to a client where they are viewed and changed. This is a rather tired pattern now, having been superseded by the following two.

A Thin Client system has all data processing and persistence on one or more servers. The client is a moderately dumb terminal, capable only of limited presentational functionality.

Layers provide a logical structure for segmenting applications into layers of abstraction or responsibility. A typical example of this is creating a layer of user interfaces which call into layers of business logic, which themselves call into layers of data manipulation, and finally into a layer of data.

Operating systems are also designed in layers as in the OSI 7 layer model described previously.

Tiered systems are logically divided into hardware partitions. Layers are sometimes grouped onto the tiers, and at other times have a one-to-one mapping across the tiers.

Object oriented solutions have long provided better building blocks than purely functional solutions. The world of objects is one of collaboration, data hiding, inheritance and polymorphism.

A Distributed solution relies on more than one server to process a piece of work or a transaction. It may also be used for fault tolerance or load balancing where the distributed parts take over the work of a failed or overburdened unit.

An Event Driven system responds to particular system events rather than processing events in order.

Batch systems are where huge updates are done periodically. Without batch processing, normal processing could be interrupted or put on hold for long periods of time. Batches include such things as global data replication. Batch may well contend for the honour of the oldest software pattern.

Pipes and filters is very useful for processing data. Each step in the process is a filter, and data is passed between the filters by pipes.

Publish and subscribe is a magazine type of pattern. First, information is published. Then it is posted out to its list of subscribers.

Anyone who has used IBM’s Smalltalk or Microsoft’s Visual C++ with MFC will know the Model View Controller pattern well, as this is how the products work. C++ uses a Document/View notation, where document is the same as model, and controller is the application. Data is held in the Model/Document, and displayed in one or more views. When data is changed in one view, all other views are updated by the application or controller. For example, if you have two views, one a grid of data, and the other a graph, then updating the data in the grid will automatically update the graph.

The idea behind the Microkernel is adaptability to changing system requirements. A minimal functional core is separate from extended functionality and customer specific parts. The microkernel also serves as a socket, into which the extensions are plugged and coordinated.

Software Application Patterns

Any product in our vast range of applications which has been repeated by others is a pattern.

Spreadsheets, word processors, databases, email systems, vector drawing, bitmap drawing, software modellers, 3D modellers, html editors et cetera ad infinitum. They are all application patterns.

Analysis Patterns

Analysis patterns are those discovered during domain analysis. The primary text in this field is Analysis Patterns[5], which discusses organisational structures and business models.

As an example, let's take the subject of resource allocation. Resource management is part of project management. It is a problem for the development manager (development people resource), test managers (test people resource, testing hardware resource), hospital ward managers (bed resource, nurse resource), hospital administrators (doctor resource, operating theatre resource) and bank managers (money). Everyone who manages, manages a resource. Utilising resources fully and avoiding resource conflicts is one aspect of management.

Fowler[5] gives us a basic resource pattern:

Figure 15.1 Resource Allocation (from [5] figure 8.13)

This shows that both Proposed Action and Implemented Action have the same basis in Action. A proposed action books a resource of a specific type with a specified quantity. Then the implemented action uses that resource. Proposed and implemented actions can be linked or separate.

Returning to our Stationer's system discussed during design, a resource might be a pencil. It is proposed (ordered) and implemented (delivered). The pencil resource must then be topped up again from the Stationer's supplier.

The resource type is a pencil. The quantity is one. The resource allocation is a line on an order.

This is an example of a resource being consumed. Buying two cups of tea from a cafe could use the same model. In fact, almost any purchase or use of a resource could use this pattern. Which is why it is a pattern in the first place.

Now let's turn our attention to a non-consumable resource. This includes things like people, rooms and video projectors. The model of figure 15.1 requires expanding to include time periods. It must also allow us to book a specific resource or a general resource. As an example, I want me and Arthur Sixpence in the meeting, but I do not require video projector serial number 2283650902374. Any one will do. I also do not mind which meeting room I get as long as it can hold two people.

Figure 15.2 General and Specific Resource Allocation with time periods (from [5] figure 8.14)

This is not a design for a system. It is a model of just what is required to be able to book a meeting room and video projector for two people at a specific time. Of course, it is more general than this and could be applied with equal success to almost any resourcing system managing specific and general resources over time.

It is also a reasonable model of a software development department. A general resource is an analyst, a programmer or a software architect. A specific resource is Tex Tully: Software Architect, or Murray Gell-Mann: Quark Namer. Each of these resources can be booked to one project after another for specific time periods. Each person can be booked as a specific or general resource, as long as the same person is not booked as both at the same time. Should conflicts occur, then bookings for general resources can be shuffled, while specific resources are left fixed.

This is just one aspect of analysis covered in Analysis Patterns. There are many more fields and methods of interactions to discover. Once they are all uncovered and documented, perhaps we will be able to converse using a pattern language in the way people run their businesses or organise their domains.

In summary, analysis patterns are generalised templates for the interrelation and interactions of elements of a domain. Most often, that domain is a business, and the interactions will be the organisation of people and assets.

Design Patterns

Design patterns are more numerous by far than all other styles of pattern. It was in design that the pattern movement in software was born. It came out of the object oriented movement, and the patterns first introduced were in the creation of object oriented solutions.

One of the goals of object-oriented software was reuse. The promise was that we would be able to buy, swap, integrate and evolve components to do almost anything. We're still a little short of goal on this one, but design patterns, hovering above the actual code and objects, do allow us a more manageable form of global reuse where we do not have to worry about licensing and support.

You have already come across one software design pattern in this book, discussed during modelling. Objects were introduced as entity, boundary and controller objects. Logically enough, this is the entity, boundary, control design pattern.

What kind of size are these patterns compared to other software axioms? I have used a few other words in the lexicon of software such as system, framework, application and algorithm. Let’s put them together and see where our patterns fit in.

Software products Process Project Conceptual size

System Architectural pattern Framework, toolkit Application Design pattern Algorithm Function Code

Software process Best practices Coding standards

Programme Project

Bigger Smaller

Figure 15.3 the relative size of software terms

Brooks[6] states:

Conceptual integrity defines ease of use.

If this is so, you will be glad of a pattern or two, where their conceptual integrity is very high, having evolved over quite some time. The likelihood of you coming up with a better solution than an evolved pattern is probably zero. When you plug together these patterns, you are also able to drill into your designs by isolating certain methods as patterns, making the whole far more conceptually simple than the bag of code it would otherwise be.

During your design, you may find a gap in the patterns market. You might create your own pattern. Others may improve upon it, fine tuning it and abstracting it into other problem areas. Thus, it will become a more general and more useful pattern. By inventing your own pattern, you will name it, explain it, and be remembered forever in the unfolding history of software.

Creation then naming is a pattern in itself. When, in my tempestuous youth, I was a rock climber, I climbed a particular route up a crag which had not been climbed before. Having done so, I got to enter it in the register (a book in our local climbing shop), name it and give it a grade. It was, in the lexicon of rock climbing, a new route, and went something like this:

Climb: Treebeard the happy rabbit First Ascent: Nigel Leeming (solo) Date: 26/6/91 Grade: E1, 5b Length: 7 metres Location: Simonside North Face, Northumberland Description: Between climbs 79 and 80. Step left onto the ledge at the start of Thunder Crack and climb the right hand side of the bulging, sandy arête.

Figure 15.4 The recording of a new route

Others have climbed it since, verifying its grade, location and description. They may have climbed it better than I, changed the grade and even the description when publishing it, but it’s my route. That’s the pattern we follow. Similarly, patterns in software need an author and a catalogue. What the patterns movement needs more than anything is a pattern for organising patterns.

The GoF divided their design patterns into three groups: creational, structural and behavioural. We shall look briefly at one of each. I do not intend to go into

design patterns in great detail. The GoF book contains 23 patterns, which is around 12 pages per pattern. The POSA books dwell even longer on a single pattern, and it is in GoF, POSA and PLoP[7] that I suggest you go for more information. In the meantime, here is a brief overview:

Creational Design Pattern

Singleton

A singleton is a class which only ever has one instantiation. There are many examples of this design pattern in schedulers such as print queues or file system manipulators. If there was more than one of such objects, they would be vying for control over files and printers and each would prevent the other from working properly.

If you make your killer application a singleton, then only one instance of it will ever be loaded into memory. It will be able to load and control multiple documents, but there will only be a single instance of it controlling and displaying the documents. The singleton approach means that when a new document is opened, it is loaded by the existing application instance and loads far quicker than loading another instance of the application along with the document.

Figure 15.5 Singleton

Structural Design Pattern

Flyweight

When people began writing word processors, the text entered by the user was held as an array of 8 bit ascii characters. Then computers became a little more complex, and as well as being able to enter text, word processor users could also format it, albeit to an extent limited to italic and a few colours. This was handled largely by inserting formatting commands into the ascii stream.

As character based operating systems waned, we were faced with more control than ever with the introduction of a selection of fonts. Not only could we select our fonts, but we could select a style of underlined, bold, italic and so on. They were also proportional, scaleable fonts and the option of holding formatting codes in the ascii stream led to ever increasing size. Increasing size, in turn, meant a longer rendering time. Size ate up scarce memory, and more complex rendering ate up clock cycles.

But what could a poor programmer do? Picking through the machine code of a competitor was laborious, and possibly illegal. Hunting through the ever expanding open source word processors for a better way could take months.

Out of this confusion came two things. First was the need for documenting the best way to handle this problem without infringing copyrights, i.e. in design patterns. Second was the creation of a method of handling the ever increasing formatting options without reducing performance, namely the pattern Flyweight.

Whether the choice of the name of a boxing category is appropriate, or whether the name fits the task is as irrelevant as questioning the name of Treebeard the happy rabbit as a rock climb. It was chosen, we understand what is meant by it, and will continue to do so.

Flyweight is based on a piece of research where word processing documents were analysed for their use of formatting. What was found was that, on average, word processing documents contained no more than a few hundred differently formatted objects.

For example, since the words Figure 15.5 above, I have used nearly two thousand characters. However, I have used only a few different characters. I have used a space, comma, full stop, new paragraph, question mark and most of the lower case letters, but only a few upper case letters. In all, with the bold and italic characters, I have used less than 80 differently displayed characters. If I ignore kerning pairs, I can recreate the above text with a set of 80 objects, each with a render() function to draw themselves. Obviously, I call some of those objects more than others. Italic capital T gets called only once. Each render repositions the cursor ready for the next character.

Each of these objects is a flyweight.

Now the whole book contains a few more character formats, such as bold, bold italic, grey text, title bars etc. There will be a few hundred different formats in all, i.e. a few hundred text objects to render this complete document of getting on for a million characters.

I shall not explore this situation mathematically, for I’m sure you can see how I could render the entire document like this:

For each characterPointer cursorLocation = characterPointer.render(cursorLocation); Next characterPointer

The characterPointer is in fact a pointer to a Flyweight object. The Flyweight holds the format of the object, i.e. the font, colour and other embellishments, and a render function. To render as quickly as possible, the Flyweight object could even contain a pre-rendered bitmap. Drawing would then be a simple bit block transfer (or blit) from the object to the screen.

Flyweight is a fairly simple concept to grasp. The GoF explain the word processor scenario above in more detail.

I once had the pleasure of writing a terminal emulator and after months of trial, change and retrial, I ended up with a flyweight factory (of sorts) which pre-rendered all possible character combinations (there weren't that many). It was the sleekest terminal emulator on the market, and I was justifiably proud of having conquered the task. Had I known about the Flyweight beforehand, I would have written it in two months less. I spent two months of my life consumed day and night in just this problem, reinventing the wheel. Then, I felt right grand. Now, I feel like I should have spent those two months learning another few design patterns to save me more months in the years which have slid between then and now. May the Lord and patterns save me from such waste ever again.

Figure 15.6 Flyweight

Flyweight uses the factory pattern to create and maintain the collection of flyweights.

Behavioural Design Pattern

Chain of Responsibility

The chain of responsibility is used in help systems and error handling. The approach to both is the same. In error handling, if an error is not handled in the routine causing the error, it is passed up the calling chain until the error is handled. If no effort is made on the part of the program to handle the error, the operating system will shut the program down. That is, it will crash. Thus, the call stack is the chain of responsibility, and shirking the responsibility will result in termination.

In context sensitive help systems, a user requests help for a particular widget. If the widget has no help available, the user will be presented with help for the widget's owner, or the widget's owner's owner. If no context is found in the help system, help for the application will appear. In this example, if responsibility is not taken for context sensitive help, then general help is offered.

Each of these passes a request for action up the chain of responsibility until something is done about it. Similarly in the army, a request for action is passed up the chain of command until someone with the appropriate level of authority can choose to do something about it.

Figure 15.7 Chain of Responsibility

Combining patterns

The Abstract Factory pattern is used when information about which class to create at run time is not known at design time. The creational call is sent to the abstract factory, which is then responsible at run time for knowing what to create.

When an application prints hard copy, it does so without knowing at design time which printers will be installed on the system. Even so, it sends out the print any way, and leaves it to the operating system to connect the call to a printer.

Figure 15.8 The Abstract Factory pattern in action

Figure 15.8 shows two abstract factories, one the printer selector, and the other the print to file selector.

This is a fairly simple example to demonstrate a single pattern. I shall now improve the design by using other patterns.

Pattern 1. I shall use a Proxy to provide a common interface out of the print selector and into the printer drivers.

Pattern 2. I shall create my proxies using a Factory. Each proxy will extend the abstract Proxy class.

Pattern 3. I shall use a Bridge to link between this operating system and another where a remote printer resides.

Pattern 4. As already mentioned, the Printer selector is a Singleton.

The proxy pattern is used to separate the concerns of the printer driver from the print request. Doing so will make my design and ultimate system cleaner as I will be coding different features of the system in different classes.

Figure 15.9 Design patterns used together

The value of designing this way is to identify the features of a system according to what has worked well for others before. By using an object oriented approach, and object oriented design patterns, the structure of my design is immediately familiar to those who speak the pattern language. Others can criticize my design immediately without having to dig into layers of UML models and code.

Organisational patterns

Where are you in the grand scheme of your business? How is it that you interact with other members of your business and customer to create your great works of software? How does the business's actions best support the creation of excellent software?

Is the way that you are structured and work the best way?

This is a question that has bothered me greatly of late. I began my working life in engineering, where software was written by engineers. I then moved into electronics, where software was written by electronics engineers. Then I moved

to robotics where software was written again by electronics engineers. Then I came into the business world, where software was written by software developers, or as the business saw it, people in IT. After that, I worked in a software house where software was written by dedicated software developers, writing for business users, their customers. Now I write words which detail software to be written in a business environment. We are led by business type people, and have a huge hierarchy in which software developers (coders) are only a small part.

All of these environments and structures of reportability have been completely different. I have noted on my various travels around companies that their accounts departments are all similar, as are their HR departments, and their canteens. Why then, are software development organisations so hugely different? Is it because our industry is still in its teenage years, battered by hormone and indecision about what to be or where to go?

Organisational patterns are the platforms for discussing these differences.

Figure 15.10 The Virtual team

Figure 15.10 shows the Virtual Team pattern. The hierarchical reporting structure of an organisation is shown as like reporting to like until the specialist managers all report to a more general manager. When it comes to creating software, a virtual team is assembled from the different disciplines to work on the project.

Organisational patterns are not limited to the organisational structure, but extend well into how the structure works to create software. In doing so, organisational patterns begin to merge with process patterns.

The Code Ownership pattern recommends code is owned by a specified individual or team. By declaring this as so, the ownership forces certain constraints on the owner. They must define and support a set of interfaces into their code. They must keep it clean and version controlled. They must love it, nurture it, and support it from attack by malevolent others. Hence code ownership keeps code in a better condition than it would otherwise be. It is not a contradiction to the principle of extreme programming, where everyone own all the code, it is merely an alternative pattern that will work in a different environment.

You must mix and match organsational patterns just as you mix design patterns. You may choose to match code ownership with Rotation, by rotating code ownership around teams. This ensures knowledge is spread between development teams rather than remaining as a black box in another.

Organsational Principles and Patterns[8], discusses a number of organisational patterns under the umbrella of VRAPS (vision, rhythm, anticipation, partnering, simplification). Each of these five principles provides a home or hierarchy for the patterns.

Process Patterns

The large scale process patterns of RUP, OPEN and OOSP have been discussed already in some detail. Beyond this, lower level process patterns are concerned with the best (and worst) ways of facilitating the various process steps and disciplines.

Process patterns cover how to make the process succeed, and how to communicate with customers in reviews and requirement gathering workshops. They can also help smooth the way between the steps of process.

Brainstorming is a useful way to create the beginnings of a vision. Such exciting work is followed up with workshops to further flesh out the loosely stated vision.

You may not consider these activities as patterns as they are part of everyday software development language. Yet they are patterns. They are patterns understood beyond software development, and using the term brainstorming can conjure up images in other's minds so you do not have to further explain what the brainstorming meeting is for. That's what patterns are for, and familiarity is the goal of the patterns movement.

Data Patterns

Data Patterns[9] discusses patterns for containing data. Anyone designing a database will have come across link tables to maintain cross references between tables of people and departments or products and suppliers. Similarly, you will know that the proficient use of indexes make stored procedures run faster, and lookups take less time. All of this best practice database design, just as best practice software design, is the subject of data patterns.

Consider the following problem: I wish to maintain a list of people in my business. To do so, I create the following data table:

Figure 15.11 People table

Later on, when my business has expanded, I add a column for Department. Now I have a problem of what to do when someone moves department. Do I overwrite the department record, or do I hold a historic record showing a record of which departments people have worked in?

There are two methods for holding the historic data. One is to create historical records in the same table, and mark the live record with a current record marker. The current record allows me to quickly extract the current information from the historical information.

Figure 15.12 The People table holding historic data

The benefit of this method is that I have not increased the complexity of my data, and can look up current and historic information from the same table. I shall refer to it as the CurrentRecord pattern.

It is a cheap and cheerful technique, fine for a small amount of data. But small amounts of data have a habit of becoming large amounts of data. If this is the case, a better pattern is to hold historic information in a History table. This allows efficient lookup and joins on the current data, but access to the full history of the People table through the larger and more awkward PeopleHistory table for the less frequent times it is needed.

Figure 15.13 People and historic data

Code Patterns

There was a brief early sojourn into the world of code patterns, but it became eclipsed, and has continued to be, by the design patterns.

Code patterns are algorithms and language constructs. It is enough to say here that there are good ways to write code, and bad ways. The good ways are patterns, the bad ones antipatterns.

User Interface patterns

User interface patterns can be divided into two classes, one presentation patterns which portray presentational methods, and the other interaction patterns that define how the user interacts with those presentations.

Presentation patterns

You are no doubt familiar with presentation patterns. A treeview is a pattern for displaying and drilling down into hierarchical information. An expanding menu is a way of letting the user at lots of options without hogging screen space. Listviews give the user options for viewing lists as icons or text items. Menu/Detail allows lists or trees on the left to give access to detailed information on the right. Header/menu/detail is often used on internet sites. It is so common now that sites without a menu down the left look odd. This is a good example of an evolutionary pattern in that it was created, improved through successive evolutions, and is now ubiquitous.

The success of a presentational pattern is how well it matches the user's conceptual model.

Interaction patterns

These are applied to user interfaces to conform to a standard way of doing things. Cut and paste, undo and load, edit and save are common interaction patterns.

The wizard allows users to traverse through a card by card display of a complex process broken down into understandable chunks. Context sensitive menus

allow users to popup a menu containing shortcuts for activities related to the one they are currently undertaking. Progress indicators allow us to see how much longer something is likely to take. Previews provide us with a view of something before we commit to it, i.e. print previews and file load previews. Hints displayed when hovering over items are expected, as is a context sensitive cursor. Often, we take these things so much for granted, that when they are absent (especially the progress indicator), using the computer becomes a painful and frustrating act.

Antipatterns

As a whole, antipatterns are more interesting, even darkly humorous on occasion. Social antipatterns are drug abuser, criminal, vagrant, fool. Software antipatterns are vendor lock-in. spaghetti systems, design by committee and the most famous and popular antipattern of all, reinventing the wheel.

Vendor lock-in means a specific vendor has such a hold over your software systems that they can increase prices and change conditions as they please. You have no alternative, so no leverage to change them, and no way out.

Design by committee creates a product of such compromise that it is bland, unclear and largely useless.

As a Software Architect, you should recognise instantly when you come up against an antipattern. Not only that, you must be able to counter it with an impeccably reasoned and thought out response. Your reply will be the problems raised downstream of the antipattern, and an explanation of the benefits of the pattern which should be used instead.

Spaghetti systems are largely the creation of time. A system starts out with a few links between disparate systems. Gradually, the links increase one at a time until the links number the factorial of the separate systems.

Figure 15.14 Spaghetti code or spaghetti systems

Spaghetti can be found in code or in systems. The solution to the problem is creating a broker. This is similar to using a broker to buy and sell stocks and shares. You sell to the broker, the broker sells to anyone else in the world who wants to buy. Similarly when you buy from a broker, you don't care where it comes from. The broker is responsible for getting the shares.

The broker pattern is useful for maintaining a centralised set of data. Many services and systems provide data and demand data, and creating a broker to maintain the central repository (using the publish and subscribe pattern), means there is no spaghetti.

Bad is better

As pointed out in Will the Real Architecture Please Sit Down[10], Software Architecture is itself a pattern. It is the solution pattern to the antipattern Bad is Better.

This classic software antipattern has grown out of the need for fast delivery. Fast delivery requires rapid methods. Rapid methods deliver software faster, at the expense of completeness. Bugs are ironed out during use. It is better from the point of view of the business to have this hastily delivered software because business gains can be made if the software is in place quickly, and business gains are good. Thus, the software is bad. It is better to have the bad software to exploit a new market. Of course, this was not the vision of the business, they merely expected reduced functionality, not reduced quality.

Hastily delivered software forgets about reuse, system design or documentation. It becomes focused solely on fast delivery. The faster the better.

Unfortunately, fast often means worse even if that was not the intention, hence bad is better.

Software architecture is the solution to this antipattern as it takes the ideals of fast delivery, but delivers within a process, and within a system structure that provides reuse and sharing.

eXtreme Programming is also a solution to this antipattern. Both extreme programming and software architecture will work providing the environment is conducive to them working.

Analysis Paralysis

Sometimes delivering bad software can be better than delivering nothing at all. Some projects get stuck in the analysis phase. You can analyse a situation forever, and never get to the bottom of it. Every layer you uncover has its own set of worms to untangle. Eventually, you become stuck in detail and conflicting statements and there is no way out. This is analysis paralysis.

The solution to this problem is to extract the stable requirements, ignore all volatility, and deliver on the stable core. The core may then be used to force crystalisation of the previously insoluble problems.

The Hero Programmer

Once upon a time, when Amalgam (a mixture of powdered silver and mercury) was first discovered, a new wave of dentists began filling holes in teeth with this

rather poisonous mixture. The more conscientious drilled out the rotting tooth first and then pushed and padded amalgam into the hole. Others got a good blob of it and pushed it down into the decaying cavity with their thumb.

A growing lobby complained loudly about putting poisonous metals into the mouths of hapless (and paying) victims, and amalgam was blamed for anything from heart attacks to miscarriages. The lobby won. Amalgam was banned in the United States for fifty years. Now, of course, we use it all the time. We know better.

Amalgam was the solution, but was seen as the problem. Because the problem lobby won, the true solution to the problem of decaying teeth was banned.

Hero programmers are the problem. Because they exist, more and more software development moves beyond the control of software developers. Long nights, early mornings, weekends, no holidays. You’ve done it, I’ve done it. It leaves behind it a behemoth of incomprehensible code which becomes ever more difficult to update. There wasn’t time for comments or design. It was typed in and fixed as fast as we could go. But we did it, by God, we did it.

But what did we do? Did it earn us large bonuses? [probably]. Did it create an extendable platform upon which excellent software could be written? [probably not]. Did it work forever without a fix [probably not], or did it need constant support and hand holding to keep going? [probably].

Relying on a hero programmer is a disease. Accountants get it in the neck at year end. Bosses get it in the neck all the time, and like to feed that frustration down, down, down, sometimes down as far as programmers. You see, programmers can go faster. All they need is a bit of pressure. Similarly artists, cooks and ovens can go a little faster. What you get is charred meat, still raw on the inside, cooks more petulant than Petula Petulant, a hurried picture that turns a queen into a frumpled old grumpwort, and yes, worse software. But you asked for it.

The hero programmer antipattern is conquered by communication, communication, more communication and a software process paid more than lip service.

The God class

Many software antipatterns are, in fact, organisational or process antipatterns. Design antipatterns are rather fewer in number than their equivalent design patterns.

As an example, the god class or blob becomes slowly responsible for the workings of most of a system. It is a result not of bad design, but of not having a design. Code gets added to the god class and it becomes ever more godly. Eventually, there are only a few other classes having minor responsibilities and everything of any significance is performed by god.

The solution to this problem is part process and part pattern. One is to use patterns, and the other is to get a better design in place.

The opposite of the god class is the proliferation of classes, also known as poltergeists. This time, there are so many intertwined classes that you can never get to the bottom of them. Every time you look, yet another class appears out of nowhere.

You can also have a mixed god class with a proliferation of classes. This time there is one central class, and hundreds of little ones doing lost of little things. A god and poltergeists are called a corncob.

Too much information

You can often hide information by presenting too much of it. This is evident in any brochure advertising insurance or pensions. There is so much information that you cannot possibly decide which product is best for you without seeking help.

An excellent example of too much information in IT systems is nuisance alarms. A nuisance alarm is one which goes off for a trivial reason. Eventually, the operator becomes so unconcerned by them coming in so often, that he ignores the one particular alarm that identifies core meltdown in the nuclear reactor. This is extraordinarily important information. Incident sunlight on the side of the building passing the 20W/m2 is less so.

Antipatterns are discussed further in Antipatterns[11].

A taxonomy of patterns

In the eighteenth century, a Swedish botanist and physician named Carolus Linnaeus (later Carl von Linné), devised a hierarchical system for naming the five million species on Earth.

Each species belongs to a genus, each genus to a family, and so on through order, class, phylum (or division), and kingdom. The associations within the hierarchy reflect evolution and are deduced from similarities in form and physiology.

At the bottom of the hierarchy, each species is given a Latin name. Each tongue is then free to assign it a common name. For example, the species homo sapiens we call human in English, humain in French, menschlich in German, and as you travel around the world others such as ninjen no and manob. All of these words represent the latin species name homo sapiens.

Common Name

Kingdom Phylum* Class Order Family Genus Species

Domesticated Dog

Animalia (animals)

Chordata Mammalia Carnivora Canidae Canis C. familiaris

Sugar Maple Plantae (plants)

Magnoliophyta Rosidae Sapindales Aceraceae Acer A. saccharum

Bread Mould Fungi (fungi)

Zygomycota Zygomycetes Mucoralis Mucoraceae Rhizopus R. stolonifer

Tuberculosis Bacterium

Prokaryotae (bacteria)

Firmicutes Actinobacteria Actinomycetales Mycobacteriaceae Mycobacterium M. tuberculosis

Pond Alga Protoctista (algae, molds, protozoans)

Chlorophyta Euconjugatae Zygnematalis Zygnemataceae Spirogyra S. crassa

Figure 15.15 A taxonomy of species

I am not suggesting we name our patterns in Latin. I am suggesting we need a structure to hold our patterns (our pieces of knowledge) in. I am also suggesting that Pattern.Code.Mathematical.Sort.Quicksort can be Quicksort.c, Quicksort.java, Quicksort.basic etc. It is a case of same thing, different dialects.

In the current vogue of togetherness, our pattern structure will be designed by committee, and there will be several disagreeing groups with their own flavours. We need a modern Linnaeus. Preferably a non-latin speaker please.

Where can I go for more patterns?

There are two catalogues of software patterns. One set is the POSA Patterns of Software Architecture (now four volumes) and the other is Pattern Language of Programming (PLoP), also in four volumes.

Doing Hard Time[12] lists many patterns particular to real time systems.

The Hillside Group[13] maintains an excellent website for pattern information.

One day, many years from now...

A software developer will be plugging together his patterns along with his UML diagrams to create a piece of software, when suddenly (oh no!) he’ll have to write a piece of code for himself.

An old hack leans over, smiling, dreaming of the day when it still used to be a chaotic bundle of fun and says:

'Old LeMessurier used to say to Architects: Any time you depart from established practice, make ten times the effort, ten times the investigation. Especially on a large project.' That’s what old Booch used to say.

I remember when any old code would do. We used to just sit there and type it in as it came into our heads.

References

1. A Pattern Language: Towns, Buildings, Construction. Christopher Alexander, Sara Ishikawa, Murray Silverstein, with Max Jacobson, Ingrid Fiksdahl-King and Shlomo Angel. Oxford University Press.

2. Using Pattern Languages for Object Oriented Programs. Kent Beck, Ward Cunningham. Technical Report CR-87-43.

3. Design Patterns: Elements of Reusable Object-Oriented Software. Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides. Addison Wesley.

4. POSA series (4 volumes). Pattern-Oriented Software Architecture. Many authors. John Wiley & Sons.

5. Analysis Patterns: Reusable Object Models. Martin Fowler. Addison Wesley. 6. The Mythical Man-Month: Essays on Software Engineering, Anniversary

Edition. Frederick P Brooks. Adison Wesley. 7. PloP series (4 volumes). A Pattern Language of Program Design. Many

authors. Addison Wesley. 8. Organisational Principles and Patterns. David M Dikel, David Kane, James

R Wilson. Prentice Hall. 9. Data Model Patterns: Conventions of Thought. David C. Hay, Richard

barker. Dorset House. 10. Will the real architecture please sit down? Thomas J. Mowbray. White

paper. 11. Antipatterns: Refactoring Software, Architectures, and Projects in Crisis.

William H. Brown, Raphael C. Malveau, Hays W. 'Skip' McCormick III, Thomas J. Mowbray. John Wiley & Sons.

12. Doing Hard Time. Bruce Powell Douglass. Addison Wesley. 13. The Hillside Group http://hillside.net/

Chapter 15a - Show me your Architecture

The focus on architecture has moved us from statements of "We have a client server architecture", or "We use an n-tier/service oriented/distributed architecture", to trying to come up with a pictorial representation of what is probably an extremely complex collection of machines and concepts.

The difficulty we face is that software architecture is an abstract concept. Some portray architecture as layers, others as block diagrams with joining lines or arrows, others again in huge 3d formations, resembling chemical factories and equally as unpleasant to have to look at. All are useful as starting points for discussion, and after some persuasion and adjustment, come to have some meaning within a community of like-minded souls.

Here are two views of a layered architecture, as described in [1].

Figure 1. a) Automation Oriented Architecture stack mapped to business departments. b) The layers mapped to a three tier pattern.

The benefit of the layered diagram on the left is the obvious delineation into areas of specialisation, and even to business departments in the larger layers of Business, Development and IT Support. The layers are further explained in the diagram on the right, which displays a path through the layers with an example of taking an order.

Figure 2 shows a layered architecture, where each of the four layers has been broken down in more detail.

Figure 2. Exploded layer diagram

In this case, more detail has been added to the layers to delineate areas of interest, without getting bogged down in detail.

Figure 3 attempts to show far more detail, including specific technologies, and a breakdown into a five tier architecture. One important aspect of this diagram is its name. A polo is a small round mint with a hole in the middle. The two polos at the bottom are used as a visual reference and memory hook so that a mention of two polos brings this architecture diagram to mind.

Figure 3. Two Polos architecture

This final example in figure 4, shows technologies, business processes, specific applications, and also lists COTS applications and the taxonomy of the enterprise, which a keen eyed viewer may recognize as a consultancy.

Figure 4 Business Process, Technology and Application Architecture

These four figures have a few things in common. One is that they are all highly specialized displays of what is considered important in a specific environment at one moment in time. It may be of interest to note that figures 3 and 4 are different views of the same enterprise. Figure 4 is a where we are now illustration, representing diverse applications tied together by export from x and import into y. Figure 3 is a concept of how it could all be tied together within a unified architecture.

Another interesting point in these four figures is that they have very little in common, other than an attempt to delineate areas of interest. However, a successful architecture must be far, far more than this simplified delineation. We require a full set of blueprints created to a set of standards which are only now emerging.

If security is important, the architecture must contain a security model. If an architecture must support 24x7 operation, there must be a model within the architecture indicating how this will be achieved. If deployment is in three geographical locations in dual-redundant server farms with firewalls (two foot thick concrete ones, not an extra server with a bit of software on), then there must be a deployment model. Only collectively, do these independent models comprise an architecture.

So instead of our depictions of colourful abstraction, what is available to us to present an entire architecture?

Here are some methods purporting to do so. For references, see the end of the chapter.

C4ISR Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance

Includes the Core Architecture Data Model (CADM), a formal model defining the data organization for a repository of C4ISR/DoDAF-compliant architecture products.

DoDAF Department of Defense Architecture Framework

Including the DoD architecture repository (DARS)

EAP Enterprise Architecture Planning

FEAF Federal Enterprise Architecture Framework

IEEE-1471 IEEE Recommented Practice for Architectural Description of Software Intensive Systems

ISO 14252 (IEEE 1003)

Developed into TAFIM, then into TOGAF

MDA Model Driven Architecture

RM-ODP The Reference Model for Open Distributed Computing

SPIRIT The SPIRIT Platform Blueprint Service Providers' Integrated Requirements for Information Technology (SPIRIT) Platform Blueprint is a specification that was developed within the Network Management Forum, now known as the TeleMangement Forum (TMF).

TAFIM Technical Architecture for Information Systems

Based on IEE1003 and superseded by C4ISR

TEAF Treasury Enterprise Architecture Framework

TISAF Treasury Information Systems Architecture Framework

Superseded by TEAF

TOGAF The Open Group Architecture Framework

Includes the Architectural Development Method (ADM)

Zachman The Zachman Framework

Figure 5. The architecture framework quagmire [n]

Instead of detailing each one of these methods, a study of one will prove more valuable. We will look at TOGAF.

TOGAF

The Open Group's Architectural Framework is gaining some momentum. Its purpose, as will all architectural frameworks, is to avoid starting with a blank sheet of paper when creating architecture. Just as with RUP, it contains tried and tested ways, best practices and foundations upon which things are built. It is an industry wisdom consensus, tool and technology neutral method.

A TOGAF enterprise architecture comprises four separate, but interrelated architectures.

1. Business architecture 2. Data or information architecture 3. Application architecture 4. Technology architecture

The Data and Application architectures are referred to collectively as the Information System Architectures.

TOGAF itself comprises three main parts

1. The TOGAF Architecture Development Method (ADM), which explains how to create a relevant architecture for your enterprise. It provides:

o A reliable, proven way of developing the architecture o Architecture views which enable the architect to ensure that a

complex set of requirements are adequately addressed o Linkages to practical case studies o Guidelines on tools for architecture development

2. The Enterprise Continuum, which is a "virtual repository" of all the architecture assets - models, patterns, architecture descriptions, etc. - that exist both within the enterprise and in the IT industry at large, which the enterprise considers itself to have available for the development of architectures. At relevant places throughout the TOGAF ADM, there are reminders to consider which architecture assets from the Enterprise Continuum the architect should use, if any. TOGAF itself provides two reference models for consideration for inclusion in an enterprise's own Enterprise Continuum:

o The TOGAF Foundation Architecture - an architecture of generic services and functions that provides a foundation on which specific architectures and architectural building blocks can be built. This Foundation Architecture in turn includes:

the TOGAF Technical Reference Model (TRM), which provides a model and taxonomy of generic platform services; and

The TOGAF Standards Information Base (SIB), a database of open industry standards that can be used to define the particular services and other components of an enterprise-specific architecture

o The Integrated Information Infrastructure Reference Model (III-RM), which is based on the TOGAF Foundation Architecture, and is specifically aimed at helping the design of architectures that enable and support the vision of "Boundaryless Information Flow".

3. The TOGAF Resource Base, which is a set of resources - guidelines, templates, background information, etc. - to help the architect in the use of the ADM.

The ADM

The Architectural Development Method (ADM), is represented in figure 6.

Figure 6. TOGAF's ADM

Each phase is defined in increasing detail, with reference to the key elements of TOGAF, and how they are created. The structure of each section is the same, detailing the objectives, approach, inputs, steps to be taken, and finally, the outputs to be used in later phases. More detail is available in the TOGAF document or website [2].

The Enterprise Continuum

The continuum itself can be thought of as a box containing all of the architectural assets: documents, descriptions, models, beliefs etc, that an organisation has created.

Foundation Architecture

TOGAF provides two reference models for inclusion in the Enterprise Continuum, the TOGAF Foundation Architecture (TFA), and the Integrated Information Infrastructure Reference Model (III-RM). III-RM is itself based upon TFA, and one is selected as a foundation upon which an architecture is created.

Technical Reference Model (TRM)

Figure 7. TOGAF's High Level and Detailed Technical Reference Model (TRM)

Standards Information Base (SIB)

The SIB[3] is a searchable online area listing IT standards documents and their publishers, with links to those available online.

III-RM

Figure 8. TOGAF's Detailed Integrated Information Infrastructure Reference Model (III-RM)

The Resource Base

The Resource Base is a collation of architectural and business know how and measurements. It contains advice on the following:

• Setting up an architecture board • Ensuring projects comply with the architecture • How to set up contracts between sponsors, developers and suppliers • Guidelines for IT governance • Assessing architectural maturity • Guidelines for using architectural patterns • Principles for the use and deployment of IT • Roles, skills and experience for architects • The use of architectural views • Business process views • A method of extracting architectural requirements from technical and

user requirements • Case studies • Architecture tool evaluation • A comparison and mapping to other architectural frameworks

TOGAF Summary

The architectural frameworks continue to align with each other, and as they do so, gain weight. It is clear that Enterprise Architecture is a large complex subject, differing greatly from one organization to another. TOGAF is well placed to evolve

into the defacto architectural framework, just as UML became the preferred design method.

The drawback, and perhaps the strength, of TOGAF is that it is not prescriptive. It does not give you an architecture out of the box. It is a method to support an experienced architect and would be of lesser value to the inexperienced.

Show me the b!**@y architecture!

The realization may have dawned by now that this is not a simple thing to do, because it is highly dependent upon an organization, and what is deems important.

Some will value business services and processes. A large insurance company's initial architecture would be highly application focused. A new company's architecture may be nice and clean, but the same could not be said for a company which had been investing in IT for the past 30 years. TOGAF's TRM points at a high level grouping of important things. The III-RM has a lot more detail which may or may not be applicable in any given environment.

Architectures are comprised of software, hardware and communications networks. Architectural Descriptions are a collation of descriptions, patterns, models and beliefs.

Any business where people do most of the work, such as the Stationer example of chapter 7, may be most interested in the following views:

• Business services, processes and workflows • Roles • Roles mapped to processes • Applications • Processes mapped to Applications • Application Availability

Robbie the Robot, also of chapter 7, wouldn't be interested in any of these views. He'd want to know about:

• Longevity • Responsiveness • Beauty • Energy efficiency • Fault tolerance

Every item of importance must be included in the architectural description, and here is a taster of the top level architecture document

Bonzo Inc. Architecture

Introduction

This is the starting point of the Architectural Description for Bonzo Inc.

The Enterprise

Bonzo Inc is located in Mumbai and sells products manufactured in house to magic stores worldwide. Production uses just-in-time stock control for raw materials, digitally controlled CNC for metals, injection moulding for plastics, and manual assembly and packing. 100 employees work at their single location and the architecture must support workflow and enable tight stock control.

The Architecture

Bonzo's architecture is based on TOGAF's technical reference model, and extended to fit our particular needs.

Models

Application model Lists all applications in use

Integration model Details how information is kept concurrent across applications

Stock control model Details how stock is kept to a minimum, but sufficient for the manufacturing pipeline

Design and Manufacture model

Details how products pass from design through manufacture

Business Processes and workflow

Defines business processes, e.g. stock of work, and maps to applications

Roles Defines roles, and maps to business processes

Data model Our logical data model for use transforming information between applications

Deployment model Shows which applications are deployed on which servers

The important architectural concerns are

Availability Our presses and injection moulding machines must be run 24x7 to keep costs low

Traceability Our employees are paid according to their output

System replacement

Applications must be able to be switched out so we can buy replacements written and supported locally

Schedulability We want to be able to make everything happen automatically

Each model identified in the standard models and architectural concerns is described in a separate document in X:\IT\Architecture.

Creating an IT architecture can be as simple as selecting a set of applications, installing them on some servers and writing a few batch import and export routines to keep each one up to date.

Or it can be a complex undertaking requiring every inch of an architect's knowledge, experience, skill, insight, inventiveness and careful planning to create an extraordinarily complex but magnificent, well-oiled machine.

The architectural description reflects the complexity of the undertaking. More complex requires more models, more views and more description. Simplicity means that more models and maps can be combined in fewer diagrams. If it all fits in one diagram, that is what you slip across the table when someone asks to see the "architecture". If it is more complex, and is highly likely to be so, you point them to the relevant area of storage, or you give them the thick wad of papers that are collectively what they requested.

So an architecture is just a pile of linked documents showing what is required and how it delivers on high level requirements?

Yes. As long as it can be understood and delivered.

But anyone can do that.

It appears not. A certain capability of abstract thought is required, and not all developers can make the disconnect between concrete and abstract deliverables. [4]

Is there a list of what I might have to focus on in my architecture?

See the next chapter.

How do I go about creating a superlative architecture?

See the next section.

The many habits of successful architectures

Reference models and architecture frameworks do not in themselves help you to be brilliant. They help you be methodical, which often has a more reliable output. To be brilliant is another challenge indeed, and requires a slightly different focus, deeper though, and an ability to weave many abstractions at the same time.

We have touched before on the seven habits of highly successful people[5]. This section reflects on these people habits and applies them to machines. It also adds in a number of habits particular to machines.

The 7 Habits

Habit 1. Be proactive

Push technology is a proactive IT mechanism. A proactive robot will sense when its battery is low, and plug itself into the mains for a recharge. Proactive software will email the CFO when a suspect payment is made rather than wait for him to go and look for it. In a proactive world, when you book a meeting, clever software will book the right room for you without you having to raise a glance at a telephone or finger to a keyboard.

Event driven architecture (EDA) is a proactive architecture. Message Queuing appears the medium of choice, allowing proactive messages to stack up and be processed at the receiver's operational speed. Message queuing's loose connection is much more resilient than relying on remote calls.

In the mist between proactivity and reactivity lie load balancing, failover and workflow based upon operator load.

Habit 2. Begin with the end in mind

An architecture has to grow. It cannot be developed in one project or delivered out of a box. It is similar to a corporate structure, which also doesn't come predefined. In business terms franchising offers an opportunity for structure in a box, which is exactly what the architectural frameworks try to deliver, although with far less detail than a business franchise. The end is a seamless, resilient architecture. Beginning with the end in mind means working towards this goal. One example of an end will be when a worker looks in only one place to see what work they have to do. At present, a particular worker may have to check on a mainframe, take verbal directions from peers and superiors, read through email, looking in a workflow environment, check the post-its on his desk, chair, screen, and finally revisit his things-to-do-list. The end is a single point of reference for things-to-do. The architecture will include work request submissions, authorizations and viewing. To build this in to the architecture requires the addition of a message format, integration technology to allow the message to travel from many systems to a point of collation. Workflow will provide redirection to a supervisor for authorization. Proactive software will balance the workload, and finally a single viewing application will allow the worker to see his things-to-do list. If he adds to it himself, the proactive workload balancing software will not overload him with other's requests. And that cryptic gap in your day will be a fire evacuation practice. This habit has begun to be applied in that written and verbal communications have been overtaken by email. Unfortunately in the interim, technologists have

invented many more lines of communication, such as workflow and texting. Beginning with the end in mind also means looking beyond the enterprise, as communications are not limited to the enterprise. In a supply chain, a worker requests an item from a store. The store, being empty then requests the item from the supplier, who in turn requests it from the wholesaler, and thence the manufacturer. The manufacturer may need parts to produce it, and the part manufacturers will need raw materials, maybe from the mining consortia. With the end in mind, this will be a seamless request, uninterrupted by human. Automated supply chains must be supported by a reflective architect.

Habit 3. Put first things first

IT systems are always prioritizing. Our microprocessors prioritize work, as do our programme managers. An architecture supporting first things first would send highly important messages before less important ones. It would list our single work view in order of importance to the enterprise, while ensuring that important sequences of process are efficiently arranged.

In software development, the first thing is the architecture. Without defining at least a skeleton to support the loftier goals of the enterprise, development will be less than holistic.

Habit 4. Think win-win

Each move in the chess-like world of enterprise architecture must benefit the architecture, benefit the developers and deliverers, and benefit the users. Everyone must win. One team must not win at the expense of the others.

It is difficult in worldly communication to create a win-win situation from opposing ideologies. In IT, we must think win-win-win for the architecture-developers-users. This is possibly the most difficult of all the trials an architect is faced with, and certainly more daunting than the technological challenge. For example, project pressures can circumvent architectural decisions, which leaves the architecture less ideal and possibly less robust.

Habit 5. Seek first to understand and then to be understood

Understanding means speaking the same language. Data needs a common (also known as a logical or canonical) data model. Messages need to use the same protocols, most likely http over tcp/ip. A common understanding of business and corporate terms must exist in a taxonomy or hierarchical ontology. If your software understands, it is able to convert the common data model into its own representation of that data. Similarly, if it wants to be understood, it must convert its internal data into the common data model before sending it out.

This conversion process can be internal to each software application, or handled by an integration layer. An integration layer uses the canonical data model and a set of transforms to convert the sender's format to the common format, then the common format to the receiver's format.

Habit 6. Synergize

Synergizing means getting more than is capable from the sum of the parts. At present, we get a fraction of a percentage of the processing power available to us on the many desktop PCs any organisation owns. Most of the time they sit idle, waiting for the excitement of a keystroke or mouse movement which will take all of one trillionth of a second to process. Then seconds of zero productivity go by until the next nudge of the mouse. At night, they are switched off - their asset value slowly decomposing as they do nothing.

Moving towards possible synergy in this example is to use grid computing, where computational tasks are shared out between all unburdened microprocessors. There is little value in one processor working flat out while many others are idly waiting for its output. Share the burden with grids. Synergy in architecture is a difficult concept to grasp, but may be better understood once we can at least move towards efficiency. Efficiency is created by the careful balancing of users needs, architectural concerns and developmental pressures to create a synergistic group of people, and in turn a useful, future-resilient architecture.

Habit 7. Sharpen the saw

To sharpen the human is to take time out. To rest, to reflect. It is difficult, or pointless, for a processor to take time out as they cannot yet reflect. Sharpening the saw in IT is to revisit every tooth in turn to make sure it is correctly aligned and performing well. Sharpening the architecture is continually revisiting the current and intended architecture and reworking in light of experience, knowledge and new applications or technologies.

Beyond the 7 habits

The 7 habits alone do not define corporate culture. They are for the individual. The corporate world is still not interdependent. It uses (in the main) a tired and fading metaphor of command and control rather than interdependent systems. IT, for once, is a small step ahead in that there is no CEO. The CEO role in Enterprise Architecture is either a shared and distributed force, or it is filled by a group of operators. I shall not delve into the conundrums this could present, nor am I suggesting the CEO role is unimportant! You may also have heard a few more mantras other than those indicated in the 7 habits. Here are some relevant to machines: Be flexible, delegate, manage time and resources better, speak a common language and promote non-formal links. OK, no-one may have heard the last one, but have you ever noticed that the smokers, those who huddle together outside cleansed offices, always know more about what’s going on than anyone else. The reason is that they form a social network reaching across company delineations. The same goes for gym members, local drinkers, or any other special interest group peopled from within an

organization. According to Ray Reagans and Ezra W Zuckerman[6]: Teams are less productive when its network remains concentrated among relatively similar persons.

Habit 8. Be flexible

How many times in Star Trek is the engineering problem solved by rerouting? It seems to bring an otherwise crippled starship back to life. Rerouting will make your architecture miraculous. But how do you do it?

Rerouting is managed through a service capable of farming out requests and rerouting them when required. Imagine the following scenario: A piece of processing work is posted to a rerouting service. The rerouting service has a glance around to look for an unburdened system capable of performing the task. It sends is along with a timeout of five minutes. Half way through the process, the computer performing the task fails. The timeout occurs without response, so the task is routed elsewhere.

A proactive rerouter may even monitor computers it has sent work to, to see if they are still up and running. It may even send the task to another service to estimate the time required for completion, to use that as an indication of the correct timeout value. In addition to rerouting, a service may be designed to be able to restart something, or even give up and try again later. It has to be flexible.

Workflow systems, as they route work requests to people, who may be off sick, or on holiday, or dead, must have such flexibility designed in.

Habit 9. Delegate

Why, in a perfectly load balanced, well distributed system, would anything have to delegate? One reason is that in an event driven world, a machine may be given a task at which it excels. If it’s busy, it should delegate its current task elsewhere. In a load balanced system, delegating back to the load balancer would be the best option.

Habit 10. Manage time and resources

In enterprise architecture, the people, the business architecture and the IT architecture will work most efficiently when they synergize. When machine can delegate to man, and vice versa, within a flexible, balanced system, operating on the first thing first within a load balanced, coordinated workflow, an enterprise will run more efficiently.

Habit 11. Speak a common language

A logical data model and a logical process model are essential in a connected world. People in an organization speak the language, hold the beliefs, and spill

the acronyms of the organization. However, these logical models are becoming domain specific rather than enterprise specific. For example, an insurance company may use the industry standard logical data model Polaris, rather than one created to their own internal beliefs. Doing so means that customers and suppliers can also speak the same language. Speaking a common language in architecture is conforming to standards that exist beyond the enterprise.

Until recently, this was achieved using the import/export functions of data and office tools, or more often by using the same tools (hence the popularity of Microsoft Office) to avoid integration problems.

Architectures and designs can speak a common language by using architectural reference models, views, and modelling notations.

Habit 12. Promote non-formal links

Has anyone seen a computer in the gym, or standing at the bus stop? Possibly not. However, a machine can look at a list every now and then to decide who it is going to delegate to. It may learn, using Bayesian or neural algorithms who might do a job best, and it may share that information with others. Sharing can be centralized, but then two pieces of information cannot exist together. Thus they cannot merge or diverge, and cannot grow genetically. Centralized information cannot hold A is best and B is best. Non-formal can.

To illustrate this concept requires another story. Two computers, A and B, learning with neural networks, decide that C and D respectively, are better at a certain task. A sends to C, believing it to be the best, while B sends to D. One day, D begins to perform slower than B is expecting, so B fires off a request to others asking who performs the same task, and who do they send it to? A replies with C, and B sends to C, which performs the task better than D. B then moves C to the top performer and D to second place. The next day, B sends work again to D, and finds that it performs better. It resends to C, and notes a slower response to D, and moves D back to the top of its performer list.

Human non-formal links are created by behavioural tendencies. In computers, the need to do so is less marked as they are all interconnected through a network. Non-formal in a machine's case mostly means non-centralized.

Habit 13. Don't work hard, work smart

IT projects get cancelled. All of the work done is wasted. Working smart would be (in order of smartness) to cancel the project earlier, to never have started the project, to never plan the project in the first place. Many organizations waste lots of effort because they are not collectively smart enough. In this case, the smartness is being smart about business desires, technological feasibility and delivery capability.

A smart architecture is a level headed, deliverable one. Expecting hard work to deliver the sub-ether faster than light communicator is not going to help deliver it.

When a single processor is running at 100% while others await its output, it is not on the smart task. Delegation is smart, not working hard. The same goes for architects.

Habit 14. Avoid inefficiency

There are three great inefficiencies: entropy, latency and procrastination. Procrastination can be overcome by workflow, human and machine load balancing, prompting mechanisms and a single place where work is displayed to the worker. A procrastinating architect can only be moved by cakes. Entropy is the energy lost to the system, or the work done by a system which provides no output. Latency is the long and often painful wait for a response.

Entropy and latency are the subject of [7].

Habit 15. Follow the paths of others

Scientist have a long, distinguished, tried and tested method of proposing and often destroying theories. The steps of this exercise are: create – peer review – publish – read - improve.

Although IT is a fairly new arena of thought, and IT architecture even newer, there are other disciplines who have waded these snark infested waters before us. Development methods, architectural frameworks, architect organisations and others' designs are all available to us and we are more effective when we refer to them.

Habit 16. Don't put your foot in it

There are certain things in IT that will cause others to avoid you. Here are two:

SPOFS and Bottlenecks

Single points of failure exist in many people and technical arenas. If the CFO is off sick, the large cheque can’t get signed. If the ship's propeller breaks, the ship won't go. If the email server is down, the email won't go out. If the office is on fire, the people will leave.

Usually, dual redundancy is used to avoid SPOFs, but there will still be a single point at which feeds the dual redundant systems. This is a smaller SPOF, but a SPOF nonetheless.

If a spoke on your bike wheel goes, the chances are good that you will still be able to pedal. If a few go, the wheel may begin to wobble, but on you will go. There will be a critical point of failure when the wheel begins to fail, but it doesn’t happen with spoke one. However, if the hub on your wheel goes, you’ve had it.

An IT hub may be load balanced and failovered to avoid the SPOF, but what happens when the hub us overloaded because every single system in the enterprise is shouting at it? It becomes a SPOF and a bottleneck. The stack of messages grows faster than can be processed, thus the bottleneck, and eventually it overflows, thus the SPOF. Architectural methods for avoiding these must be carefully designed. A hub must provide mechanisms for handing off (delegating) or readdressing work. For example, instead of saying “Hub: give me the address for person X”, you would say “Hub, where can I get the address for a person when I have their name.” The hub supplies the location of the service rather than redirecting a request to the service, and frees itself up to fail all it wants. Addresses will still be found even when the hub has failed. Hand off and readdressing make a system highly resilient. If a system asks the question “Hub where do I get…” and doesn’t get a response, it can continue using the address it was using yesterday. All applications written within an architecture that defined such operation would benefit the architect, the systems (hub and requesting application) and the user, a win-win-win situation.

References

1. Business Process Management Implementation. Nigel Leeming. http://www.softwarearchitect.biz/articles/bpm/bpm.htm

2. TOGAF. The Open Group. http://www.opengroup.org/architecture/togaf 3. Standards Information Base (SIB). The Open Group.

http://www.opengroup.org/sib2 4. Why Can't They Create Architecture Models Like 'Developer X'? An

Experience Report. George Fairbanks, Carnegie Mellon University. 5. The 7 Habits of Highly Successful People. Steven Covey. Franklin Covey

Co 6. Networks, Diversity, and Performance: The Social Capital of Corporate

R&D Units. Ezra W. Zuckerman, Ray E. Reagans. Organization Science, Jul/Aug 2001, Vol. 12 Issue 4, pp. 502-517

7. Some Elephants Can Dance. Nigel Leeming. http://www.softwarearchitect.biz/papers/elephants/elephants.htm

References for Architectural Frameworks

1. C4ISR - http://www.dsc.osd.mil/ 2. DoDAF - http://www.aitcnet.org/dodfw/ 3. EAP - Enterprise Architecture Planning : Developing a Blueprint for Data,

Applications, and Technology. Steven H. Spewak. Wiley. 4. FEAF - http://www.cio.gov/archive/bpeaguide.pdf 5. IEEE-1471 -

http://standards.ieee.org/reading/ieee/std_public/description/se/1471-2000_desc.html

6. ISO 14252 (IEEE 1003) - http://www.iso.org/iso/en/CatalogueDetailPage.CatalogueDetail?CSNUMBER=23985

7. MDA - http://www.omg.org/mda/ 8. RM-ODP -

http://www.dstc.edu.au/Research/Projects/ODP/ref_model.html 9. SPIRIT - http://www.opengroup.org/publications/catalog/sp.htm 10. TAFIM - ? 11. TEAF - http://www.ustreas.gov/offices/cio/ 12. TISAF - ? 13. TOGAF - www.opengroup.org/architecture/togaf 14. Zachman - www.zifa.com

Chapter 16 - Enterprise Architecture

The world of Enterprise architecture is one of wrestling with the network myths. It comprises the architectural concerns, how they are contained and delivered within frameworks and represented in views.

It is about architectural patterns, architecture documents and architectural insights.

You must understanding why the unit of release is the unit of reuse, how architectural vision is created and upheld, how to meld two disparate systems when companies merge, how to get from a legacy system to a new system without rewriting the whole lot at once, and how architecture will help a business reach for its goals.

The network myths or the Eight Fallacies of Distributed Computing

Deutsch[1] claims that everyone who first builds a distributed application makes the following assumptions, all of which turn out to be false.

The network is reliable

Usually, the network is reliable. It is, in modern terminology, up for a significant percentage of the time, usually measured as a percentage of recurring nines. If a network is up 99.99% of the time, it might appear to be highly adequate. The trouble is that 99.99% means the network will be down, on average, 8.64 seconds per day.

In itself, 8.64 seconds a day is quite manageable. Applications will generally wait far longer than that before timing out. Unfortunately, networks are usually up for months on end, and when the down time comes, it makes up for each of those 8.64 seconds in one hit. This means a straight run of three months will be followed by network down for 13 minutes. During those 13 minutes, the network managers will run around panicking and pressing buttons, the helpdesk phones will reach a mutual cacophony matched only by repeating proclamations from helpdesk employees that the network is, indeed, down.

What terrible things happen during down time? Can you roll back the invoicing run and the bank update? Can those whose applications have hung get back to where they were before the dreaded down? Will they have to retype data they have not enjoyed typing once already? Will they be happier with their software systems?

Should you, as architect, demand 99.9999% uptime from the network people, even though each additional nine is increasing the cost of the network exponentially? Should you write your applications to be able to recover from a disappearing network, and if so, how long should they wait before doing something about a lost connection?

Latency is zero

Latency can be though of simply as the delay between a request and a reply. Across wide area networks, it is most often measured by a ping, or sending a small packet on a round trip to and from a remote server. A typical ping to the opposite side of the world across the internet on ground based lines takes around 250ms. Straighter wires or dedicated satellites may improve this number slightly, but until we master entanglement the measure is not likely to improve significantly.

Entangled particles are the result of quantum research. If two particles are entangled, then what happens to one, happens to the other instantaneously regardless of the distance between them. Entanglement will allow us to communicate across the galaxy without waiting for the traditional delays associated with speed of light transmission. Entanglement is how those who have conquered space communicate. The SETIs are wasting their time looking for radio waves.

The causes of latency are:

1. Switches and bridges

Even within the local network, devices with store and forward mechanisms will slightly delay data packets.

2. Routers

Routers change header information. To do so, they must store, manipulate and then forward. A router is a gateway; there may be other gateways slowing the packet before it reaches the internet.

3. Transmission medium

Copper cannot transmit quite as fast as fibre-optics. Fat fibre-optics cannot transmit as fast as thin fibre-optics and require more signal boosting stations, each of which introduces delay.

4. Packet size

The larger the packet, the longer it takes to arrive. Smaller packets might arrive quicker, but you may need more of them. More than one packet usually takes longer to arrive than one packet. How do you recommend and control packet sizes when they are typically between one and four kilobytes.

5. Propagation

The speed of light is finite. A trip to the other side of the world and back at the speed of light on straight, ground based lines, takes 100ms. Propagation alone can have quite an effect on a user in Hong Kong accessing a US web application.

Time in seconds

What's happening

0 0.1 0.2 0.3 0.4 0.5 6.5

User clicks link. Request travels to server Server receives request, builds page and send out first packet First packet arrives. User's computer sends out an acknowledgement Acknowledgement arrives, next packet is sent out Packet arrives at user, acknowledgement is sent Acknowledgement is received. 30 more packets are sent, each of them requiring an acknowledgement

This simplistic approach assumes an immediate response from the user and server machines, and no other network delays. The propagation delay means the Hong Kong user will have to wait 6.5 seconds for 32 packets whereas the US users will receive them instantaneously.

In reality, the other latency and network effects magnify the propagation latency, and the Hong Kong user is more likely to wait far longer than 6.5 seconds for thirty packets of data.

6. Firewalls

The firewall stores, analyses and forwards. It analyses for malicious content, and receives many more hits than it lets through, so is not fully employed speeding through your packets.

7. Virus checkers

A virus checker has to scan through each incoming chunk of information and compare it to the signatures of thousands of viruses. This takes time.

The reason latency often escapes notice is because it is not usually present in development or test or demonstration environments. Only when an application rolls out to live does latency begin to take effect.

Bandwidth is infinite

The bandwidth of your network may be 100Mbit or 1GBit on paper, but when it comes down to moving messages around, you simply cannot use all of it.

Data must be divided up into packets, and each packet has a header and a footer, each taking up part of your bandwidth. With the size of acknowledgements, and the collision detection/retransmit nature of most networks, you will be lucky to get 70Mbit/s of pure data through a 100Mbit network. This is the first problem.

The second problem is that if you are streaming data with a specific minimum data rate, you must allow for an overlap between incoming packets, and played or displayed packets. You must ensure that packets are transmitted fast and frequently enough not to break the stream. It's no good stopping a video or audio stream on a client while the next packet comes in. You will frustrate your viewer or listener beyond their ability to view and listen.

Streaming does not require acknowledgements, but internet communication usually requires one acknowledgment per packet. This can be changed, so that a number of packets can be sent out per acknowledgement, which reduces a few latency delays.

However, if we use fewer acknowledgements, there is more traffic as packets are fired off in the hope, rather than the semi-guarantee, of being received. More traffic means there is less available bandwidth.

The network is secure

There is always a way to get into your network and onto your servers. It can be done by technological know-how, persistence and by detective work.

Technological know-how will exploit a security hole in your operating system and/or software.

Persistence means looping through endless variations of access such as login names and passwords until access is granted. Lophtcrack is one of the best known methods of persistence supported by data.

Finally, detective work will look in notepads, gather social data, search through rubbish, observe building entry and egress patterns and get into the building with a username and password.

Topology doesn't change

A piece of software might work fine one day, then not the next, because the bandwidth it requires to do its work has been reduced by a topology change. Suppose the Hong Kong office has been reduced in size and much of their work moved to the Beijing office. To avoid changing the Beijing line, Hong Kong is routed through Sydney as this is the cheapest option. Sydney has a slower line, and a different daily usage profile than the Hong Kong Office. Something has stopped working...

There is one administrator

Over the course of its life, a network will have many administrators. Each one will have their own way of doing things. If your software comes without an ikiwisi wysiwyg setup and relies on some obscure administrative tweak, then it is absolutely guaranteed that one day it will stop working.

The downtime while the magic tweak is found will quickly erode the availability below 99.99%.

Transport cost is zero

It costs a great deal of money to move information around the world. Bandwidth may be getting cheaper, but that just means we use more of it. Thus, we maintain the cost of running our applications.

Software designed with cost in mind must take into account operational costs, which are often far higher than development costs, and that includes large and/or increasing data rates.

The network is homogenous

Most large organisations have non-homogenous networks. They may use Windows and Linux desktops talking to IBM or Unix servers. Every step of non-homogeneity is a headache for someone. If your applications must work across these networks, then they will rely on both ends of the network being up, and the bridge in between them being up. If each of the networks and the bridge have individual uptimes of 99.99%, the total uptime for one application being able to talk to its server through both networks and across the bridge is 99.99 x 99.99 x 99.99%. Suddenly, your network is only 99.97% reliable. A continuous up of only one month will mean a down of 13 minutes.

Standard models

There are many models to pick from to help a viewer or designer to create or understand an architecture. Here is a list to hone your architecture from:

Roles Each employee in an organisation has one or more roles. This model explains those roles

Services and Processes

Deployment A list if servers and their geographical locations

Applications A list of applications and their purpose.

Integration The means of understanding and moving information between applications

Security Digital signatures, intrusion detection, key management, firewalls, encyption (all from TOGAF's III-RM)

Network/CommunicationsWires, leased lines, bandwidth, protocols

Technology What technologies are used where?

Technology

Technology

Mappings

Maps may exist between any of the above models. The maps may be part of other models, and help to delineate areas of responsibility, or to specifically exclude parts of work.

• Roles to applications • Processes to applications • Applications to deployment • etc

The architectural concerns

The ongoing quality of service is the ultimate measure of the quality of the architecture. To maintain the architectural quality, each relevant architectural concern must be considered in the design of the software. The architectural concerns are also referred to as non-functional requirements or the -ilities.

What must it be capable of? What might it be capable of? Each of the following topics may add to the price and quality. It is your choice which concerns are addressed, and which are not.

Automatic distribution and installation

Servers are often tended by experienced people who have a knack of getting applications working. Client machines are usually tended by people who get frustrated with continual updates and keeping them running.

Creating an install mechanism for your software can help alleviate many client headaches and frustrated help desk conversations.

Availability If you have specified 99.99% availability, how will you achieve it? Is that network or application availability? If you have offered 365 x 24 x 7, how will you achieve this?

Availability is usually achieved by building in redundancy to the system. This means that if one piece of it breaks, another can take over the function. Application availability must take into account the availability of the network, the database server, the web server, the component server, the email server and the client machine.

If your network is 99.99% available, your applications will be less available than that. Sometimes they will break when the network is fine. When the network starts up, there will be a finite delay before all the services are up and running, and your application can start.

Backup and restore What is backed up and when? How is the system restored from its backup after failure?

If availability has been quoted, how have you predicted the increasing backup and restore times with increasing information in the system? How will the volume of backed up data affect availability when data or servers need to be restored?

Efficient backup is a balancing act, for backing up information requires energy. Using energy affects the speed and response of the system being backed up. It may even mean shutting down the application to avoid synchronisation and data locking problems.

Rotating tapes, full and partial backups, and backing up database logs and data at different frequencies can improve the overall drain on the system.

Beauty The most forgotten of all, and the saddest omission.

Cost There are three costs to a software system. One is how much it costs to develop. The second is how much it costs to deploy, i.e. for the hardware your software will run on. The third is how much it costs to run. Upgrade costs are simply development costs revisited.

The highest cost is the most often the running cost. The cost focused upon above all others is the development cost.

Common interface If the system is made up from a number of applications, is the user interface consistent? This includes ensuring that keystrokes and mouse activity have the same effect in any part of the system.

Common interfaces must be defined, and if possible controlled through common views and controllers. These may take the form of compiled or script libraries and style sheets.

Compatibility Is your system compatible with existing software and hardware? If not, why not?

Sometimes prudence dictates supporting past mistakes. Other times, a clean sweep is needed. Which prudence rules this software?

Component interchangeability

How easy is it to add, remove or replace components?

A component can be as small as a single function or as large as a document management system. A component could also be increasing or decreasingly complex components that a customer may upgrade or downgrade to. Interchangeability is usually managed by carefully designed layered interfaces such as the proxy-stub or factory patterns.

Conceptual models How does the software map to the users' conceptual models? How many conceptual models are supported?

Supporting more than one conceptual model can be achieved by applying the MVC or PAC patterns.

Concurrency Occasionally, two execution paths must run concurrently to achieve an outcome which would not be realised if both paths ran sequentially. A simple example of this might be moving the arm and opening the hand to reach out for an object. In software, two disparate systems may need to be updated together to maintain two matching data sets for data redundancy. Data concurrency is usually managed within transactions. Robotic concurrency is managed in time-sliced threads

Consistency How is consistency across applications, client and server operating systems, and thin/thick client interfaces maintained?

Control Can the user gain control of your software? Can they work at different levels of control?

Coupling Loose coupling minimises change and design dependency. However, it can make interfaces more complex.

Cultural How easy it is to change icons, menus, words and layout to suit different cultures.

Adaptability, Localization

Dependability The measure of how reliable the predictability is.

Deployability, Installability

Does it require the click of a button, or a manual, fault-intolerant script to install the software? Is Configuration Management software a dream or a straightjacket?

Detail Can the user move up and down through levels of detail or abstraction?

This is most easily achieved my marking items with a detail level and showing or hiding them as appropriate. As an example, a form may require only skeleton data to be posted, yet allow full detail to be entered. A selectable option can change between the skeleton and rich data views.

Dynamic composability

Plug and play was once the coolest kid on the block. Does your software allow for such dynamic composability, in both harware(additional servers) and software (additional software modules, components, users etc)?

Dynamic resourcing Can your software decide which resource (human, hardware or software) to use at run time, based on system loads?

Dynamic negotiation Can your software negotiate dynamically with external systems? Can it argue prices, conditions? Can it negotiate if an interface or function call doesn't quite match the expected parameters. Do different external systems get different levels of trust?

Dynamic negotiation can also take place to support dynamic quality of service.

Dynamic quality of service

A quality of service recorded in the software can be maintained by dynamic negotiation between running components as to who gets what resource to maintain what level of service.

Energy use What does your system do to conserve energy? Does it have partial or complete shutdown? How does it restart itself? How will you ensure that the batteries are not depleted by the time the spacecraft reaches Mars?

Error handling How does your software trap and handle errors. Does it have categories of error, and can it recover from some errors or categories? Are errors logged in a central location for analysis? Are bad errors automatically sent to a proactive helpdesk, where problems can be resolved even before the call comes in, or user becomes aware of it?

Extendability* How easy is it to extend the system, or extend into the system for external systems?

Failure prevention What will make your software fail, and how are those failures overcome?

Fault tolerance Do you have levels of error or failure, and a means to overcome some levels of error without the user noticing?

Federation Heterogeneity

A federated system appears as one seamless whole, despite being composed of separate systems

Feedback Feedback can improve or hinder output. What does your software provide feedback on?

Flexibility, changeability, enhanceability

If there is a change required to the structure or part of the system, how have you ensured that it will be as easy as possible?

Globalisation Is there a global view of geographically disparate systems? How easy it is to deploy and support globally.

Help How is help provided, and what does it contain?

Installability What will make your system easy to install and roll back?

Integrability How easy is this to integrate with existing systems? If it is a replacement system, do the old and new run concurrently?

If you’re offering 24x7x365, you must be able to swap over systems without the user being aware.

Interoperability How does your system interoperate with others? What can be done to provide a clean and clear interoperability interface?

Learnability How easy is it for the user to absorb the ways of the software?

Localisation How the system (icons, words, ways of thinking) is targetted at a specific global area or belief system.

Logging How, where and to what level of detail does the system log information, and how are those logs viewed or searched?

Longevity How long will it last? What is the mean time between catastrophic failures? What will kill it?

Manageability How easy is it to keep the system going from the point of view of operations staff? Are there concise and timely warnings to provide the operator with information enough to prevent failures? Can it be stopped and started easily?

Memorability How easy is it for the user to remember the sequences of actions necessary to perform their work?

Memorability differs from learnability in that one is gaining the knowledge, and the other is retaining it.

Modularity Modularity is the key to being able to swap in and out parts of the system.

Natural mapping Is the process portrayed by the software the one that the user is familiar with?

Navigation What visual aids guide the eye and mind to points of navigation? Is the map of the site or application visible or easily understood?

Obsolescence When will the system become obsolete? What could advance the obsolescence?

Open Can the executing objects run on different nodes, across different platforms? Can they interact across non-homogenous environments?

Performance How fast does your system respond to users' actions? How many transactions can it process per minute or hour?

Portability What base of code is in ANSI C or HTML, portable across different operating systems? How will you maximise the ratio of portable to OS specific code?

Predictability How you can measure the expected response to all inputs.

Proactive support A software system can report errors to a centralised area. Incoming problems can then be picked up by support personnel and solved proactively even before the user is aware there is a problem.

Reliability What length of time is expected between failures? What are those failures? And how can those failures multiply to reduce availability?

Resource control What resources are available to the software, and how does it make the best, and most timely use of them?

Responsiveness How the system responds, under high and low loads, to user actions.

Responsiveness is slightly different to performance. Consider a user requesting a database write. In one system, the write is undertaken before control returns to the user. In another, the write request spawns another

process or thread. Control returns to the user, and the created thread then manages the database write.

Both systems have the same performance, but the latter is more responsive.

Reportability Does the system publish information about how well it is working against its predicted quality of service or service level agreements?

Return on investment

How long will the software take to pay for itself? This is a highly subjective area, and often very difficult to quantify reliably.

Reuse How have you built in reuse? What have you reused that already exists? How does reuse minimise the cost?

Robustness Robustness is the collation of many other architectural concerns. A robust system will fail well, and will not crumble under user pressure. It is the software's own internal defence against things going wrong.

Scalability How does your system scale? What happens when the number of users or transaction throughput is doubled or squared?

Schedulability How the system makes scheduling tasks easy

Security What kind of attack is your software secure from? How does it manage this security?

Self configuring Some software can configure itself by seeking out new hardware or software services it can make use of. For example, if a new component server or file server is added to the system, the software will find it and make us of it for load balancing, safety or backup.

Self tuning Some software systems can configure themselves to work as efficiently as possible. A simple example of this is to ping a set of servers and use the one with the fastest response. A more complex example is measuring the incoming requests and managing pools of objects to service those requests based on what is being requested most.

Standards based Standards flourish in IT. A standard is far preferable to inventing your own formats or methods, but can add complexity.

Style How do you reuse common style elements, or enforce styles?

Substitutability, Behavioural compatibility

how easy is it to replace one interface or object with another without the system being aware of the change?

Supportability How is the system supported? What helps support staff to understand what is happening from the users’ point of view?

Synchronisation Most enterprise systems require some form of data or document replication, or other types of synchronisation. One obvious example is that server clocks (of the 24 hour variety) must be synchronised so one event can be measured against another to find out which came first. Data is often replicated between data servers.

At a lower level, threads must be synchronised, for example to prevent multiple threads writing the same property to the same object at the same time.

System replacement or system interchangeability

How easy will it be to unplug all or part of the system and replace it with something else?

Tailorability, Customisability

How easy is it to change your system for different viewers? Are all user interfaces user editable?

Task completion Can tasks be stopped half way through and left until later, or resumed after errors

Task distribution Can tasks be allocated to different servers for load balancing, or different individuals if one is on holiday or overloaded with other requests?

Technological options

What are the different technologies which can be used? How do they interoperate? What do they cost?

Testability How does your system allow itself to be tested? What tests can be performed against what metrics?

Time to market Of extreme importance in the commercial world, how does the architecture affect the time to market?

Traceability, Auditability

Who are what did what, and when

Transparent The user does not have to know how the system works to be able to use it.

Understandability How is your system easily understood?

Unity How the parts of a system appear as a seamless whole. Not necessarily uniformity.

Usability How easy is it to use? Can the user find what they want without a torturous search, then use it without hesitation and unnecessary input activity?

User Access How is access granted and controlled? Must the user log in to gain access, or is it all done in the background? Do you use LDAP or have you defined your own access rights mechanism?

User satisfaction How does the quality of the system improve the quality of the users' lives?

User's performance speed

How fast can a user perform their activities? A response of less than 1ms is largely irrelevant if it takes a minute to make all the entries.

*Extensibility is often used in error for extendability. It is not a word.

Frameworks

The architectural concerns need to be delivered by a shared or common set of methods. Those method are delivered in layered frameworks.

In its simplest form, a framework is a collection of layered or otherwise related components.

On top of the distributed operating system will sit your middleware. Above that will be the runtime environment, such as the .net framework, MFC or the Java runtime. Above that sits your enterprise framework, and above that, the application frameworks.

Earlier, I introduced the concept of framework and tools. The framework is layered and distributed. The tools have no particular structure, and can be called from anywhere in the enterprise or application frameworks.

The term framework implies a structure which has certain properties, and can contain other objects. Each of the relevant architectural concerns must be allocated a place in the layered frameworks, and each framework must allow for incoming and outgoing calls, limiting the scope of its activity into specific areas of responsibility. A framework may be distributed or localised, depending on its requirements.

Figure 16.1 Enterprise and application frameworks

Frameworks are defined and communicated using views.

Views

Two definitions of the word view are:

1. That which is seen 2. A way of thinking

This is pretty much what an architectural view is. It is the part of a system which is seen, and the filter through which it is observed.

Just as a building can have a plan, front and side view, it can also have a plumbing view and an electrical view. Similarly, it can have an expandability view, or a storage view. They are not common, but could be relevant views for the sponsor or local building authority.

In software architecture, almost everything we provide is a view. Views and their descriptions make up the architectural documentation.

Software processes also contain views, or viewpoints. Unfortunately, the differences in the terminology, and what exactly goes in those views, is often difficult to comprehend.

The problem is that all software processes or houses of thought are attempting to limit the views so they can be presented in a conveniently packaged way. But we are not dealing with anything that can be conveniently packaged, and so we have a huge difference in terminology and viewing options. There are as many views as there are architecturally interesting facets to a system, multiplied by the

number of likely viewers of those facets. Even in a simple system, that’s a huge amount.

We are attempting to squeeze our views into a convenient blueprint based upon the 3rd angle projection and perspective drawing of a building, and they don’t fit. Building plans or blueprints rarely show the electricity view, the plumbing view, the insulation view, the dynamic way the passages and doors can be used view, the how to get to the lavatory in case of emergency view, yet that is what is required to sufficiently define our software systems.

We are at a point in time similar to when the renaissance gave us the perspective. The Romans, and many other fallen civilisations, managed to build great and huge buildings without perspective. We have been able to build software systems without whatever it is we are missing. When perspective came, it became easier to see a building in one picture before it was built. Will anyone ever create a software perspective?

The difference between a view, a model and a diagram

A diagram is a complete or partial view into a model from the point of view of a stakeholder. A view can be contained in a single diagram, or may require more than one diagram and accompanying text. It is unlikely that a diagram will contain multiple views.

A view provides a complete or partial representation of a system, semantically consistent with other views.

A model is an abstract form of the software. As such a view is a model, but a prototype or mock up, or textual description is also a model. A single application can have one or more models, one or more views and one or more diagrams.

The Unified Process defines a model as a self-contained view, where a system will have use a case model, an analysis model, a design model, a test model, a deployment model and an implementation model.

Figure 16.2 Diagrams, views and models

In software, a view and a model can become somewhat intertwined. It is up to you to define what you mean by a model and a view. The relationship is one of hierarchy. Here is how the software processes define views:

The 4+1 views of the Unified Process

4+1 is an odd but fitting description of the architectural views of the Unified Process.

The Unified Process contains a primary model set, comprising a process, logical, implementation and deployment view, plus the use case view. The process view is sometimes referred to as the dynamic or runtime view, and the implementation view is sometimes called the component view.

Figure 16.3 The 4+1 view of the unified process

The 4+1 views map directly to the UML diagrams:

The Use Case View comprises usually one, or maybe more, use case diagrams. It models a system as a set of high level functions.

The Process or Dynamic view contains Activity diagrams, State diagrams, Object diagrams, Collaboration diagrams and Sequence diagrams, and models control and intercommunication.

The Logical view contains Class diagrams and Package diagrams. It models packages, classes and their inter-relationships.

The Implementation view contains Component diagrams and displays the component or modular structure of the software.

The Deployment view contains Deployment diagrams, and displays the mapping of software components or objects onto hardware.

At this point, the grey line between a model and a view becomes ever more murky. The problem is our use of the word model. In building and engineering, a model is something which can be tested, such as a 1:4 scale model of an aeroplane for aerodynamic testing, or a 1:100 model of a building, which can be admired and agreed before construction begins. The views are the plan or elevation views, or the perspective view. Views are on paper. Models are things

you can hold. However, models are also mathematical static and dynamic representations in finite elements of 3D structures. The model is numbers in memory. It is rendered on a computer screen or printer as a view.

The viewpoints of the Zachman Framework

The Zachman framework comprises 30 views in a 6 x 5 (or 36 in a 6 x 6) matrix. It is primarily intended for developing business systems and is a framework to contain an enterprise architecture.

Zachman’s structure can be used to contain a set of views, allocating them to particular intersections of journalism and construction.

Figure 16.4 The Zachman framework

Zachman arouses strong feelings from its promoters and nay-sayers. It offers us a view of a software system from a slightly different mindset than our software development life cycle view so far discussed. The development moves down the rows of the framework, ensuring we are focusing on people, places and motivations as we add increasing detail to our definitions.

In all its glory, including the additional row for the users, or functioning enterprise:

What (Data)

How (Function)

Where (Network)

Who (People)

When (Time)

Why (Motivation

Scope (Contextual) PLANNER

Things important to the business

What the business does

Locations in which the business operates

Organisations important to the business

Events and cycles significant to the business

Business goals and strengths

Business model (Conceptual) OWNER

Semantic model of entities and relationships

Business process model

Business logistics system of locations and links

Work flow model, organisational charts, roles, skill sets and security issues

Master schedule

Business plan

System model (logical) DESIGNER

Logical data model (database E-R style)

Application architecture

Distributed system architecture

Human interface architecture

Processing structure of system events, dependencies and processing cycles

Business rule model

Technology model (physical) BUILDER

Physical data model

System design

Technology architecture

Presentation architecture

Control structure

Rule design

Detailed representations (out-of-context) SUBCONTRACTOR

Data definition

Application design

Network architecture

Security architecture

Timing definition

Rule specificatio

Functioning enterprise USER

Data (from the point of view of the user)

Functions or applications to use

Network, or communication facilities

Organisation, i.e. skilled people, evangelists

Schedule, or business events

Strategy, othe enforced rules

For more on the Zachman Framework, see [2], or for an introduction, see [3].

The viewpoints of RM-ODP

An alternate set of five viewpoints is specified by the Reference Model for Open Distributed Computing (RM-ODP). They are the enterprise, information, computational, engineering and technology viewpoints.

Figure 16.5 The RM-ODP Viewpoints

The Enterprise viewpoint contains the business roles, objects, scope and policies. Policies include what the roles and objects are permitted or obliged to do, and also prohibited from doing.

The Information viewpoint shows what the system contains and deals with. It may contain static, invariant or dynamic schemas for containing chunks of related information.

The Computational viewpoint contains what the software does. It shows how the system is distributed by functional decomposition into large-grained objects and interfaces.

The Engineering viewpoint shows how the system is distributed. It focuses on the mechanisms and functions which will support the distributed interactions in the computational viewpoint.

The Technology viewpoint displays how the system is implemented. It shows what choices were made and how those choices support the above viewpoints.

Views or viewpoints?

The choice of the word view or viewpoint is largely semantic. Neither word means something specific, except in the context of the methodology which contains it.

As an example, consider the security view. Where is it in each of these methodologies?

For the 4+1 view of RUP, the answer is simply wherever you want it to be.

In Zachman, it would be divided up into relevant sections within the grid. There would be more than one, but less than thirty aspects to the security view.

In RM-ODP, security is not a view, but a function. The four functions of ODP are Management, Coordination, Repository and Security.

So we find that the apparent simplicity of expecting a view for all architecturally significant aspects is not delivered by any of these methods. Neither are they wrong not to contain a security view, for Security may be offered partly by software, partly by hardware technologies, and partly by business procedures. If locking the door at night is part of the security of your software system, you would contain that information in your hybrid security view in any of the methodologies.

RM-ODP

RM-ODP functions

Figure 16.6 RM-ODP Functions

The security function contains how the security of the system is managed, how material within the system is delivered only to those authorised to see it, how integrity of the information is maintained, audited and how access rights are granted to the system's users.

The management function details how the system is managed from nodes (servers) to objects running on those servers.

The coordination function details how the significant system events are coordinated.

The repository function details how information is organised and held.

RM-ODP Distribution Transparencies

RM-ODP also contains eight distribution transparencies

Figure 16.7 RM-ODP Distribution transparencies

Access transparency

Allows an object to be created or called, regardless of its physical location or the technological boundaries which exist between the caller and the object.

Failure transparency

Ensures that failures can occur in the system, yet be trapped and corrected, invisibly to the user.

Location transparency, or Abstract Locality

Does it matter where the servers are located, or which servers are up or down at the time of use? If a user can call the help desk and manually change settings to point at another server while the local one is down, then there is a flaw in the architecture. The user must not be concerned with which server to connect to.

Abstract locality can be managed logically by recording a user's location and routing them to the most appropriate server, or by pinging a group of servers, then selecting the one with the fastest response.

Location transparency allows access to locations without knowing their name or physical address.

Migration transparency

Allows objects, servers or middleware applications to be moved yet still allowing access as before migration.

Persistence transparency, or object pooling

If an object is deactivated, it is put in a pool to be reused rather than being deleted and recreated again when needed.

Relocation transparency

Allows objects and interfaces to be relocated even during use.

Replication transparency

Allows the seamless update of many copies of the same software entity. This can lead into fragmentation transparency if the software entities are fragmented or distributed in copied parts.

Transaction transparency

Allows the coordinated activity across non-heterogeneous networks of begin/commit/rollback transactions.

RM-ODP is the most complete of all the methodologies for presenting a specific set of views for a software architectures. It covers many of the items discussed in

the architectural considerations directly, and the rest by implication. It has been accepted as an ISO standard in four parts X.901 to X.904, and its scope covers the many facets of distributed computing.

Figure 16.8 The scope of RM-ODP

Of the RM-ODP viewpoints, the Enterprise, Information and Computational viewpoints are distribution independent. These and the Engineering viewpoint are Implementation independent. Such an approach to architectural definition can help to separate out the concerns of your software system.

RM-ODP is discussed more fully in [4] and [5]

Views, patterns and architectural considerations

It is one of my greater software hopes that one day we will have a cross reference between the architectural concerns and the patterns, and that we will be able to provide views by selecting those relevant from a three dimensional matrix of architectural concerns, viewpoints and viewer.

The chain of command pattern for help was discussed in the previous chapter. The technicalities of this are of importance to the software designer and programmer. The way it works is of interest primarily to the user. Therefore, to fully understand the system in sufficient detail, the user and the designer/programmer require different views of this one feature.

Architecture Documentation

Architectures need documenting. If they are not well documented they will be compromised, leading to ever increasing entropy. When working on a project within the architecture, the project documentation needs to be as light as possible (even non-existent if you XP), leaving the architecture documentation to address all architectural issues.

The architectural documentation is concerned with high level design, creating frameworks and tools which together support the creation of applications.

The Architecture, in the form of a document or electronic storage, will contain all facets of the enterprise system. Creating a structure to hold this information in a logical way can be arduous, or you can follow RM-ODP. If you choose the RM-ODP structure, you will have viewpoints, functions and transparencies. You must also think how best to introduce the applications which sit within the architecture, and how to provide views from each viewpoint to best provide the insight the viewer will require.

Introduction

The introduction should cover about half a page. Any more will lose your reader before they begin. It will cover at least the following points:

• Who the system is for • A high level statement of what the system is intended to achieve, or

what needs it will fulfill • An overview of the existing environment or non-software practices

How the architecture supports the enterprise goals

This section is more relevant for enterprise systems, and explains how the long term enterprise goals are supported by the architecture.

The reason for it is twofold. First, it is a sales pitch to the sponsors of why the architecture is right for their given requirements. Second, it provides a defence against changes down the line. It's prominence forms a foundation in the mind of the sponsor and application designer that the architecture is not just a technical statement, but supports the wishes of the sponsors.

Viewpoints

Technology

What servers will be required, and where will they be located. What mechanism will be used to choose and introduce new locations, and what type of servers will be used. What storage is available on those servers (RAID, SAN?)

What client devices will be supported and what are the minimum requirements of each type of device?

What printing facilities will be required?

What operating systems, networking and protocols and directory servers are required? What data rates and response times must be supported?

What middleware is required? i.e. what database, email, internet access, document management system, collaboration tools, search engine, messaging?

What COTS (commercial, off the shelf) applications will be used? Which browsers will be supported?

Enterprise

What is the structure of the enterprise? What are the roles and goals, what are their guiding principles? What markets do they operate in?

What are the obligations, permissions and prohibitions of each role?

Where does the enterprise see itself in five years' time?

What are the enterprise's assets? Which of these assets are relevant to the software?

What earthly locations does the company operate in? Must the software support multiple spoken languages?

What quality of service do they expect?

What is their reporting hierarchy and who is responsible for what?

What are the enterprise objects? e.g. product, invoice, delivery note, supplier, customer, employee, timecard, expense claim. What are the relationships between these objects?

Information

How are the assets and enterprise objects created, persisted and destroyed?

How is the persisted information collated and summarised for efficient reporting?

What is the static, unchanging, or infrequently changing data the system uses as reference data?

Create a data dictionary and data model and define the basic lookup information such as countries, tax rates etc for the enterprise objects.

Engineering

How is bandwidth maintained for all users at critical times for critical applications?

How does the system pass messages among users who are not always connected to the network?

How does the system pass messages around the non-homogenous network, allowing for failure and transaction processing?

How is offline data maintained on client devices?

How are remote calls made across the network, eg COM+, Java RMI, CORBA, SOAP?

Computation

Define the object models, frameworks, layers, interfaces and shared tools of the software.

Functions

The relevant management, security, coordination and repository functions are documented.

Distribution transparency

The relevant transparencies are documented.

Applications

After defining the architecture according to RM-ODP, guidelines are given for the applications.

The architecture constrains and supports the applications. But that is not the end of the story. Applications must have a common look and feel. They must nestle comfortably in the frameworks defined in the computational view. This section introduces the applications contained within the architecture, and details how each will conform to common standards.

It must not contain any application specific detail.

A final check through all of the architectural concerns will highlight any weaknesses in the architecture and/or documentation.

Other views

You will have a requirements document, which in itself is a view of what the software must do. Each document created for analysis, design, project planning, test, deployment and maintenance is a view from a particular perspective.

You may also create traceability views from requirements into analysis, design or test.

If your system has heavy data processing capabilities you may define data flows, or if user interaction is intense, you will provide user interface and navigation views. You must decide whether they are enterprise architecture or application architecture issues and document them accordingly. As a rule of thumb, if something (an object, a style, a sequence) is used in more than one application, it is an enterprise architecture issue. It can be created in one of the applications, then promoted to enterprise level for reuse.

All views must target the audience the view is being created for. They should contain all relevant information without overloading the viewer with unnecessary or unrelated information. Views can, and should, contain notes for succinct clarification. They must ensure that logic, appearance and methods are discovered, communicated and continually agreed as development progresses.

All of these views are supported by the project's eye view of the particular methodology you are following to deliver the software.

Personally, I find RM-ODP is better for documenting architectures, and RUP's 4+1 more useful at the application level.

Architectural insights

Architectural insights are a case of if I knew then what I know now... All of these problems are highly politically charged, and fraught with high risk from the outset. Nevertheless, it is beneficial to have an idea of what stepping stones are available among the crocodiles.

These are all simplistic offerings. It will never be as easy with live systems, but at least they are foundations to build upon.

Insight 1. How to get from a legacy application to a new one without big bang implementation.

There is a need to begin moving from a system of high entropy to one of low entropy. A legacy system is working and in continuous development. It is rather worn out, having survived operating system and development environment upgrades, and a high turnover of personnel. Many different coding standards,

development approaches, aging technologies and unsupported controls exist within its firmament.

It needs to be rewritten, but times are tight and the business cannot afford the astronomical cost and long wait you have identified for their new system

A new system with a common architecture is required. The steps to get there are large scale refactors of isolation, extraction and rework.

Step 1. Understand the existing architecture and applications

This, in itself, is a huge undertaking. Most legacy systems are somewhat lacking in documentation. The architecture of the collected applications must be extracted and documented. Over time, alternative methods will have been created to do the same thing. Each method must be examined, and a successor to both chosen for the new architecture. At application level, it is better to extract common features and document each separately. It is not necessary to model the applications in any detail, for you may decide to scrap some applications, roll others into one, and extract a number of applications out of one current application. Therefore, full static and dynamic models would not be of much use. Use case models are probably sufficient.

You should resist attacking the problem from the data end just because it is easier to get hold of. Users use systems from their own point of view. Designing systems data out means users end up with a data oriented view rather than their job oriented view. As an extreme example, consider a square being drawn on a screen. You may think it has a start and end point if programming it from a mouse click and drag. A point, of course, is a pair of numbers (212,352). Now, is it a square, or a rectangle of similar side length? Is it defined by the points of its corners or its origin and side length? What is its line colour and fill colour? How thick is the line? The answer is easy: look at the data. But what if the data does not represent what the shape means to the user? Going the data route will mean you perpetuate a potential fault in the original data design, and further confuse the error when rendering it on screen. If you can't get a square right, what chance have you got with an invoice or tax return?

Step 2. Extract common components and functionality

Identify areas where methods, objects, functions, files and data are shared or similar.

Step 3. Define the architecture

At this point, the architecture is everything that needs creating before work can begin on any of the applications. It should be as sparse as possible, yet inclusive enough not to allow deviation from the architectural vision.

Step 4. Define the applications

Take all of the use cases from the application models. Add in management and system functions. Gather all of the use cases into a logical set of applications.

Step 5. Create a programme plan

Create a programme plan for creating the architecture and the applications. Each step of the programme can then be broken down into project plans with careful mapping of dependencies, so work can continue at a measured and reportable pace.

The architecture will be begun as a separate project, then finished by adding in each shared component developed with the applications. The necessity of this is so sponsors can see continuous output rather than developing the architecture in its entirety before committing to developing the applications. IT budgets are expensive, and no-one likes to see their money disappearing into the IT cloud without a visible return.

Step 6. Follow the cycle of water

In its Water method, Taoist meditation teaches us to use full effort without strain or force. That may sound rather difficult to our western ears, well versed as we are in physics equations.

For a moment, try to suspend your disbelief, and we shall go on a journey from the sea to the clouds to the sea again. In the churning sea of data that contains our structured (data) and unstructured (documents) information, we have temporarily lost sight of its higher structure, and involve ourselves in the detail of tables, columns, keys and replications.

The problem with our data is that we have the seas, the lakes and the clouds, but we have lost the raindrops. If we could get back to the raindrops, our lives might be a little easier.

A raindrop, in this case, is that piece of information held by a screen in an application, or by a section of a document. A raindrop might be an introduction. I could ask for all the introductions for wind farms and use the most suitable raindrops to create my own puddle of a document on wind farming. Otherwise, I am left wading through lakes, or delving down to the sentences (which we haven't stored). A raindrop of data might be an address, or a sales order. An address is not Customer name, company name, address line 1, address line 2, address line 3 (depending on whether or not it is empty), country and post code. That's data.

So we allow our data to precipitate up and become a rather unstructured cloud. We then allow it to rain, and catch the raindrops as they fall. Once we have our raindrops, we can easily recreate our sea. Not only can we recreate it, but keeping the raindrops means less database calls. We are not short of storage at present. It does not matter if we store an invoice as HTML, SGML, PDF, relational

data and summarised data in twenty-seven different forms. It probably mattered when the application was first written, but it is time to move on. We need the print render without having to create it from scratch every time. We need the relational data for searching and summarising. We need the summarised data for efficient reporting. There is no need to rebuild data every time it is queried right from the lowest level of relational data. Get creative with those raindrops. Create a sea with everything from atoms to huge waves. Your sponsor deserves it.

To rework the data in this way, a two way bridge between the old way and the new way is required. This is insight 2.

Insight 2. How do you redesign a database?

Redesigning data is not very difficult. What is difficult is to do so without being influenced by the demons of the past, and to create a method for switching between an old design and a new design without upsetting anyone. Data is always in a state of change. It can't be stopped, reworked then started again.

People are always trying to redesign databases, directory structures and other storage mechanisms, and they always come up against the same problem: they try to fit two eggs into the same eggshell.

The solution is one of steady migration.

Redesigning the database at the data level requires the creation of a new and separate database. Application by application, data is lifted from one database and written to the new one. It can be done two ways:

Method 1. Reporting first

All data is extracted from the old database and written to the new database, in its new formats. All reporting is then moved from the old database to the new database (hopefully dropping paper based reports and moving to browser based in the process).

Once reporting has been redirected, triggers in the old database write to the new database. The triggers continue working until each application is redirected to read from and write to the new database.

Method 2. Reporting last

Create the new database. Build a two-way bridge between the two databases so that any data written to either database is replicated to the other.

The next method for redesigning the database involves building a new object layer above the database. It is insight 3.

Insight 3. How to move from a data based system to an object based system

A wasp is a good model of an object based system. Its yellow and black striped abdomen is the data. Its magnificent multi-celled eyes, mandibles, proboscis and antennae are the interfaces to the outside world (applications). Somewhere in its thorax are a pile of objects, and the vanishingly thin wasp waist is a single component which writes data from objects to databases.

Options for the wasp-waist are:

1. a single component taking in an object as a string of XML and persisting it.

2. J2EE's entity beans using container or bean managed persistence 3. a single component that can iterate through all object properties and

read and write them from and to a database

The wasp is easy enough to understand. What is difficult is that you often have a bee. A bee has a fat waist, represented by many links from many applications at multiple levels. One application may read and write data directly using ODBC. Another might use stored procedures via JDBC or ADO.net. Yet another will write a text file which is imported by the DBAs directly into the database every Friday evening.

Getting from bee to wasp is difficult, and becomes ever more difficult as the freedom to improvise database solutions continues on an ad-hoc basis according to developers' preferences.

The only way is to create a layer in the architecture for managing all database activity, gradually including each application. Once the layer is in place, you are free to migrate to a new database. The data objects can write to both databases in an interim period.

I hope the message from insights 1, 2 and 3 is clear: Escape the straitjacket of thinking always about data at the relational level.

Insight 4. How to merge the architectures of two merging organisations

The first and foremost goal to reach is an understanding of how the businesses will work together. As long as their interwoven process remains undefined, so the software definition will be vague and unprogrammable. It is probably the worst environment in which to create software.

Mergers are cruel times. Many fail to provide a return on the investment. Heartache is high, egos are trampled, careful thought thrown to the wind. The business leaders set their date, empower IT departments to fight the good fight, and have many more things to worry about than their IT systems. They have mergers to do, processes can fall into place around the reappointed individual leaders.

The only way to define an architecture is to get into the heads of those running the merger. Who are the drivers of this merger? What are the reasons? What

working practices must be upheld and which will be discouraged? Whose ideas will form the basis of the merged organisation?

Which if the two systems best matches, and is likely to be able to support, the new organisation. How much work is required to make the new system behave exactly as the old one did - that is always the request. No-one can possible operate without the reports they are used to seeing week after week. You will need to reassure them that the information will be available.

Whichever is the best (for the business, and architecturally), you must choose. You must then analyse what those who are losing their system cannot do without, and recreate it in the chosen system. A plan must be created for the switch off of one system, and full implementation of the other. You are unlikely to have any influence over the date, and it is equally likely to be too soon.

There will be a lot of firefighting to be done. People are already unhappy that their company has evaporated, and lots of newcomers, and new managers are abroad in their world. You can only do your best.

Insight 5. How to create an architecture from a collection of loosely related applications

The Paris Guide to IT Architecture[6] argues four steps to creating an architecture. They are:

1. Define a long term plan

In defining a plan, the important aspects of an architecture are gathered into zones. A stationer may have a zone for its products, a zone for how it holds customer information, and a zone mapping customers to pricing policies. It will have another zone for how it manages itself financially and manages its employees. Other zones will summarise sales data for analysis, and manage sales and invoicing.

Enveloping these zones will be the company's vision of where they see themselves in the near and distant future. And permeating the zones will be the quality of service, the style, and the clarity of view into the collected information which they expect from their software.

This information forms the long term plan.

2. Build a stable interface infrastructure

Build into the software the concept of services. Each service is a wasp's waist through which information passes, and is filtered or augmented.

A number of services provide a limited number - though fully functional set - of points through which data passes. This limited set of pathways is the stable interface structure.

3. Appoint a Zoning Board

The zoning board is responsible for what applications belong in which zones. They also manage requested deviations from the long term plan.

4. Make the most of what you have

If you have some core applications which require little change to fit the long term plan, plug them in. Modify others to fit where possible, and make discarding and rewriting the final option after all refactors and other alterations have been considered.

Insight 6. How to create the architectural vision

The vision is what the architecture is reaching for. It is not a business vision, but a vision of how the business vision can be can contained in a software architecture. It is gathered mostly by listening to people, and reading the mission statements and behavioural guidelines produced by a company.

The vision is created by weighing up where the business wants to go, what the users, customers and suppliers want from the software, and what is technologically feasible and affordable.

Often no-one in the business will have a full knowledge of what is possible, and their personal vision will be shaped by their experiences with other software and more strongly with what they want from the software to make their lives easier.

Balancing all of this input and extracting the vision is your architectural responsibility.

The architectural design is created from the architectural vision. The vision is the bringing together of the business and employees' visions, their ideals, their beliefs and desires. It will contain quality, style, a means of conforming to common ideals, a method for drilling into its taxonomy or ontology, and a strategic statement of global, numerical and technological intent.

By a drillable ontology, I mean every time a product code is displayed on screen, its related items (customers buying it, suppliers supplying it, how well it is selling, how its sales compare to other similar items or other non-similar items) can be navigated to from that single point. All business properties such as sector and industry should provide instant views into the sector or industry, and all other related items. When viewing product sales to a business, it must be possible to get to a view of that product's sales to other businesses, other product sales to that business, product sales to other businesses in that sector, or not in that sector. Any relationship or grouped data defined in the taxonomy or ontology must allow for flow through the system from a shifting point of view. Achieving this flow is not a technological challenge, but an artistic one.

Insight 7. How the architecture helps the business reach for its goals

When a business defines its high level goals, the architecture must be designed to support those goals. If the architecture then comes under attack, a misalignment with the goals of the business can be defended against.

Defending the architecture can often be a cerebral activity argued out as points of excellence in software creation. Businesses care little for such activities. However, when someone attacks an architecture there to support business goals, they are attacking the business goals. Then, less effort is required in defense. Everyone feels free to attack the architecture. Very few will attack the business goals.

Insight 8. Why is the unit of release, the unit of reuse[7]?

Most commercial systems or applications have specific release numbers, just as software components have version numbers. The components and systems have release dates and service pack updates.

Systems written in-house for in-house consumption often suffer from its applications becoming intertwined with one another, or separated from one another.

In an intertwined system, two applications share some functionality. The functionality requires a change for one application, but that will affect the other application, which must be reworked to use the new functionality.

In separated systems, two applications will contain the same functionality, but to change the functionality, it must be updated separately in each application.

The latter example has no reuse, the former has complicated reuse to the point that the reused component cannot be changed without changing both applications.

The reused component cannot be released on its own. It requires updated releases of both applications using the component. Hence, reuse and release are intertwined.

Enterprise Architecture Case Study

In the last chapter, we looked at some example architectures. Mostly, they were software architectures combined with some high level business information. The term enterprise architecture gathers together the business structure, the wishes of the business leaders and future direction of the business, the technical architecture, and the IT infrastructure. To attempt to represent the enterprise architecture other than in the most abstract way will lead to ever increasing detail. Any attempt to represent it technically will probably be unfruitful. Here is an example of how I once chose to represent an enterprise architecture. I doubt it would do for any other enterprise, but it may give a flavour of the abstraction and focus required in comparison to the technical detail of pure software architectures.

This case study concerns one of the UK's largest insurers, where I worked for nine months on a Business Process Management initiative.

It soon became apparent that what we were attempting to do reached across the entire enterprise, encompassing the corporate goals, the architecture, the commercial vision and the process management, the latter of which was our original scope.

Only at the end of this nine month period, was I able to grasp the full picture, following conversations with many people who each supplied one part of the puzzle. It was a bottom-up process, rather than a top-down one [8], and looking back I'm not sure a top down approach would have supplied the eventual (though complicated) clarity, considering its complexity and scope.

For example, the insurer was the largest purchaser of white goods (fridges, washing machines etc) in the country; it would also like to be the largest seller. They also had a mind-boggling 1100 software systems and many other manual ones upon which their business ran.

The insurer's commercial vision, was a programme of work covering processes, people and systems, which reflected the desires of the business. It contained around 20 items of change, only four of which impinged upon more than one area of the enterprise.

• Better business activity monitoring (BAM), also better management information, specifically for offshored initiatives

• Use of an insurance industry portal (I-Market) to allow customer and broker/intermediary access to insurer's services

• Roll out and increase offshored services, most of which went to India • Process re-engineering for an electronic world

Although these were part of the work stated as moving the insurer towards its goals, the actual goals were less forthcoming and existed in the hive mind rather than being set down in hard or soft copy. All of the business goals, requirements and fears must be included in an enterprise architecture.

• Reduce account and claim handling processing costs • Reduce the number of IT systems (currently 1100) • Improve the ability to move into other markets (e.g. white goods) • Plan for a likely industry downturn in early 2006 (ripples of 9/11) • Become an organisation with care at its heart, insurance at its core • Increase the consistency of underwriting claims • Gain the reputation of being easy to do business with from a customer

point of view • Meet FSA compliance • Meet Governance requirements

The software architecture we had teased out at this point was as follows

• A service oriented architecture would allow the business process management (BPM) system to communicate with the mainframe applications

• An event driven architecture was required so that mainframe applications could inform the BPM system that events had occurred and could be processed

• A logical data model was required so that the different mainframe applications could talk to the BPM software using a common language

• All of the mainframe applications and the BPM software required integrating through an integration hub

• A taxonomy was required to identify and route claims and policies to their respective systems

From the business process management side, the following is an idealized future state, not all of which is currently available.

• Alongside the logical data model of the software architecture, a logical process model is required, created in a non-software specific language, akin to BPML

• The enterprise must have a single distributed business process rules engine.

• Running business processes must supply detailed and informative automation, monitoring and reflection.

• Workforce management workflow • Modelling, simulation, optimisation • Common user interface

These, I felt, were the important facets of the enterprise architecture. Figure 16.9 details their interdependence. I called this the smoke chamber or the flower diagram.

Figure 16.9. Initiative collaboration

This, in itself is hardly an enterprise architecture, but an important starting point as it charts the interdependence of the initiatives and hopes of the business along with programmes already underway.

It is a highly complex, even complicated picture, and that is the role of the enterprise architect: to reduce the complicated to merely complex, while tying everything together.

References

1. The Eight Fallacies of Distributed Computing. Peter Deutsch.

2. The Zachman Framework. http://www.zifa.com 3. The Zachman Framework: An introduction. David Hay.

http://www.tdan.com/i001fe01.htm 4. Architecting with RM-ODP. Janis R. Putman. Prentice Hall. 5. ISO X.901, X.902, X.903, X.904. http://www.iso.org 6. The Paris Guide to IT Architecture. Jurgen Laartz, Ernst Sonderegger,

Johan Vinckier. The McKinsey Quarterly, 2000/3. 7. Design Principles and Design Patterns. Robert C Martin.

http://www.objectmentor.com 8. Top Down Thinking for Enterprise Architects. Nigel Leeming.

http://www.softwarearchitect.biz/articles/tdt/topdownthinking.pdf

Chapter 17 - Where do we go from here?

We are fortunate to live in the dawn of our discipline. It is us who will lay the foundations of what is to become software architecture. Far more fortunate are we than those who will come and lay claim to our title, for they will be mere links in the continuity rather than forge and anvil from which it came.

And in this dawn, we travel two roads in one, the technological path, which gives us the means to understand and solve highly complex technical problems, and also the personal path, which allows us to communicate with our fellow beings and seek the goals we have set ourselves. Let us turn our face to the sun and allow the shadows of software so far, to fall away behind us. We shall create a new beginning.

Carpe Titulum

Seize the title of Software Architect!

There is no-one to stop you. We have no body of knowledge, a confusion of terms and methods, and many gurus promoting alternative visions. If you do not seize the title, the management trap will seize you, and you will be mere administrator instead of innovator.

We need software architects. We need people to be recognised as being responsible for ensuring software systems meet their purpose, and people who endeavour to create systems which uphold their own principles and the desires of their clients.

Yes, we struggle, as all did before us who were at the beginning of other things. And so we look for answers to our own confusions and wonder what history has to offer us, so our way might be clearer. We begin by contemplating contemplation itself:

Indeed, in my own experience across the traditions, I have found that the best teachers are those who do not exalt their own teachings (and, by implication, themselves) above the teachings of others. Having travelled far in their own development they have left behind such narrowness, and are able to see the

truth wherever it is to be found. Nor are they averse to criticising their own traditions where such criticism is justified. They live in openness and freedom because they are not bound by the dogma that gets in the way of wisdom.[1]

Let us follow that advice, and see what the world's major movements of thought have to offer us.

A chronology of advice for Software Architects.

Who can say how many times enlightenment came to earlier beings as unrecorded civilisations rose and fell? The great records of the Egyptians, the Babylonians and Romans showed us that they were great warriors and administrators, but they left us little on how we might conduct ourselves. It was the Greeks who began (in recorded history) to document the why rather than the how. Outside the monotheistic religions, it was Socrates who was first recorded to take the bitter hemlock, and death for his beliefs. It was the Greeks who gave us the word architecture: literally arch drawer, or arch designer.

All of these recording traditions raised civilisation, upon which we depend as software architects, but the real advice on what to do as a software architect comes from many different ancient and modern sources. Let us pass on a swift journey through two and a half thousand years, and gather things that belong to other journeys that might be applied equally to our own.

C. 500BC - The Buddha

The Buddha, or enlightened one, sought the path of peace and serenity. He suggested:

Try it and see what happens. If it works for you, use it. If it does not, discard it and look elsewhere.

This does not mean you should discard lightly. If something does not work for you, you must understand why is does not work for you.

C. 300BC - The Bhagavad Gita V16 ch 1-3 (adapted slightly)

It is difficult to date the Bhagavad Gita, or Song of the Lord. It was added to the Mahabharata as the latter began to take its final form in the reign of Asoka, the third Mauyra emperor[2]. It survived in oral form before being written down, and its origins are in the mists of civilisation.

Freedom from fear Purity of heart Constancy in learning and contemplation Generosity Self-harmony Adoration Study

Austerity Righteousness

Non-violence Truth Freedom from anger Renunciation Serenity Aversion to fault finding Sympathy for all Peace from greedy cravings Gentleness Modesty Steadiness

Energy Forgiveness Fortitude Purity A good will Freedom from pride [3]

Some say that despite our almost miraculous advances into materials, medicine and engineering, we have not discovered any better how we are meant to live. Our brief interval crammed between birth and death is jammed full of learning and examination, metered out by success and failure, pay packets, meetings and holidays.

It is a sobering thought to realise that we have moved from the age of revelation to the age of science and technology and hardly have time to breathe in the lessons of the past.

Yet to be fully effective as software architects, the life lessons are as important to us as the technology lessons. We have neglected them to our cost for long enough. Too many non-software people permeate software society because we need their man-management and personnel skills. We must learn these lessons ourselves, and use them to better deliver the excellence our professions demands.

It is not as if these lessons are hidden from us. They are out in plain view, published in books, posted on the internet. We have only to turn our view a little, and the path is wide before us.

C 360BC to early AD The seven heavenly virtues

Although we see the seven now, they were originally compiled from the four cardinal and three theological virtues.

The cardinal virtues were derived from Plato[4]. The Christian theological virtues are from St Paul[5].

Faith - believe in yourself and those around you

Hope - Believe that your will to succeed or change will have effect

Charity - Help others where and when you can

Fortitude - Do not give up

Justice - Reward those who deserve rewarding, thank those who need thanked

Temperance - Moderation can win over extremism. The tortoise can beat the hair. Relax, you're doing fine.

Prudence - Be cautious, wise and discreet.

C. 0AD The sermon on the mount

In what was arguably the most important single speech, Jesus bade us, amongst many things, to:

First take the plank out of your own eye, so that you might see clearly to remove the splinter from your brother's.

The sermon on the mount[6], in a few moments of speech gave us many proverbs and information on how we should be. Proverb for proverb, it is matched only by Solomon[7].

C. 410AD The seven contrary virtues

The contrary virtues came from Psychomachia[8], or the Battle for the Soul of Man.

Humility - Oft presented but seldom found

Kindness - Be kind, especially to those struggling to understand

Abstinence - Avoid the habits which will drag you down. Do not overindulge yourself.

Chastity - Refinement in taste and style

Patience - You will require a great deal

Liberality - Cultivate the mind for its own sake

Diligence - Be steady and earnest in the application of your knowledge

C 500 AD. The three pillars of Zen

Great faith - We do what we believe can be done

Great doubt - We carry the spirit of enquiry in our hearts

Great determination - We will do whatever we need to give us more faith, more doubt and higher achievement

1645 AD

Miyamoto Musashi[9], sitting in a cave in the mountains at the age of sixty, and considering the way of strategy, left us this:

1. Do not think dishonestly 2. The way is in training 3. Become acquainted with every art 4. Know the ways of all professions 5. Distinguish between gain and loss in worldly matters 6. Develop intuitive judgment and understanding for everything 7. Perceive those things which cannot be seen 8. Pay attention even to trifles 9. Do nothing which is of no use

Musashi was a supreme samurai swordsman and strategist. By the age of thirty he considered himself invincible, having beaten (and killed) the sixty opponents who had faced him. In feudal Japan, swordsmanship was an art, and he carried that art into drawing, carving and writing. Many of his works still exist today, yet none is the equivalent of his way of strategy. It is studied by many of the business leaders in Japan, just as it is studied by Kendo students worldwide.

Gandhi's seven sins

• Wealth without Work • Pleasure without Conscience • Science without Humanity • Knowledge without Character • Politics without Principle • Commerce without Morality • Worship without Sacrifice

Gandhi, more than anyone in recent history, embodied the greatness of the prophets of old. We certainly live in a time where commerce has no morality, and the larger the business, the less its humane character. Mergers and acquisitions are a feature of our time, where money for expensive buyouts seems easily available. The consequences of these huge buyouts rarely favour the individual.

To prove that the remarkable minds of our time also had time to take out from their scientific endeavours and enter the world of philosophy, Einstein created three rules of work:

1. Out of clutter find simplicity 2. From discord find harmony 3. In the middle of difficulty lies opportunity

Even the wags of contemporary politics can teach us something, for software and politics are both viewed as black arts peopled by those speaking fork-tongued:

Mark Twain:

I think I can say, and say with pride, that we have some legislatures that bring higher prices than any in the world.

or

It could probably be shown by facts and figures that there is no distinctly native American criminal class except Congress.

Boris Marshalov:

Congress is so strange. A man gets up to speak and says nothing. Nobody listens and then everybody disagrees.

How many words would you have to change in the above sentences to create the equivalent thoughts of those outside software on those inside?

William Churchill:

Success is moving from one failure to another without any loss of enthusiasm.

We have survived long enough on Churchill's comical vision of success. It is time we moved beyond it, and could take the advice of another leader of the 2nd world war

General MacArthur:

Have a great plan. Execute it violently. Do it today.

And finally, for we who begin something while all about us is doubt and confusion:

C. 610 - 620 AD: The Qur'an, Surah 34 verses 49 to 54. Translation by Dr Rashad Khalifa.

Say "The truth has come: while falsehood can neither initiate anything, nor repeat it." Say "If I go astray, I go astray because of my own shortcomings. And if I am guided, it is because of my Lord's inspiration. He is hearer. Near. If you could only see them when the great terror strikes; they cannot escape then, and they will be taken away forcibly. They will then say, "Now we believe it," but it will be far too late. They had rejected it in the past; they have decided instead to uphold conjecture and guesswork.

Consequently, they were deprived of everything they longed for. This is the same fate as their counterparts in previous generations. They harboured too many doubts.

Where do we go from here?

In the cold light of day, you might consider this whole book as just a list of things you might like to know. I have no doubt you knew some of them already.

We still have a long way to go in software systems and how to think about designing them. Consider the simple example of two pairs of people sitting down, one with a computer, and the other with a pad of paper and a pencil.

The benefits of the pair with the computer are that hey may have some helpful software, some of it even quite useful, like a spelling checker. They may have rendering software to draw 3d models. They are less likely to run out of disk space than their counterparts will run out of paper, nor will their keys fail as often as pencil points. They can create indexes and hyperlinks and do sorts.

The paper and pen people have invested less. They can write and draw on the same piece of paper with ease. They can tear off pieces of paper and put them in piles to represent ideas. They can write in the margin, draw lines, circles, highlights and arrows leading from one bit of the document to the other. Governing thoughts can be divided into pages far easier than onscreen. Their batteries won’t run out, and they can put pages side by side to compare them. They can even spread pages around the table into piles of related ideas.

I’ll leave you to decide who is better off.

Imagine a user interface that can draw you in. It can create new ideas in your mind, even other worlds. It can make you hang on its every word and make you want to avoid others, meals.

That user interface already exists. It is called a book. I hope this one has entertained you, and maybe introduced you to a slightly changed world.

The road ahead

It’s up to you. You can take up the gauntlet of Software Architecture, or you can leave it lying in the mud, and allow others to take up the challenge. Software desperately needs vision, and the role will be filled by Project Managers with an inkling of system design, or Business Analysts with a far better view of Business needs.

I would argue the role is solely for those with a schooling in development, and an understanding of the technical details of a wide range of technologies. If we don’t do it, someone else will.

To succeed, you must empathise with your customers. You must make every effort to communicate personally with them face to face, or failing that, by

telephone, videoconference, email or letter (in that order). You must communicate in a comfortable environment in a language you both understand.

You must understand the personalities behind the stereotypes. Here is what the Sunday Times[10] had to say about chief executives.

• No desire to fit in, but the confidence to be an individual • Unusual character, not well rounded, but has distinctive strengths and

weaknesses • Has the ability to simplify complex problems • Insecure, not cocky or confident • Driven by the need to make a difference, not by salary or status • Bright, but not the most intelligent. Too intelligent is frowned upon.

These are character traits. Not quite what you might have had in mind as a stereotype. Stereotypes are the enemy. We must dismantle them. The worst stereotype of all is our own. The techie. The propeller head. The Star Trek quoting, Dilbert reading, anorak wearing, uncommunicative, out of touch, lost in technology, software expert. We must work to destroy this stereotype if we are ever to be able to lead.

Figure 17.1 Licensed to kill the stereotype

Here is a diagram reproduced from Corporate Politics[11]. It shows how we must climb from service delivery through project delivery to have any influence on business thinking. Operating at the bottom of this graph means IT is a cost, likely to be controlled by a Financial Director and misrepresented at board level.

If we cannot kill our stereotype, then we are destined to languish in service delivery when we should be using technology to transform business thinking: to be pivots upon which the world moves.

To move upward collectively is to tread a long and difficult path. Many have achieved it individually and we must look to those who have succeeded and see what has carried them on their upward journey. The answer is highly unlikely to be technical excellence. It is embodied above, and you may find it somewhere above. Equally, you may have your own path suggested by a mentor, or come across it through luck or grind.

If you are looking for a path, [11], [12] and [13] may help you on your way.

I Love my IT department, and they love me

To borrow again from Corporate Politics, have you ever heard anyone professing love for their IT department. Has anyone ever thought they are being cared for by the department as a whole?

It is unlikely. Often, someone will find someone helpful in the department, and forever more they will become their pets. When something needs doing, they will always want to speak to the same person. The reason for this is that the IT department is usually a personality-free exchange of technical geek speak, ending with 'Have you tried rebooting your machine?'

Have you ever heard anyone say ‘My computer hates me’? Hate, a computer? A logic based on and and or can hardly hate. It may not do what the user wants at some particular moment in time, but it is emotionless silicon, or it is as far as we are yet aware. Computers simply do not hate. Yet people think they do.

People are reflected in their observations. All black dogs are mad. Making ice cubes takes no energy because the fridge is already cold. It never works for me. Modern music is rubbish. It's much slower than it used to be. My computer hates me.

Imagine you are one of your own customers. How would you feel if you rang up the helpdesk and they told you to reboot your machine then call back. How would you feel if your particular computer hadn't behaved properly for three weeks and there was a glaring announcement on the soulless pre-recorded countdown to you being connected to a real person that claimed:

99.97% of calls were responded to within the required time, and we have achieved a 99.98% up time over the last year

Would that help your particular problem? Would you think that your IT department loved you? Would you have warm feelings of love every time you felt that you could call them?

And, in the grand scheme of life, where does it leave you? As Covey[12] bids us to do: begin with the end in mind.

Figure 17.2 The Pearly Gates

If you want to know why Martin (The techie hero of Corporate Politics[11]) is a bean can with headphones and dark glasses, you know where to look.

Stop speaking in jargon

Our greatest enemy is our obfuscation of the issue, as we hide behind our clever terms for our various technologies and methods.

Here, reproduced in full is Ronald Kohl[14] on

Does anyone here speak English?

“Mr Watson, come here, I want you.” Hearing those words transmitted over an apparatus being tested by Alexander Graham Bell, Thomas A. Watson realized that his boss, Mr Bell, had at last successfully developed the telephone. Mr Bell had not intended to have the terse request be the first words uttered over his invention. However, he blurted them out in surprise when he accidentally spilled acid as he prepared for the test. The rest is history. What is not history is the conversation that followed.

WATSON: Mr Bell! We have just invented the telephone! What a glorious Day!

BELL: Yes, we have. But we aren’t going to call this apparatus a telephone. That wouldn’t fit our business model. We will call it a telecommunications device. We don’t want people to call our firm a telephone company. We want them to call us a telecommunications service provider.

WATSON: But Mr Bell, why use such a stupid obfuscation? It’s a telephone. And we will have a telephone company.

BELL: You just don’t get it. Ever since the telegraph was invented, we have been in the midst of a digital revolution. The Morse Code, Watson – that’s digital. This contraption I’ve invented is analog. We have started an analog revolution. The

whole country will be wired. Everyone will get online, or else they will be left behind.

WATSON: That means that we will have to get a means to connect these online users. We can hire ladies to operate our network and help our users log on. We can call these ladies ‘operators.’

BELL: We can’t call them ‘operators,’ Watson. What we are dealing with here is electronics. We will refer to these ladies as real-time voice interfaces. We will promote our telecommunications device as being voice-enabled. You see Watson, everyone wants to be enabled today.

WATSON: As you say sir, but down the road I’m sure we will be able to get rid of the ladies and let people get online by dialing a number on a rotary device.

BELL: Now you’re cooking, dude. And we’ll promote the dial as an enhanced functionality solution. Everyone wants functionality today. They especially want enhanced functionality, and most of all they want enhanced functionality solutions.

WATSON: When we invent the dial, we’ll have to assign numbers to our customers, and we can list the numbers in a publication called a phone book.

BELL: Wrong, Watson. That book of numbers will be called a hard copy application enabler. And if you don’t have a hard copy enabler, you can ask our voice interface to look up the number, and that will allow her to provide online help in real time. That will provide even more enhanced functionality.

WATSON: We may disagree on words, Mr Bell, but I think we agree that our telecommunications device will change how people do business.

BELL: Right. People will be able to ring up our voice interface and buy things online. This will be referred to as ‘calling in an order,’ and that, in turn, will let users form collaborative supply chains. Of course, some potential users won’t understand our telecommunications device, and they will set up cultural barriers to change. By the way, I agree that the asinine vocabulary I’ve been using makes me sound like an imbecile. But I have a premonition. Some day everyone will talk like this, regardless of whether they are users, providers, enablers, or enhancers. So get used to it. Don’t set up cultural barriers to change, Watson. Get ready for a new paradigm.

What’s on the horizon?

We have four horizons to view. The technical, process, method and business horizons.

On the technical horizon, we’re likely to see great excitement, and even some slight functionality coming out of the 3G and other related work. Certainly, it will create a higher degree of on-line access in portable devices. We will also see the continuing convergence of computers, televisions and telephones.

On the process horizon, we shall see ideas continuing to merge, and although different methodologists will continue to speak their different languages, the developers will follow the path of least resistance.

The method horizon raises a few questions. We’ve gone OO, but we’re still on the journey. Web pages have seen OO take a back seat to delivering software.

On the business horizon, we can be clear that people will want more information from more points of view, sliced and diced into ever increasing complexity. They will want business solutions quicker, and more information and functionality available on the move. Our preference is moving towards small and free over bloated and not. The true costs of free have not yet been worked out.

Claims that My kid brother can create a website will not disappear, but the tools will become ever less affordable. Kid brother websites will be insecure, database free (files are easier) and go unsupported when the said kid brother takes up rock climbing or skateboarding.

The tools we use will become ever more creative, and ever more similar. Perhaps we will give up our monolithic word processors and create a more harmonious environment to create our paper equivalents in. We must be careful that we do not stifle creativity. For the joy will leave us, of tearing out a piece of paper with a message gone wrong and tossing it in the general direction of the bin to further clutter our floor. As will the joy of the warmth in our faces when those scrapped up balls of paper burn, bringing light and life to a darkened room.

Uniformity is on all horizons. Perish the thought.

The way of strategy

Software people generally like logic. If this, then that is quite understandable. However, life is not always logical, the practices of the intellect not always predictable, measurable. There is not always a right answer. Right is only the slightly less fuzzy amongst the fuzzy.

You must study the strategy of being a software architect. So here, in the style of Musashi[9], though rather more koanish:

On paying the right price

If you pay low, or if you beat someone down to a price lower than something is worth, your reputation as miser/negotiator will grow. The next time you begin negotiation the starting price will be higher. You will end up negotiating your life away while the decent payer is offered and pays the right price every time.

If you pay high, you are cheating yourself. Your seller is left with a short term windfall which, depending on their nature, they can absorb with glee, or begin piling in extra 'free of charge' items to equate the false price with the correct price.

You may think those who absorb with glee far outweigh the balancers.

You must pay the right price. You must come to recognise it.

On choosing

In the art of software, you may have made some good choices; they may have been purchases, approaches or designs. At other times, you may have made some mediocre or bad choices. The fact is that many different methods, approaches or designs can be made to work, with varying degrees of success and ease.

To make things work better, and be easier to do, you must choose well. It is that simple.

You must first choose how to choose.

On doing

You may have heard that if a job is worth doing, it is worth doing well. Similarly, you may have heard that if a job needs doing, do it yourself. Or you may proclaim that if a job needs doing, and doing well, you should pay a professional to do it.

Doing software needs all of these doings. Only you can decide who the best doer is. If you follow one path to the exclusion of others, it is likely your software will not get done.

You must balance the triangle of doing, to do.

On winning

For you to win, another must lose. This is a falsehood, perpetuated by competition. If you love another, and they love you back, is one the victor and the other the loser? If you fight another and they fight you back, and you both die of injuries, is one the loser and one the victor.

You can deal with another, and both win. If we all work together, can we all win?

On volume

Silence in though, silence in action. Stillness in movement, or relaxation at speed, is the goal of all the great movers. Silence in thought, the volume of authority in command, the volume of equality in discussion.

Twice silence for one of speech. You must consider this well. The volume of authority is never loud.

On undermining

You can only undermine yourself. Think on this.

On talent

Talent is rare. Doing something with it is rarer still[15].

On progress

To reach the ground, the parachute and its weight are interdependent. Too light a weight will blow away in the wind. To heavy a weight will break the parachute. The ground will break the weight.

You must consider what, internally and externally, is parachute and what is weight trying for what ground. And you must consider who, what, when and where is the ground.

On navigating

To drive a car you steer along roads. You turn at turns. You accelerate and break. You consider other drivers. you avoid holes, verges. You do not drive off the road into the ditch.

Software is a complicated road. It lacks street lights, cats eyes, white lines and curbs. You must steer well.

On learning

A tree may bend in the wind, but it does not give up its branches.

On arriving

To reach a destination better, you must know the place you are travelling to, and how to recognise it when you arrive. You must know the best route to take, avoiding the many obstructions between you and there.

We are all on many journeys at once. We travel our personal journeys, our relationship, career, job and project journeys.

Beware. All journeys converge. Treat each with balance and respect for the benefit it may bring.

References

1. The Meditator's Handbook. David Fontana. Thorsons. 2. Shorter Illustrated History of the World. J M Roberts. Helicon. 3. The Bhagavad Gita. Translated by Juan Mascaró. Penguin Classics. 4. The Republic. Plato. Many imprints. Verses 427 to 434 are the source of

the four cardinal virtues. 5. *

6. Matthew verses 5 to 7. 7. Proverbs. 8. Psychomachia. Aurelius Clementis Prudentius. Many imprints. 9. Go Rin No Sho, or The Book of Five Rings. Miyamoto Musashi. Many

imprints. 10. The Sunday Times. July 1st 2001. 11. Corporate politics for IT Managers. Keith Patching and Robina Chatham.

Butterworth-Heinemann. 12. The Seven Habits of Highly Effective People. Stephen R Covey. The

Franklin Covey Co. 13. The Power of Focus. Jack Canfield, Les Hewitt, Mark Victor Hansen. Health

Communications 14. Machine Design. Ronald Kohl, Editor. June 2001. 15. Eternal truths for every artist. Harley Brown. North Light Books.

* if you know the origin of the theological virtues, or the origin of the collection into the seven heavenly virtues, I would be grateful if you would let me know.

Chapter 18 - Aardvark & Aardvark, Solicitors

Aardvark & Aardvark is an international consortium of law firms with a head office in Washington DC and regional offices throughout the world. The members of the consortium are all small, privately owned companies. The largest has around five hundred employees. They are gathered together under the umbrella of Aardvark & Aardvark to be able to compete internationally with much larger firms.

All of the members are convinced that their business model is the best, and despite each member being a small player in a big pond, collectively they manage a portfolio of accounts up among the top players.

Their local small business approach allows them to win small and medium sized jobs. The Aardvark & Aardvark consortium allows them to win the largest too. Alphabetically, they are always on top.

In this final section, picture yourself as a newly qualified, or promoted, Software Architect. You have joined a medium sized software house, Spaggot Software, and your introduction to the world of sales is about to begin.

The pitch

Your watch ticks gracefully past 10am, Monday morning. You arrived at the head office of Aardvark & Aardvark a few minutes earlier with Jim Sailmaker, the chief

salesman at Spaggot. It is a leafy July, and the air, usually quite hot at this time of the year, has held off it's asperity, making it rather more pleasant than usual.

You are nervous. Your hands are sweating. Your suit is clinging and you would rather be spending Monday morning as you usually do, buried in your comforting mountain of email. Its non-confrontational ease slides you back into the working week after the weekend world of children and games, relatives and visits.

The children this particular weekend sapped your usually boundless energy in annoying mind games where you had to get them to guess the word at the top of the card without using any of the words at the bottom. Children lack the years of waste gathering the rather pointless knowledge that would have helped them (or rather, you) succeed at this.

Figure 18.1 Hospital card

Now Jim’s got you at it again, except this time you’re not allowed to scream and pull at your hair, or stare pathetically at your teammates, saying: "Come on, it's obvious." And cheating is simply not an option.

Figure 18.2 Software system card

You may think everyone knows what a network is. They do. You know it’s a pile of cables, switches, patch panels and routers. A solicitor knows with equal conviction that it is the networked interconnection of legal professionals gathered within the umbrella of Aardvark and Aardvark, who exist to uphold law and order, and the rights of the citizen.

Bringing down the network to you means stopping a few routers. To an Aardvark & Aardvark employee it means destroying the fabric of society, business and livelihood.

Jim. Kind, friendly Jim, has brought you out prospecting. A large piece of work is in the offing, and Jim has been invited in to tender. He has worked with the company previously on quite a few assignments, so knows them well. You don’t, although you have some background provided and enthused over by Jim. He

generally exudes enthusiasm, in anything. He lives for the interaction of people, and the joy of being able to help people (i.e. make a sale). He has no worries about the logical, painfully precise world you have spent your last decade immersed in. He’s a people person. To grow, he must understand people better and spend more time understanding others' points of view. To grow, you must do the same, but you are likely to find you are far behind everyone else. They never had to master FORTRAN, C, HTML, SQL, remote procedure calls, memory models, remote method invocation and twenty different scripting languages. All the time you spent doing that, others were learning what you are now trying to learn.

The sales meeting

Jim has quite a presence and a strong personality. He would never dream of promising somebody something, and then allow you not to deliver. You and he are a team. He sells, you deliver. It’s as simple as that. There’ll be no help coding a few HTML pages in return for you helping with the sale. He is mister high integrity, low technology and you are his technological shadow. He is here to gain the trust of his customer, and will do so using nothing more than his persona. Somehow you have to be an extension of that persona, a technical foundation of nods and explanations upon which he can make the sale. And don’t mention networks. Java is a drink. C++ is a worse grade then B--. A user takes drugs, layers go in cakes and sedimentary rocks, and a table is an object upon which cups of coffee will be proffered.

You are meeting the CEO and founder of Aardvark and Aardvark, Howie Seer, and the COO, Jeff Striker. Despite their 1824 foundation claim, they were in fact founded only ten years ago, off the back of a failing solicitor's office which was indeed ... "founded even as Byron’s swamp fever overcame him. Thus, the tally of English romantic poets to have died abroad was brought to three in only four years." Jim is about to regale you with Shelley’s drowning, when Howie comes within view.

Jim and Howie have never met, but have spent some time together over the telephone, and Jim has done his research. Suddenly Jim fills the room. It’s his style. Big, loud, all inclusive. Even on the phone, everyone in the room is allowed into his conversational aura. “Howie. Great to meet you at last.” A large hand and broad smile pin Howie against his wall. He seems not to notice.

“Jim. How do you do. Come on in.” He subtly evades Jim’s magnitude and allows you to shake his hand.

“How do you do. Bim Brimbles.” You do your best not to lose your tongue in some hidden recess of your mouth as you introduce yourself. “I’m a software architect with Spaggot Software.” Logical. Complete. Unfortunately, Howie now thinks you design buildings and wishes Jim had brought someone to help with the computer system.

You are led through a set of gleaming, skillfully etched glass panels, behind which busy and important people bussle and import. In the corner with a view right down to the Citadel, you flump into a large square of leather furniture surrounding a polished white marble table. From your sunken position, you can see the tip of the Washington monument reflected, pointing at you. You do your

best to prepare your nerves of steel, swell up to the size of the others and with a deep breath, lay out your artfully folding notebook, then lunge to your feet to welcome Jeff Striker to your meeting. You’ve heard he’s the deal man. His ideal price is a single penny. He’ll probably spend the whole meeting telling you that their existing software is all they need, and any new software at this time in the financial year... Well, times are tight, they don’t need you... But why is he sitting with software sellers if this is the case?

Howie lays out the scene. “Jim, I’m going to give you an idea of what we’re trying to do, and what we need from you is a price range and idea of how long it is likely to take. We have three other software houses coming in, and we’re doing exactly the same with them. Following the first estimates, we shall invite two back and work with them to fill in the details, and then proceed with whomever we feel is most likely to deliver the best system. This will be a contract with a fixed price and fixed delivery date. We will be writing into the contract penalties for late delivery and software defects.

Your knees are trembling. You want to say but... Stop it. This is the way everyone else works.

Price ranges are dangerous things to give. As soon as you get to the and between one hundred and one million, buyers have stopped listening. They hear one hundred, and graciously use that as a starting point to beat you down.

Howie continues. “What we really want from our new software is the ability to manage jobs across our network. Although we claim to be a worldwide group, we are little more than a loose collection of independent companies. We want to be able to manage work across two or more different companies in different locations. We have been working over the past two years to get all of the A&A partners singing from the same hymn sheet. We all fill in the same timecards and expenses. We all predict our workload and manage and cost our work the same now. What we do not do is use the same software. We also hold the same information in many different places. For example we enter meetings in our calendars, represent that time in our time predictor software, and then enter it on a timecard. Our business concept is fairly simple. We work for a customer, then charge them our time and expenses. Time is costed a number of different ways, but there is little complication in that. We also want to be able to view how we are doing internationally by customer type, contract type, region, identify our leading and failing group members, and be able to distribute bonuses to our best, and by best, I mean most profitable, partners."

"It's quite straightforward really," Howie finishes.

Coffee arrives. "Excellent," announces Jim. And, indeed, excellence often is as excellence is proclaimed.

After the round of coffee presentations, Jim slides into the vacant hole in the air with a long hmmmm. He sips his coffee. No-one interrupts. Your head is feeling very light, clinging to a faraway image of palms before the white sanded shore of a tropical hideaway. "What we like to do in situations like this is to fully understand exactly what we are dealing with. Without that knowledge, our estimates will be increasingly inaccurate, and for both our sakes I think we need

to spend some time in investigation during today. If you like, you can then use what we get to present to the others coming in. Good old Jim. Big hearted, generous Jim. You feel a little uncomfortable about other software houses seeing your models. What if they have some mistakes on them? What if you don't manage to create any?

So far you have written down: Time, Expense, Bonus, Calendar. Customer - type. Filter by region, contract type, customer type. Profitability. Partner. It's not much to go on.

Jim once again vanquishes the enemy silence. "Howie, Jeff. let me run through what I believe is your business model. Jump in any time you like." He begins, having already picked up some of the jargon. "A & A is composed of one hundred and forty partners in the capitals and major cities of thirty eight countries. You supply services to your clients by way of a single costed contract, and your sales model is by isolated contract or by account, where a customer holds an open account with one of your partners and payments are made by monthly invoice. Contracts are paid either at completion or for time and expenses weekly during the contract. Some smaller contracts are paid on completion of the job."

Jeff opines: "Some of our partners even run no win no fee contracts, but we generally discourage it as all it can do ultimately is double the price of hiring a lawyer."

"Thank you for that Jeff," Jim notes, then continues. "Altogether your partners work from one hundred and eighty offices using leased line, IPVPN and dial up networking. You employ, collectively, four thousand solicitors and another thousand support staff."

Jeff comes in once more to further explain their charges. "There are a number of legal aid schemes in different countries, and our charges for those are dictated by the schemes rather than by us. We also farm out work to company formation agents, trademark and patent specialists, and our associate partners who work for us on specialist cases as external contractors. We recharge their costs to the client with or without a mark-up. Rates differ between partners and between locations."

Howie continues, getting down to the nub of the matter. "We want a complete computer system to run in our offices worldwide. It must support all of our activities through client contact to job completion, and be able to drill in to global sales information in any way we choose. What we most want from the system is the ability to be able to charge all of our clients for the current week by sending out invoices at close of play Friday. We are losing millions if not billions worldwide by delaying the issue of our invoices by anything up to a month." He leans forward slightly, as if in a secret communication. "We're not the software experts. Tell us what we need."

Slowly, as if in realisation of the fact that you are there and must have a purpose, all eyes defocus from Jim, swivel through varying degrees, and refocus on you. You gather your courage and prepare to deal to a swift death the stereotype they expect to emerge. You pull on your business savvy air, and announce: "Software is a matter of process. First we need to scope the work, then we will gather the

requirements, analyse them to make sure they are correct and agree them with yourselves. Then we shall design, write and test the software. Wherever possible, we shall work with your people to ensure that they get the software they need, but we will also keep you fully informed so that we deliver to business needs rather than losing ourselves to personal wishes." It sounded good. You move on.

"If we could spend an hour or so this morning fleshing out a high level view, we can then decide who else we might like to speak with to sharpen up any areas we are loose on. I think that would be a good starting point."

Howie announces a lunch date at 12.30 and departs with the announcement that Jeff is the operations man, and he will just be in the way. He has put first things first, synergised, and now got out of the way so the job can be done.

Taking the lead

Out of complexity, simplicity. Out of chaos, order. Out of a vacuum of leadership, a leader. You.

"Jeff, do you have an organisation chart?" you ask, bearing in mind Conway's law. It gives you a breather as Jeff finds someone to find him one. "Jim, what do you think? Shall we go ahead?"

"You're the man Bim. Do you need anything?"

You are still shaking your head when Jeff returns. He is rather fond of the organisation chart, especially his own lofty position upon it. "Jeff, thanks." You flick your eyes over it as he passes another copy to Jim, flops down into his chair and takes up his coffee again. "Why don't you talk us through the major roles represented by the people on the chart, and fill us in on the personalities as we go."

Identifying the roles and responsibilities is your first priority. It could probably be shown by facts and figures that the mapping of people to roles is anything but one to one. It is more likely that some roles will be filled by more than one person, and some roles will go unfilled. The unfilled ones are where members of related departments get to think each other are slacking, and the overfilled ones are where members of related departments get to think that they are the superior and rightful fillers of the role. You are not worried about people. People exist across the divide and lie in the problem domain of the business. Roles lie equally in both the business and your domain.

"Up at the top is Howie. He brought A&A to life and is very much the driving force behind where we are going. As Chief Operator, I report to Howie, as does the CFO, Charles Simian. I'm afraid Charles is over in Europe at the moment, but if you need to talk to him, he'll be here all next week.

Only Howie works directly for the A&A network. The rest of us sell our time. The country heads and practice heads all lead their own offices, and report to Howie as partners in the network. We have a geographically hierarchical reporting structure and also an informal skills structure, where members interested in the

same areas of law can gather physically from time to time to improve their expertise and worldliness."

"Jeff," you break in, not really wanting to break the flow. "A&A is the name of this law firm. How does that tie in with the A&A network." It's a dumb question you felt needed to be asked.

"Ah," says Jeff. This law firm is registered as A&A. We also have registered the A&A Consortium, what we refer to as the network. We are all employees of the A&A law firm, but sell our services to the A&A Consortium. Howie is our CEO, and also chairman and CEO of the A&A Consortium. Mark McMoneagle is the far sighted chap responsible for selling the A&A brand to our partners and the rest of the world. He is A&A's, that's the law firm, marketing director and fulfils the same capacity in the A&A Consortium, but only on the basis of selling his time. All partners sell their time to the network, and all partner firms pay part of their profits towards its upkeep.

What we really need is the ability to buy up smaller companies, or even other members of the network, and merge them in to our business. In some cases within the network, this has begun to happen, but many prefer to be independent yet connected."

Jim seizes the moment to guide Jeff back to the organisational chart. You had become lost in the story.

"Yes, apologies. Strayed a little there. Still, all good stuff.

Where were we? Yes. All divisions report in to me apart from finance. I have purchasing, marketing, our senior legal partners are responsible for sales." Jim squirms in his seat and begins to meditate deeply on how they survive without such key people. "Our admin staff look after our computer systems, accounts, personnel and salaries. We also have our knowledge services, which we see as a key part for expanding into a global operation for the network. We'd really like to have an internet service where anyone can go to A&A's website and be able to look up any point of law or partner in any nation worldwide. That would be a real boon for us."

You decide not to pursue this exciting opportunity yet.

"Our legal divisions break down into what I like to term 'success centres'." Jeff does the speech marks with his fingers. A success centre is a self funding part of the business, and area of law. Essentially, the owner of each success centre runs it as a separate business. They pay into the core admin and building fund, but are largely responsible for their own welfare. Some have their own administrators and software systems.

The systems are becoming a bit of a headache. Ideally we would like to see it all centralised, but many of the success centres are not keen on sharing their hard won client lists and contacts with others. From our point of view, it would be marvelous if we could maintain a global client list for the whole A&A network. Imagine being able to go to make a sale with Ford and be able to say we had just

won a similar case for them in Singapore. That would really make the A&A brand sing.

Of course, each success centre has to be a success. That means they have to be profitable, and the senior partners who drive that profitability get a significant cut of the profits. Some success centres focus on areas of law such as personal damage, litigation, and others work specifically for a client. A&A members are in thirty percent of the top hundred companies in each country where we have a presence."

You take in this vast knowledge and throw down a few words. "What about the more mundane aspects of business administration Jeff? Most software systems appease the administrators rather than the high flyers. Let's take a look at the oil for a while rather than the cogs."

Jeff gallantly returns to the chart. "HR looks after hiring, paying, training and the well being of our employees. Finance makes sure the money is flowing. Marketing advertises our services although our best marketing is done by word of mouth. Their task at the moment is the worldwide promotion of the A&A brand. They do nice brochures for specific areasof law, or even for particular clients, and look after prospecting databases. We have been discussing a worldwide effort to reach out to our customers with an email or online newsletter.

Our systems people run around fixing machines when they won't go. They swap cables and fix printers.

"Can you break down your systems people into numbers, so I can get a feel for the work they do?" You ask.

Well, we have Mike, our IT Chief. He's not overjoyed with us having you all in. He wants to do the work himself, but we think the job's too big and need specialists. He has one rather talented chap who does all of the whizzy bits." Jeff pops his fingers like fireworks to help give the impression of whizzy software. Then there's the team who look after PECAN and the VAXes. PECAN does all our invoicing work at present.

It sounds like PECAN can help you with your requirements. "Do you have any technical documentation for PECAN?" It sounds hopeful.

"Hmmm. Not really," Jeff admits rather bitterly. There's quite a bit of training material, but the developers are... Jeff tails off, not wanting to offend.

"It was all a bit of a joke really," he brightens again and leaves the documentation thorn behind." Howie was always asking for things in a nutshell, so the software became the Process Engineered Computer Accounting Nutshell. Rather meaningless. We put in our time and expenses. It puts out invoices, monthly reports and does budgets. Our predictor software runs on a UNIX box. It's a bit troublesome. Every now and then it falls over and the machine has to be taken apart. I'm not sure why. There's only one person who knows anything about it and that's Mike."

"A copy of any training manuals you can get hold of would be useful," you venture. Jeff agrees to get some to you before the end of the day.

Coffee arrives again, and an hour and a half has passed. Jeff's a little dry, and your eyes need unglazing. As Jim does the honours, you stretch your legs around the room.

"I've arranged for you to spend some time with Doug Marshall. Doug the digger, he is affectionately known as, for his ability to dig down to the core of any legal problem or dispute. He's one of the finest minds in modern law, he tells me. No doubt he'll tell you that too.

After the coffee is cleared away, you are led once again through the bustling and typing throng, then up the stairs to a quieter world of contemplation and large library sized book cases, where Doug the digger resides.

After the pleasantries Doug invites you to a cool area around a coffee table and allows you to sit down in full view of the many framed certificates he has half decorated a wall with. The other half shows him shaking hands or standing meaningfully beside the worlds dignitaries and stars. Some are recognisable, some not. All of them make you feel small and unworthy of spending time in the glow of such magnitude.

The solicitor's day

"What I do is work for any number of clients, although I make them feel as though I am their own personal lawyer. I keep a piece of paper at the top right hand corner of my desk, and every time I change from one case to another, I write down the client, and the time. At the end of every week, Fiona, my PA takes all the pieces of paper and puts my time into the system. The system then sends out invoices and somehow through the inner workings, calculates and provides my salary. I'd like a computer that I can change my client easily on and not have to worry about the paper. It's pretty good, but sometimes I am working from home, or at a client's office and my time can get a bit confused. Luckily I have a good memory, but if I didn't I'd have been stuck a few times. I'd like to record it in my watch like James Bond. That would be real useful."

I recharge expenses direct, and I don't use the damned Predictor, much to Howie's frustration. It's all in my head, or mostly Fiona's head, where I have to be and who I have to talk with."

"I would like to be able to select a client I am working for, then when I change tack to another client, I want to select the other client. I don't want to do anything more. Let the computer calculate the time I spend working for them, and how much to charge them. That's what they're good at after all."

You nod your agreement. "What about when you break for lunch?"

"I'd select lunch. When I returned from lunch, I would select another client, or continue with the same one. When I go home, I'll select home. Although I leave

every day at 6pm, so if I forget to choose home, then I expect the computer to record in my absence that I have gone at my usual time."

"How about the time predictor," you ask. "What do you use instead?"

"The reason for the Predictor," Doug observes, "is to find someone who is not allocated to a piece of work, and allocate them. I am booked up months in advance. When I get booked up too many months in advance, I put up my fees."

"And claiming expenses," Doug continues without prompting, "is really so tedious. Incidental expenses are built into my fees. I only recharge flights and hotel bills. I believe it should be beneath one's dignity to waste time claiming for such petty amounts. We all live well enough. You rarely see a poor lawyer. And when I say I claim, I mean Fiona claims. The bills are sent straight to her. Fiona..," he rings out in a sonorous cal, and moments later Fiona has appeared. "These gentlemen would like to know what you do with my expenses."

"Doug's expenses come straight to me. I enter expense items singly as they come in, rather than let them pile up, and they are recharged to the client each week," Fiona elicits.

Doug is a rare soul. He has polished efficiency to a high art to enable him to fit life in with his highly important career. He goes straight for the heart of the matter every time. Even his door is paper thin so Fiona can can overhear every conversation, and none of Doug's time is spent reiterating conversations which were hardly interesting the first time around. His philosophy is the older you get, the more efficient you have to be. Older people have less time to play with than the young.

"If an expense is for a Government job," Fiona continues, "I record the expense against the task."

"Can you define a task?" you ask, catching on to Doug's method.

"All work for the Government is broken down into tasks, and time and expenses must be booked against a task. Mostly we just book to the job, which is for a specific client, but the Government has its own rules. Any changes to time booked on a Government job must be recorded with reasons for the change."

You see a long road developing and head it off sharply. Perhaps we can meet this afternoon Fiona, and you can run us through your weekly interaction with your systems. We have a slot three-thirty until four. How does that sound?

Fiona agrees and vanishes through the door which billows and puffs as it opens and closes, exactly the way paper would.

"What about knowledge, Doug?" you ask. What would help you find the information you are looking for, and how would you go about adding to the knowledge held by the firm?"

"Well the knowledge I need is rather superficial. Conversation in chambers is as much contact as I need with the letter of the law. My work is with people, and the rather diverse ways in which the truth can be extracted from them. A single word at the right moment can swing a case. I am the expert of the word and the moment, not dreary legal proceedings."

"Let me run you through a typical case. I get a call, usually from the person rather than the representative - he sweeps his arm across his array of faces. They tell me what they want. I tell them I'm expensive. They say price is not an issue. I ask them about the case. They tell me. If it sounds interesting, or I get a twitch in the back of my neck, I take the case. If not, I refer them to our highly skilled and rather impressionable pool of talent.

By this time you have decided that Doug may be a master of the law, but he is not much help defining a computer system. It is not a path you have discovered, however, more one you have been led down.

"Bim, Jim, I want to thank you for your time, but now I believe Fiona will of far more use than I explaining the machinations of our company.

Fiona appears and leads you through the paper-thin door.

She leads you through Predictor, the time and expenses system, and from there through the management reports and general expectations of the partners and senior personnel of A&A. You are dotting your last i when Jeff arrives for lunch.

Lunch

You are taken to a rather agreeable place only a short drive from A&A's offices. Over lunch, Howie asks about your approach to writing software.

You resist the urge to leap to your feet and start drawing diagrams on a whiteboard. "As I said earlier Howie, it's a matter of process. Software development only ever becomes a problem when we lose sight of the process.

First, we scope the work and create a definition of the project for the project team. By project team, I also include yourself and your people. We are in this interdependently and neither of us will create a decent system without the other. Together we agree at a large grain, what is in, and what is out of the system.

With a good idea of what you want, I will create the architecture, and we match the architecture with a programme plan. This is essentially a list of projects we need to do to build your software, with risks, assumptions and dependencies mapped out.

When we have the architecture and the programme plan, we create a set of projects. Each project is mapped across your business initiatives and also attached to an application, so that each project will build a partial or complete software application.

At this point, we estimate the size of the projects using a technique which ascribes complexity points to each piece of work. Once we have the size, we have a price, and we can then further negotiate on timescales, cost and scope.

If you go for our quote, we can then begin the projects. We have two mechanisms for delivering those projects, based upon your preferences. Our preferred method is to get our software developers to come and sit alongside the people they will be developing software for. By doing so, the communication is allowed to flow free, and the resulting software better suits the needs of the user.

This technique is often, but not always the best. We wouldn't put software developers in with air traffic controllers. We may choose not to put them in with your sales teams. Better they concentrate on their work, not on software. When this is the case, we come in and do some requirement gathering, usually as interviews or workshops.

We analyse requirements to ensure they are complete, logical and deliverable and that expensive requirements, or those which increase the risk of non-delivery or poor performance, are reconciled. Analysis is followed by design, coding and testing. Where we can, we approach this iteratively, ensuring at each step that we are on target and delivering the software you need. You may find during the development your business changes. There is little point us continuing to deliver software that meets out of date requirements. As I said, we're in this together, and we want you to get a system that reflects your needs upon delivery, not at the point in time that we happened to come in and gather requirements.

Whatever method we use, and there are benefits and drawbacks to each, we create your software with as much interaction as you prefer. Sometimes you may wish to dictate approaches, sometimes you may be happy to let others. As long as we continue to interact and visit each project as it is being written, you will get your software.

Jim has a tear in his eye. He thought all techies were nerds. His stereotype of you is dissolving. You haven't publicly disagreed with him. You're not a liability. You have even helped him with his sale.

Two rare things have just passed. Jim is speechless. Howie and Jeff have an almost clear, if short lived, view into another world. In their minds' eyes the black art of software was momentarily transformed transparent.

The afternoon interviews

The first stop post lunch is with Eli, the chief whip. You failed earlier to extract her true job designation as Howie and Jeff were sharing a private joke you could not penetrate. The beady eye she fixes you with adds to your discomfort and she manages to not let you settle properly in your chair. Jim has gone, Jeff vanished like a will o' the wisp. It is you versus Eli, round one. The bell rings...

Don't speak until you're spoken to was the cruelest trick played on us as children. It was designed to keep us out of trouble, but somehow stretched into our

adulthood where it doesn't belong. It is a dragon in need of a St George. "So Eli, where's the best place to start?" You don't know. She might. It's worth a try.

"At the beginning." A sterner eye, though in it is a glint of mischief, even humour. You catch it only because it sits on your own face while you are pummeling your children into a sense of what might be construed as tolerable behaviour. You share the joke. "OK Eli. The beginning for us is having people like you who pay us money. For your gracious reward, we deliver you software which we expect will save you significantly more in the long term than you have paid us for in the short."

The scowl deepens. The sense of outrage spreads across her face at the mere suggestion of money going out rather than coming in. It is a deep personal affront to her to see the scales of commerce tipped, however briefly and for whatever reason, toward the outgoing. Income was what she cared about. Income and the people around her, and the business she ran, and the people for whom she ran it. But the stony face is betrayed by the mirth beneath it.

Most businesses run along similar lines. They sell. They supply a product or service to their customers. Their customers are sent invoices, which with varying degrees of success, get paid. All of the products delivered, services rendered, invoicing and state of payment is recorded and used for accounting and business intelligence. Internally, people join, are gathered into functional groups, rewarded for their contribution, and leave. In the business, that's about it. Beyond the business, the balance of income and outgoings is fed to the world for taxation, shareholders and financial analysts.

"So tell me about your customers."

You may equally have opened by asking about customer records, but that would have been starting with data. When an artist paints a picture, he begins by defining shapes, then light, then colour, then detail such as highlights. When an engineer designs a vehicle, he defines the shape, and the human compartments long before worrying about the fact that something has to move it along a road. An artists does not draw the reflections on the concave and convex surface and then create a bottle around it. The engineer doesn't worry about the pattern in the tyre treads, or the type of carburetor before the vehicle itself. You are artist and engineer. You begin with shapes.

"Well, Bim. You might be one of our customers. Is it Bim? Is that correct?"

"It is. Bim is short for Bimford, my mother's maiden name."

"You might have committed some dastardly crime and require our representation. You might have been accused of doing so, and need our help to prove your innocence, or you may have a legal dispute with a third party and want us to argue your case. You may also use our services for marital problems, medicare, employment. We used to have an old brochure which listed everything we did. We're a little more special these days though..." She rummages through her papers and produces a folding brochure, stating quite clearly what they used to advertise as what they did.

"Have you changed the way you work since this brochure was produced?" you ask.

"No, we just don't say what we do any more. We do everything. Jack of all trades. Master of all too, with our network. We can do anything for anyone, in any country. And often we do too."

The brochure states:

Aardvark & Aardvark: local service, global reach. There is also an attempt at a cool but friendly how we can help you.

Litigation Health care Business

Appeals Complex Commercial Litigation Employment Fraud and Abuse Medicare and Medicaid Reimbursement Medical Malpractice Product Liability

Anti-Trust Academic Medical Centers Employment Fraud and Abuse Medicare and Medicaid Reimbursement Medical Malpractice Mental Health Regulatory Compliance

Anti-Trust Employment Medicare and Medicaid Reimbursement Organizational Structures Regulatory Compliance Intellectual Property

Eli continues. "We also do USAID work and Government work, and all jobs are run through our computer system. Would you like me to show you how it all works?"

You prepare to take some notes. "First we have our client database." Eli opens the client database on her screen. It looks basic, but appears to do the job.

"Do you hold contact relationships, such as Jack Hawkins, Manager, and Sophie McGregor, Jack Hawkins' Secretary, so that when you search for Jack Hawkins, you get both?"

"No. Information like that is held by the practices. Jeff calls practices success centres, the rest of us call them practices." Some of the partners don't enter their contacts in the system as they don't want to share them, or they don't want someone else getting at their clients, unbeknown to them. Many people guard their contact information very carefully. This contact information is quite out of date now. Most practices use their own contact lists, on paper or spreadsheets, or systems they have bought or written themselves."

"So no-one uses the contacts, but they do use the clients?" you ask.

"Yes, all charges are booked to a client. That is done by administrators in the Booker system or by employees using TEX, which books time and expenses. Predictor is used by most of our lawyers so we can see who is available for upcoming work. We run Invoicer every Friday, and depending on the client or the piece of work, invoices are raised and mailed out close of play Friday. Invoicer

also creates files which are imported into the accounting system. Booker also creates files for payroll, which is run in the HR system. Most of the management information comes from the Reporter which is a series of reports we print off each week and mail to the senior partner group."

"We also create invoices on Fridays for work we do with other A&A members. That is a manual process at present

"Then there is the infoweb, where we store case histories, publications, details of all the members of the network and their specialities. It's a place where network members can sell skills and time to each other, so keeping the maximum amount of work inside the network as possible. Some of our members only work in one area of law and have joined the network as single person companies or husband and wife teams. It's a great way to get the best minds inside the network, for the best lawyers usually work for themselves.

You are guided expertly through infonet, and see how legal professionals can find and make contact with people whose skills are valuable to them. Unfortunately, the text based searching means correct legal terms are required, and you are forced to write at double speed to get a few of them down so you can look up their meaning later.

You decide to avoid the spiralling thoughts of legal terms being different in different countries and the implications of this for the rights of man, or less obviously, beast.

Then you dive into document control, how people create, check in and check out documents to work on them, how absolute control over every document and every print or copy of every document in the system is attempted. It is a rigorous system, and seen as hugely important as documents and legal argument are the output of A&A. Each edition of every document is in there, controlled, and as watched over as mouse holes guarded by hawks.

You go through tendering, getting work and following it up. You are shown time, expenses, predictor, invoicer, budgeter, managing a client, a contract (piece of work), a person. You see how people join, leave, how they are paid, rewarded and departed. You see how an office is managed, and a state, and a country. You see how they purchase legal forms, send out invoices, and do their accounting. You watch goods come in, and services go out. You see how two members of the network can work on the same contract, how contacts are kept, how accounts are managed at the restaurant where you ate lunch - the shock in the eye of the beholder of the price - and how coffee comes out of the machine to your left.

What you didn't see was how any other member of A&A worked, other than A&A itself. Following your meeting with Eli, Jeff rather unwillingly deposits you with Mike, the IT chief.

Mike explains: "You see Bim, they never let us do it properly. Everything has to be delivered yesterday, and we're under such pressure to deliver we hardly have time to test anything and make sure it works properly. The users therefore perceive our software to be rather buggy."

It is often the case that business leaders will pay consultancies to work and deliver their own way. It is usually very successful in that the business gets what it needs, and both parties trust enough to allow the job to be done. And yet the same business leaders can cripple internal departments with daily requirement changes, a refusal to negotiate on improbable requirements, and a demand for delivery dates which cannot be met realistically. The solution to the internal software conundrum is the long, careful, painful and arduous building of trust. Trust is built by process and quality. Process and quality are undermined by a lack of trust and immediate needs.

You sympathise with Mike. It is a difficult situation to be in, and it wasn't long ago that you lived under the same clouds. Those very clouds were the ones that made you embark upon your voyage of self discovery. It lifted you out of the purely technical arena and made you understand how your personal interaction was every bit as important in delivering software as your technical skills. This difference between you and Mike is as thin as the road not yet taken.

Howie appears once more on the scene to bid you farewell. His departing thought for you is to suggest that you have seen who they are and what they've got. He wants to know how they can be even more successful through the use of technology. He also wants at a glance knowledge of how every facet of A&A is performing.

Back at the office

By the time you get back to the office, the sun is a pounding fireball. Your supplies of energy have been eroded but you have a good feeling about this work. It is big enough for you to make a difference on, and not too onerous in terms of technology. You slink up the stairs, relieve yourself of your by now vapid jacket and pour yourself a cup of overripe coffee.

All in all, the day has gone well. To check for any gaps typical for a business, you check off the activities of A&A against game theory.

Game theory

The games afoot are how the Aardvarks fit into the overall world of practicing law. The other game is how they attract business from, and lose business to their competitors.

Game theory is a mathematical study of the strategies a player can invoke to create change. A similar mathematical study can be made of group theory, where the individuals in groups act for the good of the individual and the good of the group. Acting for the good of the individual may create an antagonistic group, or a synergetic group. Clearly Aardvark & Aardvark see themselves as a highly distributed but nonetheless synergetic group. Game theory contains parts. The parts are more than a collection of separate pieces to the game, but provide a complete set of levers for the game. They are: players, added values, rules, tactics and scope. The players are those involved in the game. The added values are what each player brings to the game. Rules give a structure to the game. Tactics are what the players use to change the game. Scope defines the bounds of the game

The players

Players in this case are the A&A members, and the rest of the legal world such as judges, defendants and law courts. It is also Aardvark's customers, suppliers, external contacts, and, of course, competitors.

A&A is a law firm with 200 legal professionals and 150 administrators. A&A is also an umbrella organisation for another 139 different companies around the world, collectively containing 4000 legal professionals and 1000 administrators.

A&A's clients are much the same the world over. They are people or companies.

Their competitors are other law firms.

Their collaborators are other legal professionals such as judges.

Their suppliers are stationers, printers and building maintenance companies, and hopefully Spaggott.

Perhaps some method of locating players and identifying their strategies would be useful to them as an extension of their infonet?

The added values

The added values are the knowledge, experience and contacts contained within the Aardvark members. To leverage those added values, member companies must be able to access the information held by other members with the greatest ease. The greater the ease, the higher the added value. That is unless access to that data creates antagonism either between legal professionals, or between A&A members.

The rules

Rules are laws, and the interpretation of those laws, which generally change over time and from country to country. They are also the way a business must work, again changing from country to country. Rules also cover how Aardvark may go about getting business. Game rules often come on a pamphlet; business rules change day to day on the whim of the players.

An A&A office has a specific boundary beyond which they may not trespass into another office's or member's region. Free space, i.e. that unassigned to an office is open territory.

The tactics

Tactics are the means Aardvark uses to get customers, and/or prevent one of their competitors winning the customer or piece of work. Tactics are also used by the members in the law courts to pursue a good settlement for their clients.

Use of the A&A name gives a global brand to an otherwise local company. Their global brand helps them win large or international work. Their local offices let them win local work. The expertise they have acquired by getting people who work for themselves into the A&A network is also a tactic.

The scope

Scope defines the extent of the world in which Aardvark works. It is their individual businesses, their collective business and matters of law.

They only do legal work.

They are geographically global, though localised through offices.

You also compare Aardvark & Aardvark to your standard business model, created from years of observing the trials and tribulations of other businesses. It contains nine core parts: sales, service, people, storage, ontology, technologies, 3rd party applications, knowledge, and reporting. Sales and service is a choice of one or both business models, one sells items to a customer, the other provides a service to a customer. The sales and service items are cyclic, as is the employee item. The rest, at this point, are just lists, or in the case of the ontology, possibly a hierarchical list.

Figure 18.3 Spaggott's standard business and systems model

Aardvark - the business and systems model according to Bim B.

A&A fit the service model. The service they provide is legal representation.

Service

Contact management

At present their contact management is not global. Locally, their contact management, certainly at their head office is done by diverse means, and their supposedly centralised system is so out of date, it is only used by a few die hards.

Account management

Account management is better. Pieces of work are sold to a location which belongs to an account.

Procurement

They do little procurement other than building services, stationery and legal forms.

Sales

Sales are made by the person intending to lead the work.

Assign personnel

the availability of each employee is available from the Predictor reports. Their skills, at A&A the law firm, are held in their personal profiles

Project management

Some risk mitigation exercises are attempted. Other than that, accounts are managed by the senior personnel

Success monitoring

Little success monitoring is done from the point of view of the success on each assignment, but Howie wants to see success globally by any number of measures. These are usually financial or by cases won and lost (both to clients and in court). A successful lawyer is not one who delivers the truth, but who delivers the truth of his client, and wins the case.

Follow up

Follow up is by a repeat order from an impressed customer. It is not a managed task.

Knowledge

Their knowledge is held in case histories, which are collated in the infonet, and indexed using a search engine.

Reporting

All reports come from PECAN, and are mostly of a financial nature. Some manual compilation of data from other A&A members is reported on also.

The rest of the business is not yet relevant, as you have been tasked with creating a holistic environment. What stays and what goes will be decided on later.

Aardvark - the business model

Finally, you tick off the subjects of general business activity: marketing, ethics, accounting, sales, customer management and satisfaction, finance, and the business model items (from Chapter 12).

Figure 18.4 A Business model

Each part of each model is accounted for, so you are content in your understanding of A&A the law firm. Other law firms within the A&A empire are yet to come within scope.

The remains of the day

You spend the last few moments of the day gathering your thoughts, making a few notes on post-its, then finally abandon them to the evening and limp home exhausted.

The Gathering Architecture

Over the next few days you begin to put together a few ideas, based upon the desires of A&A head office. You have an uneasy feeling about not being in contact with other A&A members and get in touch with Howie in the hope that he will point you towards some helpful souls.

This he does, and they confirm largely what you gleaned from Howie and Jeff, but leave you in no doubt that they do not want their businesses dictated to by an

overseeing head office. This is not the way they work. They value their independence, and sell that independence to their customers. They also want to be free to decide on whether they keep their existing systems or choose to go the A&A way. They also want to be able to take their computer systems with them should they part company with A&A> Many are not happy with Howie's dictates, and his suggestion that their current software systems may not be adequate.

It is up to you to create a system good enough for them to choose over what they have.

The A&A partners also throw in a few areas of law that are new to you. Your rather sketchy model of the law now looks like this:

Figure 18.5 The Law

High level architecture

For the A&A Consortium, you decide to go for a fully distributed system, where every law firm has their own computer hardware running their own software. You will add to that, a single centralised hub, where information from all law firms is collated and presented for reporting and job sharing. Some of that information will be able to be pulled down from the hub for purposes such as invoicing. This

will allow a client to have a single invoice when work is being done at more than one law firm within the A&A Consortium.

Figure 18.6 A&A Distributed information by firm and office

Computer users in a law firm's office will push updates (1) to a centralised (for that firm) data location. The centralised data will then push out updates (2) to lookup information held at each site or office. The lookup information will be simple lists, of employees, clients and ontology etc.

The central location will also push out offline data to remote or disconnected users (3), and those users, when they connect, will push updates (4) to the central data source of their firm.

All law firms will promote selected information, such as employees, skills, and work done on shared jobs to the Consortium's data hub (5a, b, c). Firms will then pull down information (6) for such things as invoicing work done between two or more consortium members. Firms will also pull down information collated and shared (7) for viewing, as will offices of other firms (8).

Distributed system

Each Aardvark consortium firm will have their own complete system. Each firm's offices will maintain one centralised set of information, a subset of which will be distributed to each of their offices to provide instant lookup across the LAN (local area network). Pushing and pulling information will be performed through a security layer based on security tokens. The hub will also provide information, again based on security tokens, to whomever is entitled to get it.

Spagsoft

Spaggot software is one of many software houses who have realised that developing the same applications over and over again is tedious for them, and expensive for their clients. They have clubbed together and created a controlled source movement (Consom) and from that, created a code core. Much of that code core is wrapped with Spaggott's own code to create Spaggott's software offering to the world. Some of that software will provide for A&A's needs straight off the shelf. Other parts of it will need to be modified, and yet other parts will be written from scratch.

To come up with a price, you will identify the use cases required to deliver A&A's needs. Jim will then add in licensing costs for the off the shelf items. Spaggott's standard terms are 17% of the cost price per year for support and maintenance, fixed for three years, and negotiable thereafter.

Spaggott also has a 5:3:2 pricing expectation for this scale of work for software : hardware : implementation and training. This means that the software price will be doubled and presented to A&A as the starting point for negotiations.

Business objects and the SpagSoft Object Database

According to your business models, information about A&A can be confined within the following business objects:

A&A member Location Person - Employee or Contact ExpenseClaim WorkDiary - timecard, meetings, appointments, predictor, things to do Account Assignment Publication - document (with sections) or website (with pages) Report - internal report on some facet of business or system Asset Role Organisation - A law firm, or a department or group within. A&A is the highest level of organisation Invoice

Each business object will be held in Spaggott's own object database as an XML string containing hierarchical and grouped information.

The Spagsoft Services Framework, and other Spagsoft software

The framework is a set of layers of objects, leading down into the object database, which allow you to develop a bespoke business application without having to write all of the more basic code or business functions.

Figure 18.7 The Services Framework

The Services Framework is a collection of objects in a layered architecture which provide the building blocks upon which a service oriented organisation may build its software. Many difficult areas of software have already been built for use by other Spagsoft customers, and indeed many other Consom members' customers, so less angst is gone through developing each subsequent system. The framework will be added to by the following software components:

Spagsoft Server Software

Spagsoft Operating System

A choice of a commercial or open source operating system, wrapped in the Spagsoft colours.

Spagsoft Web and EMail Server

An internet, email and instant messaging service.

Spagsoft Message Queue

A guaranteed message delivery system for intermittently connected users.

Spagsoft Workflow and Decision Engine

The workflow engine routes forms and text messages around the employees, deciding on which employees need to be included in the loop through a decision engine. The decision engine assesses risk and authorisation structures according to the reporting hierarchy and decision making process of the business. There are four versions of the Decision engine, the manually programmed, the fuzzy logic, the neural net, and the artificially intelligent.

Spagsoft Application Software

Spagsoft Human Resources (HR)

The HR system looks after hiring, firing, and the ongoing support of employees. It records training, disciplinaries and rewards. It hold information about internal people, their addresses and affiliations. Futhermore, it allows people to trace others by searching their skills and experience records, their attendance at meetings, as well as internal or external functions or training courses.

Spagsoft Financials

Financials looks after ledgers, budgets, daybooks, aged debt, creditors, debtors, and payment management.

Spagsoft Collaboration Workspace

When teams of people are dispersed across the country, or globe, the collaboration workspace provides a means for them to interwork electronically. It is used mostly as a cataloguing or library service, where each assignment and each account has its own space to hold emails, messages, files and collaborative thoughts.

Spagsoft Account Manager

Used to manage accounts. All work is booked to, and charged to, an account.

Spagsoft Contact Manager

Used to manage all contacts within and beyond A&A. It interfaces with the HR system to extract information about fellow employees and fellow Aardvarks. All contacts belong to an account, including A&A employees.

Spagsoft Procurement

A system used to hold and search suppliers' catalogues. It creates purchase orders, and monitors their status. It links with Financials to match incoming invoices with outgoing purchase orders.

Spagsoft Client Software

Spagsoft Office

Basic word processing and spreadsheets.

Spagsoft Email, Browser and Collaboration clients

The client end of the email, web and collaboration experience.

Spagsoft Offline workflow and data synchronisation engine

For those disconnected from the Spagsoft network, they can still lookup information offline. This feature keeps it available and up to date.

Spagsoft system software

Finally, there are the system administration duties, taken care of, as always by the Spagsoft core. The system applications keep all of the information in check, and systems operating efficiently:

Spagsoft Ontology Manager

The ontology manager enables system administrators to create new ontological definitions, hierarchies and lists. It is a mechanism used to create front-end drill downs into the raw and summarised data created by the Object Morpher.

Spagsoft External Systems Integrator

Many external systems need to integrate with the information held in the object database. Examples of this are payroll and telephone systems. The object database is the single point of information input, which feeds out using text formats such as XML, CSV and fixed formats to all external systems.

Spagsoft Backup

Backup saves the state of the object database at the end of each day. You recommend dumping the objects without their indexes to make the backup as efficient as possible. Indexes can be rebuilt on XML data fairly quickly. Document indexes will take longer, typically a day to build considering A&A's volumes.

Spagosft Helpdesk

All user queries and problems are logged and managed in the helpdesk. The use of Spagsoft Helpdesk allows calls to be raised through second and third line support directly to Spaggott support staff.

Spagsoft System Monitor

The system monitor takes the results and agreements that are part of the contract for response time and transactional throughput, and records the actual numbers as the system is used, upgraded and reconfigured. It also records changes to hardware, so changes in response for new disks, memory upgrades etc, may be observed.

Spagsoft Role Manager

As shown in figure 18.7, access to the information is filtered through a security layer. This is role based, and tasks are assigned to roles in multiple hierarchies and allowed certain information privileges. The role manager is the method of assigning roles to people, and thus defining their security access.

Spagsoft Object Database

Sapgsoft's object database contains a set of technologies to support their application software.

Spagsoft Template Manager

All information is held in business objects, which are based on definitions held in templates. The Template Manager allows creation, alteration and retirement of business objects. Retirement is a method of removing information no longer required, but without losing it forever.

Spagsoft Event Initiator

When objects in the database change state or are otherwise updated, or derived values for risk or profitability are hit, the event initiator raises a specified event. In most cases, the action is to send a note to someone, or update values for dashboard viewers.

Spagsoft ObjectMorpher

The object morpher takes one set of objects and interlinks them with others to create even more objects. Typically, the resulting objects will be OLAP cubes or reports.

Spagsoft Index Builder

The index builder builds indexes for the object database.

Spagsoft Search & List Creator

When information is added to the object database, lists and search indexes must be updated. This can be done live, or in the case of huge data inputs, done

by a system administrator or automatic process on a regular cycle. Documents are usually indexed daily. Data inserts can be indexed as part of the insertion procedure.

New or Modified A&A Software

Other software will have to be written specifically for A&A, or modified from the core Spagsoft applications to provide the functionality requested by A&A.

Document Manager

Although a document is stored in the object database in a fairly unstructured way, A&A want document management to be a specific activity, done though a specific interface.

Document management is the managing of complex and multiple hierarchical views of documents. For example, the same document might show up in Client, Skill or Practice views. It also controls document security, and check in/check out services. It must do version and change control, and manage authorization when documents need to be authorized before release to the client. It must also attempt to manage printouts of highly classified or important documents.

Figure 18.8 Document Management Use Cases

Assignment Manager

Assignment Manager is used to record and track the progress on each assignment.

Figure 18.9 Assignment Management use cases

Work Record

The work record is the fundamental measure of effort output by A&A employees. Each meeting and each piece of time spent working for a client is recorded, and recharged to the client.

Furthermore, all locations are recorded for tax purposes, and meetings must interface with the asset information to be able to book meeting rooms and equipment.

Figure 18.11 Work Record and Asset Management

Asset Manager

The Asset Manager allows the business to keep track of all its assets such as meeting rooms (locations), computers, servers, telephones, printers, projectors, desks, floor space, cupboards, keys, vehicles and all of the other paraphernalia that businesses accumulate.

Spagsoft's basic Asset Manager will be extended to deliver the required functionality.

Expenses

The Spagsoft expense system does not fit A&A's requirement to book expenses to tasks within assignments. Other A&A requirements also make using their existing expenses system difficult, so you quote for a bespoke system. Expenses allows an employee to make a claim for incidental expenses such as travel, hotel bookings, meals and entertainment.

Invoicing

All payments for work done are requested by sending an invoice to the client. Various electronic methods of contact have been specified, along with mail. A record of invoices printed and sent must be kept within the system.

SkillSearch

Finding someone with a particular skill is used by those wanting advice or help from someone, and equally by those booking people onto future assignments.

Skill search can be run within the scope of an office, city, business, or A&A network.

Dashboard and Reporting

Reporting is provided mostly by an Ontology view built upon a set of data and cubes created by the Morpher. There are some specific views of data requested by senior managers which are built into a dashboard to provide a real-time view of the business.

The dashboard shows incoming satisfaction surveys, payments, outgoing invoices, and the lag between request and payment. It shows a sales prediction, mapped against the actual income, the keenness and willingness of the staff reflected in their assessments and online staff surveys.

All reports can be viewed in numerous graphical formats, as summary or detail data, or as grid data, ready to copy and manipulate in other applications.

Procurement

All purchases made on behalf of the company from recognised and recorded suppliers, are done through the procurement software

Success Monitor

The success review is sent out at the end of an assignment to gauge how the client felt the work went.

Membership Register (for all A&A members) and information promotion and pull down

Each member of the A&A network has access to the global information, depending on their rights. They can promote information, such as time and expenses recorded against an assignment shared between two ore more members. They can also pull down that information if they are sending out the compiled invoices. Finally, they can view other information on A&A members, such as their offices, skills, staff, and many other things you might find on an intranet.

Figure 18.10 Membership Register and Administration Use Cases

The Architectural Concerns

The important architectural concerns for A&A are as follow:

Availability, Dependability, Reliability, Robustness

The system must be there whenever it is required.

Backup and restore

Legal professionals do not want to lose any versions of their documents.

Globalisation, Localisation

A&A is (or wants to be) a global brand.

Longevity, Changeability

Howie has not asked for a system for five years (no-one ever does). However, he sees it as being a differentiator for A&A, being able to work together.

Security

Many items of information are of a controlled nature, especially work for the Governments of the world.

System replacement, Deployability, Installability

It must be easy for a wholly disconnected law firm using a set of disparate systems to be able to replace their systems with yours, with ease. This must be able to be done, in extreme cases, without anyone on the ground with the recipient.

Function Point Count for Effort Estimation

The total Unadjusted Function Point count for all applications = 715.

From the equation Time = 20 x UFP x Technical Difficulty x Environmental Difficulty, time = 20 x 715 x .7 x .8 = 8333 hours. At 20 hours per developer per week, that is 35 weeks work for 12 people.

Based on your own fiddle factor of 1.5, which you have observed between the ideal and Spaggott's output, you have 48 weeks for twelve people.

Thus, you have a one year project for three teams of four developers.

Your cost per developer hour is calculated to include all on costs (floor space, machines, analysts, managers, testers etc) at $120, making this a $2.5M development project.

Jim adds in licence costs for the server, client and framework software, then multiplies by 2 to take account of hardware, system installation and support, and ends up with $12M. That sits about right on his shoulders, but he spends another week going over and over the numbers to satisfy himself and Spaggott's Chief Executive, Mike Spaggot, that the measure meets the mandate, then puts his name to it, and delivers it to Howie.

The wait

Weeks tick by nervously. Howie and Jeff have had a raft of questions and discussions with yourself, Jim and Mike, to ensure themselves of the integrity of your offer. They're about to launch into a high risk manoeuvre within their company and the A&A network, to begin to seize greater control of all outlying partners by stealth through their software. They will also fundamentally change the way they work, relying on external software development for the first time in their careers. It's a rocket that could launch them or burn them to a crisp.

The decision

You have been resummoned. It's either you or one of the other software houses, and Jeff is delighting in his role of chief negotiator. He keeps talking about money and Jim has cunningly countered his every ploy. Fortunately, there are many "would likes" on the negotiating table and Jim's poker face, his unimpressed disinterest in changing the price or introducing new features are one moment bathing you in an icy blast, and another moment stewing you in your own steam bath. It is a tense meeting.

You haven't said much. You haven't had anything to say. The architectural case hasn't changed since you stated it a month earlier, and despite the hammering out of detail, nothing much has changed in the applications either. Into this general awkwardness, Jeff drops the bomb.

"Bim. You have recommended a distributed architecture. The other guys have recommended centralisation into three hub sites. They reckon the maintenance costs will be a fraction of the fully distributed architecture you have recommended. They even went as far as to scoff at your suggestion and claimed that the internet age was yet to dawn on Spaggott Software." Jeff smiles. He says it without malice, but wants to know what the reality will be. Ongoing costs are dear to the heart of all business operations, and your opponents have at least partially convinced him of the truth of their claim.

It may be a ragged bone, but at least you have chewed this one many times before. With the ancient words of Sun Tzu[1] ringing in your ears, you marshall your neural troops to attack not the enemy, but his strategy.

"Of course, we considered various degrees of centralisation from a single box on down. The choice of three hubs is one possible design as it will bring you the benefits of semi-centralised data, limited to a few continents each such as US, Europe, the Middle East and Africa, and Asia Pacific. However, this design will still lead to data consolidation issues across the three sites. Neither will it give your users the instant response a fully distributed system will.

On the business side, it could lead to antagonism between you and an A&A partner, for they will perceive you as the controller of the hubs whenever there is a breakdown or dispute. Better to have the IT support staff in direct control of the partner, then at least they are arguing amongst themselves.

Our distributed service also allows job sharing at any level between the A&A partners, and keeps the operational data summaries away from them. The three hub approach will still need to share jobs across partners, but at that level, you will have less control over what else they share, leading to cliques of partner groups within the larger group. Similarly, when it comes to overall information for A&A, drilled by country, sector or any of the other breakdowns you have specified, the information still has to be collated, so one of the three hub sites would still be the master site, with information gathered from the other two.

And what happens if your internet service provider has problems, or the internet goes down? There would be no service at all. The largest problem with the centralised approach is what we call latency. Imagine you posted a letter to someone in Hong Kong, and every country it went through opened your letter, rewrote the envelope, and posted it on again to the next country. On a much faster scale, this is what happens across the internet, eating away response time. A local service will give an instant response. A user in Eastern Russia accessing a centralised service in Europe or Hong Kong will have some appreciable delay, which gradually erodes any satisfaction in using these centralised systems. The partners nearest the hub sites would be happy; those furthest away, less happy."

Jim doesn't want to get lost in the technicalities. You have given him at least a start upon which he can build his sale, and he launches into a highly charged closing statement.

"Howie, I want you to picture A&A in a few years time. The system we build has been installed and in operation for quite some time. Your business is doing extremely well both locally and globally. Our system matches your business model perfectly, and each time a new partner joins, you have a well practiced team to get them up and running in the minimum amount of time.

Business is booming. You successfully work with your partners worldwide, sharing jobs, resources and skills. A&A is a well known, global brand. There you are Howie, on the cover of Forbes, helped there by the fact that you chose our software and were left to concentrate on A&A.

You were kind enough to share your vision with us, and now we want to return the favour by becoming a part of your vision. We can help you do it today. Right now. Right now. I know you have concerns, and I want you to know we can overcome them. Together we will create exactly what you need to put A&A right where you want it to be.

Jeff once again seizes the jugular. "Clearly, we are now in the position where we want to balance cost and features."

"Yes, we can do that. And we can continue for the rest of the day to do so. And we can do it tomorrow. And each day we do it will add another day to the cost. No win no fee lawyers will double the price of lawyers you said when we first met. Discussing software also increases its price. Add the price of each day for your business operating without your new software and you can clearly see how the price is increasing every moment. Every precious moment.

We can start now. Today. Right this minute. We can decide, together, to start building you a superb software system that you can clearly see will meet your current and future needs. Can we leap over the initial hurdle together and put it behind us? Can we get the paperwork out of the way and get on with the job?

This powerful imagery[2] ends with Jim pointing, Jeff nodding, and Howie signing. The job is yours.

Questions

1. What would you do? 2. Do you agree with the use cases? Would you add any, or remove any? 3. Assigning 1, 2 or three use case points to each actor, and 5, 10 or 15

points per use case depending on their complexity, what is your unadjusted function point count for all of the applications?

4. Using the transaction count, does your function point count differ significantly?

5. What price would you charge for developing the software?

References

[1] The Art of War. Sun Tzu. Many imprints. [2] Unlimited Selling Power - How to master hypnotic selling skills. Donald Moine, Kenneth Lloyd. Prentice Hall.