Programming Techniques

Lecture 3 Agile Development

Based on: Software Engineering, A Practitioner’s Approach, 6/e, R.S. Pressman

School of Software

Programming Techniques

Manifesto for Agile Software Development

Proposes that it may be better to value:

• Individuals and interactions over processes and tools

• Working software over comprehensive documentation

• Customer collaboration over contract negotiation

• Responding to change over following a plan

Programming Techniques

Agile Software Engineering

• Represents a reasonable compromise between the conventional software engineering for certain classes of software and certain types of software projects;

• Embraces a philosophy that encourages customer satisfaction, incremental software delivery, small project teams (composed of software engineers and stakeholders), informal methods, and minimal software engineering work products;

• Guidelines stress on-time delivery of an operational software increment over analysis and design.

Programming Techniques

Agile Development

• Can deliver successful systems quickly;

• Stresses continuous communication and collaboration among developers and customers.

Programming Techniques

What is “Agility”?

• Effective (rapid and adaptive) response to change

• Effective communication among all stakeholders

• Drawing the customer onto the team

• Organizing a team so that it is in control of the work performed

Yielding …

• Rapid, incremental delivery of software

Programming Techniques

Agility

• An agile team is able to respond to changes during project development

• Agile development recognizes that project plans must be flexible

• Agility encourages team structures and attitudes that make communication among developers and customers more facile

Programming Techniques

Agility

• Agility eliminates the separation between customers and developers

• Agility emphasizes the importance of rapid delivery of operational software and de-emphasizes importance of intermediate work products

• Agility can be applied to any software process as long as the project team is allowed to streamline tasks and conduct planning in way that eliminate non-essential work products

Programming Techniques

Agility Principles

1. Highest priority is to satisfy customer through early and continuous delivery of valuable software.

2. Welcome changing requirements even late in development, accommodating change is viewed as increasing the customer's competitive advantage.

3. Delivering working software frequently with a preference for shorter delivery schedules (e.g., every 2 or 3 weeks).

4. Business people and developers must work together daily during the project.

Programming Techniques

Agility Principles

5. Build projects around motivated individuals, given them the environment and support they need, trust them to get the job done.

6. Face-to-face communication is the most effective method of conveying information within the development team.

7. Working software is the primary measure of progress. 8. Agile processes support sustainable development,

developers and customers should be able to continue development indefinitely.

Programming Techniques

Agility Principles

9. Continuous attention to technical excellence and good design enhances agility

10. Simplicity (defined as maximizing the work not done) is essential.

11. The best architectures, requirements, and design emerge from self-organizing teams

12. At regular intervals teams reflects how to become more effective and adjusts its behavior accordingly.

Programming Techniques

An Agile Process

Are based on three key assumptions:

1. It is difficult to predict in advance which requirements or customer priorities will change and which will not. It is also difficult to predict how customer priorities will change as a project proceeds.

2. For many types of software, design and construction activities are interleaved (construction is used to prove the design). It is difficult to predict how customer priorities will change as a project proceeds.

3. Analysis, design, and testing are not as predictable from a planning perspective as one might like them to be.

Programming Techniques

An Agile Process

• An agile process, must be adaptable to manage unpredictability

• Agile processes must be adapted incrementally to progress

• Incremental adaptation requires customer feedback based on evaluation of delivered software increments (executable prototypes) over short time periods

Programming Techniques

An Agile Process

• Is driven by customer descriptions of what is required (scenarios)

• Recognizes that plans are short-lived

• Develops software iteratively with a heavy emphasis on construction activities

• Delivers multiple ‘software increments’

• Adapts as changes occur

Programming Techniques

Human Factors

• Agile development focuses on the talents and skills of individuals, molding the process to specific people and teams

• The process moulds to the needs of people and team, not the other way around.

Programming Techniques

Agile Team Member Traits

• Competence – innate talent, specific software related skill, knowledge of the process applied.

• Common focus – to deliver a working software increment to the customer within the time promised.

• Collaboration – with one another, with the customer and with business managers.

• Decision-making ability – for both technical and project issues.

• Fuzzy-problem solving ability - have to deal with ambiguity. • Mutual trust and respect • Team self-organization:

- organises itself for work to be done.- organises the process to best accommodate its local environment- organises the work schedule to best achieve delivery of the software increment.

Programming Techniques

Agile Process Models

• Extreme Programming (XP)

• Adaptive Software Development (ASD)

• Dynamic Systems Development Method (DSDM)

• Scrum

• Crystal

• Feature Driven Development (FDD)

• Agile Modeling (AM)

Programming Techniques

What is Extreme Programming

• The most widely used agile process, originally proposed by Kent Beck

• “Extreme Programming (XP) is a lightweight, efficient, low-risk, flexible, predictable, scientific, and fun way to develop software" -- Kent Beck, Extreme Programming Explained

• Relies on object-oriented approach

Programming Techniques

XP Values

• Communication: – developer and customer work closely together

– pair programming

• Simplicity: just satisfy current design.

• Feedback: from development process, e.g. prototype.

• Courage: do the right thing, e.g. throw away code.

Programming Techniques

XP Key Activities

• Planning

• Design

• Coding

• Testing

Programming Techniques

XP Planning

• Begins with the creation of “user stories” – placed on an index card

• Customer assigns a value (i.e. a priority) to the story

• XP team assesses each story and assigns a cost – measured in development weeks

• If the story will require more than 3 weeks, the customer is asked to split the story into smaller stories – and assign new value etc…

Programming Techniques

XP Planning

• Stories are grouped for a deliverable increment

• A commitment is made on delivery date

• After the first software increment, “project velocity” – number of user stories implemented during the first release, is used to help define subsequent delivery dates for other increments

• Customer can add stories, change the value of an existing story, split stories or eliminate them.

• The XP team then reconsider all remaining releases and modifies its plans accordingly

Programming Techniques

XP Design

• Follows the keep it simple (KIS) principle

• Encourage the use of Class-Responsibility-Collaborator (CRC) cards

• CRC cards are only design work produced

• For difficult design problems, suggests the creation of “spike solutions”

• Spike solution – helps to lower risk when true implementation starts and validate the original estimates.

Programming Techniques

XP Design

• Encourages “refactoring” - an iterative refinement of the internal program design

• “Refactoring is a change made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behaviour.”[Martin Fowler]

Programming Techniques

XP Design

• XP design is viewed as a transient artefact that can and should be continually modified as construction proceeds

• Refactoring is to control these modifications by suggesting small design changes that ‘can radically improve the design

Programming Techniques

XP Coding

• Recommends the construction of a unit test for a store before coding commences

• Encourages “pair programming” – two people work together at one workstation to create code for a story

• Continuous integration and smoke testing is utilized

Programming Techniques

XP Coding – Smoke Testing

• A common approach for creating “daily builds” for product software

• Smoke testing steps:– Software components that have been translated into code

are integrated into a “build.”

– A series of tests is designed to expose errors that will keep the build from properly performing its function.

– The build is integrated with other builds and the entire product (in its current form) is smoke tested daily.

Programming Techniques

XP Testing

• Unit tests created before coding are implemented using an automated testing framework to encourage use of regression testing;

• All unit tests are executed daily• Integration and validation testing done on daily basis • “Acceptance tests” - customer tests, are defined by the

customer and executed to assess customer visible functionality

Programming Techniques

XP Principles

• Provide rapid feedback: feedback improves design.

• Assume Simplicity: only produce requirement.

• Incremental changes: reduces the risk.

• Embrace change: being an iterative approach change can be accommodated .

• Do Quality work. Using frequent testing and customer feedback. Writing the tests for each scenario provided by the users before writing the programs and keeping all of the tests running at all times. Writing acceptances tests when new functionality is identified. Tests grow with system;

Programming Techniques

XP core practices

1. Planning2. Small Releases - putting a minimal system into production

quickly and growing it in whatever directions that prove most valuable

3. Simple Design - starting projects with a simple design that constantly evolves to add needed flexibility and to remove unneeded complexity;

4. Continuous Testing - integrating and testing the whole system several times a day; writing tests for new function before writing function helps make code testable also it decouples form its environment; i.e. a method/class to be tested is independently callable. Tests act as documentation.

Programming Techniques

XP core practices

5. Refactoring - improving design of existing software;

6. Pair Programming - producing all software in pairs, two programmers at one screen;

7. Collective Code Ownership

8. Continuous Integration

9. 40-Hour Working Week

10. On-site Customer

11. Coding Standards

Programming Techniques

Extreme Programming (XP)

unit test continuous integration

acceptance testing

pair programming

Release

user stories values acceptance test criteria iteration plan

simple design CRC cards

spike solutions prototypes

refactoring

software incrementproject velocity computed

Programming Techniques

XP Issues

• Extreme Programming is somewhat controversial, for example:- team members are not forced to be specialists such as analysts or integrators;- documentation separate from the code is discouraged as communication is done face to face or through carefully written code and its tests.

• XP is a method favored for small to medium-sized teams developing software in the face of vague or rapidly changing requirements.

Programming Techniques

XP Issues

• Customers generally do not want have time or resources to be continuously involved in a project

Programming Techniques

Summary

• It seems natural to introduce XP to an upper level course in software engineering.

• Less of an Computer Science focus more Software Engineering approach.

• Some of the practices of XP should be introduced earlier in the curriculum.

• XP is not a ‘silver bullet’, it should be compared and contrasted with other approaches.

Programming Techniques

Adaptive Software Development

• Originally proposed by Jim Highsmith

• Emphasizes self-organizing teams, interpersonal collaboration, and both individual and team learning.

Programming Techniques

ASD Features

• Mission-driven planning

• Component-based focus

• Uses “time-boxing” - the practice of deciding a priori the fixed amount of time that can be spent on each task. When the task's time limit is exceeded, development moves on to the next task (with the hope that a majority of the critical work was completed before time ran out).

• Explicit consideration of risks

• Emphasizes collaboration for requirements gathering

• Emphasizes “learning” throughout the process

Programming Techniques

ASD Phases

• Speculation

• Collaboration

• Learning

Programming Techniques

ASD Phases - Speculation

• The project is initiated and adaptive cycle planning is conducted.

• Adaptive cycle planning uses:

- project initiation information – the customer’s mission statement,

- project constraints (e.g. delivery dates or user descriptions)

- basic requirements – to define the set of release cycles (software increments).

Programming Techniques

ASD Phases - Collaboration

• Requires teamwork from a jelled team, joint application development is preferred

• People working together must trust one another to:

- criticise without animosity;

- to assist without resentment;

- work as hard or harder as they do;

- have the skill set;

- communicate problems in a way that leads to effective action.

Programming Techniques

ASD Phases - Learning

• Focus groups – the customer and/or end-users provide feedback on software increments

• Formal technical reviews – ASD team members review the software components, improving quality and learning as they proceed.

• Postmortems – addressing its own performance and the process with intent of learning and then improving its approach.

Programming Techniques

Adaptive Software Development

adaptive cycle planning uses mission statement project constraints basic requirements time-boxed release plan

Requirements gathering JAD mini-specs

components implemented/ tested focus groups for feedback formal technical reviews postmortems

software incrementadjustments for subsequent cycles

Release

Programming Techniques

Dynamic Systems Development Method (DSDM)

• Promoted by the DSDM Consortium (www.dsdm.org)

• Provides a framework for building and maintaining systems which meet tight time constraints using incremental prototyping in a controlled environment

• Uses Pareto principle (80% of project can be delivered in 20% required to deliver the entire project)

Programming Techniques

Dynamic Systems Development Method (DSDM)

• Each increment only delivers enough functionality to move to the next increment

• Uses time-boxes to fix time and resources to determine how much functionality will be delivered in each increment

Programming Techniques

DSDM Features

• Similar in most respects to XP and/or ASD• Guiding principles

» Active user involvement is imperative. » Teams must be empowered to make decisions.» The focus is on frequent delivery of products. » Fitness for business purpose is the essential criterion for

acceptance of deliverables.» Iterative and incremental development is necessary to converge

on an accurate business solution.» All changes during development are reversible.» Requirements are baselined at a high level» Testing is integrated throughout the life-cycle.

Programming Techniques

DSDM Life Cycle Activities

• Feasibility study (establishes requirements and constraints);

• Business study (establishes functional and information requirements needed to provide business value);

• Functional model iteration (produces set of incremental prototypes to demonstrate functionality to customer);

• Design and build iteration (revisits prototypes to ensure they provide business value for end users, may occur concurrently with functional model iteration);

• Implementation (latest iteration placed in operational environment);

Programming Techniques

Dynamic Systems Development Method

Programming Techniques

Scrum

• Originally proposed by Schwaber and Beedle

• Scrum—distinguishing features– Development work is partitioned into “packets”

– Testing and documentation are on-going as the product is constructed

– Work occurs in “sprints” and is derived from a “backlog” of existing requirements

– Meetings are very short and sometimes conducted without chairs (stand-up meetings)

– “demos” are delivered to the customer with the time-box allocated

Programming Techniques

Scrum principles

• Small working teamed used to maximize communication, minimize overhead, and maximize sharing of informal knowledge.

• Process must be adaptable to both technical and business challenges to ensure bets product produced.

• Process yields frequent increments that can be inspected, adjusted, tested, documented and built on.

• Development work and people performing it are partitioned into clean, low coupling partitions.

• Testing and documentation is performed as the product is built. • Provides the ability to declare the product done whenever

required.

Programming Techniques

Scrum Activities

• Backlog - a prioritized list of requirements or features the provide business value to customer, items can be added at any time;

• Sprints - work units required to achieve one of the backlog items, must fit into a predefined time-box, affected backlog items frozen;

• Scrum meetings - (15 minute daily meetings) addressing these questions: What was done since last meeting? What obstacles were encountered? What will be done by the next meeting?

• Demos - deliver software increment to customer for evaluation).

Programming Techniques

Scrum

Scrum Process Flow (used with permission)

Programming Techniques

Crystal

• Proposed by Cockburn and Highsmith

• Crystal—distinguishing features– actually a family of process models that allow

“maneuverability” based on problem characteristics

– face-to-face communication is emphasized

– suggests the use of “reflection workshops” to review the work habits of the team

Programming Techniques

Crystal principles

• Its always cheaper and faster to communicate face-to-face

• As methodologies become more formal teams become weighed down and have trouble adapting to project work vagaries

• As projects grow in size, teams become larger and methodologies become heavier

• As projects grow in criticality some degree of formality will need to be introduced in parts of the methodology

Programming Techniques

Crystal principles

• As feedback and communication become more efficient the need for intermediate work products is reduced

• Discipline, skills, and understanding counter process, formality, and documentation

• Team members not on the critical project path can spend their excess time improving the product or helping people who are on the critical path

Programming Techniques

Crystal

• Incremental development strategy used with 1 to 3 month time lines

• Reflection workshops conducted before project begins, during increment development activity, and after increment is delivered to review the work habits of the team

Programming Techniques

Feature Driven Development

• Originally proposed by Peter Coad et al

• Practical process model for object-oriented software engineering

• FDD—distinguishing features– Emphasis is on defining “features”

» a feature “is a client-valued function that can be implemented in two weeks or less.”

Programming Techniques

FDD Philosophy

• Emphasizes collaboration among team members • Manages problem and project complexity using feature-

based decomposition followed integration of software increments

• Technical communication using verbal, graphical, and textual means

• Software quality encouraged by using incremental development, design and code inspections, SQA audits, metric collection, and use of patterns (analysis, design, construction)

Programming Techniques

FDD Framework Activities

• Develop an Overall Model - contains set of classes depicting business model of application to be built

• Build a Features List - features extracted from domain model, features are categorized and prioritized, work is broken up into two week chunks

• Plan By Feature - features assessed based on priority, effort, technical issues, schedule dependencies

Programming Techniques

FDD Framework Activities

• Design By Feature - classes relevant to feature are chosen, class and method prologs are written, preliminary design detail developed, owner assigned to each class, owner responsible for maintaining design document for his or her own work packages;

• Build By Feature -- class owner translates design into source code and performs unit testing, integration performed by chief programmer

Programming Techniques

Feature Driven Development

Programming Techniques

Agile Modeling

• Practice-based methodology for effective modeling and documentation of software systems in a light-weight manner

• Originally proposed by Scott Ambler

• Modeling is essential for all systems, but that the complexity, type and size of the model must be tuned to the software to be built.

Programming Techniques

Agile Modeling Principles

• Model with a purpose – have a specific goal

• Use multiple models – that provide value to their intended audience

• Travel light (only keep models with long-term value)

• Content is more important than representation

• Know the models and tools you use to create them

• Adapt locally – the modeling approach should be adapted to the needs of the agile team

Programming Techniques

Agile Philosophy

• The importance of self-organising teams that have control over the work they perform;

• Communication and collaboration between team members and between practitioners and their customers;

• A recognition that change represents an opportunity;

• Emphasis on rapid delivery of software that satisfies the customer.

Programming Techniques

Refactoring

• “Behaviour-preserving transformations are known as refactorings. Refactorings are changes whose purpose is to make a program more reusable and easier to understand, rather than to add behaviour” William Opdyke[Opd92].

• XP encourages refactoring – a construction technique that is also a design technique.

Programming Techniques

Refactoring

• “Refactoring is the process of taking a running program and adding to its value not by changing its behaviour but by giving it more of these qualities that enable us to keep developing at speed” [Kent Beck]. Where the qualities Beck refers to are easy to read and understand code, no duplicate logic, sensible hierarchies.

Programming Techniques

Refactoring

• Reduce redundancy, eliminate unused functionality and simplify design.

• With appropriate tools, it removes fear of change and builds confidence.

• Nothing is set in stone, refactoring facilitates change.

Programming Techniques

Refactoring

• It can be considered as a technique that does not produce new functionality but incorporates best practice into evolving designs.

• RF segments development into two stages:

1) adding or designing functions and

2) refactoring those designs.

• RF considers code as a manifestation of design, so code and design are related.

Programming Techniques

Refactoring

• Making software flexible also makes it more complex. The excessive flexibility may never be used, so another approach is to build a system that meets the immediate requirements and can then be refactored to meet future requirements.

• Design goal is to build systems that can fairly easily be refactored.

• Refactoring can lead to simpler design without sacrificing flexibility.

Programming Techniques

Refactoring

• A reasonable pool of knowledge exists.

• Many common and well known refactorings are catalogued:

–rename variables, methods;

–extract portions of methods into separate methods;

–push methods, variables up and down the hierarchy;

–move methods into other classes through references.

Programming Techniques

Refactoring

• Also there is a set of guidelines that help identify when existing design/code need refactoring:

- duplicate code,

- long methods,

- large classes,

- long parameter lists, often called the ‘smell’ of bad code which often indicates bad design.

Programming Techniques

Refactoring

• RF, like testing, should be part of the process, there is no ‘refactoring stage’

• Guidelines when to refactor:

- when a function is being added.,

- when a bug is fixed,

- after code review (code review help spread knowledge throughout the team).

Programming Techniques

Refactoring

• Refactoring is a key technique in ‘agile-development’ processes in contrast with complete up front design.

• RF does not replace design, must do some design, as usual ‘no silver bullet’. Trick is to find the correct balance between design and refactoring.

Programming Techniques

Refactoring

• Large and small scale refactoring (LF,SF) differ in both a technical and management perspective. Technically is not possible (or desirable) to completely refactor a large system.

• From a management point of view LF requires agreement between entire developers and manager.

• Currently most tools and catalogues address SF.

Programming Techniques

Advantages of RF

• RF permits change of design in the light of experience and in light of changing requirements. Maybe a past decision was wrong and almost certainly there will be new requirements.

• Reduce redundancy, eliminate unused functionality and simplify design.

• With appropriate tools, it removes fear of change and builds confidence.

• Nothing is set in stone, refactoring facilitates change.

Programming Techniques

Advantages of RF

• A reasonable pool of knowledge exists.

• The assumption is that each automatic refactoring is safe and fast.

Programming Techniques

Advantages of RF

• Refactoring is relatively easy and natural in an OO environment like Java.

• You can refactor for performance reason, however RF differs from pure performance enhancement in that is should reduce complexity.

• Without refactoring, as changes are made, the design will decay.

• Helps eliminate duplicate code.

Programming Techniques

Advantages of RF

• RF makes software easier to understand. If RF is part of the process it could be included as part of the code review.

• The refactoring catalogues can be considered as a set of ‘best practice’

• RF can make design insight more explicit by using the catalogue as design guideline.

• RF can reduce/remove the need to attempt to predict every possible the way in which system might change in the future

• Reduce redundancy, eliminate unused functionality and simplify design.

Programming Techniques

Disadvantages of RF

• Systems not just program code a persistent data aspects need to be considered.

• Some refactorings change interface. If the interface is published, then the two interfaces must be maintained.

• Refactoring catalogues and tools maturing, no universally accepted standards.

• Refactoring difficult where code ownership is held by individuals rather team based.

Programming Techniques

Disadvantages of RF

• Can there be designs that cannot be refactored? Open issue.

• Sometimes the existing code base is so poorly understood or in such a mess that it is better to re-write from scratch rather than refactor.

• Buggy code should not be refactored. Refactoring works on the assumption that the existing system works.

Programming Techniques

Disadvantages of RF

• When not to refactor:

- if the team/programmer do not understand the technique;

- if the benefits are too vague and long term;

- refactoring is an overhead activity , programmers are paid to write new code;

- refactoring might break a new program.

Programming Techniques

Disadvantages of RF

• How does refactoring fit in with configuration and change management where several lines and/or versions of software exist.

• Cost of refactoring increases in non-OO systems (e.g. COBOL). Then we must focus more heavily on design prior to coding.

• Fairly new technique, RF needs to be proved on large system in order to gain wider industry support.

Programming Techniques

Disadvantages of RF

• Refactoring is not a standalone activity, must integrate with process and tools.

• Good software tools are needed for testing, debugging, code management, deployment (e.g. JBuilder, JUnit, Ant, Cactus, VisualWorks, ENVY etc). How will the tools be chosen? Will the tools work well together?

Programming Techniques

References

• William F. Opdyke. Refactoring Object-Oriented Frameworks. PhD thesis, University of Illinois, 1992. Available at: ftp://st.cs.uiuc.edu/pub/papers/refactoring/opdykethesis.ps.Z

• Extreme Programming Explained: Embrace Change By Kent Beck, ISBN 0201616416, Addison-Wesley 1999

Programming Techniques

2005F

Instructor

John D Lin - johndlin@hotlinemail.com

Lectures

Tues & Thurs 10:00-11:30 RM. 100, Lower Block