May, 2004William (Bill) Myers1 Agile Methodologies and Chaos Revisited William (Bill) Myers Cinergy...
-
Upload
wendy-mckenzie -
Category
Documents
-
view
216 -
download
2
Transcript of May, 2004William (Bill) Myers1 Agile Methodologies and Chaos Revisited William (Bill) Myers Cinergy...
May, 2004 William (Bill) Myers 1
Agile Methodologiesand
Chaos RevisitedWilliam (Bill) Myers
Cinergy Corporation
Information Builders Summit 2004 User Conference
May 2004
* The statements and opinions herein are those of the individual author. They are provided for informational purposes only and should not be relied upon for decision making.
May, 2004 Page 2 William (Bill) Myers
Introduction• Audience & Flow
• Agile Methodologies • Philosophy & Methods
– Management & Coding
• General Agile reporting• Questions
– Follow-up questions at end
• Appendix • Links • Additional Details / starts at References slide• Additional points
• Supplemental Handout
May, 2004 Page 3 William (Bill) Myers
What is Agility
• Ability to adapt • to Changing circumstances• to Changing requirements
• Impact• Improves business climate• Provides competitive edge• Quick gratification
» Customers & Developers» Faster Return on Investment
May, 2004 Page 4 William (Bill) Myers
Point of View - Concerning Risk
• Traditional Methods• Risks for IT
– Errors– Delivery on budget, on time, on scope
• Business risks generally ignored
• Agile Methods• Business risks
– Competitive timeliness– Effectiveness / the right thing
• IT risks– Errors– Budget, time, scope
May, 2004 Page 5 William (Bill) Myers
Agile Methodologies / Perspective
• How to Decide
• Relationships & Responsibilities
• 80 / 20 rule• Feature driven• Customer stories• Baseline high level• Emerge/Converge
• Reduce reporting risks
Note: Crystal Methodology IS NOT RELATED to the reporting product
May, 2004 Page 6 William (Bill) Myers
What is Agile Development?• Set of policies and conventions
• Controls risk• Puts project into safety zone
• Agile Methodologies• Changing environments • Philosophies
– Products, Personal relationships, Timing, Documentation
• Methods– Approach & Principles– Project management– Programming – low level practices
• Traditional Methodologies• Rigid, known environments• Static processes• Structured Methods
– Waterfall project management & clones
May, 2004 Page 7 William (Bill) Myers
Agile Philosophieshttp://www.agilealliance.org
• Agile Manifesto– Individuals and interactions
– over processes and tools
– Working software– over comprehensive documentation
– Customer collaboration– over contract negotiation
– Responding to change – over following a plan
Quote from consortium: “While there is value in the items on the right (sub-topics above), we value the items on the left (major topics) more”.
May, 2004 Page 8 William (Bill) Myers
General Agile Principles• Deliver something useful to customer
• Nothing counts until you deliver software
• Deliver frequently• Four to eight weeks
• Encourage collaboration• “Active customer involvement is imperative!” – find a champion• Daily meetings – very short
• Provide technical excellence• Rely on the skills of your people• Do the simplest thing possible
• Balance flexibility and structure• Self-correcting teams
• Don’t let people suffer• Install when business needs it & technically feasible
May, 2004 Page 9 William (Bill) Myers
Agile Manifesto - Highlights
• Nothing counts until you deliver software• Deliver every four to eight weeks• Direct contact - customers & developers• Daily contact and communication• Let champions drive projects • Maintain a constant pace• Agile processes enhance quality• Teams self-correct• Never forget the goal - software
May, 2004 Page 10 William (Bill) Myers
Adaptability ScaleRemember Sarbanes-Oxley
May, 2004 Page 11 William (Bill) Myers
Project Management Styles• Command & Control
• Defined process• Plan what you expect – make predictions• Enforce the predictions• “Change Control”
– to manage change
• Leadership – Collaboration• Empirical process• Start with a plan• Change plan frequently• Inspection & Adaptation
– To control change
May, 2004 Page 12 William (Bill) Myers
Waterfall Methodsand clones
• Process• Sequential• Analysis, design, construct, test, implement• Rigid• Paper intensive• Good with 1960’s mainframes with cards
• When Useful• Known processes
– Hardware & vendor software upgrades
• Little creativity required
May, 2004 Page 13 William (Bill) Myers
Many Agile Methods • What methodology to use ?
• Crystal & Crystal Light ( helps with decision )
• Project management• Scrum ( broadest, variable end-point )• DSDM Model ( construction oriented )• Lean Development (domain, 80/20 approach )• Feature-Driven Development - FDD (most structured)• Adaptive• Extreme Programming – XP
• Programming oriented• Extreme Programming - XP
» Customer Stories, programs, working conditions
• Agile reporting ( core & option )
May, 2004 Page 14 William (Bill) Myers
Agile Methodologies• Agile Software Development Ecosystems
• Reference book & web site• by Jim Highsmith• http://www.jimhighsmith.com
• Tom DeMarco: • Agility: 1• everything else: 0
• Numerous books• Highlights only
May, 2004 Page 15 William (Bill) Myers
Observations - Agile Conference• Easy to go wrong
• Mis-interpretation, mis-implementation, missed point
• No silver bullet• One size does not fit all
• Beware too much documentation• Offload legal requirements from developers
• Always “on time, on scope, on budget”• Leads to mediocrity ( no risks taken )
• Cultural change• from - military Command & Control• to – cooperative Leadership & Collaboration
• Productivity leaps• 77 function points / month when industry average 2 pts / mo • WebFocus fits agile development very well
May, 2004 Page 16 William (Bill) Myers
Reasons for Failure• One Size Methodology Fits All• Intolerant Methodology• Embellished
• Do frivolous tasks• Ignore necessary tasks
• Heavy• Untried• Tried Once
• Limited applicability• Champion
• Failure to Tune• Distractions
• Dates can’t be met
May, 2004 Page 17 William (Bill) Myers
Crystal Methodology – Theory
( NOT RELATED to Crystal Reporting )
• Self-adapting "shrink-to-fit," software development methodologies
• Different set of policies & conventions for each project• Projects are sensitive to people issues • Crystal is people-centric
– other methodologies may be: » process-centric» architecture-centric» tool-centric
• Keys• Communication intensive
– Masses of paper do not improve communication• Experiment based • Requires feedback loops
• Theory of Methodologies
May, 2004 Page 18 William (Bill) Myers
Crystal – Communication
• Effectiveness of different methods
• Courtesy of Alistair Cockburn • http://alistair.cockburn.us/crystal/articles/cpanfocisd/characterizingpeopleasnonlinear.html
People are Non-Linear
First-Order Components in Software Development
May, 2004 Page 19 William (Bill) Myers
Crystal - Model• Model
• Risk to end-user, Size, Complexity– Tables present questions to ask
» To establish processes & controls» When to change
• Weight is costly– Heavy methods for large teams– More ceremony for critical deliverables– Avoid frivolous tasks– Efficiency is expendable
» Non-bottleneck activities & Rarely used items
• Crystal Methodologies• Reference books & web site• Alistair Cockburn – Author• http://alistair.cockburn.us/crystal
May, 2004 Page 20 William (Bill) Myers
SCRUM• Principles
• Do the right thing (from customer’s view)• Deal with reality, not artifacts or fixed plans• Short iterations
– Usually four weeks– Be prepared to implement at the end of an iteration
• Allow changes• Keep everything visible
– Concentrate on time remaining - not time spent
• Highlights and pinpoints responsibilities• Self organization
– Creates sense of obligation and teamwork– Extensive collaboration– Changes role of Project Leader
» Certification available
May, 2004 Page 21 William (Bill) Myers
SCRUM• Broad application
• Iterative, lightweight project management method• Software AND non-software projects• Based on manufacturing Process Control theory
• Adaptable – Can be implemented
• Beginning of a project• Middle of a project• Product development effort that is in trouble
• More information• Ken Schwaber and Mike Beedle • http://www.controlchaos.com/
May, 2004 Page 22 William (Bill) Myers
SCRUM – Concepts
Courtesy of Ken Schwaberhttp://www.controlchaos.com/Scrumo.htm
May, 2004 Page 23 William (Bill) Myers
SCRUM• Full visibility
• Priority list• Items in development cycle ( sprint )• Short, daily meetings ( 15 minutes )• Work hours remaining in sprint• Assignments, who & what
• Responsibility & Authority• Anyone can add to priority list• Business manages priorities• Developers determine what can be done• Project Leader removes impediments
• Removes developers from politics
May, 2004 Page 24 William (Bill) Myers
SCRUM - Impact• Backlog
• Scope Creep – no such thing• No request ever lost
» It may never get a high enough priority
• Business owner only one who can remove it
• Return on Investment• Install option available
– Software completed to date available / immediate return
• Option to stop– Save development costs
• Perfect for fixed price projects– Reorder deliverables list – no cost– Substitute deliverables with equal effort – no cost– “Stop Development” clause easy to apply– Big competitive advantage
May, 2004 Page 25 William (Bill) Myers
Scrum - Definitions• Product Backlog
• Prioritized list of items» Business Product Owner – sets priorities» List of Functional requirements,
Nonfunctional requirements, Issues» Contains preliminary estimate» Changes constantly» Items in a sprint: locked down
• Time-box• Period of time that can not be exceeded
• Sprint• Time-box of 30 days• Sprint backlog defines delivery list
» Product item» Internal support item (i.e. database, system doc)» Work remaining & burndown graph
May, 2004 Page 26 William (Bill) Myers
Scrum - Players• Scrum
• Product Owner» Adds items to product backlog list» Set priorities
• Team» Determines sprint list» Develops software
• Stakeholder» Funded project, will use it, affected by it» Requests enhancements
• ScrumMaster» Responsible for Scrum process» Removes impediments
• Other interested parties• IT Management
» Manpower allocation» Budgets & Billing
• Senior Business Management» Best use of corporate resources
May, 2004 Page 27 William (Bill) Myers
Scrum - Milestones
• Sprint Planning meeting ( two 4 hour segments )» Product owner & Team: identifies backlog items to be
delivered in 30 day sprint / Team commits» Team plans details in Sprint Backlog
• Daily Scrum Meeting ( 15 minutes )» Team: Review prior day, identify impediments, plan day
• Sprint Review Meeting ( 4 hours )» Team: demonstrates completed functionality
- to Product Owner and other interested parties
• Sprint Retrospective meeting ( 3 hours )» Team: how to improve next sprint or
make more enjoyable
Sprint PlanningMeeting
( 8 Hours )
Sprint( 30 cal days )
Sprint ReviewMeeting
( 4 hours )Daily Scrum
Meeting( 15 minutes )
SprintRetrospective
Meeting( 3 hours )
May, 2004 Page 28 William (Bill) Myers
SCRUM: database - goals• Software development
• Collect new requests• Manage product backlog• Manage Scrum process• Reduce administrative load on developers
» Reduce or eliminate most status reporting needs
• Support Management• Provide data for manpower planning• Provide status to everyone
» Priority in queue» Currently being worked on» Remaining effort
• Provide historical information» Time tracking mechanism for billing» Answer “What did you do for us last year”
May, 2004 Page 29 William (Bill) Myers
SCRUM – Process FlowSCRUM
Bu
sine
ssP
rod
uct
Ow
ner
De
velo
pm
en
tT
ea
mA
ppl
icat
ion
Re
po
rts
Use
r
Request newfeature
Accept orRejectrequest
ProcessResponse
Request
via Web Page
e-Mailresult
Backlog
Result
Makehigh levelestimate
PrioritizeBacklog
e-Mail notification
Estimate
Sprint List
Determine whatcan be done
by priority(Team Commits)
Commitsbusinessresources
PlanningMeeting
Hours Workremaining
Chargeback
Daily SCRUM Meeting:
1. Completed yesterday2. To complete today3 Impediments
Log time forbilling only
DemonstrateResults
Daily
Putinto
Production?
Put intoProduction
Yes
Status Change
ProjectedManpower
Needs
StatusReports
InvoicedHours
CurrentAssignments
Work inProcess
DeliveredProducts
Priorities
RetrospectiveMeeting
REPORTS:
Timely - reports always show current status, but can show historical strips Agile - reports are self-building, self-summarizing, include many filters Visible - only a confidential project would be hidden
Backlog Planning Sprint Demonstration
Accepted
William W Myers - June 2004
May, 2004 Page 30 William (Bill) Myers
SCRUM – Database ModelProjects
Project_ID
Comments
Comment_ID
RequestFromUser
UserReq_ID
Features
Feature_ID
Assignments
Project_IDPerson_IDFeature_ID
ProjectTeam
Project_IDPerson_ID
People
Person_ID
ProjectManagement
ProjMgt_ID
HoursCharged
Charge_ID_IDPerson_IDSource
ModuleSprint
Module_IDSprint_ID
ProductionModule
Module_ID
Sprints
Sprint_ID
SprintFeatures
Sprint_IDFeature_ID
Sprint_to_Feature
Comment_IDFeature_ID
ProjectType
ProjectType
SCRUMDatabase
Model
William W Myers - June 2004
May, 2004 Page 31 William (Bill) Myers
SCRUM: demo app – Priority Page
May, 2004 Page 32 William (Bill) Myers
DSDM Model
• Principles• Iterative & incremental development• Changes during development are reversible• Requirements baselined at high level
– Minimum details to continue
• Testing throughout lifecycle– Developers AND business users
• More Information• Developed in UK• Grew out of RAD approach• http://www.dsdm.org/
May, 2004 Page 33 William (Bill) Myers
Lean Development Model• Most strategic oriented of Agile methods• Principles
• Domain instead of point solutions• 80% today better than 100% tomorrow• Everything is changeable• Minimal is essential• Needs determine the technology• Best value for money
• More information• http://www.poppendieck.com/• http://www.leanconstruction.org/
May, 2004 Page 34 William (Bill) Myers
Adaptive Software Model• Software development
• Complex adaptive system• Like stock market or flock of birds• Circumstances generate emergent capabilities
» Total skills exceeds sum of parts » Only emergent capabilities tames complexity
• Software organizations:Must create the proper circumstancesfor emergenceof superior capabilities
May, 2004 Page 35 William (Bill) Myers
Adaptive Software Model
• Principles– Leadership – Collaboration
– Speculate, collaborate, learn– Drop militaristic Command and Control (stifles emergence)
– Cycle through phases many times, converging– Tasks not listed in plan– Components at end of plan are listed– Mission directed, feature driven, iterative
time boxed, risk driven, change tolerant
May, 2004 Page 36 William (Bill) Myers
Feature-Driven Development (FDD)
• Process– One-time
– Quick, high value, low dollar
• Develop overall domain model• Build feature list• Plan by feature
– Iterative• Design by feature• Two week cycles building components
• More information• http://www.featuredrivendevelopment.com/• http://www.nebulon.com/fdd/
May, 2004 Page 37 William (Bill) Myers
Extreme Programming - XP• Project management
• Collect customer stories• Estimate & prioritize• Small iterative development cycles
• Development• Test-first development
» Write test cases before coding• Collective code ownership• Refactoring• Pair programming• Daily standup meetings
• Many processes & rules• More information
• http://www.extremeprogramming.org• http://www.objectmentor.com/home
May, 2004 Page 38 William (Bill) Myers
Agile - Review• Positives
• Software delivered• No death marches
• Dangers• Process itself becomes the goal• Placebo effect• Key point is missed
• SCRUM• Immediately useful• Becoming broadly popular
» Classes around the world» Hundreds certified
May, 2004 Page 39 William (Bill) Myers
Reporting Population
• Super User » Technical skills » Understands their piece of the data» May be tactical or strategic
• Strategic User » Executive or analyst» Needs highly summarized and graphed snapshots» Limited technical skills
• Tactical User» Needs up-to-minute detailed data to do their job» Limited technical skills
Super-Users(ad-hoc)
Strategic Users( historical cubes )
Tactical Users(core & options )
May, 2004 Page 40 William (Bill) Myers
Normal Report Development Process
• Demand that customer predict the future• Forecast list of needed reports• Requires visualization / no data available• Data may support 20,000,000 variations
• Design signoff in blood• Early code freeze
• Rigid Change Control• More signoffs
• Complain about scope creep• Accuse users of changing their mind
• List all changes as bugs• Falsely implies a defective product Requirements List
May, 2004 Page 41 William (Bill) Myers
Agile Reporting• Bounded Chaos
• Random actions inside of the boundary
• Self Building• Developer builds 90%• User checks options• Program finishes writing itself• Program creates report
• Broad Scope• Can replace millions of fixed reports• Requirements can be hazy up front
May, 2004 Page 42 William (Bill) Myers
Agile Reporting - Definitions• Report (definition)
• Any output where user can not change data in database• More than printer
• Core & Options• Core – on every report, fixed • Options – added to report based on user’s selections:
columns & totals
• Filters• Limits input data
• Medium• Web page, paper, excel, pivot table, PDF file, lists,
graphs, etc
• Measures• Changes the core - unusual
May, 2004 Page 43 William (Bill) Myers
IBI Sample - ggorder2.fex
• As Installed• Units by Product• For one date
• Agile Modification• Add &sort1• Standard front-end• Add dollar value
» Immaterial to agile
May, 2004 Page 44 William (Bill) Myers
Core & Option - Web Page
• Shows total sales of each product by state & sales rep• Checking no boxes: one number only
May, 2004 Page 45 William (Bill) Myers
Agile ggorder2.fex
• Core only
• Boxes checked• None
May, 2004 Page 46 William (Bill) Myers
Agile ggorder2.fex
• Same fex• No changes
• Boxes checked• Vendor Name• Product
May, 2004 Page 47 William (Bill) Myers
Agile ggorder2.fex
• Same fex• No changes
• Boxes checked• Product
– Totals
• Store Name
May, 2004 Page 48 William (Bill) Myers
Agile ggorder2.fex• Same fex
• No changes
• Boxes checked– State, City,
Store Name (totals), Vendor Name, Product
• Options– 362,889 formats
– ONE program
– Risks managed
May, 2004 Page 49 William (Bill) Myers
Agile Reporting – Core & Options
• Core– Units Ordered
• Options– State– Contact– Product– Package
– Totals on: Contact, State
May, 2004 Page 50 William (Bill) Myers
Core & Option Executive Demo• IBI Sales Prototype
• Used Production databases• Not standard IBI demo
» Specs and Core & Option HTML were supplied
• 4 Days development• Clear to all
• Reports• Simple list – Volumes & Dollars (agile core & option)• Delta – Change since yesterday (agile core & option)• Complex Delta – Changes of averages• 3-D Graphs• Pies with drill downs
May, 2004 Page 51 William (Bill) Myers
Exclusions
• Power not demonstrated• Combined databases• Reporting Cubes• Scheduling & Mailing• Super-user interface
May, 2004 Page 52 William (Bill) Myers
Risk Mitigation
• End-User reporting
– Normal recommendation
– Risks
– Mitigation for average user
– Super-users
May, 2004 Page 53 William (Bill) Myers
Normal Reporting Recommendation
• Reporting is simple, use…– MS Access– Excel– Pivot Tables– WebFocus– Crystal Reports– Etc
• Training– Help– Manuals– Limited classes
May, 2004 Page 54 William (Bill) Myers
Risks
• Complexity– Database structure– Queries
• All tests against production databases– Overload servers & network
• Over-reporting– Double counting data
• Under-reporting– Not including data
Deal Price
Deal Price
May, 2004 Page 55 William (Bill) Myers
Mitigation• Target the skill sets
– Developers• Handle complexity• Insure accuracy• Insure efficiency• Identify possible options• Set up “Core & Option” reports
– End Users• Decide on core & options during development• Test during development• Understand the results
May, 2004 Page 56 William (Bill) Myers
Products & Languages
• Effort• Easy• Varying levels of difficulty
– Brute force –vs- fits vendor’s product well
• Power• Is agile• Appears to be agile• Partly agile• Beware of pretty –vs- power
May, 2004 Page 57 William (Bill) Myers
Regular File Structure - WebFocus
Pre-load ( fex )
GetCore & Options
( HTML )
Report ( fex )
Report Caster ( fex )
VariablesLists
OptionsFilters
Output Medium
Pre-setOptions, FiltersOutput Medium
Menu Item ( fex )
Pre-setOptions, FiltersOutput Medium
HTMLPDFExcel
Files
Standard Reporting ProgramRelationships
May, 2004 Page 58 William (Bill) Myers
Core & Option Structure - WebFocus
• Build JavaScrips• Options & Template• Options – SQL database• Template & Fields – text file
• Hidden Fields fairly static
May, 2004 Page 59 William (Bill) Myers
Report ( fex )
DEFINE FILE TRADES
Cost/P12.2CB = VOLUME * Price;
Value/P12.2 = VOLUME * STRIKE_PRICE;
Net/P12.2 = Value - Cost;
END
TABLE FILE TRADES
SUM VOLUME AS 'Volume'
Cost
Value
&sort1-* ***** add other BY statements here *****
END
May, 2004 Page 60 William (Bill) Myers
The End( References )
• http://www.agilealliance.org - Agile Alliance
• http://www.jimhighsmith.com - Agile overview
• http://alistair.cockburn.us/crystal - Crystal
• http://www.controlchaos.com/Scrumo.htm - Scrum
• http://www.dsdm.org - DSDM
• http://www.poppendieck.com - LEAN
• http://www.leanconstruction.org - LEAN
• http://www.santafe.edu - Complex Adaptive
• http://www.featuredrivendevelopment.com - FDD• http://www.nebulon.com/fdd - FDD• http://www.extremeprogramming.org - EX (Extreme)• http://www.objectmentor.com/home - EX (Extreme)
May, 2004 Page 61 William (Bill) Myers
Agile Manifesto (1)Follow these principles for Developing Software:
• Satisfy customerswith valuable software
• The highest priority is to satisfy the customer through early and continuous delivery of valuable software.
• Welcome change requests at any time • Making changes quickly can be a competitive
advantage• Welcome changing requirements, even late in
development. Agile processes harness change for the customer's competitive advantage.
May, 2004 Page 62 William (Bill) Myers
Agile Manifesto (2)Follow these principles for Developing Software:
• Deliver working software frequently • Don’t let people suffer• Deliver working software frequently, from a couple
of weeks to a couple of months, with a preference to the shorter timescale.
• Daily Contact • Customers & developers• Avoid intermediaries• Business people and developers must work
together daily throughout the project.
May, 2004 Page 63 William (Bill) Myers
Agile Manifesto (3)Follow these principles for Developing Software:
• Use Champions• Build projects around motivated individuals. Give
them the environment and support they need, and trust them to get the job done.
• Team Communication • Avoid long or large meetings• The most efficient and effective method of
conveying information to and within a development team is face-to-face conversation.
May, 2004 Page 64 William (Bill) Myers
Agile Manifesto (4)Follow these principles for Developing Software:
• Working software is primary measure• Models, documentation, use cases do not count• Working software is the primary measure of
progress
• Maintain a constant pace indefinitely• No Death Marches • Agile processes promote sustainable development.
The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
May, 2004 Page 65 William (Bill) Myers
Agile Manifesto (5)Follow these principles for Developing Software:
• Quality enhances agility• Late and post-implementation changes easier• Continuous attention to technical excellence and
good design enhances agility.
• Self-organizing teams increase quality• Volunteers produce better results• Members look out for and help each other• The best architectures, requirements, and
designs emerge from self-organizing teams.
May, 2004 Page 66 William (Bill) Myers
Agile Manifesto (6)Follow these principles for Developing Software:
• Simplicity• Do less – get more• The art of maximizing the amount of work not
done -- is essential.
• Teams self-correct • Not continuously – will drive people nuts• At regular intervals, the team reflects on how to
become more effective, then tunes and adjusts its behavior accordingly.
May, 2004 Page 67 William (Bill) Myers
File Naming Standards - WebFocus
• Application prefix + Name + Suffix– Drilldown can be called from many reports– Name is first report written
• Program field names – UpperLower_Case
Function Program Name Suffix
Pre-load Scr_Status_h.fex hGet Core & Options Scr_Status.htm
Report Scr_Status.fex
Drill Down Scr_Status_dr1.fex dr(+)Report Caster Scr_Status_c1.fex c(+)
Menu Item Scr_Status_m1.fex m(+)