Emergent architecture- a casestudy TREDS
-
Upload
syed-rayhan -
Category
Documents
-
view
1.355 -
download
0
description
Transcript of Emergent architecture- a casestudy TREDS
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
Emergent Architecture on Agile Projects
Syed Rayhan
Co-founder, Code71, Inc.Contact: [email protected]: http://blog.syedrayhan.comCompany: http://www.code71.comProduct: http://www.scrumpad.com
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
2
Agenda
Recap
Initial Architecture
Section 2
Section 4
Section 3
Evolving initial architecture
Project summary
Section 5
IntroductionSection 1
Q&ASection 6
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
3
Agenda
Introduction
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
4
My Background
Expertise
Career
Iterative incremental development
Agile technology planning and architecture
On-shore/Off-shore software development using Agile/Scrum
Interests
Co-founder, Code71, Inc. 15+ years of total experience Co-author of “Enterprise Java with UML”
Cultural aspect of self-organizing team Scrum for small projects delivered remotely Agile engineering practices
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
5
What to Expect
Focus
Context
How architecture fits in the new, new world of agile development
How emergent architecture actually is in practice
Candid look at what is working, what is not working, and how we have adjusted to succeed
Key Takeaways
How to do architecture on agile projects
Who does architecture
What practices to customize and why
Iterative, incremental development a.k.a. Agile software development
Traditional architecture practice is inherently waterfall
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
6
The Case Study• Department of Motor Vehicle (custodian agency)
• Department of Transportation
• State and local law enforcement agencies
ProjectJustification
Clients
Architecture Landscape
• Ensure public safety on roads
• Reduce traffic collisions
• System under consideration is dependent on other systems that are also in the process of being replaced
• Infrastructure is provided by VITA
• New technology
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
7
Agenda
Project Summary
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
8
Problem Statement and Project Goals
• The current backlog of reports 3-6 months• Duplicate data stores is causing poor data quality• Manual data entry is causing poor data quality as well as
increased processing time and costs• Lack of data visibility and access to data contributes to
wrong or delayed policy changes and or enforcement
Problems
Goals
• Streamlining and simplification of data collection• Elimination of paper reports• Increasing the efficiency and improving data quality• Elimination of duplicate data entry• Electronic data reporting• Elimination of data entry backlogs
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
9
Project Facts
• Contractors and consultants
• Core team has 12 members with cross-functional
• Located on-site
Funding
Staffing
Timeline
• Federal grant
• $4.5 M
• Project initiated in 2006
• Development started in 2007 May
• 60 sprints completed to-date
• First release was in March 2008
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
10
Project Context
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
11
Project type
well understood Not in agreement
expert
new
requirements
tech
nolo
gy
simple
Zone of death
complex
complex
Our project
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
12
Existing Architecture
data entry
Image db 2
Image db 1
data entry
copy
update
image
image
update
update
Agency 2Agency 1
Agency 3
data export
Main App
App 1 App 2
App 3
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
13
Existing Enterprise Architecture Standards
Technology Platform
Security
Infrastructure
VB.Net, MSSQL & Oracle
Two-factor authentication (FOB based)
Strict 3-tier architecture
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
14
Agenda
Initial Architecture
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
15
Architecture Principles
• Minimize custom components
• Reuse existing components
• Leverage COTS components
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
16
Needs to Technology Map
Crash form redesign
Electronic submission
Increase dataaccuracy and reducedata correction needs
Workflow automation
Information exchangewith various externaland internal systems
Standard & ad-hocReporting
Analysis and datamining
Authentication &authorization(security)
Expanded and modifieddata model
Support for multiple waysto send in reports (scan,Web, batch)
Automated data validation
Workflow Management
Integration tool that cansupport multipleformats, protocols, andmodes (batch & real-time)
Reporting tool forbusiness & technicalusersAnalytic tool forBusiness users
Security that spansIntranet, Extranet, &Internet
XML
Report Beam,
Taskmaster
BizTalk
BRE
WF
SQL Server
SSIS, SSRS
LDAP based Store (ADAM)
Functional Technical Proposed
Requirement Assessment Component
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
17
Architecture Patterns
TREDS User Interface
TREDS business logic controller
CRASH Data Model
TREDS Integration Layer
HTRIS RNS
CSSCAP
RMSRMS
Report Beam
MVC
Hub & Spoke
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
18
Logical Architecture- Initial Vision
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
19
Roadmap (partial view)
Nov. 15, 2007 - CAP Expansion
•CAP expansion to accommodate new fields
•Scan bubble data into CAP•Automate data presentation forSafetynet•Build TREDS/SFTP Interface•Build system to system authorization and authentication
Milestones Capabilities Impact
Jan. 15, 2008•Revised FR300 Scanning•TREDS/Safetynet Integration
•Allow automated commercial datareporting to Safetynet•Reduce manual data entry with scanning•Business Process Impact:: - Scan data validation - Automation of some current manual operations
•Expand database for new fields•Expand UI for new field data entry
April 15, 2008- TREDS/RNS Integration
•Build TREDS/CAP interface•Build TREDS/RNS interface•Build CAP/Broker interface
•Eliminate RNS dependency on HTRIS•Remove requirement for FR300forwarding to VDOT•Business Process Impact: - DMV/VDOTFR300 forwarding process
Nov. 15, 2008•TREDS Core Deployment•TREDS Client – Pilot 1
•Build TREDS data management•Build centralized rules engine•Build TREDS Data Warehouse(Havana)•Scan bubble data into TREDS(CUBA)•CAP data migration to TREDS•Build human to system authorization and authentication
•Back Office operations will use
TREDS for data management
•TREDS/ CAP data synchronization will provide system fall back during burn in•Policy and standards enhancements may occur due to human to system authentication and authorization•Business process impact – IT systems support for TREDS – operations will expand
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
20
Agenda
Evolution of Architecture
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
21
1st Release Architecture
Controller WS
BT Orchestration
WS
Broker
FTP
data migration
App 1
Main App
unprocessed data
update
WS-API
Image db1image
Scanning App
Image db2
App 3
image
data
DW
data
image
App 2
scannedreport
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
22
1st Release Recap
Reduce data entry time by 50%
Reduce architectural risk around infrastructure,
Existing system is still the system of records
Electronic data capture, not delivery
Goals
Capabilities
Lessons Learned
Form redesign should have followed data modeling and scanning technology selection
Windows environment for efficient data entry poses different challenge
Result
OCR accuracy was lower than expected
Initial data processing speed went down
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
23
2nd Release Architecture
Controller WS
BT Orchestration
WS
Broker
FTPApp 1
Main App
data
update
WS-APIImage db1
image
Scanning App
Image db2
App 3
image
data
DWprocessed
data
App 2Workflow
Model
Web Client
DB
BRE
DB
AzManscannedreport
Processed data
AuditDB
trigger
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
24
Defining Workflow using State Machine
stm Crash State Diagram
PO submits the original/revised report
Superv isor rev iew
Under Correction
Pending
Under Rev iew On hold
DMV Approv ed
Assigned Pending
Anoriginal/revisioncrash wasscanned
Pending Manager Rev iew
Accepted
BO manager/clerkreaccess crash
An approved crash is submitted
PS rejects crashto PO
PS approves crash
PO submits for re-approval
system assignsreportables or BOclerk pulls
Non-reportable by pass the crash processing
BO manager/clerkputs in on hold
BO clerk sends crash to manager
A new revision created
Manager reassigns crashto pending or a user
VDOT accepts crash with updated GPS
VDOT rejects the crash
BO clerk processes crash
Time out or logoff
BO manager processes crash
Crash lifecycle ends
Manager/clerk accepts crash
Auto-approved
Revision is created
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
25
2nd Release Recap
Existing system is still the system of records
Automated work management of data entry clerks
Goals
Capabilities
Lessons Learned
Too many data validation rules were too restrictive
Lack of robust exception handling resulted in premature termination of workflows
Result
Automated workflow was well received
Improved the quality of management of the supervisors
Introduce automated workflow for data processing
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
26
3rd Release Architecture
Controller WS
BT Orchestration
WS
Broker
FTP
App 2
Main App
update
WS-APIImage db1
image
Scanning App
Image db2
App 4
image
data
DW processed data
App 3Workflow
Model
Web Client
DB
BRE
DB
AzMan
Report WS
scannedreport
AuditDB
trigger
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
27
3rd Release Recap
Integration with App2
Canned reporting from new DW
Main reporting from the new system
Complete data entry from the new system
Goals
Capabilities
Lessons Learned
Longer processing time at certain steps in the workflow caused user frustration
Web service interface to BizTalk was causing timeouts
Result
Reporting time reduced from a few weeks to a few minutes
Reports requiring manual check reduced to 70%-80% due to auto acceptance
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
28
4th Release Architecture
Controller WS
BT Orchestration
WS
Broker
FTP
App 2
Main App
update
WS-APIImage db1
image
Scanning App
scannedreport
Image db2
App 4
image
data
DWprocessed
data
App 3Workflow
Model
Web Client
DB
BRE
DB
AzMan
Report WS
Req Q Req Q
Client App
electronicreport
scanner
AuditDB
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
29
Workflow with Asynch. System Processing
stm Crash State Diagram
PO submits the original/revised report
Superv isor rev iew
Under Correction
Pending
Under Rev iew On hold
DMV Approv ed
Assigned Pending
Anoriginal/revisioncrash wasscanned
Pending Manager Rev iew
Accepted
System Processing
BO manager/clerkreaccess crash
An approved crash is submitted
PS rejects crashto PO
PS approves crash
PO submits for re-approval
system assignsreportables or BOclerk pulls
Non-reportable by pass the crash processing
Auto-approved
BO manager/clerkputs in on hold
BO clerk sends crash to manager
A new revision created
Manager reassigns crashto pending or a user
VDOT accepts crash with updated GPS
VDOT rejects the crash
Revision is created
BO clerk processes crash
Time out or logoff
BO manager processes crash
Crash lifecycle ends
BO manager/clerk accepts the crashCSS post errors
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
30
4th Release Recap
Improve response time using asynchronous processing
Electronic delivery of reports from the field
Direct submission of reports (no scanning) to the system
Complete data entry from the new system
Goals
Capabilities
Lessons Learned
Change in workflow caused versionites
Accelerate adoption of electronic delivery of reports from the field
Result
Improved user satisfaction from reduction in response time
Reports requiring manual check further reduced to 50%-60% due to auto acceptance
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
31
5th Release Architecture
Controller WS
BT Orchestration
WS
Broker
FTP
App 2
Main App
update
WS-APIImage db1
image
Scanning App
Image db2
App 4
image
data
DW processed data
App 3
Workflow
Model
Web Client
DB
BRE
DB
AzMan
Report WS
Req Q Req Q
Client App
electronicreport
scanner
scannedreport
AuditDB
WCF
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
32
5th Release Recap
Integration with App3“Revision of Reports” handlingAd-hoc reporting from new DW
New system is self-sufficient
Asynchronous processing of reports delivered electronically
Goals
Capabilities
Lessons Learned
The definition of “revision of a report” changed (new capability changes business practices)
ResultIntegration with other reporting tools for electronic reporting
Old system is ready for sunset
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
33
What’s next?
• Keep up with the technology upgrades and changes• Upgrade to Biztalk 2009• Upgrade to Visual Studio 2008
• Keep an eye on new technology for new capability or easier maintainability• ASP.Net MVC• Entity Framework or Fluent NHibernate to replace Nhibernate
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
34
Agenda
Recap
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
35
Recap lessons learned
Start with existing architecture standards, identify gaps, and know
upcoming changes
Influence / introduce new standards to fill the gap in existing
architecture
Align with upcoming changes to the existing architecture
Let architecture evolve top-down (architect is responsible) and
bottom-up (team is responsible)
Use working “spikes” to determine how to adopt new technologies
Manage evolution of architecture in small increments (a.k.a refactor)
Document architecture using FAQ style and just-in-time
Architecture evolves; when unmanaged, it becomes sprawling.
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
36
Q&A
Guided evolution of architecture improves Agility!
Copyright 2010 Code71 Inc. All rights reserved.http://www.code71.com http://www.scrumpad.com
37
Contact
Please contact for on-site
training/coaching or Webinar:
Contact: [email protected]: http://blog.syedrayhan.comCompany: http://www.code71.comProduct: http://www.scrumpad.com