© 2006, iTKO, Inc. | All rights reserved.
Achieving 99.99% Qualityin SOA Applications –
The “Headless” Dilemma
John Michelsen
StickyMinds WebinarNovember 7, 2006
Agenda
SOA development and QA trends
Strategy/best practices - the Three C’s
“Headless” Testing Fundamentals
LISA Intro, and LISA WS-Testing Offer
Who are we?
1999 . 2003 . 2005 . 2006 .2001.
1999 – iTKO Incorporates (Privately Held)
2001 – LISA under development – (4) Patents Pending
2003 – LISA 1.0 Released to Market
2004 – LISA 2.0
2006 - LISA 3 & 3.5
A History of Q
uality Leadersh
ip
2004 .
• Entered the market to deliver an SOA testing solution, LISA
• Current version 3.5 (2006)
• Key value propositions:• Increase Test Coverage• Lower Cost per Test• Faster Discovery & Resolution
Everyone should own quality™.
Mainframe Client Based SOA/Composite Apps
1980 . . . 1985 . . . 1990 . . . 1995 . . . 2000 . . . 2005 . . . 2010 . . . 2015
3270/CICS data stream testing
User-Interface oriented testing
Component, heterogeneous, distributed testing
Platform Evolution Causes Testing Evolution
The Promise of SOA
• Reduce integration cost• Through loosely-coupled forms of integration and industry standards
• Increase asset reuse• Build new business workflows from existing Services to form
composite applications
• Increase business agility• Better control business process definition and management to meet
customer needs
• Reduce business risk• Governance, compliance, and risk reduction through increased
business visibility and agility
Server (database)
BrowserUIs
Standardenterprise apps
ClientUIs
…before SOA
You had a standard delivery platform.And a standard infrastructure.
Transaction service
Datawarehouse
BI tools
Core app
PartnerReseller
Customer buyer
Business rules
Order management
Outsourcedsupplier
Transactionprovider
PartnerReseller
Customerbuyer
Your extendedenterprise
Division 1
Division 2
SOA promises a composite workflow
SOA was supposed to simplify extending workflows.
Transaction service
Datawarehouse
BI tools
Your AppWorkflow
Channel Partner
RMI objects
Customer company
CRM Web App
BusinessRules
Outsourced firm
Legacy App
SOAP objects
Web Services
.NET OrderingService
ContentDatabaseLegacy App
Web interface
Messagingservice
File System
Legacy Data
Division2
FinancialsMainframe
Division 1 Your Company
ESB
But heterogeneous technology is complex
Standards and components are still evolving
Composite apps have multiple owners
It’s Continuously Changing
Why A New Strategy?
• Few changes at the database level, but multiplies issues downstream
• For every middle tier change, there are hundreds of front-end impacts possible at the UI layer
• Limitless front end options• Any client, multiple customers• Mix of technologies –
XML/RPC, Swing, AWT, JS, DOM, now AJAX
• Services consume business logic
Database/mainframe= 1 change
UI level=1000x
Middle-tierlevel= 100x
Best Practices: “Three C’s” of SOA Testing
Complete Testing• Every heterogeneous layer of
architecture• Test UI verify in system of record• Reuse functional test for
performance testing
Collaborative Testing• Test early before UIs are created• Not just dev, business analysts and
QA should verify processes
Continuously Test• Regress on existing functionality• Add testing of new services to
existing testing workflows
Complete >single component all technologies
Continuous >Phased tests constant testing
Collaborative >Dev/QA test silos everyone tests, every phase
Data / Applications
Process and services layer
Complete: SOA Testing across every layer
Presentation layer
Integration layer
Web UI
RMI
DatabaseLegacy App File System
ASP.NET
MQ
Custom API
Swing
J2EE
SOAP/XML
CORBA
ESB
BPM .NET WS
AdditionalApps Now& Future
AdditionalApps Now& Future
AdditionalApps Now& Future
CORBA App
Typical Trading System Example
OrderingSwing App
ES
B
ProcessingWeb App
BusinessProcessAppA
BusinessProcess AppB
AdditionalApps Now& Future
Database
DB
RMI
Data is correlated in here.
1
9
App Server
EJB, WS,POJO
Collaborative: SOA Requires Agile
QA expands dev tests and create functional and load
tests
Complete Test CoverageLower Cost per TestFaster Discovery & Resolution
Developers unit test and jump-start QA
Business Analysts outline a business process and ‘not implemented’ test
cases
Production test their implementation and report
issues to support
Support rapidly debug issues and communicates appropriately
Collaborative Testing in an SOA environment
BPM Validation
Orchestration Tests
Service Validation
Component Tests
Service 3
Dev/QE
Order Acquisition
Check InvCheck Credit
Place Order
QA/BA
Place OrderService 1 Service 3
Service 2
QE QA Con
tinuo
us S
OA
Tes
t Pla
tform
Continuous: SOA is never done
• SOA development is constantly evolving
• Project-based development had a “test phase”
DatabaseCreate
componentsCreate UI
AcceptanceTest
DeployAppUnit test Integration
New service
Database Create UI
Database
Create UI
Create UI
New service
Servicechanged
New service
Serviceretired
ESB
Continuous: Minimize Unintended Consequences
APP 1
Database
Registry MDM Transactions
InternalService
LegacyApp
PartnerService
APP 2
???
???
?
?
NEW
ESB
Continuous: Minimize Unintended Consequences
APP 1
Database
Registry MDM Transactions
InternalService
LegacyApp
PartnerService
APP 2
A data service is currently failing
due to a change in the internal
service
ContinuousTesting
Continuous: SOA at 5-9’s
Business Investment
Ava
ila
bil
ity
100%99.999%systemuptime Physical
Infrastructure
Sca
lab
ilit
y
100%
99.999%applicationavailability
Inte
gri
ty100%
?????%functionalintegrity
ApplicationInfrastructure
BusinessApplications
Fault Tolerant HWDiesel PowerRedundant Bandwidth
App ServersTransaction MonitorsClustered Services
SOA-aware Dev/QA ProcessComprehensive Testing ToolsContinuous Testing Service
Availability to the business is the product of all three
Continuous Levels: Build & Deploy
Component/Service Teams
Development
QA/QE
Pre-production
Production
product component product component
Dev build
test bed
Staging
Production
Dev build
test bed
Con
tinuo
us S
OA
Tes
t Pla
tform
Deployment Level
BuildLevel
Requirements are King -- On-line Music Scenario
• Use Case: (UC1) Customer with existing account does a search for ‘Rush’, selects the Grace Under Pressure tour DVD and uses the 1-click purchase feature to buy.
• Business: (BR1) We need a flexible mechanism to change these criteria at our whim. After discussion: for now partners get 20% of net proceeds on their product sold on the first $100,000 / mo, 10% thereafter.
• PSR: (PSR1) Search can’t take any longer than 3 seconds for a user on a typical broadband connection.
• Policy: Punt for now. This is security, interop standards, corp dev standards, etc.
• Functional: (FR1) Search must include results from CDs, DVDs, Books, and Memorabilia, with CDs on the top (UC1).
• Technical: (TR1) Create a search service that aggregates CD, DVD, Book, and Mem data (FR2). (TR2) Search service execution must be under 2 seconds (PSR1).
Service Level Guidelines
Service Validation
Component Tests
Service 3
Dev/QE
• UC & BR: Validation mechanism
• PSR: Load/performance test at the service level
• FR: Likely 80% of your requirements can be nailed right here
• TR: Dev test at the component level
UC=Use Case, BR=Business Requirements, FR=Functional Requirements, TR=Technical Requirements
Orchestration Level Testing Guidelines
• UC & BR: Not usually
• PSR: Often the best way to prove PSR
• FR: Invoke a behavior, validate outcomes
• TR: Simulate unstable/unavailable services
Orchestration Tests
Place OrderService 1 Service 3
Service 2
QE QA
UC=Use Case, BR=Business Requirements, FR=Functional Requirements, TR=Technical Requirements
Composite/BPM Level Testing Guidelines
• UC: Primary focus, performs a cause that we validate at the service/component level
• BR: Many times same as UC, sometimes not relevant
• PSR: A few PSR scenarios proved at this level, but continuous testing is the key here
• FR & TR: Hopefully very fewBPM Validation
Order Acquisition
Check InvCheck Credit
Place Order
QA/BA
UC=Use Case, BR=Business Requirements, FR=Functional Requirements, TR=Technical Requirements
Typical SOA Test Map -- Dramatically Simplified
Content
Presentation Layer
Content Mgmnt
Offers & Specials
Search OrderMgmnt
CommissionRules Svc
CD DVDBooks Mem
CRM System
Media Delivery Billing
Media Repo
AP
CRMFacade
Batch Feeds
UC1
UC1
UC1
UC1: Search & Buy
Typical SOA Test Map -- Dramatically Simplified
Content
Presentation Layer
Content Mgmnt
Offers & Specials
Search OrderMgmnt
CommissionRules Svc
CD DVDBooks Mem
CRM System
Media Delivery Billing
Media Repo
AP
CRMFacade
Batch Feeds
BR1*
BR1
BR1: Commissions
Typical SOA Test Map -- Dramatically Simplified
Content
Presentation Layer
Content Mgmnt
Offers & Specials
Search OrderMgmnt
CommissionRules Svc
CD DVDBooks Mem
CRM System
Media Delivery Billing
Media Repo
AP
CRMFacade
Batch Feeds
TR2*
PSR1PSR1: 3 sec search
Typical SOA Test Map -- Dramatically Simplified
Content
Presentation Layer
Content Mgmnt
Offers & Specials
Search OrderMgmnt
CommissionRules Svc
CD DVDBooks Mem
CRM System
Media Delivery Billing
Media Repo
AP
CRMFacade
Batch Feeds
FR1: Search is aggregate, CDs top results
FR1
Typical SOA Test Map -- Dramatically Simplified
Content
Presentation Layer
Content Mgmnt
Offers & Specials
Search OrderMgmnt
CommissionRules Svc
CD DVDBooks Mem
CRM System
Media Delivery Billing
Media Repo
AP
CRMFacade
Batch Feeds
TR1 & 2
TR2: 2 sec search
TR1: Create search on 4 repos
TR1
The Three Dimensions of Quality maturity
Complete >single component all technologies
Continuous >Phased tests constant testing
Collaborative >Dev/QA test silos everyone tests, every phase
LISA meets SOA Testing Challenges
• LISA is Complete - Test heterogeneous SOA• LISA can test every tier of an application, regardless of location• LISA can test every service technology with one tool, one test• LISA’s framework approach creates bridges to legacy services
that means ‘you can get there from here’
• LISA is Collaborative – Everyone owns quality• LISA shines at testing all components before UI even exists, as
well as integrating with the team’s ALM lifecycle and process• Unit, regression, system, load, and monitoring in one tool• Leverages diverse skill sets in Dev, QA/QE, and BA roles
• LISA is Continuous – Enables Agile SOA• Tests constantly evolving, interdependent systems during and
after deployment• Traps for the “what-if” unintended system-wide consequences of
making changes or corrections to components.
LISA WS-Testing $5,000 Off for next 50k users
• LISA WS-Testing Edition is THE tool for Web Services testing
• All our web services goodies are in there:• WSDL-based, point and click• WS-I Interop testing• Functional testing and service simulation• UI and Command Line execution
Thank you for your interest!
? ? ?
• iTKO Web Site Resources• URL: http://www.itko.com or http://www.ws-testing.com• White papers• Support FAQs, Tech Notes, and Forums (once registered)• Download WS-Testing or start a larger evaluation
• More from me• See above• Blog: http://itko.blogspot.com• Email: Start with [email protected]; I’m [email protected]
Top Related