a case study - Capgemini Sogeti Danmark A/S · -a case study And then a reflective discussion . ......
Transcript of a case study - Capgemini Sogeti Danmark A/S · -a case study And then a reflective discussion . ......
An agile journey with Nordea - a case study
And then a reflective discussion
My Background ”Why I'm here”
Dan Borring Larsen
Sr. Project Manager, Consultant & Agilist
Working IT industry since 1995 – roles from System
Specialist to Test Manager and final Project Manager
Specializing in helping companies in Agile/QA best
practices (deliver value early). Experience with both
frontend and backend software development on
various platforms. Using the whole-team approach
onshore/offshore. Domain knowledge on investment
banking and business administration. Current assigned as an IT Project
Manager at Nordea in the SWO
department (Christiansbro).
• 04/12/2013
3 •
http://www.linkedin.com/in/borring
THE MISSION ”what nordea wanted”
• 04/12/2013
5 •
High Quality
High Test awareness
High Process awareness
Fast Time-to-Market
Monthly deliveries
Resource Scalability
Offshore development
Case facts
• 04/12/2013
6 •
The Product (classified) An online solution for Nordea’s customers,
and is launched this year
Project size (IT) 7+ local resources
16 offshore resources
Technology Responsive web design
HTML5, Bootstrap, Less, Apache Wicket, Java
Test Tools Cucumber, Selenium, HP Quality Center, Adobe Edge Inspect,
CA-LISA , Jenkins, Sonar Qube
DAY ONE ”does this boat cross an ocean”
The boat
• 04/12/2013
8 •
Master Test Plan
Master Test Plan
Requirements
Design
Code
Test
Trying to test everything manually
..and way too late!
THE VISION ”rethink”
The Test Vision based on an Agile Approach
• 04/12/2013
10 •
Chapter 16 Quality
Integrate Testing into the Process
Mike Cohn
Test Vision
Automated Unit tests
Automated
Integration tests
Automated
Functional test
UAT
• 04/12/2013
11 •
Test Automation Tailoring the Test vision with the Master test plan
Manual
UAT
Automated Unit Tests
Automated
Integration Tests
Automated
Functional Test
Requirements
Business Sign-off
Test
Resu
lts
Test automation
Manual Test
- Look & feel
(Every 30 minute)
Exploratory
The Test Vision
• 04/12/2013
12 •
The Test Vision Test Vision
Test Objectives • Test more in lesser time
• Test early
• Deliver fast
• Predictable value
• High customer satisfaction
• Fun at work
Test Strategy • Evolution
The Test Vision Master Test Plan
Working quality process over comprehensive documentation
THE CREW ”bring in the right skills”
Test Team
• 04/12/2013
14 •
New competencies
• Test execution by Scripts instead of Keyboards
• From Functional Test to digging deeper (yes unit test)
Test Scripting &
Integration Test
(4 resources)
Business UAT
(2 resources)
Manual Functional Test
(2 resources)
OJT
DIVING ”going deeper”
What is a Feature?
• 04/12/2013
16 •
Behaviours written in the form of Scenarios
Written in simple English and written by non technical people
Feature: Signing up for TextExpo [title]
As a tester [role]
I want to attend TestExpo [feature]
So that I get inspired for my work [benefit]
Scenario: Signing up successfully
Given I go to www.sogeti.dk/Events/TestExpo [context]
When I click the button labelled 'Tilmeld dig her’ [event]
Given there are <free seats> seats [another context]
Then I should see a form with fields: Navn | E-mail | … [outcome]
And I fill in Navn | E-mail | … [another context]
And I click the button 'Tilmelding’ [another event]
I should see 'Du er nu tilmeldt' message. [another outcome]
The order of Given, When, Then, And can be interchanged or used multiple times or sparingly used.
A template for Cucumber, tool used for creating automated acceptance tests
NAVIGATION ”humans get quickly disoriented”
Test Flow
• 04/12/2013
18 •
Requirement walk-
trough by business
Documented
business
requirements
Test Team
Write Feature files
Business
Sign-off
on feature files
Development Team
Sprint Review
Meeting
Test Team
Test feature
files manually
Test Team
Test automation
Candidates that can’t
be automated
Test execution
by CD process
Test execution
before release
CODE Working Software
How does it fit with Sprints
2/4/2014
19
SPRINT 2
SPRINT 2
SPRINT 3
Test Team
Development Team
SPRINT 1
TEST
SPRINT1 CODE
TEST TEST
SPRINT 3
FIX CODE CODE
REQ
B
REQ
A
REQ
C
End-to-End feature flow
• 04/12/2013
20 •
Ready for
Development
Start IT
Development
Business
Demo
Develop
features
Business
UAT
New Version
Required
Assign
Assign
Test Report
- Passed
- Failed
Business IT
Sign-off
Product
Business
Priority
Project
Backlog
Backlog
Feature
Refine
Requirement
Feature
Complete
DoD
Onshore
Sprint
Backlog
Offshore
Sprint
Backlog
Develop
features
Test
Implementation
& Execution
DoD
DoD = Unit tested, Reviewed & features deployed in test environments
Sprint
Demo Dispatch
Sprint
Demo
Write Test
Cases
(feature files)
CRUISE CONTROL ”the quality engine”
Extend Automation with Continuous integration (The Hook)
- End-to-end quality
• 04/12/2013
22 •
Chapter 16 Quality
Integrate Testing into the Process
1. Automate Test (Test vision)
2. Build a pipeline
3. Extend the pipeline
David Farley
How long time does it take to deploy one line of code to production?
Continuous integration
• 04/12/2013
23 •
• Technical Debt
• Unit Tests Coverage
• Unit test success
• Code Compliance
• Compare with the past (trends)
SonarQube™ an open platform to track code quality
Automated Unit tests
Automated
Integration Tests
Continuous integration
• 04/12/2013
24 •
• Functional Test - behavior driven
• It’s based on business requirements and
end-user behaviour
• Features it’s written in plain text (DSL)
• The test is a cycle of interactions with
well-defined outputs, resulting in the
delivery of working tested software
Automated
Functional test
Visualize the process
• 04/12/2013
25 •
RETROSPECT ”so far, so good”
• 04/12/2013
27 •
KPI
Reduced Wait Times
Greater Availability of Services
Decreased Software Cycles
Increased Test Coverage
Shorter Test Cycle Times
Improved Time-to-Market
Fewer Production Defects
• Testers are not waiting for the
builds anymore – single click
deployment
• Regression test is always
running
• Requirements are analyzed up
front by testers due to feature file
creation (increase awareness)
• Developers help testers and
testers help developers
• Testers become more technical
aware
• Testers help deliver software
successfully
FORECAST ”the storm is coming”
Pipeline spinning out of control
• As more test cases and systems are added in the pipeline the complexity will increase!
• Test case execution is often unsuccessful due to failures outside of under-test application (external dependencies)
• Rapidly increasing number of test cases resulting in increased levels of failure
• 04/12/2013
29 •
Number of automated tests
Number of integration points
Automated testing success rate
Environmental complexity
Time
Volume
“I can’t do anything until I have everything… and I never have everything!”
Continuous integration: What are the constraints?
• 04/12/2013
30 •
INCOMPLETE DEVELOPMENT
SYSTEM UNAVAILABLE
NO ACCESS TO 3RD PARTY
INVALID OR MISSING DATA
LIMITED ACCESS TO OTHER SYSTEMS
BURNED DATA
Same problems as every other project I guess!
VISION v2 ”reduce risk of damage”
Objectives for Vision v2
32
• Build an ‘isolated’ test environment using Service Virtualization to provide virtualized dependencies
• Key tasks – Integration into continuous integration
• Record virtual services from the End-to-End (E2E) test and store images in version control
• Extract and deploy CA LISA virtual services from version control.
• Build an isolated test environment configured to use LISA virtual services and execute test cases
Isolated Test environment
The new pipeline with Service Virtualization support
• 04/12/2013
33 •
Code
guidelines
Give full control to “legacy”
and test data
Application
Test
E2E environment used to
capture service images
CA LISA Recorder
CA LISA VSE
VS VS VS
E2E Test
Functional Application
Test
Legacy
systems
Integration
test Code Quality
Compile
Unit test
SVN
Additional step executed
against virtual services
THE END ”of this presentation”
DISCUSSION How many of you work with:
• Agile methods?
• Testing on all levels?
• Test automation?
• Continuous integration?
• Service virtualization?
Does it work for you?
Can we do better than this?
THE END ”every end is is new beginning”