How to manoeuvre as test/QA responsible in agile teams to get the "right" product quality

AGILE DEVELOPMENT IN SAXO BANKHOW TO MANEUVER AS TESTER TO GET THE RIGHT QUALITY

BY KNUD HANGAARD

FACTS ABOUT ME

AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY

2

Master of Science, Electronics and Computer Engineering

Certifications:ISTQB, PRINCE2, ACI Dealing/Trading

My working sectors:• HW/SW production industry• Banking industry

My current role:Front Office test lead:• Release test responsible• Test environments• Development and test

processes

Contact:[email protected]: www.linkedin.com/in/khangaardSKYPE: khangaard

mailto:[email protected]

PRESENTATION FOCUS AREAS


3

Introduction to Saxo bank and the software

Agility challenges and finding the “Right” quality

What we have done to become more agile: process, tools, people

What to bring home:

o Key focus areas in the test process for achieving right level of quality

o Key skills that has been necessary to manage testing in a hyper agile

organisation

INTRODUCTION TO SAXO BANK


4

Established 1992

Employees >1300 (700 in IT)Offices 25 countriesFX trades daily 170,000Daily av. turnover 20 billion USDCountries with retail clients 190Financial instruments > 25,000Received prices daily 5-6 billionDevelopment in Denmark and IndiaThe “Front Office” in Saxo Bank

headquarter.

HUGE SOFTWARE PLATFORM

o Software components: 350+

o 11 million lines of code


THE DEVELOPMENT CONDITIONS


6

o FOCUS ON LEADING ROLE, DELIVERY AND

COST: oBE PRODUCTIVE• Ongoing business

development, large project portfolio - demand for new functionality (DEVELOPING THE BANK)

oBE GOOD• Ongoing technical and

performance improvements – minimum number of bugs in Live. (RUNNING THE BANK)

oBE ECONOMICAL• Cost awareness: (off shore

development and test)

http://www.google.dk/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=Rd-1T7meMWB5bM&tbnid=QLfHFKRUjUyvuM:&ved=0CAUQjRw&url=http://louisdietvorst.wordpress.com/2011/12/02/&ei=ZzJgUun7EKWh4gSO3IGwAg&bvm=bv.54934254,d.bGE&psig=AFQjCNEB235Aegoq8XWKDZEZwo7l4vw4KA&ust=1382122203850967

SO WE WANT TO BE AGILE…


7

Pro

ject

s


8

Project N Platform Production

Defects

Incidents from Production

Project Test Environments

Release Test Environment

Simulation Production

Project testers

Release testersProject 1

Project 2

Quality gate

Quality gate

Scrum/Kanban

Merge to main

(Prince2)/Scrum/Kanban

SAXO BANK DEVELOPMENT MODEL

DEVELOPMENT TRENDS IN SAXO BANK, BECOMING MORE AGILE

Delivery cycle has been reduced over timeo Idea is less changes per cycleo No forcing of last minute changes (follow the process!)

Development has been divided into sectors, aligned with business areaso More focus on “the customer”


9

CHALLENGES FOR QUALITY ASSURANCE AND TEST

o Huge code baseo Huge amount of componentso Test basis (documentation) is limitedo Limited time for writing test caseso Limited time for testingo Limited time for writing test reports.o What would we really like to test - test coverage?


10

SO WHAT IS THE RIGHT QUALITY?

Right Quality:o Happy Business (more competitive products, earnings – less

problems in Live, losses)oHappy Customers (more products, opportunities – less bugs found)

Deliveries/changes to Business (x)

Live Incidents and roll backs (y)

High quality (less new changes, only few Live incidents)

“Right” quality (enough number of new changes, reasonable number of incidents in Live)

Low quality (many new changes, huge amount of Live incidents)

y=v*x where v: development velocity (throughput)


BECOMING MORE AGILE


12

KEY POINTS COVERED:

oThe Process

oThe Tools

oThe People

DEVELOPMENT AND TEST PROCESS – KEY POINTS


Time boxed development and time boxed testing (backlogs for each area)

Incremental approach:o release in small steps - backward compatible - possible to roll

back

Risk based testing:o Spend more time on high risk changes (both functional and

regression)

Flexible development and test process:o Quality gates/ verification points can be decided on low level

Cross functional teams:o Shared test responsibility between developers and testers

13

PROCESS

RELEASE TEST CYCLE (TIME BOXED)

o Monday: Baseline o Tuesday: Deploy DB changes and verifyo Wednesday: Deploy new services and verifyo 6 days for test and reportingo Next Thursday: Go/No meetingo Friday: Follow-up and planning next release

Monday

•Release Test

Tuesday Wednesday

•Release Test

•Reporting

Thursday

•Go/No Go meeting

•Release to Simulation

Friday

•Planning next release test

Saturday

•Release to Live

Sunday

Backup of Live

Monday

•Baseline of test environment

Tuesday

•DB release candidates

Wednesday•Apps og Services rel. candidates

Thursday

•Release Test

Friday

•Release Test

Saturday

•DB Releases to Live

Sunday

•Release Test

AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY PROCESS

RELEASE DOCUMENTATION BUILT-UP

A tree structure of work items:

o Release item: Parent for the full release

o Change items: One for each Component – release request for production

o Delivery items: One from each team or project.

o User Story items: The functional changes

o Bug items: The bug fixes.


15

PROCESS

Release, week xChange (Component 1)

Delivery (Project A)

User Story a

Bug Fix y

Change (Component 2)

User Story b

Delivery (Project A)

User Story c

Delivery (Project C)

…

STATE TRANSITIONS AND TEST FLOWS

Development User Story states (Kanban):• Not Done, Active, Resolved and Closed

Independent Test stages (Test Flows):• NA, Not Done, Done

Test flows at 3 stages: • development phase, • integration phase, • system integration phase (release test) Auto generated test report


16

PROCESS

THE TOOLS – ALL IN ONE (Team Foundation Server along with Microsoft Test Manager)

User Stories/Features

Tasks

Test Cases/Test

Scripts

CodeBugs

Test Execution

Plan

Test Results

Test Reports


17

TOOLS

THE TEST ENVIRONMENTS

o Test environments that simulates Live (Production) environment• Release and project test environments

o Baseline of test environments with Live data every two weeks

o Intensive scripting for adapting Live data to test environment (object)

o One-click deployment of new software


18

TOOLS

AGILITY - IMPACTS ON QUALITY ASSURANCE THINKING

o Adapt to the situation and plan very much based on RISK

o Find the “right” test depth (effort) – use your time optimal

o Share results early and adjust when you know more• early feedback both ways is important

o Find the right level of test documenting and reporting• so you get the message out but do not overdo


19

PEOPLE

A NEW TESTER ROLE

o Test certifications help as a basis - but is not sufficient in “the field”

o Be able to handle many different stakeholders (developers, project managers, delivery managers, business owners, business support)

o Be technical well founded on multiple areas (less time for study, understand fast, react fast)

o Be RISK focused (find the serious bugs in the beginning)o Be Proactive - ask questions!o Make plans but expect them to change and look for the new test

opportunitieso Spend more time on exploratory testing instead of just running

the same test caseso Do early and precise test reporting (only few lines to explain the

problem)


20

PEOPLE

EFFECTS OF THE APPROACH


21

EFFECTS

Business Effects:• More releases going to Live • Less high severity bugs found in Live

(roll back)

Process Effects:• Better transparency and traceability• Effective documenting and reporting• The flexible dev. model makes it easy to

adapt to new situations• Benefit from time difference: Build late

in Denmark - test early in India

Test Effects:• ”Exploratory testing” generates more

bugs than executing test cases prepared in advance

22


LESSONS LEARNED


23

Simple, clear and flexible processo Clear responsibilitieso Clear expectations

Teamworko Daily scrums with testers and

developerso Avoid being dependent on other

team’s priorities.

Effective and clear communicationo To avoid time wasted o To avoid bugs being overseen

Effective documentation

o Find the ”right” level of detailso Be focused on key points –

make it easy for the reader

Effective toolingo Easy planning, test execution

and reportingo Easy setup of test

environments


24

KEY FOCUS AREAS IN THE TEST PROCESS FOR ACHIEVING RIGHT LEVEL OF QUALITY

KEY SKILLS NECESSARY TO MANAGE TESTING IN A HYPER AGILE ORG.

o Great communication skills - be able to communicate with many different stakeholders

o Knowledgeableo Need to have both good business and

technical/development knowledgeo Need to know the ”Go to Guys” to get more knowledge

quickly o Great team player - share knowledge – get knowledge

o Open mindset - be ready to change plans quickly

o Tester’s mindset – explore, ask questions

o Responsible mindset – Be the one that follows-up


25

CULTURAL DIFFERENCES BETWEEN OFF-SHORE TEAMS

Different thinking: Communication path - hierarchical vs. lateralControl - centralized vs. decentralized

Planning - detailed vs. less detailed

Different communication form:Direct vs. indirect (verbally)Direct vs. indirect (in writing – sugar coating)language barriers

Different job goals:Grow hierarchical vs. grow expert wise


26

CHALLENGES FOUND AND HOW TO OVERCOME

o Culture differenceso Discuss the cultural differences for better understanding

and learningo Engage with team members (non-work related)o Travel often both ways

o Unclear goals/expectationso Ensure clear communication (combine verbal and writing)o Team SPOCs both places for ”translating” the messageso Be very concrete on goals and expectationso Regularly status meetings retrospectives and feedback

o Having the feeling of being far awayo Electronic boards for status sharing o Video conferencing a must


27

NEXT PLANS FOR SAXO BANK


28

o Continuous delivery

o Even more effective processes

o Even more automation

o Even more auto generated reports

CONCLUSION – IMPORTANT FOCUS AREAS IN AGILE TEAMS

Clear Communication

Effective Processes

Effective Tools

Team Player Mindset


Risk Mindset

THANK YOU FOR LISTENING!

30

Feel free to contact me:LINKEDIN: WWW.LINKEDIN.COM/IN/KHANGAARD


How to manoeuvre as test/QA responsible in agile teams to get the "right" product quality

Technology

Transcript of How to manoeuvre as test/QA responsible in agile teams to get the "right" product quality