When Testers Feel Left Out in the Cold
-
Upload
techwellpresentations -
Category
Technology
-
view
42 -
download
2
description
Transcript of When Testers Feel Left Out in the Cold
nt Session
Presented by:
Hans Buwalda
Brought to you by:
340 Corporate Way, Suite Orange Park, FL 32073 888‐2
W1 Concurre4/9/2014 10:30 AM
“When Testers Feel Left Out in the Cold”
LogiGear
300,68‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com
Hans Buwalda LogiGear
Hans Buwalda has been working with information technology since his high school years. In his thirty year career, Hans has gained experience as a developer, manager, and principal consultant for companies and organizations worldwide. He was a pioneer of the keyword approach to testing and automation, now widely used throughout the industry. His approaches to testing, like Action Based Testing and Soap Opera Testing, have helped a variety of customers achieve scalable and maintainable solutions for large and complex testing challenges. Hans is a frequent speaker at STAR conferences and is lead author of Integrated Test Design and Automation: Using the Testframe Method.
2/5/2014
1
STARCANADA 2014Toronto, Canada
"Test Management" Track Session W1
Wednesday, April 9, 10.30 – 11.30
© 2014 LogiGear Corporation. All Rights Reserved
Hans BuwaldaLogiGear
hans @ logigear.com
Who is your speaker
Software testing company, around since 1994Testing and test automation services:
www.logigear.comwww.testarchitect.com
LogiGear Corporation
Testing and test automation services:− consultancy, training− test development and automation services− "test integrated" development services− focus on big and complex testing projects
Products:− TestArchitect™, TestArchitect for Visual Studio™− integrating test development with test management and automation− based on modularized keyword-driven testing
© 2014 LogiGear Corporation. All Rights Reserved
Dutch guy, in California since 2001Background in math, computer science, managementSince 1994 focusing on automated testing− keywords, agile testing, big testing
Hans Buwalda hans @ logigear.comwww.happytester.com
2/5/2014
2
Scope of this talk
Introduction to the management aspects specific for a test projectNot a project management co rseNot a project management courseA personal view, based on experiences with my own methodCollection of ideas and experiences, not necessarily a complete pictureInput and discussions welcome (also talk with each other!)
© 2014 LogiGear Corporation. All Rights Reserved
Input and discussions welcome (also talk with each other!)The ideas are coming from mixed European, American and Asian experiences, apply with care
The Challenges for a Test Process
testing should be fun
testing sho ld be effecti etesting should be effective
testing should be efficient
testing should be under control
© 2014 LogiGear Corporation. All Rights Reserved
2/5/2014
3
Keywords, Action Words
number name quantitynew product P-9009 Sledge Hammer 5
number quantityadd quantity P-9009 20
number name quantitynew product P-9009 Sledge Hammer 5
number quantityadd quantity P-9009 20
actions, each with a keyword and arguments read from top
to bottom
fragment from a test with actions
Common in automation tools nowadays (but with different styles)
Identify tasks for both test development and automation
q yadd quantity P-9009 3add quantity P-9009 6
number quantitycheck quantity P-9009 34
q yadd quantity P-9009 3add quantity P-9009 6
number quantitycheck quantity P-9009 34
"34" is the expected value here
to bottom
© 2014 LogiGear Corporation. All Rights Reserved
y p
The test developer creates tests using actions
Each action consists of a keyword and arguments
The automation task focuses on automating the actions
Each action is automated only once
Test Module Plan
Test Module 1 Test Module 2 Test Module N
Overview
Tests
Objectives
Tests Tests
Actions
. . .Objectives Objectives
© 2014 LogiGear Corporation. All Rights Reserved
AUTOMATION
user passwordlog in jdoe StarEast
first last brand modelrent car John Renter Ford Escaperent car John Renter Chevrolet Volt
last totalcheck bill Renter 140.42
interaction test business testwindow control value
enter log in user name jdoeenter log in password StarEast
window control property expectedcheck prop log in ok button enabled true
car guy
2/5/2014
4
Discussing costs and benefits
© 2014 LogiGear Corporation. All Rights Reserved
Examples of costs and benefits
Involvement of users and business specialists
Preventation of incidentsShorter time to market
Costs? Benefits?
Need for investmentsBuild up of a test organisationLicense costsTraining, coachingMaintenance of (much) testware...
Increased quality and certaintyLesser dependence on users and expertsElimination of dull workImproved documentation...
© 2014 LogiGear Corporation. All Rights Reserved
Business Benefits• Time-to-Market• Quality-to-Market• Control
2/5/2014
5
What can make testing miserable . . .
Discussions about costs and benefits
Resistance
Commitment
Politics
Dependence
Unrealistic expectations
© 2014 LogiGear Corporation. All Rights Reserved
Unrealistic expectations
Difficulty
Motivation
Practical issues and problems
Resistance is Normal
“Let's reconsider” “I don't understand this”
WHEN THEY SAY : THEY COULD THINK :
Let s reconsiderNow is not the timeAll the time new objections“Fine, but are we ready for this?”Saying nothingSaying yes, acting noThe method is good but in
I don t understand this“I didn't expect all this” “This is going to cost me my job”I'm the star here, I don't need the competition“We can't achieve this”We will become too
© 2014 LogiGear Corporation. All Rights Reserved
The method is good, but in this specific case ….
We will become too dependent on those guysNow they will find out how bad we are testing
2/5/2014
6
GETTING COMMITMENT...
© 2014 LogiGear Corporation. All Rights Reserved
Management Commitment
Offer solutions not additional problems
Testing is often not popular . . .Nobody wants an extra task (extra problem)
Bug are bad
Testing is good
Offer solutions, not additional problemsTell managers that a good tested system creates a positive image (not only negative reasons for testing)Present/show what you're doing (glass box)Try to get clarity about policies and directions for testingKeep in mind: managers want things to be under control
© 2014 LogiGear Corporation. All Rights Reserved
Keep in mind: managers want things to be under control − give clear and timely information about (1) progress and (2) results
Use outsiders and/or books/articles to make your caseTry to find some bugs . . .
2/5/2014
7
Commitment, specifics
“No time, no money, ...”− back to the problem− you should not become the problem owner!you should not become the problem owner!
“It is so expensive/it is so difficult”− testing is expensive and difficult− test automation is difficult
“The others should do the testing”figure this out
© 2014 LogiGear Corporation. All Rights Reserved
− figure this out− you can't deal with this yourself (strategic context mismatch)− "doing things right" versus "doing the right things"
General vagueness− hidden problems and conflicts
Politics . . .
Who is to blame when things go wrongSystems are sometimes released while not finishedTesters tend to get the bullet
Notify the responsible managers early about what is going to happenMake clear written down procedures, especially for the test execution phaseNot every bug is equally important (pick your battles)
© 2014 LogiGear Corporation. All Rights Reserved
Make the test process transparentMaintain an atmosphere of cooperation and communication with allAsk for help, sound the alarm, − don’t underestimate the “politics of failure”, it can hurt you
2/5/2014
8
The Life Cycle of an IT Project
An (Old) Joke, Still Valid
EnthusiasmDisappointment! ! ! PANIC ! ! !Looking for guilty ones ("blame-storming")
© 2014 LogiGear Corporation. All Rights Reserved
Punishing the innocentRewarding those who had nothing to do with it
The 3 questions for a bug
Ask these question, in the given order
1. Is it a problem?
2. What is the root cause?
© 2014 LogiGear Corporation. All Rights Reserved
3. Why did we not catch it?
2/5/2014
9
Dependencies . . .
Testing and automation is dependent on many factors:• Working systems• Test environments (like virtual machines)
S ifi ti i t d i k l d
Make clear arrangements with everybody about everything, as early as possible− written down
• Specifications, requirements, domain knowledge• People• ...
© 2014 LogiGear Corporation. All Rights Reserved
Keep in touch with the rest of the projectMake the high level test products as early as possible
Dependencies, specifics
“The system under test isn't available”− The automation will be the first in trouble− Discuss this (early)Discuss this (early)− Allocate resources when really needed
“The system under test doesn't work”− The automation will be the first in trouble− Create a “health check” (smoke test to see if all
functionalities that are relevant to the automation work)
“Th i i ”
© 2014 LogiGear Corporation. All Rights Reserved
“There is no test environment”− Start planning this as early as you can− Make clear how important this is− “No test environment, no test”
2/5/2014
10
Expectations Around AutomationThe impression can arise that automated testing (in particular with keywords) is just a push of a button . . .
even if nobody said so!
Message: testing is difficult, automated testing is even more difficult
Make clear what is happening
Communicate manage expectations
. . . even if nobody said so!
© 2014 LogiGear Corporation. All Rights Reserved
Communicate, manage expectations
Don't put too low figures in a test plan− let somebody else lower them (in writing)
Ask managers and other people involved what they expect
Difficult to Keep People Motivated
Watch the signals
Motivation of team members can erode when time progressesHappens to both test developers and automation engineers
Watch the signals
Make sure that the right people are assigned− Automation engineers should have a programming background− Testers should have a testing and/or business background
Avoid “monks work” (Dutch expression for tedious repetitive work)
Differentiate the work
Avoid isolation of the test group
© 2014 LogiGear Corporation. All Rights Reserved
Be prepared to switch roles
Consider agile approaches
Be ready for this problem in advance
Create operational and professional communication structures− for example special interest groups to regularly discuss professional topics
2/5/2014
11
Complexity: The Work is Difficult
Making tests is supposed to be "difficult":• Finding bugs of others• Making concrete examplesAutomating them is difficult as well
Don't be over-confident, use coaching from experienced peers or outsidersConsider reference visits to other projects or sitesTalk about the work, even create special interest groupsKeep in touch with others doing the same work
Automating them is difficult as well
© 2014 LogiGear Corporation. All Rights Reserved
Keep in touch with others doing the same workOrganize interactions (meetings!), for example with:− users− developers− auditors
Delegate tasks (better lazy then crazy...)
Practical Issues and ProblemsDoes the test tool work here ? ?Do we have authorizations on the test environment ? ?Where do we keep our test products ? ? Which tests do we have ? ?
Be prepared for this, automated testing is a “cloud of details”Put suitable team members in leading roles, and delegate responsibilities to themF l t t j t t t t i f ll ti j b
Questions, chores, details, . . .
© 2014 LogiGear Corporation. All Rights Reserved
For larger test projects test management is a full time job, make sure you get this timeTry to find easier ways, organize things better, or automate more− “If you have a difficult task, ask a lazy man. He will find an easier way”
2/5/2014
12
Some final remarks
Changing a process can mean:− changing of work processes− a learning curvea learning curve− risks and benefits
It is not the same as: buy a tool and all will be well
Changing any process is more than anything an
© 2014 LogiGear Corporation. All Rights Reserved
organisational change and should be managed as such
Thank you for listening
…and...
keep cool, even in the cold
© 2014 LogiGear Corporation. All Rights Reserved