KW3 Keynote

5/1/2013 4:30:00 PM

Lightning Strikes the Keynotes

Presented by:

Lee Copeland

Software Quality Engineering

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Lee Copeland

With more than thirty years of experience as an information systems professional at commercial and nonprofit organizations, Lee Copeland has held technical and managerial positions in applications development, software testing, and software process improvement. Lee has developed and taught numerous training courses on software development and testing issues and is a well-known speaker with Software Quality Engineering. Lee presents at software conferences in the United States and abroad. He is the author of the popular reference book, A Practitioner’s Guide to Software Test Design.

5/13/2013

1

Lightning Strikes

The Keynotes

Moderated by

Lee Copelandlee@sqe.com

Featuring

Michael BoltonJon Bach

Jennifer Bonine

Griffin Jones

John Fodeh

Dawn Haynes

Hans Buwalda Bob Galen

Geoff Horne

James Bach

5/13/2013

2

James Bach

5/13/2013

3

Jon Bach

5/13/2013

4

Michael Bolton

Attention,

Deficit,

Disorder.

Michael Bolton

http://www.developsense.com

5/13/2013

5

aircanada.com

aircanada.com

5/13/2013

6

Error MessageVia Rail, between Montreal and Toronto, 2007

But I can’t contact my… oh, never mind.

Error MessageVia Rail, between Montreal and Toronto, 2007

5/13/2013

7

Hyatt Regency, 2008

If you can’t do math, it’s a nickel extra.

Hyatt Regency, 2008

5/13/2013

8

Why you shouldn’t let an unsupervised

algorithm choose your sponsored links (1).

Vimeo’s Web PageSpring 2010

Why you shouldn’t let an unsupervised

algorithm choose your sponsored links (2).

Vimeo’s Web PageSpring 2010

5/13/2013

9

Why you shouldn’t let an unsupervised

algorithm choose your sponsored links (3).

Vimeo’s Web PageSpring 2010

Google Chrome

5/13/2013

10

Google Calendar Update

OK, fine.

Don’t Know Who This Was

5/13/2013

11

Adobe Acrobat

Adobe Acrobat

5/13/2013

12

Microsoft Outlook

Intuit Quicken

5/13/2013

13

Intuit Quicken

Intuit Quicken

5/13/2013

14

Here’s another err.

Intuit Quicken

ibm.com

5/13/2013

15

Windows

I think I’ll shoot my own trouble for now.

Windows

5/13/2013

16

Windows

I don’t know!

Windows

5/13/2013

17

iTunes

Testing is more than checking.

5/13/2013

18

Welcome to the new world.

“The World of Heineken”Heineken Experience, Amsterdam, 2006

Jennifer Bonine

5/13/2013

19

Hans Buwalda

5/13/2013

20

Misconceptions of

Test Automation(and keyword testing)

Hans Buwalda

Comments:

" automation is easy, no need to think about it much "

� I'm still waiting to see my first "easy" automation project

� Development is hard, testing is harder, automated testing is the

hardest

� If you can't do automation well, be ready to lose time and money

� If you can do automated testing well... you're in an enviable position

5/13/2013

21

Comments:

" test automation means automating manual tests "

� A car is not the same as a carriage with an engine

� Good automated testing is not the same as good automation of good

manual testing

� Automating manual test designs tends to be cumbersome, uninspiring,

maintenance sensitive, and hard to scale

� How you organize and design your tests is the main driver for automation

success

+ =

Comments:

" keywords is a method "

� Keywords are not much more than a format to write tests in, in itself

not much different from (good) coding of test cases

� Some of the worst tests I have seen were keyword tests

� I do believe however that keywords are just about the only way to go

for big and complex projects (in addition to exploratory testing). They

just need a method

� In my approach "test modules" play a central role for organizing test, to

achieve effective test development and successful automation

5/13/2013

22

Test Module Plan

Tests

Objectives

Test Module 1

Tests

Test Module 2

Tests

Test Module N

Actions

. . .

AUTOMATION

Objectives Objectives

user password

log in jdoe StarEast

first last brand model

rent car John Renter Ford Escape

rent car John Renter Chevrolet Volt

last total

check bill Renter 140.42

Example of a method with keywords:

"Action Based Testing"

interaction test business test

window control value

enter log in user name jdoe

enter log in password StarEast

window control property expected

check prop log in ok button enabled true

Comments:

" to do automated testing you need to be a good programmer "

� The focus should be on testing and test design, not on programming tests

� A good tester is not automatically a good programmer, or vice versa� I don't believe we should replace all testers with programmers "in test"

� A good programmer can contribute greatly to an airplane control system,

but is not automatically a good airline pilot

� Even automation itself is a profession, quite different from regular

programming

5/13/2013

23

Comments:

" if there are automation problems, tests should be debugged "

� "Thou should never debug tests"

� If observed results are not the expected results, it is not an

automation problem, either the tester or the developer were off

� If the application under tests UI isn't accepting your input, run lower

level tests first

� If actions aren't working, test them and make them better, before

running your tests with them

user passwordlog in jdoe StarEast

first last brand

model

rent car John Renter FordEscape

rent car John Renter Chevrolet Volt last

totalcheck bill Renter 140.42

Comments:

" you need an ROI analysis to determine which tests to automate "

� One of the most commonly found statements on test automation

� I consider In my view, in a good automated testing effort (my definition

of 'good'), automation is a secondary practical matter� may have to address technology issues to interface with a system under test

� good automation is cheap and re-usable, and has a high payoff in time and money

� I would rather see an ROI on the tests than on their automation

$$

$$$

$$

$

$$$

$$$

$$

$

$

5/13/2013

24

Comments:

" automated tests are dumb "

� They often are very mechanical, and boring, in particular when 1 on 1

based on requirements or specifications, but they don't have to be

� Distinguish between an analytical activity ("what to test") and creative

activity ("how to test"), and be mean...

� It is the responsibility of the testers to ensure tests are not dumb� automation is not an excuse

� Lame tests are not likely to find interesting bugs

" homework "

Are these misconceptions ???

� "test automation is the same as programming"

� "the most important activity in an automation effort is selecting a tool"

� "automation is most suitable for regression testing"

� "test automation is a technical challenge"

� "if you use keywords your test automation will be successful"

� "to have more automation, you just need more people"

5/13/2013

25

Bob Galen

5/13/2013

26

John Fodeh

Ways to Increase

Our Value as Testers

John Fodeh

5/13/2013

27

Value Proposition

Testing

QualityCost Productivity

5/13/2013

28

7 Quality Enhancing Features

Domain insight

Modeling

Programming fundamentals

Testing techniques

Automation

Lateral & system thinking

Leadership skills

5/13/2013

29

5/13/2013

30

Dawn Haynes

5/13/2013

31

DDDDDDDD

AAAAAAAA

RRRRRRRR

TTTTTTTT

SSSSSSSS

DDDDDDDDDataData

AreasAreasAAAAAAAA

RulesRules

RRRRRRRR

TrailsTrails

TTTTTTTT

StatesStates

SSSSSSSS

DDDDDDDDataataataataataataataata

AAAAAAAAreasreasreasreasreasreasreasreas

RRRRRRRRulesulesulesulesulesulesulesules

TTTTTTTTrailsrailsrailsrailsrailsrailsrailsrails

SSSSSSSStatestatestatestatestatestatestatestates

Follow the Follow the Follow the Follow the popcorn trail popcorn trail popcorn trail popcorn trail of data …of data …of data …of data …

SSSSame data, ame data, ame data, ame data, different areasdifferent areasdifferent areasdifferent areas

Check rules Check rules Check rules Check rules in various in various in various in various statesstatesstatesstates

Evaluate Evaluate Evaluate Evaluate interacting interacting interacting interacting state modelsstate modelsstate modelsstate models

5/13/2013

32

Geoff Horne

5/13/2013

33

Griffin Jones

5/13/2013

34

WRESTWorkshop on Regulated Software Testing

Software subject to review by an internal or external regulatory body

Purpose and Format

• Share and generate ideas and techniques

• Provide a forum for people interested in the topic

• Participation is free and open to all

• LAWST-style rules of engagement

5/13/2013

35

Workshop Structure

• Facilitated

• Series of experiential presentations and group discussions

• Atmosphere is collaborative, supportive and constructive

• Focus on the practical and useful

More Information

• Next WREST: Friday, May 3rd

2013Hosted by SQE here at STAREAST

• Contact:Karen N. Johnson, John McConda, Griffin Jones

• Website: wrestworkshop.com

5/13/2013

36

Our Thanks To

Michael BoltonJon Bach

Jennifer Bonine

Griffin Jones

John Fodeh

Dawn Haynes

Hans Buwalda Bob Galen

Geoff Horne

James Bach