Pursuing quality?

(you won’t get there by testing)

James Whittaker Engineering Director, Google

+docjamesw (Google+) @docjamesw (Twitter)

2011 j f m a m j j a s o n d

2011 j f m a m j j a s o n d

2011 j f m a m j j a s o n d

2011 j f m a m j j a s o n d

2011 j f m a m j j a s o n d

2011 j f m a m j j a s o n d

2011 j f m a m j j a s o n d

2011 j f m a m j j a s o n d

2011 j f m a m j j a s o n d

2011 j f m a m j j a s o n d

“Want to make 30% less money? Put the word „test‟ in your title.”

-- Alberto Savoia aka Testivus

(GTAC 2011 Keynote Test is Dead)

2011 j f m a m j j a s o n d

Software is getting better!

Vs.

Why software is better ...

Immediate cost of low quality

Competition

Why software is better ...

Post-ship bug fixing

Auto update

Why software is better ...

Self repairing software

Crash recovery

Why software is better ...

Goodbye server config

Cloud deployment

Why software is better ...

Reduction of dependencies

Standards

Why software is better ...

Elegant programming languages

Languages

Why software is better ...

Eradication of certain bug species

Extinction

Why software is better ...

Better code management

Initial code quality

Why software is better ...

Continuous build/integration/release/test

No More Big Bang

Why software is better ...

Convergence of the user and test community

Crowd Source

Where is testing in this mix?

“Your quality is not good”

So what‟s the problem here?

We have an overinflated opinion of ourselves

When do you appreciate car mechanics?

When they find a problem? … Or when they fix it?

So what‟s the problem here?

We identify with our role instead of our product

This is anti-team, anti-company

So what‟s the problem here?

We think testing provides something of value

The value of testing is in the activity,

not the artifacts

So what‟s the problem here?

We‟ve made developers lazy

I can mow my own lawn, but why would I?

Test is development’s crutch

So what‟s the solution here?

“If you are not part of the solution …

…there is good money to be made in being part of the problem”

If you can‟t beat „em, join „em!

If you can‟t beat „em, join „em

What is the value of a single, great tester?

One user? Ten? A hundred? Ten thousand? More?

We are paying testers to act like users? Really? Users don‟t have to act and they have all the devices we‟re interested in Stop testing, start enabling those who do

If you can‟t beat „em, join „em

Who says developers can‟t test?

They can‟t? … Or they don‟t have to? They can‟t? … Or its too hard to be worth the effort?

Stop testing, start enabling those who should

2011 j f m a m j j a s o n d

2011 j f m a m j j a s o n d

2011 j f m a m j j a s o n d

Shipped new APIs every few weeks since mid August

• More than half of the developers have „test‟ in their title

• Zero dedicated testers, everyone is a dev

• One developer owns the testing feature

• Dogfood and trusted tester programs aggressively managed

• No production defect has lived more than a few hours

2011 j f m a m j j a s o n d

2011 j f m a m j j a s o n d

What does this do to testing?

• Quality is now “just another feature” of the product

• Testing is focused on enabling, tracking … not doing

• The “crowd” takes center stage

• Specialization is a career opportunity

developer

tester

user

developer

tester

user

• treat testing as a feature

• gets managed in dev workflow

• product is the focus, not the role

developer

tester

user

• treat testing as a feature

• gets managed in dev workflow

• product is the focus, not the role

• it doesn't matter who does the testing, only that it gets done

• establish test goals, measure progress toward these goals

• specialized testing is focus

James Whittaker Engineering Director

Google

If following my work appeals to you: +docjamesw (Google+) @docjamesw (Twitter)

googledevspot.blogspot.com

googletesting.blogspot.com