9/05/14

1

1

Disruptive Forces In Testing

Dr Kelvin Ross Australian Test Managers Forum, Sydney 8 May 2014

2

Disruptive Technology

9/05/14

2

3

Disruptive Technology

4

Disruptive Technology

9/05/14

3

5

Disruptive Technology

Biggest Disruptors The Last 5 Years? Agile

Offshoring Virtualisation

9/05/14

4

7

Disruptive Forces

Customer Testing

Dev Ops

“You can't just ask customers what they want and then try to give that to them. By the time you get it built, they'll want something new.”

Steve Jobs

9/05/14

5

9

FAIL FAST

10

Customer Testing

9/05/14

6

Secured $80m in VC funding

Applause!Testing In The Wild

9/05/14

7

Verification “Are we building the product right?”

Validation “Are we building the right product?”

Barry Boehm, 1979

Barry Boehm

Scientific Method for Customer Experiments

to Guide Software Development

9/05/14

8

15

Customer Testing

A / B Tests

Source: Advanced Topics in Continuous Deployment Mike Brittain,Engineering Director, Etsy @mikebrittain mikebrittain.com/talks E.g.“Google Analytics Content Experiments”

9/05/14

9

Source: Principles and Practices in Continuous Deployment Mike Brittain,Engineering Director, Etsy @mikebrittain mikebrittain.com/talks

Feature Toggles

9/05/14

10

Source: Advanced Topics in Continuous Deployment Mike Brittain,Engineering Director, Etsy @mikebrittain mikebrittain.com/talks

Feature Toggles

DEV OPS

9/05/14

11

21

No Silos

22

CI CD CD

Continuous Integration

Continuous Deployment

Continuous Delivery

9/05/14

12

Source: Principles and Practices in Continuous Deployment Mike Brittain,Engineering Director, Etsy @mikebrittain mikebrittain.com/talks

Rapid Releases

9/05/14

13

CD Platform

Source Control

Build System

Pipeline Management

& Visualization

Artifact/Package

Repository

Automated Configuration Management

(IaC)

Test infrastructure

Automated Deployment

System

Source: Continuous Delivery at Scale John Esser, Director Engineering Productivity, Ancestry.com ALM Forum Seattle, 2014

Git Flow

9/05/14

14

Source: Principles and Practices in Continuous Deployment Mike Brittain,Engineering Director, Etsy @mikebrittain mikebrittain.com/talks

Release Pipeline

Source: Steve Neely, sneely@rallydev.com @stenvoon http://bit.ly/cdpaper ALM Forum Seattle, 2014

Release Pipeline

9/05/14

15

Source: Advanced Topics in Continuous Deployment Mike Brittain,Engineering Director, Etsy @mikebrittain mikebrittain.com/talks

Release Pipeline

Source: Principles and Practices in Continuous Deployment Mike Brittain,Engineering Director, Etsy @mikebrittain mikebrittain.com/talks

Release Pipeline

9/05/14

16

9/05/14

17

33

In 5 Years…

“Be a yardstick of quality. Some people aren't used to an environment where excellence is expected.”

Steve Jobs

9/05/14

18

35

Q&A