@wakaleo

The evolution of automated testing practicesTake me on a journey

@wakaleo

@wakaleo

John Ferguson

Smart

Introductions

“I help teams of smart people learn to work together more efficiently,

to deliver better software faster”

@wakaleo

Test AutomationDo it well, or don’t do it at all

@wakaleo

How much are your tests worth?

Weekly cost vs

weekly savings$0

$20,000

$40,000

$60,000

$80,000

$100,000

$120,000

$140,000

$160,000

$180,000

1/09/2015 1/10/2015 1/11/2015 1/12/2015

Test0suite0cost0vs0savings0(per0week)

Cost0saved0par0week Test0suite0cost0per0week

@wakaleo

How much are your tests worth?

Weekly cost vs

weekly savings$0

$20,000

$40,000

$60,000

$80,000

$100,000

$120,000

$140,000

$160,000

$180,000

1/09/2015 1/10/2015 1/11/2015 1/12/2015

Test0suite0cost0vs0savings0(per0week)

Cost0saved0par0week Test0suite0cost0per0week

Team cost is constant

@wakaleo

How much are your tests worth?

Weekly cost vs

weekly savings$0

$20,000

$40,000

$60,000

$80,000

$100,000

$120,000

$140,000

$160,000

$180,000

1/09/2015 1/10/2015 1/11/2015 1/12/2015

Test0suite0cost0vs0savings0(per0week)

Cost0saved0par0week Test0suite0cost0per0week

Team cost is constant

Time saved increases with the number of executed

@wakaleo

How much are your tests worth?

$0

$200,000

$400,000

$600,000

$800,000

$1,000,000

$1,200,000

1/09/2015 1/10/2015 1/11/2015 1/12/2015

What0is0your0test0suite0worth?

Test0suite0ROI Test0suite0cost

Cumulated cost vs

cumulated savings

@wakaleo

How much are your tests worth?

$0

$200,000

$400,000

$600,000

$800,000

$1,000,000

$1,200,000

1/09/2015 1/10/2015 1/11/2015 1/12/2015

What0is0your0test0suite0worth?

Test0suite0ROI Test0suite0cost

Cumulated cost vs

cumulated savings

Cumulated value should increase faster than cumulated cost

@wakaleo

Now factor in high maintenance

40-60%Typical maintenance overhead for poorly-written test suites

@wakaleo

Now factor in high maintenance

Number of tests

Main

tena

nce

over

head

@wakaleo

Now factor in high maintenance

Number of tests

Main

tena

nce

over

head

@wakaleo

$0$50,000

$100,000$150,000$200,000$250,000$300,000$350,000$400,000$450,000$500,000

1/09/2015 1/10/2015 1/11/2015 1/12/2015 1/01/2016

What0is0your0test0suite0worth?

Test0suite0cost Test0suite0ROI

Now factor in high maintenance

High-maintenance costs reduce the number of tests a team can write

@wakaleo

$0$50,000

$100,000$150,000$200,000$250,000$300,000$350,000$400,000$450,000$500,000

1/09/2015 1/10/2015 1/11/2015 1/12/2015 1/01/2016

What0is0your0test0suite0worth?

Test0suite0cost Test0suite0ROI

Now factor in high maintenance

High-maintenance costs reduce the number of tests a team can write

High maintenance costs can cause your test suite to lose value

@wakaleo

Narrative Coding Style

Open-Closed Principle

Single Responsibility Principle

Separation of Concerns

Software Engineering Principles can help you write more maintainable tests

@wakaleo

Narrative Coding Style

Open-Closed Principle

Single Responsibility Principle

Separation of Concerns

Software Engineering Principles can help you write more maintainable tests

@wakaleo

Narrative Coding Style

Open-Closed Principle

Single Responsibility Principle

Separation of Concerns

Software Engineering Principles can help you write more maintainable tests

@wakaleo

Narrative Coding Style

Open-Closed Principle

Single Responsibility Principle

Separation of Concerns

Software Engineering Principles can help you write more maintainable tests

@wakaleo

Narrative Coding Style

Open-Closed Principle

Single Responsibility Principle

Separation of Concerns

Software Engineering Principles can help you write more maintainable tests

@wakaleo

Separation of Concerns - Layers

Goals

Tasks

Actions

@wakaleo

Separation of Concerns - Layers

Goals

Tasks

Actions

What are you trying to achieve?

@wakaleo

Separation of Concerns - Layers

Goals

Tasks

Actions

What are you trying to achieve?

What do you need to do to achieve this goal?

@wakaleo

Separation of Concerns - Layers

Goals

Tasks

Actions

What are you trying to achieve?

What do you need to do to achieve this goal?

What interactions with the system do you need for each

task?

@wakaleo

Separation of Concerns - Layers

Scenario: Add a new todo entry on the todo home page Given I am on the Todo application home page When I enter 'Buy some milk' into the New Todo field And I press ENTER Then the todo list box should contain 'Buy some milk'

@wakaleo

Separation of Concerns - Layers

Scenario: Record a new todo action for future use Given I need to buy some milk When I add the todo action 'Buy some milk' Then 'Buy some milk' should be recorded in my todo list

@wakaleo

Feature documentationDetailed feedbackHigh level feedback

Communication and documentation

@wakaleo

Feature documentationDetailed feedbackHigh level feedback

Communication and documentation

@wakaleo

Serenity BDD

“Because life is too short to h a v e t o m a i n t a i n u g l y automated test suites”

@wakaleo

Living documentation

Strong Selenium WebDriver support

JUnit, Cucumber or JBehave

A layered approach

Serenity BDD

@wakaleo

The Serenity layered architecture

Goals

Tasks

Interactions

@wakaleo

The Serenity layered architecture

Goals

Tasks

Interactions

Tests or scenarios

@wakaleo

The Serenity layered architecture

Goals

Tasks

Interactions

Tests or scenarios

“Steps”

@wakaleo

The Serenity layered architecture

Goals

Tasks

Interactions

Tests or scenarios

“Steps”

Page Objects or other technical components

@wakaleo

The Serenity layered architectureFeature: Add new todos James needs to be able to jot down actions he needs to do as he thinks of them Scenario: Record a new todo action for future use Given James needs to buy some milk When James adds the todo action 'Buy some milk' Then 'Buy some milk' should be recorded in his todo list

@wakaleo

The Serenity layered architectureFeature: Add new todos James needs to be able to jot down actions he needs to do as he thinks of them Scenario: Record a new todo action for future use Given James needs to buy some milk When James adds the todo action 'Buy some milk' Then 'Buy some milk' should be recorded in his todo list

A goal

@wakaleo

The Serenity layered architectureFeature: Add new todos James needs to be able to jot down actions he needs to do as he thinks of them Scenario: Record a new todo action for future use Given James needs to buy some milk When James adds the todo action 'Buy some milk' Then 'Buy some milk' should be recorded in his todo list

A goal

A task

@wakaleo

The Serenity layered architectureFeature: Add new todos James needs to be able to jot down actions he needs to do as he thinks of them Scenario: Record a new todo action for future use Given James needs to buy some milk When James adds the todo action 'Buy some milk' Then 'Buy some milk' should be recorded in his todo list

A goal

A task

@Steps ATodoUser james; @When(“^(?:.*) (?:adds|has added) the todo action '(.*)'$") public void i_add_the_todo_action(String actionName) { james.adds_an_action_called(actionName);}

@wakaleo

The Serenity layered architectureFeature: Add new todos James needs to be able to jot down actions he needs to do as he thinks of them Scenario: Record a new todo action for future use Given James needs to buy some milk When James adds the todo action 'Buy some milk' Then 'Buy some milk' should be recorded in his todo list

A goal

A task

@Steps ATodoUser james; @When(“^(?:.*) (?:adds|has added) the todo action '(.*)'$") public void i_add_the_todo_action(String actionName) { james.adds_an_action_called(actionName);} The task definition

@wakaleo

The Serenity layered architectureFeature: Add new todos James needs to be able to jot down actions he needs to do as he thinks of them Scenario: Record a new todo action for future use Given James needs to buy some milk When James adds the todo action 'Buy some milk' Then 'Buy some milk' should be recorded in his todo list

A goal

A task

@Steps ATodoUser james; @When(“^(?:.*) (?:adds|has added) the todo action '(.*)'$") public void i_add_the_todo_action(String actionName) { james.adds_an_action_called(actionName);} The task definition

TodoPage onTheTodoHomePage; @Steppublic void adds_an_action_called(String actionName) { onTheTodoHomePage.addAnActionCalled(actionName);}

@wakaleo

The Serenity layered architectureFeature: Add new todos James needs to be able to jot down actions he needs to do as he thinks of them Scenario: Record a new todo action for future use Given James needs to buy some milk When James adds the todo action 'Buy some milk' Then 'Buy some milk' should be recorded in his todo list

A goal

A task

@Steps ATodoUser james; @When(“^(?:.*) (?:adds|has added) the todo action '(.*)'$") public void i_add_the_todo_action(String actionName) { james.adds_an_action_called(actionName);} The task definition

TodoPage onTheTodoHomePage; @Steppublic void adds_an_action_called(String actionName) { onTheTodoHomePage.addAnActionCalled(actionName);}

An interaction

@wakaleo

Serenity for Living DocumentationFeature: Add new todos I need to be able to jot down actions I need to do as fast as I think of them Scenario: Record a new todo action for future use Given I need to buy some milk When I add the todo action 'Buy some milk' Then 'Buy some milk' should be recorded in my todo list

@wakaleo

Serenity for Living DocumentationFeature: Add new todos I need to be able to jot down actions I need to do as fast as I think of them Scenario: Record a new todo action for future use Given I need to buy some milk When I add the todo action 'Buy some milk' Then 'Buy some milk' should be recorded in my todo list

@wakaleo

Coding demo

@wakaleo

Hard to apply to Single Page Apps

Need modifications to extend

Multiple Responsibilities

Bloat

Page Objects considered harmful

@wakaleo

Introducing the Journey Pattern

@wakaleo

Highly readable style

Encourages reuse and maintainable code

Small, reusable interaction components

User-centric

The Journey Pattern

@wakaleo

Origins of the Journey Pattern in Serenity

Antony Marcano

2007

*

* Based on actual events

@wakaleo

Origins of the Journey Pattern in Serenity

Antony Marcano

“Page Objects kinda suck. Why not try this pattern instead?”

Agile Alliance Functional Test Tools Workshop 2007

2007

*

* Based on actual events

@wakaleo

Origins of the Journey Pattern in Serenity

Antony Marcano

2007 2009

Andy Palmer

@wakaleo

Origins of the Journey Pattern in Serenity

Antony Marcano

2007 2009

More people should know about this pattern

Andy Palmer

@wakaleo

Origins of the Journey Pattern in Serenity

Antony Marcano

2007 2009

More people should know about this pattern

JNarrate

Andy Palmer

Let’s write a reference implementation!

@wakaleo

Origins of the Journey Pattern in Serenity

Antony Marcano

2007 2009

“Coronation Streetrocks

2012

Andy Palmer

@wakaleo

Origins of the Journey Pattern in Serenity

Antony Marcano

2007 2009

“Coronation Streetrocks

2012

Andy Palmer

Eureka! “Soap Opera Personas”

@wakaleo

Andy Palmer

Origins of the Journey Pattern in Serenity

Antony Marcano

2007 2009 2012

Jan Molak

@wakaleo

Andy Palmer

Origins of the Journey Pattern in Serenity

Antony Marcano

2007 2009 2012

Jan Molak

You should call this “The Journey Pattern”

@wakaleo

Andy Palmer

Origins of the Journey Pattern in Serenity

Antony Marcano

2007 2009 2012

Jan Molak

You should call this “The Journey Pattern”Damn!

@wakaleo

Andy Palmer

Origins of the Journey Pattern in Serenity

Antony Marcano

2007 2009 2012

Jan Molak

You should call this “The Journey Pattern”Damn!

Why didn’t we think of that!

@wakaleo

Andy Palmer

Origins of the Journey Pattern in Serenity

Antony Marcano

2007 2009 2012

Jan Molak

2015

John Smart

@wakaleo

Andy Palmer

Origins of the Journey Pattern in Serenity

Antony Marcano

2007 2009 2012

Jan Molak

2015

John Smart

Serenity is cool

@wakaleo

Andy Palmer

Origins of the Journey Pattern in Serenity

Antony Marcano

2007 2009 2012

Jan Molak

2015

John Smart

Serenity is cool

But it needs the Journey Pattern

@wakaleo

Andy Palmer

Origins of the Journey Pattern in Serenity

Antony Marcano

2007 2009 2012

Jan Molak

2015

John Smart

Serenity is cool

So let’s make it happen!

But it needs the Journey Pattern

@wakaleo

The Journey Pattern - A User-Centric model

Actors have goals

@wakaleo

The Journey Pattern - A User-Centric model

Actors have goals

Meet James

@wakaleo

The Journey Pattern - A User-Centric model

Actors have goals I’d like to be able to recall all the things I need to do

Meet James

@wakaleo

The Journey Pattern - A User-Centric model

Actors have abilities

@wakaleo

The Journey Pattern - A User-Centric model

Actors have abilities I can browse the web with my browser

@wakaleo

The Journey Pattern - A User-Centric model

Actors have abilities I can browse the web with my browser

I can also query REST services

@wakaleo

I’d like to be able to recall all the things I need to do

The Journey Pattern - A User-Centric model

Actors perform tasks

@wakaleo

I’d like to be able to recall all the things I need to do

The Journey Pattern - A User-Centric model

Actors perform tasks

I’ll add ‘Buy some milk’ to my todo list

@wakaleo

I’d like to be able to recall all the things I need to do

The Journey Pattern - A User-Centric model

Actors may interact with the system to perform these tasks

I’ll add ‘Buy some milk’ to my todo list

@wakaleo

I’d like to be able to recall all the things I need to do

The Journey Pattern - A User-Centric model

Actors may interact with the system to perform these tasks

I’ll add ‘Buy some milk’ to my todo list

@wakaleo

I’d like to be able to recall all the things I need to do

The Journey Pattern - A User-Centric model

Actors may interact with the system to perform these tasks

I’ll add ‘Buy some milk’ to my todo list

Type ‘Buy the milk’

@wakaleo

I’d like to be able to recall all the things I need to do

The Journey Pattern - A User-Centric model

Actors may interact with the system to perform these tasks

I’ll add ‘Buy some milk’ to my todo list

Type ‘Buy the milk’

@wakaleo

I’d like to be able to recall all the things I need to do

The Journey Pattern - A User-Centric model

Actors may interact with the system to perform these tasks

I’ll add ‘Buy some milk’ to my todo list

Type ‘Buy the milk’Press ‘ENTER’

@wakaleo

I’d like to be able to recall all the things I need to do

The Journey Pattern - A User-Centric model

Actors may interact with the system to perform these tasks

I’ll add ‘Buy some milk’ to my todo list

Type ‘Buy the milk’Press ‘ENTER’

@wakaleo

The Journey Pattern in Serenity

Actors have goals I’d like to be able to recall all the things I need to do

@wakaleo

The Journey Pattern in Serenity

Actors have goals

Feature: Add new todos James would like to be able to recall all the things he needs to do Scenario: Record a new todo action for future use Given James needs to buy some milk When James adds the todo action 'Buy some milk' Then 'Buy some milk' should be recorded in his todo list

I’d like to be able to recall all the things I need to do

@wakaleo

A User-Centric model

Actors have abilities I can browse the web with my browser

Actor james = Actor.named(“James");

@ManagedWebDriver hisBrowser; … james.can(BrowseTheWeb.with(hisBrowser));

@wakaleo

A User-Centric model

Actors perform tasks

@StepsAddItem addATodoItem; … james.attemptsTo(addATodoItem.called("Buy some milk"));

@wakaleo

A User-Centric model

Actors perform tasks I’ll add ‘Buy some milk’ to my todo list

@StepsAddItem addATodoItem; … james.attemptsTo(addATodoItem.called("Buy some milk"));

@wakaleo

A User-Centric model

Actors perform tasks

@StepsAddItem addATodoItem; … james.attemptsTo(addATodoItem.called("Buy some milk"));

actor.attemptsTo( Enter.theValue(thingToDo).into(ToDoList.NEW_TODO_FIELD), Hit.the(RETURN).keyIn(ToDoList.NEW_TODO_FIELD) );

@wakaleo

A User-Centric model

Actors perform tasks I’ll add ‘Buy some milk’ to my todo list

@StepsAddItem addATodoItem; … james.attemptsTo(addATodoItem.called("Buy some milk"));

actor.attemptsTo( Enter.theValue(thingToDo).into(ToDoList.NEW_TODO_FIELD), Hit.the(RETURN).keyIn(ToDoList.NEW_TODO_FIELD) );

@wakaleo

A User-Centric model

Actors perform tasks I’ll add ‘Buy some milk’ to my todo list

@StepsAddItem addATodoItem; … james.attemptsTo(addATodoItem.called("Buy some milk"));

actor.attemptsTo( Enter.theValue(thingToDo).into(ToDoList.NEW_TODO_FIELD), Hit.the(RETURN).keyIn(ToDoList.NEW_TODO_FIELD) );

Type ‘Buy the milk’

@wakaleo

A User-Centric model

Actors perform tasks I’ll add ‘Buy some milk’ to my todo list

@StepsAddItem addATodoItem; … james.attemptsTo(addATodoItem.called("Buy some milk"));

actor.attemptsTo( Enter.theValue(thingToDo).into(ToDoList.NEW_TODO_FIELD), Hit.the(RETURN).keyIn(ToDoList.NEW_TODO_FIELD) );

Type ‘Buy the milk’

Press ‘ENTER’

@wakaleo

Coding demo

@wakaleo

References• http://www.serenity-bdd.info• https://github.com/serenity-bdd/serenity-journey-demo• http://bit.ly/rg-journey

@wakaleo

Questions?John Ferguson Smart

john.smart@wakaleo.comwakaleo

http://www.wakaleo.comhttp://www.serenity-bdd.info