1

Agile and DevOps teams: test value first!!!

LAURENT PY

@py_laurentlaurent.py@hiptest.nethttps://hiptest.net

VINCENT PRETRE

vincent.pretre@hiptest.nethttps://hiptest.net

HIPTEST COMMUNITY

2

37%15% 22%

5%

13%

4%

4%

CountriesOrganizations

+1000 +130

users

20000

PLAN

3

Testing inDevOps

Step 1 :Testing theidea

Step 2 :Testing thecode

Step 3 :Testing inproduction

Step 4 :GOTOstep 1

Conclusion :Benefits

TESTING IN DEVOPS

4

5

DEVOPS : BIGGEST OPPORTUNITY?

SPEEEEEED

6

DEVOPS : BIGGEST OPPORTUNITY

TESTING IN DEVOPS

7

1) Which value do we want to deliver?

2) Is the feature’s implementation aligned with that?

3) Does the feature have a real impact on the end user?

4) Is the UX great and what’s the feature usage in time ?

STEP 1: TESTING THE IDEA

8

WHY?

Challenge business assumptions

10

• Use personae to create intimacy between users and the team

• Define business assumptions

• Behavior Driven Development (BDD) to capture the behavior

WHICH VALUE DO WE WANT TO DELIVER?

11

• Create a shared understanding based on examples• Using a common business terminology• Define the stop• Once automated the examples become a living

documentation

BDD IN A NUTSHELL

12

User interview +

BDD – CAPTURE THE BEHAVIOR

13

• Using Gherkin syntax with declarative style• Using a consistent business terminology

• Refactoring continuously your tests

GOOD PRACTICES

CONTINUOUS TESTING

14

Test

Testcontinuouslyyoursoftwarefromideatoproduction

Idea Dev Production

Test Test

STEP 2: TESTING THE

CODE

15

16

1. In step 1 we have created a shared understanding of the feature

2. The dev team can develop a first increment of the feature

3. Integration to CI/CD pipeline

DEVELOPMENT

2 - Refine with examples1 – Manage Backlog 3 - Implement the stories

4 – Create builds and run tests5 – Deploy in production

Results

TESTING PROCESS AT HIPTEST

17

18

100% of our checks (scenarios) are automated

AUTOMATION

AND

It is expensive

19

Credit to Martin Fowler

• 550 tests BDD

• 1600 tests Ember

• 3700 tests Rspec

OUR AUTOMATED TESTS

CONTINUOUS TESTING

20

Test

Testcontinuouslyyoursoftwarefromideatoproduction

Idea Dev Production

Test Test

STEP 3: TESTING IN

PRODUCTION

21

22

1. In Step 2 we have validated the implementation

2. The feature is rolled out using a switch (GA or progressive)

MAKING THE FEATURE AVAILABLE

23

Appsignal,Pingdom,Scalingo etLogmatic/datadog

APPLICATION PERFORMANCE MANAGEMENT

24

THE MATCH

Correctness Availability

25

MEASURE USAGE CONTINUOUSLY

• Measure usage- NB users- NB requests…

• Measure impact- Sign-up- Revenue- Retention…

26

Notification feature• Usage- NB clicks NB of notifications- NB users

• Business: none

Refactoring feature• Usage- NB refactoring actions- NB of impacts

• Business: retention

EXAMPLES

27

LIVE CHAT FOR REAL TIME FEEDBACK

28

LIVING DOCUMENTATION

CONTINUOUS TESTING

29

Test

Testcontinuouslyyoursoftwarefromideatoproduction

Idea Dev Production

Test Test

ETAPE 4: LET’S ITERATE

30

31

1. In step 3 we have measured the impact

2. The feature can be removed (with the tests), enhanced or validated

3. GoTo step 1

LET’S ITERATE

CONCLUSION

32

33

•Test value first

• Incremental investment in quality

•Context suitable to experiments

BENEFITS OF TESTING IN DEVOPS

Making something nobody want

THE BIGGEST RISK

A HUGE THANK YOU!

LAURENT PY

@py_laurentlaurent.py@hiptest.nethttps://hiptest.net

VINCENT PRETRE

vincent.pretre@hiptest.nethttps://hiptest.net