Testing an Erlang Backend

IntroductionWhy should I test?

Types of tests

Good Practices

TestingTechniquesUnit tests

Integration tests

Acceptance Tests

Performance Tests

Making ithappenTDD & BDD

Automatingeverything

Q&A

Testing an Erlang Backend

Enrique Paz

Senior Backend Developer @ Team Services

EUG-NL / 12-07-2012

1/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Introduction

Testing Techniques

Making it happen

Q&A

2/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Reasons for testing

• It increases your confidence in the code you write• Tests themselves are useful documentation• It makes refactoring really an option• It helps finding bugs earlier, so the impact is much lower

3/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Yeah, but...

• “My code is so clear/simple it doesn’t need tests”

I simplicity and clarity are not universal wordsI The simpler your code is, the easier for you to test it

• “It takes me much longer testing the code than writing it”I Was your original code working or you spent the time fixing it?I Have you consider refactoring?

• “Setting up the scenario for testing my code is too complex”I So. . . you do it manually everytime?I Automating that scenario for testing will be expensive. . . but only once!

• In short, stop arguing and start testing!

4/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Yeah, but...

• “My code is so clear/simple it doesn’t need tests”I simplicity and clarity are not universal words

I The simpler your code is, the easier for you to test it• “It takes me much longer testing the code than writing it”

I Was your original code working or you spent the time fixing it?I Have you consider refactoring?



4/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Yeah, but...

• “My code is so clear/simple it doesn’t need tests”I simplicity and clarity are not universal wordsI The simpler your code is, the easier for you to test it




4/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Yeah, but...


• “It takes me much longer testing the code than writing it”

I Was your original code working or you spent the time fixing it?I Have you consider refactoring?



4/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Yeah, but...


• “It takes me much longer testing the code than writing it”I Was your original code working or you spent the time fixing it?

I Have you consider refactoring?• “Setting up the scenario for testing my code is too complex”

I So. . . you do it manually everytime?I Automating that scenario for testing will be expensive. . . but only once!


4/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Yeah, but...





4/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Yeah, but...



• “Setting up the scenario for testing my code is too complex”

I So. . . you do it manually everytime?I Automating that scenario for testing will be expensive. . . but only once!


4/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Yeah, but...



• “Setting up the scenario for testing my code is too complex”I So. . . you do it manually everytime?

I Automating that scenario for testing will be expensive. . . but only once!


4/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Yeah, but...





4/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Types of tests

• Unit tests What you have written is exactly what you were trying to write• Integration tests The API of your component can interact with other APIs• Acceptance tests The API you have implemented fits the bussiness

requirements• Performance tests Your solution is suitable for the expected scenario

5/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

General Good Practices

• Tests should be repeatable (setup and cleanup)

• Tests should be independent from each other (use sequences whennecesary)

• Tests should be isolated from each other (leave the environment as youfound it)

• Tests should test specs and not depend on the implementation details• Separate the tests from the code. If not possible, refactor!

6/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A


• Tests should be repeatable (setup and cleanup)• Tests should be independent from each other (use sequences when

necesary)

• Tests should be isolated from each other (leave the environment as youfound it)


6/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A



necesary)• Tests should be isolated from each other (leave the environment as you

found it)


6/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A




found it)• Tests should test specs and not depend on the implementation details

• Separate the tests from the code. If not possible, refactor!

6/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A




found it)• Tests should test specs and not depend on the implementation details• Separate the tests from the code. If not possible, refactor!

6/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Introduction

Testing Techniques

Making it happen

Q&A

7/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Main tools

• EunitI The classic *unit approachI Starting to use it is very easyI Out of the box integration with rebarI But. . . Might be annoying to debug it

• Common TestI Much more powerfulI Setting it up can be painfulI It shines in complex scenarios, so usually is better for integration tests

8/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

A different approach: property testing

Finding good testcases might be hard:• There might be lots of them• They might be just hard to predict• They might be expensive / hard to generate

9/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Property testing tools

PropEr / Quickcheck Let’s use some math!

• You fed them with properties your code should satisfy• You specify how the input should look like• They smartly generate input and try to verify whether the properties are

satisfied• If they find a counterexample, they simplify it for you!• They can also verify the transitions in your gen_servers or gen_fsms

10/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A


PropEr / Quickcheck Let’s use some math!• You fed them with properties your code should satisfy

• You specify how the input should look like• They smartly generate input and try to verify whether the properties are


10/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A


PropEr / Quickcheck Let’s use some math!• You fed them with properties your code should satisfy• You specify how the input should look like

• They smartly generate input and try to verify whether the properties aresatisfied

• If they find a counterexample, they simplify it for you!• They can also verify the transitions in your gen_servers or gen_fsms

10/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A


PropEr / Quickcheck Let’s use some math!• You fed them with properties your code should satisfy• You specify how the input should look like• They smartly generate input and try to verify whether the properties are

satisfied

• If they find a counterexample, they simplify it for you!• They can also verify the transitions in your gen_servers or gen_fsms

10/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A



satisfied• If they find a counterexample, they simplify it for you!

• They can also verify the transitions in your gen_servers or gen_fsms

10/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A




10/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Unit Tests ExamplesTo be tested

−spec remove_dupl icates ( [ i n t e g e r ( ) ] ) −> [ i n t e g e r ( ) ] .

remove_dupl icates ( [ ] ) −>[ ] ;

remove_dupl icates ( L ) when i s _ l i s t ( L ) −>l i s t s : uso r t ( L ) .

r e mo ve_ du p l i ca te s_ f r o m_ l i s t s _ f i l e ( F i l e ) −>case l o a d _ l i s t s _ f r o m _ f i l e ( F i l e ) o f

{ e r ro r , Reason } −>{ e r ro r , Reason } ;

{ ok , L i s t s } −>{ ok , [ remove_dupl icates ( L ) | | L <− L i s t s ] }

end .

%% I n t e r n a l f u n c t i o n s

l o a d _ l i s t s _ f r o m _ f i l e ( F i l e ) −>case f i l e : consu l t ( F i l e ) o f

{ e r ro r , Reason } −>{ e r ro r , Reason } ;

{ ok , Terms } −>{ ok , [ L | | L <− Terms , i s _ l i s t ( L ) ] }

end .

11/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Unit Tests ExamplesEunit

simple_cases_test_ ( ) −>[ ?_assertEqual ( [ ] , t s _ l i s t s : remove_dupl icates ( [ ] ) ) ,

% Bad tes tcase ( dependent / f o r c i n g on the implementat ion ! ! )?_assertEqual ( [ 1 , 2 , 3 , 4 , 5 ] , t s _ l i s t s : remove_dupl icates ( [ 1 , 2 , 2 , 3 , 4 , 4 , 5 ] ) ) ] .

l o a d _ l i s t s _ f r o m _ f i l e _ t e s t _ ( ) −>{ setup ,

% SETUP creates a v a l i d f i l e w i th 2 l i s t s and ensures I n v a l i d F i l e does not e x i s tfun ( ) −> s e t u p _ f i l e s ( ) end ,fun ( { V a l i d F i l e , _ I n v a l i d F i l e } ) −> ok= f i l e : de le te ( V a l i d F i l e ) end ,fun ( { V a l i d F i l e , I n v a l i d F i l e } ) −>

[ ?_assertEqual ( { ok , [ [ 1 , 2 , −2, −2, −1] , [−2, 2 , 2 , 1 ] ] } , t s _ l i s t s : l o a d _ l i s t s _ f r o m _ f i l e (V a l i d F i l e ) ) ,

?_assertEqual ( { e r ro r , enoent } , t s _ l i s t s : l o a d _ l i s t s _ f r o m _ f i l e ( I n v a l i d F i l e ) ) ]end

} .r e m o v e _ d u p l i c a t e s _ f r o m _ l i s t s _ f i l e _ t e s t _ ( ) −>

{ setup ,fun ( ) −> s e t u p _ f i l e s ( ) end ,fun ( { V a l i d F i l e , _ I n v a l i d F i l e } ) −> ok= f i l e : de le te ( V a l i d F i l e ) end ,fun ( { V a l i d F i l e , I n v a l i d F i l e } ) −>

[ ?_assertEqual ( { e r ro r , enoent } , t s _ l i s t s : r em ov e_ dup l i ca t es _ f r om _ l i s t s_ f i l e ( I n v a l i d F i l e ) ) ,?_assertMatch ( { ok , [ L1 , L2 ] } when i s _ l i s t ( L1 ) and i s _ l i s t ( L2 ) , t s _ l i s t s :

r e mo ve _du p l i ca te s_ f r o m_ l i s t s _ f i l e ( V a l i d F i l e ) ) ]end } .

12/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Unit Tests ExamplesProper

prop_remove_dupl icates_keeps_al l_elements ( ) −>numtests (1000 , ?FORALL( L , n e _ l i s t ( i n t e g e r ( ) ) ,

beginProcessed= t s _ l i s t s : remove_dupl icates ( L ) ,?WHENFAIL(

i o : format ( " L=~p , Processed=~p~n " , [ L , Processed ] ) ,l i s t s : a l l ( fun (N) −> l i s t s : member (N, Processed ) end , L )

)end

) ) .prop_remove_dupl icates_conta ins_no_dupl icates ( ) −>

numtests (1000 , ?FORALL( Or i g i na l , n e _ l i s t ( i n t e g e r ( ) ) ,begin

L= t s _ l i s t s : remove_dupl icates ( O r i g i n a l ) ,Seq= l i s t s : seq (1 , leng th ( L ) ) ,?WHENFAIL( i o : format ( " O r i g i n a l =~p , Processed=~p~n " , [ O r i g i na l , L ] ) ,

l i s t s : a l l ( fun (N) −>Elem= l i s t s : nth (N, L ) ,{ _Before , [ Elem | A f t e r ] } = l i s t s : s p l i t w i t h ( fun (X) −> X/=Elem end , L ) ,not l i s t s : member ( Elem , A f t e r )

end , Seq ))

end) ) .

13/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Integration Tests

Common Test A flexible tool for complex scenarios

• per_suite and per_testcase setup and teardown• Easy to automatically avoid certain testcases under certain conditions

({skip, Reason})• Also possible to run the test suites against a configurable set of running

nodes• . . .

14/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Integration Tests

Common Test A flexible tool for complex scenarios• per_suite and per_testcase setup and teardown

• Easy to automatically avoid certain testcases under certain conditions({skip, Reason})

• Also possible to run the test suites against a configurable set of runningnodes

• . . .

14/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Integration Tests

Common Test A flexible tool for complex scenarios• per_suite and per_testcase setup and teardown• Easy to automatically avoid certain testcases under certain conditions

({skip, Reason})

• Also possible to run the test suites against a configurable set of runningnodes

• . . .

14/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Integration Tests



nodes

• . . .

14/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Integration Tests



nodes• . . .

14/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Common TestThe way we use it

1. We use the SUITE setup to prepare the pair of projects we want tointegrate

2. Still in the SUITE setup, we use ct_slave to start the VMs for those projects3. We use per testcase setup for creating specific conditions in each node

via RPC4. Being A and B the nodes we want to integrate, testcases execute

something in A that requires it to interact with B5. Testcases assert the obtained response matches the expected one given

the testcase preparation

15/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A



2. Still in the SUITE setup, we use ct_slave to start the VMs for those projects

3. We use per testcase setup for creating specific conditions in each nodevia RPC

4. Being A and B the nodes we want to integrate, testcases executesomething in A that requires it to interact with B

5. Testcases assert the obtained response matches the expected one giventhe testcase preparation

15/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A




via RPC

4. Being A and B the nodes we want to integrate, testcases executesomething in A that requires it to interact with B


15/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A





something in A that requires it to interact with B


15/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A





something in A that requires it to interact with B5. Testcases assert the obtained response matches the expected one given

the testcase preparation

15/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

General Considerations

• These tests should check the whole stack, top to bottom• People without technical knowledge about the project should be able to

write them• They can be written in any language• They don’t look for bugs, they validate the requirements

16/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A


• These tests should check the whole stack, top to bottom

• People without technical knowledge about the project should be able towrite them

• They can be written in any language• They don’t look for bugs, they validate the requirements

16/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A



write them

• They can be written in any language• They don’t look for bugs, they validate the requirements

16/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A



write them• They can be written in any language

• They don’t look for bugs, they validate the requirements

16/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A



write them• They can be written in any language• They don’t look for bugs, they validate the requirements

16/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

What we use at Team Services

• Since we are developing an HTTP stack, we use JMeter for validation• We could use any of the already described erlang tools, but. . .

I We want QA people from other teams to be able to black-box test our codeI We want to avoid corrupting the acceptance tests with our knowledge of the

system internals

17/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A



I We want QA people from other teams to be able to black-box test our code

I We want to avoid corrupting the acceptance tests with our knowledge of thesystem internals

17/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A



I We want QA people from other teams to be able to black-box test our codeI We want to avoid corrupting the acceptance tests with our knowledge of the

system internals

17/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

What we use at Team ServicesJMeter

• Simple assertion-based mechanism• Nice management interface

18/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Some other tools used inside Spil GamesCucumber

• Scenario descriptions in Gherkin (close to natural language)• Multi-language support (Ruby, Php, Python. . . )

# language : enFeature : Payment Method Count

As a user , I want to see a c e r t a i n number o f payment methodsf o r my count ry .

Scenario Out l i ne : Payment Method ChoiceGiven I am a user i n "<Country > "And I open the PSSThen I should see "<PaymentMethodCount> " payment methodsExamples :

| Country | PaymentMethodCount || Poland | 6 || France | 5 || B r a z i l | 6 |

19/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Some other tools used inside Spil GamesCucumber

• Scenario implementation/∗∗ @Given / ^ I am a user i n " ( [ ^ " ] ∗) " $ / ∗ /p u b l i c f u n c t i o n iAmAUserIn ( $country ){

$country = s t r t o l o w e r ( $country ) ;$ t h i s−>session−>setRequestHeader ( ’SPILTESTIPADDRESS ’ , Sp i lTes t IP : : addressFor ( $country ) ) ;

}/∗∗ @Given / ^ I open the PSS$/ ∗ /p u b l i c f u n c t i o n iOpenThePss ( ){

r e t u r n $ t h i s−>iOpenThePaymentSelectionScreen ( ) ;}/∗∗ @Then / ^ I should see " ( [ ^ " ] ∗) " payment methods$ / ∗ /p u b l i c f u n c t i o n iShouldSeePaymentMethods ( $nummethods ){

$page = $ t h i s−>session−>getPage ( ) ;$e l = $ t h i s−>getDomNodeByXpath ( $page , ’ / /∗ [ @id=" pss_method_col lect ion " ] ’ ) ;$foundmethods = count ( $e l−>f i n d A l l ( ’ css ’ , ’ d i v . pss_method ’ ) ) ;i f ( $nummethods ! = $foundmethods ) {

throw new Except ion ( " Found payment methods does not equal $nummethods " ) ;}

}

20/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Basic Benchmarking

• Testing a function doesn’t have to be painful• The sooner you realize is bad, the less impact

−spec tes t_ fun ( any ( ) , fun ( ( ) −> ok ) , i n t e g e r ( ) , i n t e g e r ( ) ) −> { f l o a t ( ) , f l o a t ( ) } .

t es t_ fun (Name, TestFun , ParCount , SeqCount ) −>RS = tes t_ fun_x_t imes (Name, TestFun , SeqCount ) ,RP = t e s t _ f u n _ x _ t i m e s _ p a r a l l e l (Name, TestFun , ParCount , SeqCount ) ,{RS, RP} .

21/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Basic BenchmarkingSync Execution

tes t_ fun_x_t imes (Name, Fun , SeqCount ) −>Tick = now ( ) ,l i s t s : foreach ( fun ( _ ) −>

Fun ( )end , l i s t s : seq (1 , SeqCount ) ) ,

Tock = now ( ) ,AvgTime = t imer : now_d i f f ( Tock , T ick ) /1000 ,Speed = SeqCount∗1000/AvgTime ,i o : format ( " ~40p req / sec ~.1 f ~n " , [Name, Speed ] ) ,Speed .

22/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Basic BenchmarkingAsync Execution

t e s t _ f u n _ x _ t i m e s _ p a r a l l e l (Name, Fun , ParCount , SeqCount ) −>Tick = now ( ) ,pmap( fun ( _ ) −>

l i s t s : foreach ( fun ( _ ) −>Fun ( )

end , l i s t s : seq (1 , SeqCount ) )end , l i s t s : seq (1 , ParCount ) , 250000) , %t imeout

Tock = now ( ) ,AvgTime = t imer : now_d i f f ( Tock , T ick ) /1000 ,Speed = ParCount∗SeqCount∗1000/AvgTime ,i o : format ( " ~40p req / sec ~10.1 f ~n " , [Name, Speed ] ) ,Speed .

23/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Tsung

• It can be used to generate waves of load according to different parameters• It generates graphics showing how the target system reacted to the load• Uses XML especifications

24/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Tsung

• getToken waves of 100 and 500 req/s during 600 secs<?xml vers ion=" 1.0 " ?>< !DOCTYPE tsung SYSTEM " / usr / share / tsung / tsung −1.0. dtd " [ ] ><tsung l o g l e v e l = " debug " dump t ra f f i c = " p ro toco l ">

< c l i e n t s >< c l i e n t host= " l o c a l h o s t " use_cont ro l le r_vm=" t rue ">< / c l i e n t >

< / c l i e n t s ><servers>

<server host= " ap i . spi lgames . com" po r t = " 443 " type=" s s l ">< / server>< / servers><load>

< a r r i va lphase phase=" 1 " du ra t i on=" 600 " u n i t = " second "><users a r r i v a l r a t e =" 100 " u n i t = " second ">< / users>

< / a r r i va lphase >< a r r i va lphase phase=" 2 " du ra t i on=" 600 " u n i t = " second ">

<users a r r i v a l r a t e =" 500 " u n i t = " second ">< / users>< / load>

<sessions><session name=" ap i " p r o b a b i l i t y = " 100 " type=" t s _ h t t p ">

<request><dyn_var iab le name=" guesttoken " jsonpath=" auth . token " / >

< h t t p u r l = ’ / account / getToken / ’ vers ion= ’ 1.1 ’ contents= ’ {&quot ; type&quot ; :&quot ;sample&quot ; } ’ content_ type= ’ t e x t / json ’ method= ’POST ’>

< / h t t p >< / request>< / session>

< / sessions>< / tsung>

25/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Introduction

Testing Techniques

Making it happen

Q&A

26/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

The purist approach

1. A developer receives some requirements from his manager, who ideallyhas writting some acceptance tests

2. He designs the simplest (flexible) solution fitting those requirements3. He documents and specs the API, and implements only the exported

functions headers all returning unimplemented4. He writes as many unit and integration tests as necesary for that API5. He hands over his work to another developer, who after agreeing with the

design fills the implementation until all the tests pass

27/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

The purist approach


2. He designs the simplest (flexible) solution fitting those requirements

3. He documents and specs the API, and implements only the exportedfunctions headers all returning unimplemented

4. He writes as many unit and integration tests as necesary for that API5. He hands over his work to another developer, who after agreeing with the


27/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

The purist approach



functions headers all returning unimplemented

4. He writes as many unit and integration tests as necesary for that API5. He hands over his work to another developer, who after agreeing with the


27/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

The purist approach



functions headers all returning unimplemented4. He writes as many unit and integration tests as necesary for that API

5. He hands over his work to another developer, who after agreeing with thedesign fills the implementation until all the tests pass

27/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

The purist approach



functions headers all returning unimplemented4. He writes as many unit and integration tests as necesary for that API5. He hands over his work to another developer, who after agreeing with the


27/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

A more practical approach

When writting new code• Write the tests ASAP, if possible before writing the code itself• While implementing, keep in mind everything you write you have to test it• Start executing your tests ASAP, specially to validate those small simple

functions you use all around (yes, those that can never fail)

When changing existing code• Even if the existing coverage is poor, write tests for the changes you

introduce• Make sure all the tests are running before touching a single line!

28/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

A more practical approach

When writting new code• Write the tests ASAP, if possible before writing the code itself• While implementing, keep in mind everything you write you have to test it• Start executing your tests ASAP, specially to validate those small simple

functions you use all around (yes, those that can never fail)When changing existing code• Even if the existing coverage is poor, write tests for the changes you

introduce• Make sure all the tests are running before touching a single line!

28/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

TDD refactorOriginal Tests

• One of the following tests crashed on June 23get_age_test_ ( ) −>

[?_assertEqual (47 , su_user : get_age (# date { year =1965 , month=2 , day=1 } ) ) ,?_assertEqual (27 , su_user : get_age (# date { year =1984 , month=6 , day=27 } ) )

] .

29/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

TDD refactorOriginal Code

• The implementation had an obvious flaw−spec get_age (# date { } ) −> non_neg_integer ( ) .

get_age (# date { year =0 , month=0 , day=0 } ) −>0;

get_age (# date { year=Y, month=M, day=D} ) −>Bir thGregor ianDays=calendar : date_to_gregor ian_days ( {Y, M, D} ) ,{ NowDate , _ } = calendar : now_to_datetime ( os : timestamp ( ) ) ,CurrentGregorianDays=calendar : date_to_gregor ian_days ( NowDate ) ,( CurrentGregorianDays−Bir thGregor ianDays ) d i v 365.

30/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

TDD refactorUpdated Tests

• We update the tests with the requirements for a human-like approachget_age_test_ ( ) −>

[?_assertEqual (47 , su_user : get_age (# date { year =1965 , month=2 , day=1 } ) )

] .i s_b i r thday_gone_tes t_ ( ) −>

Dob=#date { year =1972 , month=4 , day=21 } ,[

?_assertEqual ( fa l se , su_user : is_b i r thday_gone (Dob , 4 , 20) )?_assertEqual ( t rue , su_user : is_b i r thday_gone (Dob , 4 , 22) ) ,?_assertEqual ( t rue , su_user : is_b i r thday_gone (Dob , 4 , 21) ) ,

] .

31/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

TDD refactorUpdated Code

• The resultant code is still very clean. And now it works!−spec get_age (# date { } ) −> non_neg_integer ( ) .get_age (# date { year =0 , month=_ , day=_ } ) −>

0;get_age (# date { year=Y, month=_ , day=_ } =Dob) −>

{ {NY, NM, ND} , _ } = calendar : now_to_datetime ( os : timestamp ( ) ) ,case is_b i r thday_gone (Dob , NM, ND) o f

t r ue −> NY − Y;f a l s e −> NY − Y −1

end .

−spec is_b i r thday_gone (# date { } , pos_ in teger ( ) , pos_ in teger ( ) ) −> boolean ( ) .i s_b i r thday_gone (# date { month=M, day=D} , CurrentMonth , CurrentDay ) −>

(M∗100+D) =< ( CurrentMonth∗100+CurrentDay ) .

32/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

CI Environment

• Builds and tests a release your projects after every commit (Unit,Integration and Acceptance)

• This automatic V&V process should never take longer than 5 minutes• If the process fails, immediatly blames the originator of the change• Long duration test suites can be put together in a regression task

performed during the night

CI + AutomaticDeployment = ContinuousDelivery

33/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

CI Environment


• This automatic V&V process should never take longer than 5 minutes

• If the process fails, immediatly blames the originator of the change• Long duration test suites can be put together in a regression task



33/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

CI Environment


• This automatic V&V process should never take longer than 5 minutes• If the process fails, immediatly blames the originator of the change

• Long duration test suites can be put together in a regression taskperformed during the night


33/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

CI Environment


• This automatic V&V process should never take longer than 5 minutes• If the process fails, immediatly blames the originator of the change• Long duration test suites can be put together in a regression task



33/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

CI EnvironmentJenkins

34/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Keeping an eye on quality

• Most CI systems have desktop or browser plugins• Screens on the wall can be an option as well• The quality of the final product starts with quality of your code

35/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Measuring code quality

• Cover can be integrated with eunit and Common Test to show you the testcoverage

• Dialyzer performs static type analysis and clear specs help a lot whentesting

• Sonar analyses your code looking for risksI Acts as an interface for cover and dialyzer resultsI Points you to the weakest parts of your sourceI Shows nice graphics about test coverage, unreachable code, nested

code. . . over the time

36/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A




• Sonar analyses your code looking for risks

I Acts as an interface for cover and dialyzer resultsI Points you to the weakest parts of your sourceI Shows nice graphics about test coverage, unreachable code, nested


36/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A




• Sonar analyses your code looking for risksI Acts as an interface for cover and dialyzer results

I Points you to the weakest parts of your sourceI Shows nice graphics about test coverage, unreachable code, nested


36/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A




• Sonar analyses your code looking for risksI Acts as an interface for cover and dialyzer resultsI Points you to the weakest parts of your source

I Shows nice graphics about test coverage, unreachable code, nestedcode. . . over the time

36/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A




• Sonar analyses your code looking for risksI Acts as an interface for cover and dialyzer resultsI Points you to the weakest parts of your sourceI Shows nice graphics about test coverage, unreachable code, nested


36/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Quality control with Sonar

37/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Quality control with Sonar

38/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Introduction

Testing Techniques

Making it happen

Q&A

39/40


Types of tests

Good Practices


Integration tests

Acceptance Tests

Performance Tests



Q&A

Thanks Everyone!

• Questions?• Comments?• Suggestions?

40/40

Testing an Erlang Backend

Technology

Transcript of Testing an Erlang Backend