Testing with Style - Andreas Neumann

Senior Software Engineer Search

‹Nr.›

CONTENTS / ASSERTIONS

‹Nr.›

Aims

▪ Tests are code that test other code ▪ Tests prove or falsify certain assumptions ▪ general structure for tests ▪ Readability is important => doubles as documentation ▪ Examples two main Scala Testing Frameworks: Specs2 ,

ScalaTest

‹Nr.›

Contents

▪ Structuring your project : A project Blueprint ▪ Run tests : I have written the code, what now ? ▪ Testing styles : So many to chose from ▪ Test Results : Get and interprete results

‹Nr.›

PROJECT STRUCTURE

‹Nr.›

Project Structure

▪ Scala/JavaProject:

▪ /src/test ▪ /src/it

▪ Subfolders:

▪ scala : Tests written in Scala ▪ java : Tests written in Java ▪ resources: Files needed for

testing ▪ these folders are not reachable out of main but can use

everything in main ▪ will normally not be included in Artifacts

‹Nr.›

Structuring Tests

▪ tests can be structured in packages ▪ tests should mimic the main package ▪ to allow for portability there should be 1 .. n test files for 1

implementation file

‹Nr.›

RUNNING YOUR TESTS

‹Nr.›

Running Tests

▪ Tests can be run in many different ways

▪ we will look at

▪ CI ▪ IDE ▪ sbt

‹Nr.›

CI

▪ CI does that for you ▪ runs complete set of tests ▪ run by git push ▪ run by hand ▪ for all Devs

‹Nr.›

IDE

▪ built in support in every major IDE ▪ often through JUnit Integration ▪ can run single files or suites ▪ tad slow, no looping

‹Nr.›

SBT

▪ Run all tests : test ▪ Run all tests in a specific project: <PROJECT>/test ▪ Run only one specific test: testOnly

▪ Run only tests failed before: test-quick ▪ Looping: ~

activator testing_styles/test Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 [info] Loading global plugins from /Users/anneumann/.sbt/0.13/plugins [info] Loading project definition from /Users/anneumann/test-wars/project [info] Set current project to test-wars (in build file:/Users/anneumann/test-wars/) [info] PersonFeatureSpec: [info] As a Developer [info] I want to have a Person [info] which can be part of the Test Wars Universe [info][info] Total for specification PersonSpecMutable [info] Finished in 56 ms [info] 7 examples, 0 failure, 0 error [info] [info] ScalaTest [info] Run completed in 1 second, 404 milliseconds. [info] Total number of tests run: 14 [info] Suites: completed 3, aborted 0 [info] Tests: succeeded 14, failed 0, canceled 0, ignored 0, pending 0 [info] All tests passed. [info] Passed: Total 36, Failed 0, Errors 0, Passed 36, Pending 3 [success] Total time: 2 s, completed Apr 15, 2015 1:03:14 PM

‹Nr.›

IDE with SBT and CI

‹Nr.›

DIFFERENT WAYS OF WRITING YOUR TESTS - BY EXAMPLES

‹Nr.›

What we want to test

Person can be created without exception

Person default values is Neutral by default is no Jedi by default has default aiming prob of 50%

Person can change sides ad libitum to the empire to Rebels

‹Nr.›

Classic : JUnit like

▪ Looks like normal code

▪ Workhorses ▪ Scalatest : FunSuite ▪ test(„description") { … }

▪ assert( cond => Boolean )

▪ pro: nothing new to learn, just functions

▪ con: looks like code, can get messy

class PersonTestClassic extends FunSuite { test("Person can be created without exception") { assert( Person("Nobody").isInstanceOf[Person] == true ) } test("Person is Neutral by default") { assert( person.side == Neutral ) } test("Person is no Jedi by default") { assert( person.isJedi == false ) } test("Person has default aiming prob of 50%") { assert( person.aim == Probability(0.5) ) } test("Person can change sides ad libitum to the Empire") { val anakin = Person("Anakin", isJedi = true) anakin.side = Empire assert( anakin.side == Empire ) } test("Person can change sides ad libitum to the Rebels") { val han = Person("Han Solo") assert( han.side == Neutral ) han.side = Rebels assert( han.side == Rebels ) } def person = Person("Honk") }

‹Nr.›

QA loves this - FeatureSpec

▪ Focus on description

▪ Workhorses ▪ FeatureSpec ▪ Given - When - Then

▪ pro: Focus on Functionality / Requirements

▪ con:,mixes description and code, hard to debug, ordering

class PersonFeatureSpec extends FeatureSpec with GivenWhenThen { info("As a Developer") info("I want to have a Person") info("which can be part of the Test Wars Universe") info("which has defaults and can change sides") feature("Person") { scenario("Creating a Person gives Person with defaults") { Given("a Person created with just the name") val person = new Person("Honk") Then("the person should have defaults") assert(person.isJedi == false) assert(person.aim == Probability(0.5)) } } feature("Keep it interesting") { scenario("A person can change sides") { Given("Han Solo") val han = new Person("Han Solo") assert(han.side == Neutral) When("han sees the good in the Rebellion and befriends Luke he changes sides") han.side = Rebels Then("Han is part of the Rebellion") assert(han.side == Rebels) } }}

‹Nr.›

A little DSL - FlatSpec with Matchers

▪ More like natural language

▪ Workhorses ▪ Scalatest FlatSpec ▪ ShouldMatchers : DSL

▪ pro: more natural, can be understood by non programmers,not boolean centric

▪ con: What is tested hidden by code

class PersonFlatTest extends FlatSpec with ShouldMatchers { "A Person" should "be created without exception" in { Person("Nobody") shouldBe a [Person] } it should "be Neutral by default" in { person.side should be(Neutral) } it should "be no Jedi by default" in { person.isJedi should be(false) } it should ("have default aiming prob of 50%") in { person.aim should be( Probability(0.5) ) } "Person can change sides ad libitum " should "to the Empire" in { val anakin = Person("Anakin", isJedi = true) anakin.side = Empire anakin.side should be( Empire ) } it should "to the Rebels" in { val han = Person("Han Solo") han.side should be( Neutral ) han.side = Rebels han.side should be ( Rebels ) } def person = Person("Honk") }

‹Nr.›

Another little DSL: Specs2 mutable

▪ like natural language

▪ Workhorses ▪ mutable.Spec ▪ specs2 DSL

▪ pro: natural, can be understood by non programmers, not boolean centric

▪ con: code can hide tests

class PersonSpecMutable extends Specification { "A Person" should { "be created without exception" in { Person("Nobody") must not throwA(new Exception) } } "Person default values" should { "be Neutral by default" in new TestPerson { side mustEqual Neutral } "be no Jedi by default" in new TestPerson { isJedi must beFalse } "have aiming prob of 50% " in new TestPerson { aim mustEqual Probability(0.5) } "have default chance of evading 50%" in new TestPerson { evade mustEqual Probability(0.5) } } "Person can change sides ad libitum" should { "to the empire" in { val anakin = Person("Anakin") anakin.side mustEqual Neutral anakin.side = Empire anakin.side mustEqual Empire } "to Rebels" in { val han = Person("Han Solo") han.side mustEqual Neutral han.side = Rebels han.side mustEqual Rebels } } class TestPerson(name: String = "Honk") extends Person(name) with Scope}

‹Nr.›

Functional Acceptance Style

▪ personal favorite

▪ Worhorses: ▪ Specs2 Specification

(immutable) ▪ String interpolation ▪ DSL with Matchers

▪ pro: clear distinction requirements/description code, functional, readable by no coders ( upper Part )

▪ cons: Learning curve

class PersonSpec extends Specification {def is = s2"""Person can be created without exception $createPerson default values is Neutral by default $defaultSide is no Jedi by default $isTheForceWithHim has default aiming prob of 50% $defaultAim has default chance of evading 50% $defaultEvadePerson can change sides ad libitum to the empire $becomeEvil to Rebels $becomeRebel""" def create = Person("Nobody") must not throwA(new Exception) def defaultSide = person.side mustEqual Neutral def isTheForceWithHim = person.isJedi must beFalse def defaultAim = person.aim mustEqual Probability(0.5) def defaultEvade = person.evade mustEqual Probability(0.5) def becomeEvil = { val anakin = Person("Anakin") anakin.side.mustEqual(Neutral).and { anakin.side = Empire anakin.side mustEqual Empire } } def becomeRebel = { val anakin = Person("Han Solo") anakin.side.mustEqual(Neutral).and { anakin.side = Rebels anakin.side mustEqual Rebels } } def person = Person("Honk") }

‹Nr.›

TEST RESULTS

‹Nr.›

Test-Results

▪ Test Results need to be readable by humans ▪ Tests Results need to be machine readable !

▪ we will look at ▪ Terminal ▪ HTML ▪ JUnit-XML

‹Nr.›

Test Results: Terminal

▪ for humans ▪ some color support

▪ green, yellow, blue, red

▪ result at the end

> testOnly SpaceShipSpec [info] Passed: Total 0, Failed 0, Errors 0, Passed 0 [info] No tests to run for test-wars/test:testOnly [info] ScalaTest [info] Run completed in 12 milliseconds. [info] Total number of tests run: 0 [info] Suites: completed 0, aborted 0 [info] Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0 [info] No tests were executed. [info] Passed: Total 0, Failed 0, Errors 0, Passed 0 [info] No tests to run for universe/test:testOnly [info] SpaceShipSpec [info] A spaceship [info] + has shield [info] + has attack power [info] + has a aim, which defaults to 50% accuracy [info] + has a chance to evade, which defaults to 50% [info] + belongs to a side which by default is Neutral [info] [info] Spaceship Shield [info] + a ship with shield left is ok [info] + a ship with shield below 0 is broken [info] + ship armor can be changed which affects the isOK state [info] [info] Spaceship Battle [info] * a ship can engage another ship will not end in an endless loop PENDING [info] * it will not engage if it is not Ok PENDING [info] * after being engaged by another ship it will engage the other ship [info] it will engage the other ship till one ship is no longer ok PENDING [info] [info] Total for specification SpaceShipSpec [info] Finished in 46 ms [info] 11 examples, 0 failure, 0 error, 3 pending [info]

‹Nr.›

Test Results: HTML

▪ depends on Testing Framework ▪ may need A LOT OF project configuration ▪ can also be used to create documentation ▪ console-output needs to be readded

//HTMLOutput(testOptions in Test) ++= Seq( Tests.Argument(TestFrameworks.Specs2, "html"), Tests.Argument(TestFrameworks.ScalaTest, "-h", "target/scalatest/html") )

‹Nr.›

Tests Results: Machine Readable

▪ machine readable ▪ used by many CIs

<?xml version="1.0" encoding="UTF-8" ?><testsuite errors="0" failures="0" hostname="mb0141417118.local" name="person.PersonFlatTest" tests="6" time="0.059" timestamp="2015-04-15T13:19:05"> <properties> <property name="jline.esc.timeout" value="0"> </property> <property name="java.runtime.name" value="Java(TM) SE Runtime Environment"> </property> <property name="sun.boot.library.path" value="/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib"> </property> <property name="java.vm.version" value="25.25-b02"> </property> <property name="user.country.format" value="DE"> </property> <property name="gopherProxySet" value="false"> </property> <property name="java.vm.vendor" value="Oracle Corporation"> </property> <property name="java.vendor.url" value="http://java.oracle.com/"> </property> <property name="path.separator" value=":"> </property> <property name="java.vm.name" value="Java HotSpot(TM) 64-Bit Server VM"> </property> <property name="file.encoding.pkg" value="sun.io"> </property> <property name="user.country" value="US"> </property> <property name="sun.java.launcher" value="SUN_STANDARD"> </property>

‹Nr.›

Thank YOU for YOUR participation

code can be found at:

https://github.com/daandi/test-wars/

‹Nr.›

More to come, let me know what you want to hear about:

▪ (Mocking, Stubbing and Stabbing) ▪ Tests composability and inheritence ▪ Test Factories ▪ How do I test xy ( JSON, Futures, Web) ▪ Stubs, Mocks and Fixtures * ▪ Integration ▪ Polyglot testing ▪ Write testable code * ▪ Property based Testing (you won’t get away :)

‹Nr.›

TRIVIA - How is that connected to the talk ?