Post on 09-Feb-2022
QuickCheckClojureCheckScalaCheck
Dave Gurnell
Property-based testing with...
Testing(it’s good ...but it’s time consuming)
cost(maintenance) > cost(development)
test("reverse an empty list") { assert(List().reverse === List())}
test("reverse a list of length 2") { assert(List("a", "b").reverse === List("b", "a"))}
test("reverse a list with a null in it") { assert(List("a", null).reverse === List(null, "a"))}
test("reverse a palindrome (?!)") { assert("TACOCAT".toList.reverse === "TACOCAT".toList)}
Lots of tests = lots of maintenance...
How can we get similar coveragewith less code?
Properties(aka “Invariants”)
reverse(a ++ b) == reverse(b) ++ reverse(a)
reverse(reverse(a)) == a
endpoint(request).status != 500
i > 0 ==> i/2 < i
∀a,b reverse(a ++ b) == reverse(b) ++ reverse(a)
∀a reverse(reverse(a)) == a
∀request endpoint(request).status != 500
i > 0 ==> i/2 < i∀i
Problem - these are all universally quantified...
We can’t test all possible inputs to our properties.
Can we find a representative sample instead?
Generators(define inputs to properties)
import org.scalacheck.Gen
val ints = Gen.oneOf(1, 2, 3) // Gen[Int]
ints.sample // => Some(1)ints.sample // => Some(3)ints.sample // => Some(2)
Generators samples values from large datasets
import org.scalacheck.Gen
val lists = Gen.listOf(Gen.oneOf(1, 2, 3))
lists.sample // => Some(List(1, 3, 2, ... lists.sample // => Some(List())lists.sample // => Some(List(2, 3, 3, ...
Generators samples values from large datasets
import org.scalacheck.Genimport org.scalacheck.Prop.forAll
val lists = Gen.listOf(Gen.oneOf(1, 2, 3))
val prop = forAll(lists) { list => list.reverse.reverse == list}
Use generators to approximate universal quantification
import org.scalacheck.Genimport org.scalacheck.Prop.forAll
val lists = Gen.listOf(Gen.oneOf(1, 2, 3))
val prop = forAll(lists) { list => list.reverse.reverse == list}
prop.check // stdout: OK, passed 100 tests.
Use generators to approximate universal quantification
import org.scalacheck.Genimport org.scalacheck.Prop.forAll
val lists = Gen.listOf(Gen.oneOf(1, 2, 3))
val prop = forAll(lists, lists) { (a, b) => (a ++ b).reverse == b.reverse ++ a.reverse}
prop.check // stdout: OK, passed 100 tests.
Use generators to approximate universal quantification
import org.scalacheck.Genimport org.scalacheck.Prop.forAll
val lists = Gen.listOf(Gen.oneOf(1, 2, 3))
val prop = forAll(lists, lists) { (a, b) => (a ++ b).reverse == a.reverse ++ b.reverse}
prop.check // stdout: Falsified after 2 tests. // ARG_0: List(0) // ARG_1: List(1)
Use generators to approximate universal quantification
“Arbitraries”(materialize generators from types)
import org.scalacheck.Arbitrary.arbitrary
val ints = arbitrary[Int] // Gen[Int]
ints.sample // => Some(2147483647)ints.sample // => Some(-1)
val lists = arbitrary[List[Int]] // Gen[List[Int]]
lists.sample // => Some(List(2120334425, 0, 0, ...lists.sample // => Some(List(-1639755040, 1, ...
import org.scalacheck.Prop.forAll
forAll { (a: List[Int]) => a.reverse.reverse == a}
forAll { (a: List[Int], b: List[Int]) => (a ++ b).reverse == b.reverse ++ a.reverse}
forAll can grab Arbitraries implicitly
But wait, there’s more...
Automatic test case reductionDSL for testing side-effects
Generator and property combinators
See the ScalaCheck web site for info!
References
Haskell QuickCheckhttp://fsl.cs.illinois.edu/images/3/3d/QuickL.pdf
http://www.haskell.org/haskellwiki/Introduction_to_QuickCheck1
ScalaCheckhttps://github.com/rickynils/scalacheck
Java QuickCheckhttps://bitbucket.org/blob79/quickcheck
ClojureCheckhttps://github.com/jbondeson/clojurecheck