What is doctest?Python testing frameworks overview

1 / 26

What am I doing here?

2 / 26

What am I doing here?Facts

Jachym CepickyGIS (maps in computer)Open source software

Not facts

not a testernot educated software engineernor GIS educated

3 / 26

What am I doing here?

Me: confused py.test with doctest

4 / 26

Let's get started

5 / 26

Testing motivation

6 / 26

Everybody loves to tests.

7 / 26

Everybody has heard, that using TDD is the only way how to writecode loves to tests.

8 / 26

TDD

9 / 26

The fact

Writing tests

...

does make sense

10 / 26

Summary

Making sure, that the system

meets the requirements that guided its design and development,responds correctly to all kinds of inputs,performs its functions within an acceptable time,is sufficiently usable,can be installed and run in its intended environments, andachieves the general result its stakeholders desire.

https://en.wikipedia.org/wiki/Software_testing

11 / 26

Testing in Python - overview

12 / 26

It's pretty cool and easy to test in Python

13 / 26

Unittest

https://docs.python.org/3/library/unittest.htmloriginally inspired by JUnit (inspired by SUnit for Smalltalk, since 1998)https://shebanator.com/2007/08/21/a-brief-history-of-test-frameworks/

Pros

Native frameworkTesting methodsSupported everywhereAuto discovery of test classes and functions

Cons

ClassesTesting methodsComplicated (for some people)Lack of Group fixtures (specified environment for a whole group)Still better to separate testing code from business logicUnit-test oriented

14 / 26

import unittest

class TestStringMethods(unittest.TestCase):

def test_upper(self): self.assertEqual('foo'.upper(), 'FOO')

def test_isupper(self): self.assertTrue('FOO'.isupper()) self.assertFalse('Foo'.isupper())

def test_split(self): s = 'hello world' self.assertEqual(s.split(), ['hello', 'world']) # check that s.split fails when the separator is not a string with self.assertRaises(TypeError): s.split(2)

if __name__ == '__main__': unittest.main()

15 / 26

$ python my_unittest.py .F.======================================================================FAIL: test_split (__main__.TestStringMethods)----------------------------------------------------------------------Traceback (most recent call last): File "neco.py", line 14, in test_split self.assertEqual(s.split(), ['hellox', 'world'])AssertionError: Lists differ: ['hello', 'world'] != ['hellox', 'world']

First differing element 0:'hello''hellox'

- ['hello', 'world']+ ['hellox', 'world']? +

----------------------------------------------------------------------Ran 3 tests in 0.001s

FAILED (failures=1)

16 / 26

py.test

http://doc.pytest.org/en/latest/

Pros

PluginsHTML outputSimpler and more straight-forward syntaxAuto discovery of test functionsDistributed

Cons

Not nativeCan get complicated too (decorators)

17 / 26

def inc(x): return x + 1

def test_answer(): assert inc(3) == 5

18 / 26

$ python3 -m pytest my_pytest.py============================================ test session starts ============================================platform linux -- Python 3.5.2+, pytest-3.0.6, py-1.4.32, pluggy-0.4.0rootdir: /tmp, inifile: collected 2 items

neco.py F.

================================================= FAILURES ==================================================________________________________________________ test_answer ________________________________________________

def test_answer():> assert inc(3) == 5E assert 4 == 5E + where 4 = inc(3)

neco.py:5: AssertionError==================================== 1 failed, 1 passed in 0.03 seconds =====================================

19 / 26

Doctest

https://docs.python.org/3/library/doctest.html

Pros

NativeTests are (can be) part of class and method documentationCan be in separated text (e.g. README) filesSupported by Sphinx

Cons

Not suitable for larger testing systemsNo fixtures

20 / 26

def unique_words(page): ''' Returns set of the unique words in list of lines of text

Example:

>>> from StringIO import StringIO >>> fileText = """the cat sat on the mat ... the mat was ondur the cat ... one fish two fish red fish ... blue fish ... This fish has a yellow car ... This fish has a yellow star""" >>> file = StringIO(fileText) >>> page = file.readlines() >>> words = unique_words(page) >>> print sorted(list(words)) ["This", "a", "blue", "car", "cat", "fish", "has", "mat", "on", "ondur", "one", "red", "sat", "star", "the", "two", "was", "yellow"] >>> '''

return set(word for line in page for word in line.split())

def _test(): import doctest doctest.testmod()

if __name__ == "__main__": _test()

21 / 26

$ python2 my_doctest.py **********************************************************************File "neco.py", line 16, in __main__.unique_wordsFailed example: print sorted(list(words))Expected: ["This", "a", "blue", "car", "cat", "fish", "has", "mat", "on", "ondur", "one", "red", "sat", "star", "the", "two", "was", "yellow"]Got: ['This', 'a', 'blue', 'car', 'cat', 'fish', 'has', 'mat', 'on', 'ondur', 'one', 'red', 'sat', 'star', 'the', 'two', 'was', 'yellow']**********************************************************************1 items had failures: 1 of 6 in __main__.unique_words***Test Failed*** 1 failures.

22 / 26

nose (nose2)

nose extends unittest to make testing easiersmart developer should get familiar doctest, unittest, pytest, and nosehttps://pypi.python.org/pypi/nose/ --> new projects should use py.test ornose2https://github.com/nose-devs/nose2

Pros

Build on top of unittestsOverpasses some of it's limitsCommand line tool

Cons

nose not maintained any more --> nose2Not nativeUnder development (nose2)

23 / 26

Python unittests overview

24 / 26

ConclusionDoctests != py.testDoctests are the natural way, how to test your code and how to documentitDoctests should be used for documentation, not for testingpy.test seems to be current leading testing frameworkEvent /me knows, what is the difference between doctests and py.test

25 / 26

That's all folks

#jachymcjachym.cepicky at opengeolabs dot cz

26 / 26