Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL...
Transcript of Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL...
![Page 1: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/1.jpg)
Speed up execution ofDjango tests
Igor DavydenkoUA PyCon 2011
1
![Page 2: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/2.jpg)
I am...
• Python developer at oDesk Professional Services
• Like write tests, but not only in TDD manner
• github.com/playpauseandstop
• djangonaut.blogspot.com
• djangonaut.posterous.com
2
![Page 3: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/3.jpg)
Django tests
3
![Page 4: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/4.jpg)
What I’ve used?
• Django 1.3.1
• SQLite 3.7.5
• PostgreSQL 9.1
• MySQL 5.5.15
• Did not try to speed up tests on other database backends like Oracle or NoSQL solutions
4
![Page 5: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/5.jpg)
Test settings
• TEST_RUNNER
• DATABASES• TEST_NAME
• TEST_USER
• TEST_CHARSET
• TEST_COLLATION
• TEST_DEPENDECIES
• TEST_MIRROR
5
![Page 6: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/6.jpg)
Test settings$ python manage.py test --settings=test_settings app ...
test_settings.py
from settings import *
DATABASES = { ... }EMAIL_BACKEND = ‘django.core.mail.backends.locmem.EmailBackend’
if ‘south’ in INSTALLED_APPS: INSTALLED_APPS = list(INSTALLED_APPS) INSTALLED_APPS.remove(‘south’)
try: from local_test_settings import *except ImportError: pass
6
![Page 7: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/7.jpg)
Test project
• 2 test packages
• 34 test modules
• 621 tests
• near 11300 lines of code in test modules
7
![Page 8: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/8.jpg)
Test machine
• Intel Core 2 Duo E6300 1.86 GHz
• 4GB Ram
• 7200rpm HDD
• openSUSE 11.4 x86_64
• Python 2.7
8
![Page 9: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/9.jpg)
First results
Database TEST_NAME Time
SQLite :memory: 157.527s
SQLite test_sqlite3.db 169.026s
PostgreSQL test_postgresql 216.585s
MySQL test_mysql 218.406s
9
![Page 10: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/10.jpg)
First results
Note: In-memory SQLite database is fastest solution for running tests out of box
Time
0 55 110 165 220
218.406
216.585
169.026
157.527
SQLite :memory: SQLite PostgreSQL MySQL
10
![Page 11: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/11.jpg)
Speeding it up
11
![Page 12: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/12.jpg)
1st Solution
Obvious. Improve hardware
• From Intel Core 2 Duo E6300 1.86 GHz to Intel Core i5
• From 4GB 667 GHz to 4GB 1333 GHz DDR3
• From 7200rpm HDD to SSD
• Do not forget to upgrade software too :) Tests in Django 1.3 run a bit faster than in 1.2 and a lot faster than in 1.1
12
![Page 13: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/13.jpg)
New results
Database TEST_NAME Time
SQLite :memory: 118.932s
SQLite test_sqlite3.db 125.789s
PostgreSQL test_postgresql 165.814s
MySQL test_mysql 150.909s
13
![Page 14: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/14.jpg)
New results
Old Time
New Time
0 55 110 165 220
150.909
218.406
165.814
216.585
125.789
169.026
118.932
157.527
SQLite :memory: SQLite PostgreSQL MySQL
14
![Page 15: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/15.jpg)
2nd Solution
Run tests on internal machineJenkins continuous integration system
http://jenkins-ci.org/
# local machinegit push origin <branch>
# Jenkins machineruntests.sh # or you should config how to run tests after each commit
Connect Jenkins and Django with django-jenkins
https://github.com/kmmbvnr/django-jenkins
15
![Page 16: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/16.jpg)
2nd Solution
Run tests on internal machineLocal test server
Spend some money for new shiny hardware (Quad Core, SSD, etc)and run tests there with Jenkins or git hook or whatever else.
16
![Page 17: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/17.jpg)
3rd Solution
Obvious. Find for fat tests
• Do not forget that tests is just a Python code
• All tips for speeding up Django ORM can be affected in tests too
• https://docs.djangoproject.com/en/1.3/topics/db/optimization/
17
![Page 18: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/18.jpg)
4th Solution
PostgreSQL + RamFS (tmpfs)
• Mount a tmpfs
• Create second PostgreSQL instance on a different socket/port using this mount as data dir
• Tell Django to use it
• More info by click
18
![Page 19: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/19.jpg)
Results
Time
0 55 110 165 220
172.809
216.585
169.026
157.527
SQLite :memory: SQLite PostgreSQL PostgreSQL + RamFS MySQL
19
![Page 20: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/20.jpg)
4th Solution
MySQL + RamFS (tmpfs)
• Mount a tmpfs
• Create second MySQL instance on a different socket/port using this mount as data dir
• Tell Django to use it
• More info by click
20
![Page 21: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/21.jpg)
Results
Time
0 55 110 165 220
218.406
216.585
169.026
157.527
SQLite :memory: SQLite PostgreSQL MySQL MySQL + RamFS
21
![Page 22: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/22.jpg)
5th Solution
Multiprocessing
• pip install nose
• pip install django-nose
• Update test settings with:
TEST_RUNNER = ‘django_nose.NoseTestSuiteRunner’NOSE_ARGS = [‘--processes’, ‘2’] # For dual-core CPU
22
![Page 23: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/23.jpg)
New results
Database TEST_NAME Time
SQLite :memory: 94.333s
SQLite test_sqlite3.db 151.588s
PostgreSQL test_postgresql 6.296s
MySQL test_mysql 127.044s
23
![Page 24: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/24.jpg)
New results
Old Time
New Time
0 75 150 225 300
127.044
218.406
6.296
216.585
151.588
169.026
94.333
157.527
SQLite :memory: SQLite PostgreSQL MySQL
24
![Page 25: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/25.jpg)
New problems
• It’s totally mess with data, you don’t know how things going on next test run
• “Database is locked” errors
• Needs only multi-core CPU
• Official BEWARE from nose documentation
25
![Page 26: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/26.jpg)
6th Solution
Mix it up!
• The most obvious choice is to continue use SQLite in-memory database, but with multiprocessing
• Or, really, buy new MacBook Pro (that would be introduced on next week :) )
26
![Page 27: Speed up execution of Django testsWhat I’ve used? • Django 1.3.1 • SQLite 3.7.5 • PostgreSQL 9.1 • MySQL 5.5.15 • Did not try to speed up tests on other database backends](https://reader033.fdocuments.in/reader033/viewer/2022042400/5f0ece947e708231d44108bc/html5/thumbnails/27.jpg)
Questions?
27