Faster Python Code in 5 minutes

Anoop Thomas MathewProfoundis Labs

@atmb4u31st August, 2013PyCon India 2013

➢@atmb4u➢ Co-Founder and CTO @ Profoundis Labs➢ Avid user of open source since 2003➢ Python and Django for last 4 years

AB

OU

T M

E

Quick Profiling●%timeit and %prun in ipython●Try line_profiler● Not premature optimization

TIP

1

use Iterators as possible

●Iterators and Generators are memory efficient and scalable●use itertools as much as possible

TIP

2

list comprehension > for loop >

while loop

consider numpy and tuples for huge data

TIP

3

xrange instead of rangeC version of range, designed to work with int

TIP

4

map is a wonderful idea,

so is reduce and filtermap(function, data)

TIP

5

Function calls are costly

●pass entire list to a function than iterating on each element and calling the function●Recursive loops are dangerous with quadratic complexity

TIP

6

Local variables are faster

they are faster than global variables, builtins and attribute lookups

TIP

7

delete the unnecessary

●use proper namespacing ●weakref is a good idea●use gc.collect() to expicitly collect the unused variables

TIP

8

Threads for I/Octypes for CPU(GIL)

●use threads for I/O bound processes. ●For CPU bound with GIL lock, use gevent or C extensions to override it

TIP

9

cProfilecStringIOcPickle ...

use C versions of library. They can be 100x faster.

TIP

10

Standard Library &

builtin functions are always faster

TIP

11

use try than if; given mostly True(exeptions exists: builtin functions are always faster)

NB: Bad practice!

TIP

12

immutable types when possible

TIP

13

●''.join('<string>') is faster than +●insert in a list is O(n)●Faster lookup in dict, set●deque for double sided operations

TIP

14

Slides @http://www.slideshare.net/atmb4u/faster-python

More Infohttp://infiniteloop.in/blog/quick-python-performance-optimization-part-i/http://infiniteloop.in/blog/quick-python-performance-optimization-part-ii/http://wiki.python.org/moin/PythonSpeed/PerformanceTipshttp://wiki.python.org/moin/PythonSpeed

MO

RE

INFO

TH

AN

K Y

OU

on twitter: @atmb4u