Introduction to Python - michael-kraus.orgmichael-kraus.org/teaching/python_w11/Python_IPP_1.pdf ·...
Transcript of Introduction to Python - michael-kraus.orgmichael-kraus.org/teaching/python_w11/Python_IPP_1.pdf ·...
Introduction to Python
Part 1: The Python Programming Language and the Cython Extension
Michael Kraus ([email protected])
Max-Planck-Institut fur Plasmaphysik, Garching
27. February 2012
About Python
high-level, general-purpose, interpreted
multi-paradigm programming language (structural, object-oriented,functional, meta programming)
open source (interpreter, libraries, IDEs)
portable (available on Linux, Mac, Windows, ...)
powerful data structures and practical shortcuts
large and comprehensive standard library
lots of freely available third-party modules
extendable with C/C++, Fortran, Javaembed code (weave, f2py)wrap libraries (SWIG, ctypes)wrappers for common libraries (Qt, GTK, VTK, FFTW, MPI, ...)
Cython: extension of the Python languageprovides static typed functions and variablesgenerates efficient C code for fast computations
IDEs: IDLE, PyDev Plugin for Eclipse
Python and Scientific Computing
numpy: numerical array objects and routines to manipulate them
scipy: high-level data processing routines (regression, interpolation, ...)
matplotlib: 2D visualisation (LaTeX support, publication-ready plots)
mayavi: 3D visualisation
netcdf/hdf5: efficient data storage for scientific applications
cython: fast computations, shared-memory parallelisation
� glue between existing components
couple already-written codesadd new functionality to your codes (e.g. a GUI),using the plethora of available python modules
� powerful but simple reading and writing of data in different file formats
� analysis and visualisation of data (interactive, automated)
References
Practical Programming: An Introduction to ComputerScience Using Python (2009)Jennifer Campbell, Paul Gries, Jason Montojo, Greg Wilson
A Primer on Scientific Programming with Python (2011)Hans Petter Langtangen
Python Essential Reference (2009)David Beazley
References
Think Python: How to Think Like a Computer Scientisthttp://greenteapress.com/thinkpython/thinkpython.html
Python Scientific Lecture Noteshttp://scipy-lectures.github.com/
Software Carpentryhttp://www.software-carpentry.org/
Google’s Python Classhttp://code.google.com/edu/languages/
google-python-class/
Pythonhttp://www.python.org/
Cythonhttp://www.cython.org/
Python on the Linux Cluster
default: Python 2.6 (most packages outdated)
optional: Python 2.7 (regular updates, compiled with Intel)
module load python27/python
module load python27/nose
module load python27/numpy
module load python27/scipy
module load python27/cython
module load python27/ipython
module load python27/pyqt
module load python27/matplotlib
module load python27/h5py
module load python27/netcdf
prerequisites:
module load intel /12.1
module load mkl /10.3
Outline
Python
interpreter and shell
basic language features
data types and containers
control flow
functions
scripts and modules
input/output
exceptions
object oriented programming
Cython
compilation
static types
parallelisation
Python Basics: Interpreter and Shell
interpreted (as opposed to compiled) language
� Python code is not compiled prior to execution
interactive mode (advanced calculator)
� type in Python code directly into command line shell
� gives immediate feedback for each statement
> python
Python 2.7.2 (default , Dec 15 2011, 14:57:19)
[GCC Intel(R) C++ gcc 4.3 mode] on linux2
Type "help", "copyright", "credits" or "license" for ...
>>> print("Hello World!")
Hello World!
IPython: enhanced Interactive Python
� enhanced Python shell providing a rich toolkit
� improved interactive data visualisation (pylab)
> ipython -pylab
Python Basics: Interpreter and Shell
� quick help: help(function), help(module), dir(module), dir(variable)
>>> import math
>>> help(math)
help on module math:
[...]
DESCRIPTION
This module is always available. It provides access to
the mathematical functions defined by the C standard.
[...]
run a Python script in the interpreter:
> python script.py
run a Python script and enter interactive mode afterwards:
> python -i script.py
Python Basics: Indentation
blocks of code are delimited by indentation levels
� instruction under a flow-control keyword (if, else, while, for) or in thebody of a function must be indented
you can use as many spaces/tabs as you want (but consistently!)
guarantees some structure of the program and allows to visualise thisstructure quite easily
� correct:
>>> letters = [’a’,’b’,’c’]
>>> for s in letters:
>>> print(s)
� wrong (will throw an exception, IndentationError: unexpected indent):
>>> letters = [’a’,’b’,’c’]
>>> for s in letters:
>>> print(s)
>>> letters = [’a’,’b’,’c’]
>>> for s in letters:
>>> print(s)
Python Basics: Types
dynamic typing: an object’s type can change during runtime
>>> a = ’Hello ’
>>> type(a)
<type ’str’>
>>> a = 3
>>> type(a)
<type ’int’>
scalar data types: integer, float (64bit), complex (native!), boolean:
>>> b = 3.
>>> type(b)
<type ’float ’>
>>> c = 1. + 0.5j
>>> type(c)
<type ’complex ’>
>>> d = (3 > 4)
>>> type(d)
<type ’bool’>
>>> print(a)
3
>>> print(b)
3.0
>>> print(c.real)
1.0
>>> print(c.imag)
0.5
>>> print(d)
False
Python Basics: Containers
lists: ordered collection of objects, that may have different types
>>> l = [1, 2, 3, 4, 5]
>>> type(l)
<type ’list’>
� indexing: access individual objects contained in the list (starting at 0)
>>> l[2]
3
� counting from the end with negative indices:
>>> l[-1]
5
� slicing (sublists): l[start:stop:stride] (note: l[start:stop] containsthe elements with index i such that start <= i < stop)
>>> l[2:4]
[3, 4]
>>> l[3:]
[4, 5]
>>> l[::2]
[1, 3, 5]
Python Basics: Containers
� append elements:
>>> l.append (6)
>>> l
[1, 2, 3, 4, 5, 6]
� extent by another list:
>>> l.extend ([7 ,8])
>>> l
[1, 2, 3, 4, 5, 6, 7, 8]
� reverse order:
>>> l.reverse ()
>>> l
[8, 7, 6, 5, 4, 3, 2, 1]
� sort:
>>> l.sort()
>>> l
[1, 2, 3, 4, 5, 6, 7, 8]
Python Basics: Containers
� pop last element:
>>> l.pop()
8
>>> l
[1, 2, 3, 4, 5, 6, 7]
� remove some element:
>>> l.remove (4)
>>> l
[1, 2, 3, 5, 6, 7]
� number of elements:
>>> len(l)
7
� elements can be of different type:
>>> l.append(’s’)
>>> l
[1, 2, 3, 5, 6, 7, ’s’]
Python Basics: Containers
tuples: immutable lists (lets you create simple data structures)
� written between brackets, e.g. u = (x,y):
>>> u = (0,2)
>>> u
(0, 2)
or just separated by commas:
>>> t = 12345, 67890, ’hello ’, u
>>> t
(12345 , 67890 , ’hello’, (0, 2))
� tuples are unpacked using list syntax:
>>> u[1]
2
� elements of a tuple cannot be assigned independently:
>>> u[1] = 3
TypeError: ’tuple’ object does not support item assignment
Python Basics: Containers
sets: unordered collection with no duplicate elements
>>> basket = [’apple ’, ’orange ’, ’apple ’, \
’pear’, ’orange ’, ’banana ’]
>>> fruit = set(basket)
>>> fruit
set([’orange ’, ’pear’, ’apple’, ’banana ’])
� eliminates duplicate entries
� membership testing:
>>> ’orange ’ in fruit
True
>>> ’melon ’ in fruit
False
Python Basics: Containers
� create sets of characters:
>>> s = set(’abcd’)
>>> s
set([’a’, ’c’, ’b’, ’d’])
>>> t = set((’e’, ’f’, ’g’))
>>> t
set([’e’, ’g’, ’f’])
� add elements:
>>> s.add(’a’)
>>> s
set([’a’, ’c’, ’b’, ’d’])
>>> t.update ([’a’, ’b’])
>>> t
set([’a’, ’b’, ’e’, ’g’, ’f’])
� remove an element:
>>> t.remove(’g’)
>>> t
set([’a’, ’b’, ’e’, ’f’])
Python Basics: Containers
� support for mathematical operations like union, intersection, difference,and symmetric difference
� letters in s but not in t:
>>> s-t
set([’c’, ’d’])
� letters in either s or t:
>>> s | t
set([’a’, ’c’, ’b’, ’e’, ’d’, ’f’])
� letters in both s and t:
>>> s & t
set([’a’, ’b’])
� letters in t or s but not both:
>>> s ^ t
set([’c’, ’e’, ’d’, ’f’])
Python Basics: Containers
dictionaries: hash table that maps keys to values (unordered)
>>> tel = { ’peter ’: 5752, ’mary’: 5578 }
>>> tel[’jane’] = 5915
>>> tel
{’jane’: 5915, ’peter’: 5752, ’mary’: 5578}
>>> tel[’mary’]
5578
>>> tel.keys()
[’jane’, ’peter ’, ’mary’]
>>> tel.values ()
[5915, 5752, 5578]
>>> ’peter ’ in tel
True
� dictionaries can have keys and values with different types:
>>> d = { ’a’: 1, ’b’: 2, 3:’hello ’ }
>>> d
{’a’: 1, 3: ’hello’, ’b’: 2}
Python Basics: Control Flow
if/elif/else:
>>> if 2**2 == 4:
... print(’Obvious!’)
...
Obvious!
� no brackets enclosing the conditional statement
>>> a = 10
>>> if a == 1:
... print(’one’)
... elif a == 2:
... print(’two’)
... else:
... print(’a lot’)
...
a lot
Python Basics: Control Flow
for/range
>>> for i in range (3):
... print(i)
...
0
1
2
� for always loops through a sequence of elements� range([start,] stop[, step]) creates lists of arithmetic progressions� the arguments must be plain integers� the start argument defaults to 0, the step argument defaults to 1
while
>>> z = 1. + 1.j
>>> while abs(z) < 100:
... z = z**2 + 1.
...
>>> z
( -134+352j)
Python Basics: Control Flow
break
>>> z = 1. + 1.j
>>> while abs(z) < 100:
... if abs(z.imag) > 10:
... break
... z = z**2 + 1.
...
>>> z
(-11-16j)
continue
>>> a = [1, 0, 2]
>>> for element in a:
... if element == 0:
... continue
... print 1. / element
...
1.0
0.5
Python Basics: Control Flow
iterate over any sequence (list, set, dictionary, string, file, ...)
>>> for word in (’cool’, ’powerful ’, ’readable ’):
... print(’Python is %s’ % word)
...
Python is cool
Python is powerful
Python is readable
iterate over a dictionary:
>>> d = {’a’:1, ’b’: 2, ’c’: 1j }
>>> for key , val in d.iteritems ():
... print(’Key: %s has value: %s’ % (key ,val) )
...
Key: a has value: 1
Key: c has value: 1j
Key: b has value: 2
� for key in d: only gives you the keys, not the values(but of course, values can be accessed via d[key] within the loop)
Python Basics: Functions
defining functions with def:
>>> def test ():
... print(’This is a test!’)
...
>>> test()
This is a test!
passing arguments and returning a value:
>>> def disk_area(radius ):
... return 3.14 * radius * radius
...
>>> disk_area (1.5)
7.0649999999999995
� in the arguments, Python passes references to objects by value
� all values that the function should return... must be returned
� by default, functions return None
Python Basics: Functions
in Python everything is an object, variables hold references to objects� every variable assignment is an assignment of a reference
a = 2
h = [1,2,3]
� the value of a is a reference to an object of type int
� the value of h is a reference to an object of type list
Python passes references to objects by value� assigning a new instance to a variable (i.e. a reference to a new object)
within a function does not change the original variable outside thefunction
>>> def some_function(s):
... s = set(’xyz’)
...
>>> u = set(’abcd’)
>>> some_function(u)
>>> print(u)
set([’a’, ’c’, ’b’, ’d’])
Python Basics: Functions
immutable objects: numbers, strings, tuples, booleans
>>> def square(x):
... print(id(x))
... x *= x
.... print(id(x))
... return x
...
>>> a = 2
>>> b = square(a)
4312871488
4312871440
>>> print(a)
2
>>> print(b)
4
� altering immutable objects inside a function creates a new instance
� the original instance outside the function is not changed
Python Basics: Functions
mutable objects: lists, sets, dictionaries, classes
>>> def set_first_element_to_zero(l):
... l[0] = 0
...
>>> c = [1,2,3]
>>> print(c)
[1, 2, 3]
>>> set_first_element_to_zero(c)
>>> print(c)
[0, 2, 3]
� such changes inside a function are preserved when the function returns� the referenced object is changed (e.g. list entries, class variables), but
>>> def some_function(l):
... l = [4,5,6]
...
>>> b = [1,2,3]
>>> some_function(b)
>>> print(b)
[1, 2, 3]
Python Basics: Functions
multiple values can be returned as a sequence:
>>> def integer_divide(x,y):
... integer_part = x/y
... the_rest = x%y
... return integer_part , the_rest
...
>>> int , rest = integer_divide (10 ,3)
>>> print(int)
3
>>> print(rest)
1
map(f,s) applies the function f to the elements of s and returns a list:
>>> def square(x):
... return x*x;
>>> l = [1, 2, 3, 4]
>>> map(square ,l)
[1, 4, 9, 16]
Python Basics: Functions
optional parameters (keyword or named arguments):
>>> def say_my_name(name=’How should I know?’):
... return name
...
>>> say_my_name ()
’How should I know?’
>>> say_my_name(’Mike’)
’Mike’
� keyword arguments allow you to specify default values
� the order of keyword arguments does not matter:
>>> def slicer(seq , start=None , stop=None , step=None):
... return seq[start:stop:step]
...
>>> l = [1,2,3,4,5,6]
>>> slicer(l, start=1, stop =4)
[2, 3, 4]
>>> slicer(l, step=2, stop=5, start =1)
[2, 4]
Python Basics: Functions
functions are objects� functions can be assigned to a variable:
>>> func = test
>>> func()
This is a test!
� functions can be put into a list (or any collection):
>>> l = [1, ’a’, test]
>>> l
[1, ’a’, <function test at 0x10698e050 >]
>>> l[2]()
This is a test!
� functions can be passed as an argument to another function:
>>> def call_func(function ):
... function ()
...
>>> call_func(test)
This is a test!
Python Basics: Functions
docstrings: documentation about what the function does and it’sparameters
>>> def funcname(params ):
... """
... Concise one -line sentence describing the function.
...
... Extended summary , can contain multiple paragraphs.
... """
... # function body
... pass
...
>>> help(funcname)
funcname(params)
Concise one -line sentence describing the function.
Extended summary , can contain multiple paragraphs.
� the docstring conventions webpage documents the semantics andconventions associated with Python docstrings
Python Basics: Scripts
you do not want to type everything into the shell, especially for longersets of instructions
� write a text file containing code using an editor with syntax highlighting
� execute by starting the interpreter with the filename as argument
script: file with extension .py containing a sequence of instructions thatare executed each time the script is called
test.py
message = "How are you?"
for word in message.split ():
print(word)
� execute the script:
> python test.py
How
are
you?
Python Basics: Modules
put definitions in a file and use them as a module in other scripts
div.py
def integer_divide(x,y):
integer_part = x/y
the_rest = x%y
return integer_part , the_rest
� importing the module gives access to its objects via module.object:
import div
int , rest = div.integer_divide (10 ,3)
� import objects from modules into the main namespace:
from div import integer_divide
int , rest = integer_divide (10 ,3)
� prescribe a shortcut for the module:
import div as d
int , rest = d.integer_divide (10 ,3)
Python Basics: Modules
create modules if you want to write larger and better organisedprograms
� define objects (variables, functions, classes) that you want to reuseseveral times in your own modules
demo.py
"""A demo module."""
def print_b ():
""" Prints b."""
print ’b’
def print_a ():
""" Prints a."""
print ’a’
print("Module demo got loaded!")
c = 2
d = 2
Python Basics: Modules
� import the demo module in another script and call its functions:
demo test.py
import demo
demo.print_a ()
demo.print_b ()
print(demo.c)
print(demo.d)
> python demo_test.py
Module demo got loaded!
a
b
2
2
Python Basics: Modules
� take a look at the documentation:
>>> help(demo)
Help on module demo:
NAME
demo - A demo module.
FILE
/Users/mkraus/Lehre/Python/code/demo.py
FUNCTIONS
print_a ()
Prints a.
print_b ()
Prints b.
DATA
c = 2
d = 2
Python Basics: Modules
you can define a __main__ function that is executed if the module isexecuted as a script but not when the module is imported:
demo2.py
""" Another demo module."""
def print_a ():
""" Prints a."""
print ’a’
if __name__ == ’__main__ ’:
print_a ()
� execute the module on the shell:
> python demo2.py
a
� load the module in interactive mode:
>>> import demo2
>>>
Python Basics: Modules
a directory that contains many modules is called a package� a package is a module with submodules (which can have submodules
themselves, etc.)� a special file called __init__.py (might be empty) tells Python that the
directory is a Python package, from which modules can be imported� a package might be organised as follows:
Graphics/
__init__.py
Primitive/
__init__.py
lines.py
text.py
...
Graph2D/
__init__.py
plot2d.py
...
Graph3D/
__init__.py
plot3d.py
...
....
Python Basics: Input/Output
reading and writing of strings from and to files
� write to a file:
>>> f = open(’tempfile ’, ’w’)
>>> type(f)
<type ’file’>
>>> f.write(’This is a test \nand another test’)
>>> f.close ()
� read from a file:
>>> f = open(’tempfile ’, ’r’)
>>> s = f.read()
>>> print(s)
This is a test
and another test
� numpy features more powerful reading/writing routines
Python Basics: Input/Output
� iterate over a file:
>>> f = open(’tempfile ’, ’r’)
>>> for line in f:
... print(line)
...
This is a test
and another test
� file modes:
r : read-onlyw : write-only (create a new file or overwrite existing file)a : appendr+: read and write
Python Basics: Exceptions
exceptions are raised by different kinds of errors arising when executingPython code and normally terminate program execution:
>>> 1/0
ZeroDivisionError: integer division or modulo by zero
>>> 1 + ’e’
TypeError: unsupported operand type(s) for +: ’int’ and ’str’
>>> d = { 1:1, 2:2 }
>>> d[3]
KeyError: 3
>>> l = [1,2,3]
>>> l[3]
IndexError: list index out of range
>>> l.foobar
AttributeError: ’list’ object has no attribute ’foobar ’
� you may (and often should) catch errors or define custom error types
Python Basics: Exceptions
catching exceptions: try/except
exceptions1.py
while True:
try:
x = int(raw_input(’Please enter a number: ’))
break
except ValueError:
print(’That was no valid number. Try again ...’)
> python exceptions1.py
Please enter a number: a
That was no valid number. Try again ...
Please enter a number: 1
Python Basics: Exceptions
catching exceptions: try/finally
exceptions2.py
try:
x = int(raw_input(’Please enter a number: ’))
finally:
print(’Thank you for your input.’)
> python exceptions2.py
Please enter a number: 1
Thank you for your input.
> python exceptions2.py
Please enter a number: c
Thank you for your input.
ValueError: invalid literal for int() with base 10: ’c’
� important for resource management (e.g. closing a file)
Python Basics: Exceptions
raising exceptions to pass messages between parts of the code
exceptions3.py
def achilles_arrow(x):
if abs(x - 1) < 1e-3:
raise StopIteration
x = 1 - (1-x)/2.
return x
x = 0
while True:
try:
x = achilles_arrow(x)
except StopIteration:
print(’Iteration has stopped ’)
break
print (" Result: " + str(x))
> python exceptions3.py
Iteration has stopped
Result: 0.9990234375
Python Basics: Exceptions
reraise an exception:
exceptions4.py
while True:
try:
x = int(raw_input(’Please enter a number: ’))
break
except ValueError , e:
print(’That was no valid number.’)
raise e
> python exceptions4.py
Please enter a number: a
That was no valid number.
ValueError: invalid literal for int() with base 10: ’a’
> python exceptions4.py
Please enter a number: 1
Python Basics: Object Oriented Programming (OOP)
python supports object oriented programming
� helps to organise your code
� simplifies reuse of code in similar contexts
objects consist of internal data and methods that perform various kindsof operations on that data
example: harmonic oscillator
� attributes: mass m, spring constant k, initial displacement x0
� methods: total_energy(), period(), frequency(), acceleration(x),velocity(x), kinetic_energy(x), ...
Python Basics: Object Oriented Programming (OOP)
classes gather custom functions and variables
student.py
class Student:
def __init__(self , name):
self.name = name
def set_age(self , age):
self.age = age
def set_major(self , major):
self.major = major
def info(self):
print(’%s is a student in %s and %d years old.’ \
% (self.name , self.major , self.age) )
>>> from student import Student
>>> anna = Student(’anna’)
>>> anna.set_age (21)
>>> anna.set_major(’physics ’)
>>> anna.info()
anna is a student in physics and 21 years old.
Python Basics: Object Oriented Programming (OOP)
inheritance allows to derive new classes from old ones that have thesame methods and attributes but might be extended by new ones
>>> class MasterStudent(Student ):
... internship = ’mandatory , from March to June’
...
>>> james = MasterStudent(’james ’)
>>> james.internship
’mandatory , from March to June’
� you can access all methods and attributes defined in Student as well:
>>> james.set_age (23)
>>> james.age
23
>>> james.set_major(’math’)
>>> james.info()
james is a student in math and 23 years old.
Cython
Cython is a programming language based on Python
� provides extra syntax allowing for static type declarations
(remember: Python is generally dynamically typed)
� the source code gets translated into optimised C/C++ code andcompiled as Python extension modules
� Cython can compile (most) regular Python code but the optional statictype declarations usually achieve major speed improvements
� allows for very fast program execution and tight integration withexternal C libraries
� combines the benefits of Python with the speed of C
References:http://www.cython.org/
http://docs.cython.org/
Cython: Compilation
Cython code must, unlike Python, be compiled:
a .pyx source file is compiled by Cython to a .c file, containing the code ofa Python extension modulethe .c file is compiled by a C compiler to a .so (shared object library) filewhich can be imported directly into a Python session
� several ways to build Cython code:
use pyximport, importing Cython .pyx files as if they were .py files (usingdistutils to compile and build the background)write a distutils setup.pyrun the cython command-line utility manually to produce the .c file fromthe .pyx file, then manually compile the .c file into a shared object libraryuse the Sage notebook which allows Cython code inline
Cython: Compilation
imagine a simple “hello world” script:
hello.pyx
def say_hello_to(name):
print("Hello %s!" % name)
implicit compilation using pyximport:
>>> import pyximport
>>> pyximport.install ()
>>> import hello
>>> hello.say_hello_to("Mike")
Hello Mike!
� this allows to automatically run Cython on every .pyx that Python istrying to import
� use this for simple Cython builds where no extra C libraries and nospecial building setup is needed
Cython: Compilation
the following could be a corresponding setup.py script:
setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules = [Extension("hello", ["hello.pyx"])]
setup(
name = ’Hello World App’,
cmdclass = {’build_ext ’: build_ext},
ext_modules = ext_modules
)
� you can specify additional build settings (include dirs, linker flags, ...)
ext_modules = [Extension("hello", ["hello.pyx"],
include_dirs = [numpy.get_include ()],
extra_compile_args = [’-fopenmp ’],
extra_link_args = [’-fopenmp ’] )]
Cython: Compilation
the script can be called with:
> python setup.py build_ext --inplace
after compilation, Cython modules can be imported like every otherPython module:
>>> import hello
>>> hello.say_hello_to("Mike")
Hello Mike!
Cython: Static Types
Cython can compile normal Python code without changes (with a fewexceptions of some as-yet unsupported language features), but forperformance critical code, it is often helpful to add static typedeclarations
� they allow Cython to step out of the dynamic nature of the Pythoncode and generate simpler and faster C code (sometimes faster byorders of magnitude)
� however, type declarations can make the source code more verbose andthus less readable
� it is discouraged to use them without good reason, i.e. only inperformance critical sections where they really make the codesubstantially faster
Cython: Static Types
consider the following pure Python code:
def f(x):
return x**2-x
def integrate_f(a, b, N):
s = 0
dx = (b-a)/N
for i in range(N):
s += f(a+i*dx)
return s * dx
� simply compiling this in Cython gives a 35% speedup
� adding some static types can make a much larger difference
Cython: Static Types
with additional type declarations, the former example might look like:
def f(double x):
return x**2-x
def integrate_f(double a, double b, int N):
cdef int i
cdef double s, dx
s = 0
dx = (b-a)/N
for i in range(N):
s += f(a+i*dx)
return s * dx
� typing the iterator variable i with C semantics, tells Cython to compilethe for-loop to pure C code
� typing a, s and dx is important as they are involved in arithmetic withinthe for-loop
� this results in a 4x speedup over the pure Python version
Cython: Static Types
Python function calls can be expensive: one needs to convert to andfrom Python objects to do the call
� in our example, the argument is assumed to be a C double both insidef() and in the call to it, yet a Python float object must be constructedaround the argument in order to pass it
� Cython provides the cdef keyword for declaring a C-style function:
cdef double f(double x):
return x**2-x
� speedup: 150x over pure Python
� but: now the function is no longer available from Python-space, asPython wouldn’t know how to call it
� using the cpdef keyword instead, a Python wrapper is created as well
Cython: Static Types and NumPy Arrays
you can use NumPy from Cython exactly the same as in regularPython, but by doing so you are loosing potentially high speedups
� Cython has support for fast access to NumPy arrays:
import numpy as np
cimport numpy as np
def update(np.ndarray[double , ndim =2] f):
cdef unsigned int i, j
cdef np.ndarray[double , ndim =2] h = \
np.zeros ([xmax , ymax], dtype=double)
for i in range(0, f.shape [0]):
for j in range(0, f.shape [1]):
...
� typing the contents of ndarray by specifying the datatype and thenumber of dimensions enables access to the data buffer directly at Cspeed, otherwise the [] operator still uses full Python operations
Cython: Faster Code
if you’re confident you’ve done everything right, there are a fewdirectives that can further speed up your code
� boundscheck=False disables array boundary checks
� wraparound=False disables negative indexing of NumPy arrays
there are several ways of setting these compiler directives
� globally, with a special header comment at the top of the file:
#cython: boundscheck=False
#cython: wraparound=False
� locally, as function decorators or in a with statement:
cimport cython
@cython.boundscheck(False)
def update(np.ndarray[double , ndim =2] f):
...
with cython.boundscheck(True):
...
Cython: Parallelisation
Cython supports native parallelism via OpenMP (more backends mightbe supported in the future)
by default, Python’s Global Interpreter Lock (GIL) prevents thatseveral threads use the Python interpreter simultaneously
� to use this kind of parallelism, the GIL must be released
� parallel loops can be created by usingprange([start], stop[, step], nogil=False, schedule=None)
� OpenMP automatically starts a thread pool and distributes the workaccording to the schedule used
static: approx. equally sized chunks, one for each threaddynamic:iterations are distributed as threads request them, chunk size of 1guided: iterations are distributed as threads request them, the chunk sizeis proportional to the number of unassigned iterations divided by thenumber of threads, decreasing to 1auto: the decision regarding scheduling is delegated to the compiler orruntime system
� you have to add the NumPy includes and OpenMP flags to setup.py
Cython: Parallelisation
thread-locality and reductions are automatically inferred for variablesexample with reduction: the values from the thread-local copies of thevariable will be reduced with the operator and assigned to the originalvariable after the loop
from cython.parallel import prange
cdef int i
cdef int sum = 0
for i in prange(n, nogil=True):
sum += i
example with a shared NumPy array:
from cython.parallel import prange
def func(np.ndarray[double] x, double alpha):
cdef unsigned int i
for i in prange(x.shape[0], nogil=True):
x[i] = alpha * x[i]
Outlook: Scientific Computing and Plotting
NumPy: fundamental package for scientific computing with Python
powerful N-dimensional array object and various derived objects (matrices)routines for fast operations on arrays (selecting, sorting, shapemanipulation, mathematical, I/O, ...)modules for linear algebra, statistics, Fourier transforms, random numbersit’s fast: most routines are optimised, pre-compiled C code
SciPy: numerical routines, data manipulation and analysis
Matplotlib: 2D plotting
allows generation of plots, histograms, power spectra, bar charts, errorcharts, scatter plots, etc., with just a few lines of codefull control of line styles, font properties, axes properties, etc., via anobject oriented interface or via a set of functions similar to MATLAB
MayaVi: 3D plotting and interactive scientific data visualisation
HDF5/NetCDF: self-describing and efficient storage of scientific data
Outlook: Advanced Topics
calling and embedding C and Fortran code
weave: embed C code within your .py filesSWIG: automatic wrapper generation for C/C++ codectypes, cython: wrap librariesf2py: wrap and compile Fortran routines to behave like Python modules
GUI programming with PyQt and PySide
overview and simple examplesbasic widgetssignals and slotsmenus, toolbars, statusbarlayoutdialogsembedding matplotlib plotscreating UIs with Qt Designer
Symbolic Calculations with Sage