Object Oriented Programming in Python

Juan Manuel Gimeno Illa

[email protected]

Curs 2009-2010

J.M.Gimeno ([email protected]) OOP in Python Curs 2009-2010 1 / 49

Outline

1 Introduction

2 Classes

3 Instances I

4 DescriptorsReferencing AttributesBound and Unbound MethodsPropertiesClass-Level Methods

5 InheritanceMethod Resolution OrderCooperative Superclasses

6 Instances II

Introduction

Programming Paradigms

A programming paradigm consists in the basic concepts into whichour programs are made of

Procedural Modules, data structures and procedures that operateupon them

Objectural Objects which encapsulate state and behaviour andmessages passed between these objects

Functional Functions and closures, recursion, lists, ...

Python is a multiparadigm programming languageI this allows the programmer to choose the paradigm that best suits the

problemI this allows the program to mix paradigmsI this allows the program to evolve switching paradigm if necessary

Introduction

Classes

Python classes

A class is a python object with several characteristics:

You can call a class as it where a function and this call returns a newinstance of the class

A class has arbitrary named attributes that can be bound, unboundan referenced

The class attributes can be descriptors (including functions) or normaldata objects

Class attributes bound to functions are also known as methods

A method can have special python-defined meaning (they’re namedwith two leading and trailing underscores)

A class clan inherit from other classes, meaning it delegates to otherclasses the look-up of attributes that are not found in the class itself

Classes

Python classes

Classes

Python classes

Classes

Python classes

Classes

Python classes

Classes

Python classes

Classes

Object models

Since Python2.2 there co-exist two slightly different object models inthe language

Old-style (classic) classes This is the model existing prior toPython2.2

New-style classes This is the preferred model for new code

Old-style

>>> class A: pass>>> class B: pass>>> a, b = A(), B()>>> type(a) == type(b)True>>> type(a)<type ’instance’>

New-style

>>> class A(object): pass>>> class B(object): pass>>> a, b = A(), B()>>> type(a) == type(b)False>>> type(a)<class ’ main .A’>

Classes

Object models

Old-style

New-style

Classes

Object models

Old-style

New-style

Classes

Object models

Old-style

New-style

Classes

Object models

Old-style

New-style

Classes

New-style classes

Defined in the type and class unification effort in python2.2

(Introduced without breaking backwards compatibility)

Simpler, more regular and more powerfulI Built-in types (e.g. dict) can be subclassedI Properties: attributes managed by get/set methodsI Static and class methods (via descriptor API)I Cooperative classes (sane multiple inheritance)I Meta-class programming

It will be the default (and unique) in the future

Documents:I Unifying types and classes in Python 2.2I PEP-252: Making types look more like classesI PEP-253: Subtyping built-in types

Classes

New-style classes

Classes

New-style classes

Classes

New-style classes

Classes

New-style classes

Classes

New-style classes

Classes

New-style classes

Classes

New-style classes

Classes

New-style classes

Classes

New-style classes

Classes

New-style classes

Classes

New-style classes

Classes

New-style classes

Classes

The class statement

class classname(base-classes):statement(s)

classname is a variable that gets (re)bound to the class object afterthe class statement finishes executing

base-classes is a comma separated series of expressions whosevalues must be classes

I if it does not exists, the created class is old-styleI if all base-classes are old-style, the created class is old-styleI otherwise it is a new-style class1

I since every type subclasses built-in object, we can use object tomark a class as new-style when no true bases exist

The statements (a.k.a. the class body) define the set of classattributes which will be shared by all instances of the class

1We are not considering metaclass nowJ.M.Gimeno ([email protected]) OOP in Python Curs 2009-2010 7 / 49

Classes

The class statement

Classes

The class statement

Classes

The class statement

Classes

The class statement

Classes

The class statement

Classes

The class statement

Classes

The class statement

Classes

Attributes of class objectsAttributes can be bound inside or outside the class body.

>>> class C1(object):... x = 23>>> print C1.x23

>>> class C2(object): pass>>> C2.x = 23>>> print C2.x23

Some attributes are implicitly set:

>>> print C1. name , C1. basesC1, (<type ’object’>,)>>> C1. dict [’z’] = 42>>> print C1.z42>>> print C1. dict [’x’]23

Classes

Accessing class attributesIn statements directly inside the class’ body:

>>> class C3(object):... x = 23... y = x + 19

In statements in methods of the class:

>>> class C4(object):... x = 23... def amethod(self):... print C4.x

In statements outside the class:

>>> class C3(object):... x = 23>>> C3.x = 42

Classes

Class-private attributes

When a statement in the body (or in a method in the body) uses anidentifier starting with two underscores (but not ending with them)such as private, the Python compiler changes it toclassname private

This lets classes to use private names reducing the risk of accidentallyduplicating names used elsewhere

By convention all identifiers starting with a single underscore aremeant to be private in the scope that binds them

>>> class C5(object):... private = 23>>> print C5.__privateAttributeError: class A has no attribute ’ private’>>> print C5. C5 private23

Classes

Function definitions in a class body

Most class bodies include def statements since functions (called methods inthis context) are important attributes for most class objectsA method defined in a class body has a mandatory first parameter(conventionally called self) that refers to the instance on which the methodis called (staticmethods and classmethods are not considered now)A class can define a variety of special methods (two leading and two trailingunderscores) relating to specific operation on its instances

>>> class C5(object):... """This is the docstring of the class.... It can be accessed by C5. doc """... def hello(self):... "And this the docstring of the method"... print "Hello!!"

Classes

Instances I

Creating Instances 1

To create an instance of aclass, call the class object asif it were a function

If it defines or inheritsinit , calling the class

object implicitly calls it toperform any neededinstance-specificinitialisation

You can give an instance anattribute by binding a valueto an attribute reference

>>> anInstance = C5()>>> isinstance(anInstance, C5)True>>> class C6(object):... def init (self, n):... self.x = n>>> anInstance = C6(42)>>> print anInstance.x42>>> anInstance.z = 8>>> print anInstance.z8

Instances I

Attributes of Instance ObjectsAttributes can be bound inside or outside class methods

>>> class C1(object):... def amethod(self, n=8):... self.n = n>>> c = C1()>>> c.amethod()>>> print c.n8

>>> class C2(object):... pass>>> d = C2()>>> d.n = 15>>> print d.n15

Some attributes are implicitly set (both can be rebound but not unbound):

>>> print d. class<class ’ main .C2’>>>> d. dict [’z’] = 42>>> print d.z42>>> print d. dict [’n’]15

Instances I

Descriptors

A descriptor is any new-style object whose class supplies a specialmethod named get

Descriptors that are class attributes control the semantics ofaccessing and setting attributes on instances of that class

If a descriptor’s class also supplies method set then it is called anoverriding descriptor (a.k.a. data descriptor)

If not, it is called non-overriding (a.k.a. non-data) descriptor

Function objects (and methods) are non-overriding descriptors

Descriptors are the mechanism behind properties, methods, staticmethods, class methods, and super (cooperative super-classes)

The descriptor protocol also contains method delete forunbinding attributes but it is seldom used

Descriptors

A Descriptor Example

>>> class Area(object):... """An overriding descriptor"""... def get (self, obj, klass):... return obj.x * obj.y... def set (self, obj, value):... raise AttributeError>>> class Rectangle(object):... """A new-style class for representing rectangles"""... area = Area()... def init (self, x, y):... self.x = x... self.y = y>>> r = Rectangle(5, 10)>>> print r.area50

Descriptors Referencing Attributes

Attribute Reference Basics

An attribute reference is an expression of the form x.name, where x isan expression and name is an identifier

Many kinds of Python objects have attributes, but an attributereference when x refers to a class or an instance has special richsemantics

The mechanics of attribute getting is defined in the special methodgetattribute

The predefined behaviour is defined in the implementation of thismethod in the type (for class attributes) and object (for instanceattributes) built-in types

Getting an attribute from a class

When you use the syntax C.name to refer to an attribute on a class objectC, the look-up proceeds in two steps:

1 When ’name’ is a key in C. dict , C.name fetches the value vfrom C. dict [’name’].

I If v is a descriptor (i.e. type(v) supplies a get method), thentype(v). get (v,None,C) is returned

I Otherwise, ir returns v

2 Otherwise, it delegates the look-up to its base classes (in methodresolution order)

When these look-ups steps do not find an attribute, Python raises anAttributeError exception.

Getting an attribute from an instance I

When you use the syntax x.name to refer to an attribute of instance x ofclass C, the look-up proceeds in three steps:

1 When ’name’ is found in C (or in one of C’s ancestor classes) as thename of an overriding descriptor v (i.e. type(v) supplies bothget and set ), then the value of x.name is

type(v). get (v,x,C)

2 Otherwise, when ’name’ is key in x. dict , x.name fetches andreturns x. dict [’name’]

3 Otherwise, x.name delegates the look-up to x’s class (looking intoC. dict or delegating to C’s bases) and

I if a descriptor v is found, the overall result is againtype(v). get (v,x,C).

I if a nondescriptor value v is found, the result is v.

Getting an attribute from an instance II

When these look-ups steps do not find an attribute, Python either:

raises an AttributeError exception, or

if C defines or inherits special method getattr , callsC. getattr (x, ’name’)

Setting an attribute

The attribute look-up steps happens when referring to an attribute,not when binding an attribute (e.g. with x.name=v)

When binding an attribute whose name is not special, only thedict entry for the attribute is affected

Unless a setattr method is defined (or inherited) in x’s class

Or name corresponds to an overriding descriptor in x’s class (or one ofits ancestors)

Descriptors Bound and Unbound Methods

Functions are descriptorsThe get methodof a function eitherreturns

I an unboundmethod object, or

I a bound methodobject

that wraps thefunction.The key difference isthat a unboundmethod is notassociated with aparticular instancewhile a bound isOther callables (suchas built-ins) are notdescriptors

>>> class C(object):... def init (self, x): self.x = x>>> def fun(obj): return obj.x>>> fun<function fun at 0xb7d89c6c>>>> C.met = fun>>> o = C(8)>>> o< main .C object at 0xb7d9046c>>>> C.met<unbound method C.fun>>>> o.met<bound method C.fun of

< main .C object at 0xb7d9046c>>

Unbound methodsC.met is fun. get (None, C)It has also three attributes: im class, im self, im funcYou can call it as its im func, but the first argument must be an instance ofim class (or descendant)

>>> C.met == fun. get (None, C)True>>> C.met.im class is CTrue>>> C.met.im self is NoneTrue>>> C.met.im func is funTrue>>> class D(object): pass>>> C.met(D())Traceback (most recent call last):File " <stdin> ", line 1, in ?

TypeError: unbound method fun() must be called with Cinstance as first argument (got D instance instead)

>>> C.met == fun. get (None, C)True>>> C.met.im class is CTrue>>> C.met.im self is NoneTrue>>> C.met.im func is funTrue>>> class D(object): pass>>> C.met(D())Traceback (most recent call last):

File " <stdin> ", line 1, in ?TypeError: unbound method fun() must be called with C

instance as first argument (got D instance instead)

Bound methods

o.met is fun. get (o, type(o))It has three attributes: im class, im self, im funcWhen called, it calls im func passing im self as first argument

>>> o.met == fun. get (o, type(o))True>>> o.met.im class is CTrue>>> o.met.im self is oTrue>>> o.met.im func is funTrue>>> o.met.im func(o.met.im self)8

Bound methods

Simulation of the built-in function type

class Function(object):"""Simulation of the built-in function type"""

def __init__(self, fun):self.fun = fun

def __call__(self, *args, **kwargs):return self.fun(*args, **kwargs)

def __get__(self, obj, cls):return InstanceMethod(self.fun, obj, cls)

def __repr__(self):return "Function-wrapper of %s" % (self.fun,)

Simulation of the built-in instancemethod type

class InstanceMethod(object):

def __init__(self, fun, obj, cls):

self.im_func = fun

self.im_self = obj

self.im_class = cls

def __call__(self, *args, **kwargs):

if self.im_self is None: # Unbound method call

if not isinstance(args[0], self.im_class):

raise TypeError,

"%s should be called with an instance of %s" % \

(self, self.im_class.__name__)

return self.im_func(*args, **kwargs)

else: # Bound method call

return self.im_func(self.im_self, *args, **kwargs)

def __repr__(self):

if self.im_self is None:

return "UnboundMethod %s.%s" % \

(self.im_class.__name__, self.im_func.__name__)

return "BoundMethod %s.%s of %s" % \

(self.im_class.__name__, self.im_func.__name__, self.im_self)

Closures, Bound-methods and callable objectsdef make_adder_as_closure(augend):

def add(addend):return addend + augend

return add

def make_adder_as_bound_method(augend):class Adder(object):

def init (self, augend):self.augend = augend

def add(self, addend):return addend + self.augend

return Adder(augend).add

def make_adder_as_callable(augend):class Adder(object):

def call (self, addend):return addend + self.augend

return Adder(augend)

return add

Descriptors Properties

Defining propertiesThe built-in

property(fget=None, fset=None, fdel=None, doc=None)simplifies the creation of descriptors for calculated attributesRemember that descriptors only work their magic if defined as classattributes

>>> class Rectangle(object):... """A new-style class for representing rectangles"""... def __init__(self, x, y):... self.x = x... self.y = y... def getArea(self):... return self.x * self.y... area = property(fget=getArea, doc="Area of the rectangle")... del getArea>>> r = Rectangle(5, 10)>>> print r.area50

A Property Implementation in Pythonclass Property(object):

"Emulate PyProperty_Type() in Objects/descrobject.c"

def __init__(self, fget=None, fset=None, fdel=None, doc=None):self.fget = fgetself.fset = fsetself.fdel = fdelself.__doc__ = doc

def __get__(self, obj, objtype=None):if obj is None:

return selfif self.fget is None:

raise AttributeError, unreadable attribute"return self.fget(obj)

def __set__(self, obj, value):if self.fset is None:

raise AttributeError, can’t set attribute"self.fset(obj, value)

def __delete__(self, obj):if self.fdel is None:

raise AttributeError, can’t delete attribute"self.fdel(obj)

Descriptors Class-Level Methods

Built-in non-overriding descriptors

Python supplies two built-in non-overriding descriptors types which give aclass two distinct kinds of class-level methods:

static method is a method you can calll on a class or on any instancewithout the special behaviour of the first parameter of amethod

class method is a method you can call on a class or on any instance andthe first parameter gets bound to the class (or the object’sclass)

Descriptor Called from an Object Called from a Classfunction f(obj, *args) f(*args)staticmethod f(*args) f(*args)classmethod f(type(obj), *args) f(klass, *args)

Example of Static-Method

>>> class E(object):... def f(x):... return x... f = staticmethod(f)... @staticmethod... def g(x):... return 2*x>>> print E.f(15)15>>> print E().g(8)16

Example of Class-Method

>>> class Dict(object):... ........ def fromkeys(klass, iterable, value=None):... "Emulate dict_fromkeys() in Objects/dictobject.c"... d = klass()... for key in iterable:... d[key] = value... return d... fromkeys = classmethod(fromkeys)>>> Dict.fromkeys(’abracadabra’)’a’: None, ’r’: None, ’b’: None, ’c’: None, ’d’: None

Inheritance Method Resolution Order

Method Resolution Order

With multiple inheritance comes the question of method resolutionorder: the order in which a class and its basses are searched lookingfor a method of a given name

In classic python the rule is left-to-right depth-first:

def classic lookup(cls, name):"""Searches name in cls and its base classes"""if name in cls. dict :

return cls. dict [name]for base in cls. bases :

try:return classic lookup(base, name)

except AttributeError:pass

raise AttributeError, name

Problems with classic mro

class A:def save(self):

....class B(A):

....class C(A):

def save(self):....

class D(B,C):....

This is an exemple of a diamond inheritancediagram

C overrides A’s save

(maybe C.save calls A.save)

When invoking save on a D instance, A.saveis called (completedly ignoring C.save)

But in classic python diamond diagrams areseldom found because

I most hierarchies use single inheritanceI multiple inheritance is usually limited to

mix-in classes

....class B(A):

....class C(A):

def save(self):....

class D(B,C):....

mix-in classes

....class B(A):

....class C(A):

def save(self):....

class D(B,C):....

mix-in classes

....class B(A):

....class C(A):

def save(self):....

class D(B,C):....

mix-in classes

....class B(A):

....class C(A):

def save(self):....

class D(B,C):....

mix-in classes

....class B(A):

....class C(A):

def save(self):....

class D(B,C):....

mix-in classes

....class B(A):

....class C(A):

def save(self):....

class D(B,C):....

mix-in classes

....class B(A):

....class C(A):

def save(self):....

class D(B,C):....

mix-in classes

Classic MRO and New-classes

In the classic model C inherits the modified behaviour of getattrBut in the new model C inherits the default behaviour defined in built-inobject

Classic-classesclass A:

....class B:

def setattr (self, name, val):....

class C(A, B):....

Classic MRO and New-classes

In the classic model C inherits the modified behaviour of getattrBut in the new model C inherits the default behaviour defined in built-inobject

New-classes# Predefined object defines the# default behaviour of getattrclass A(object):

....class B(object):

def setattr (self, name, val):....

class C(A, B):....

New MRO algorithm (naıve description)

>>> class A(object):... pass>>> class B(A):... pass>>> class C(A):... pass>>> class D(B,C):... pass

Classic look-up would visit classes inthis order:D, B, A, object, C, A, object

New algorithm (naıve python 2.2version) scans the list and eliminateselements repeated further in the list

So, the new order is:D, B, C, A, object

The real algorithm (since python 2.3)named C3 is described here.

New-style classes get a special attributenamed mro

Monotonicity and Order Disagreements

>>> class A(object):... pass>>> class B(object):... pass>>> class X(A,B):... pass>>> class Y(B,A):... pass>>> class Z(X,Y):... passTypeError: .....

Monotonicity: If class C1 precedes classC2 in the look-up order from class D,then C1 precedes C2 in the look-uporder of any subclass of D

Local precedence: The look-up orderdoes not contradict the ordering of thebases in the class definition

In the example, Z’s mro would beZ, X, Y, B, A, object

But X’s mro would beX, A, B, object

And monotonicity would not bepreserved

Inheritance Cooperative Superclasses

Cooperative super call

This is one of the coolest and one of the most unusual features ofthe new-class model

Cooperative classes are written with multiple inheritance in mind,using a pattern known in other languages as call-next-method

It is more powerfull than the super call found in single-inheritancelanguage such as Java

First, let’s review the traditional non-coperative super call:

class A(object):def m(self):

print "Saving A’s data"class B(A):

def m(self):print "Saving B’s data"A.m(self)

Non-cooperative super and multiple inheritance

class C(A):def m(self):

print "Saving C’s data"A.m(self)

class D(B, C):def m(self):

print "Saving D’s data"B.m(self)C.m(self)

This example is a case ofdiamond inheritancediagram

A.m gets called twice !!!I at best this is

inefficientI at worst it is an error

Traditional ¿solution?

class A(object):def _m(self): print "Saving A’s data"def m(self): self._m()

class B(A):def _m(self): print "Saving B’s data"def m(self): self._m(); A._m(self)

class C(A):def _m(self): print "Saving C’s data"def m(self): self._m(); A._m(self)

class D(B, C):def _m(self): print "Saving D’s data"def m(self): self._m(); B._m(self); C._m(self); A._m(self)

The proliferation of extra methods and calls is a problem

It creates an undesisable dependency in the derived classes on thedetails of the inheritance graph of its base classes

Using cooperative super

class A(object):def m(self): print "Saving A’s data"

class B(A):def m(self): print "Saving B’s data"; super(B, self).m()

class C(A):def m(self): print "Saving C’s data"; super(C, self).m()

class D(B, C):def m(self): print "Saving D’s data"; super(D, self).m()

the first argument to super is the class in which it occurs

the second is always self (and it is not repeated in the argument list)

How does super work?

Consider the MRO of each of the classes

A. mro == (A, object)

B. mro == (B, A, object)

C. mro == (C, A, object)

D. mro == (D, B, C, A, object)

super(C, self)I should only be used inside the implementation of methods in class CI self is an instance of CI self. class may not be C but one of the classes derived from C

The expression super(C, self).m1 first, searches self. class . mro for C2 and then, starts looking for an implementation of m following that point

(super(class, subclass) also works to allow cooperative newand other static methods. It uses subclass. mro in the first step.)

Implementing super in Python

class Super(object):

def __init__(self, type, obj=None):

self.__type__ = type

self.__obj__ = obj

def __getattr__(self, attr):

if isinstance(self.__obj__, self.__type__):

startype = self.__obj__.__class__

startype = self.__obj__

mro = iter(startype.__mro__)

for cls in mro:

if cls is self.__type__:

# mro is an iterator, so the second loop picks

# up where the first one left out

for cls in mro:

if attr in cls.__dict__:

x = cls.__dict__[attr]

if hasattr(x, "__get__"):

x = x.__get__(self.__obj__)

return x

raise AttributteError, attr

Instances II

Each new-style class has (or inherits) a static method named new

When you do x = C(*args, **kwargs) Python executes:(link to Metaclasses: this code is in type(C). call )

x = C.__new__(C, *args, **kwargs)type(x).__init__(x, *args, **kwargs)

The predefined object. new :I creates a new, uninitialized instance of the class it receives as its first

argumentI it ignores other arguments if that class has (or inherits) an init

methodI it raises an exception if it receives other arguments beyond the first but

the class does not have an init method

When you override new inside the class it is not necesary to addnew = staticmethod( new )

Instances II

Subclassing immutable built-in

Does not work !!!class inch(float):

"Converts from inch to meter"

def __init__(self, arg=0.0):

return float.__init__(self,

arg*0.0254)

print inch(12) # prints 12

It works !!!class inch(float):

def __new__(cls, arg=0.0):

return float.__new__(cls,

arg*0.0254)

print inch(12) # prints 0.3048

float. init ignores its arguments to preserve immutability

Types which ignore init ’s arguments: int, long, float,complex, str, unicode, tuple

Types which ignore new ’s arguments: dict, list, file, super,classmethod, staticmethod, property

Built-in type object ignores extra arguments in both methods

Built-in type type is special in many respects

Instances II

arg*0.0254)

Instances II

arg*0.0254)

Instances II

arg*0.0254)

Instances II

arg*0.0254)

Instances II

arg*0.0254)

Instances II

arg*0.0254)

Instances II

Some rules for new

A new method that overrides base class’s new may call thatbase class’s new method

In doing so, the first argument in the call must be the class argumentto the overriding new and not the base class

Unless you want to do caching, new ’s only way to create aninstance is to call base class’s new

I subclass’s new can pass different arguments to the base class’snew , or

I modify the resulting object after it is created by the call

new ’s must return an object but there’s nothing that requires it tobe a new object nor an instance of its class argument

Instances II

Some rules for new

Instances II

Some rules for new

Instances II

Some rules for new

Instances II

Some rules for new

Instances II

Some rules for new

Instances II

A version of the Singleton Pattern

class Singleton(object):_singletons = {}def __new__(cls, *args, **kwargs):

if cls not in cls._singletons:cls._singletons[cls] = \

super(Singleton, cls).__new__(cls, *args, **kwargs)return cls._singletons[cls]

Any subclass of Singleton which does not override new has atmost one instance

If the subclass defines an init method it must be idempotent(safe when called repeatedly)

Instances II

Exercicies

Hands-on work

1 Implement a class for a ring-buffer with a fixed size so that, when otfills up, adding another element overwrites the oldest. It has threemethods: init (self, size max), append(self, x) andtolist(self)

2 Implement a binary-search-tree class. Include a method for anin-order traversal (returning a list of elements)

3 Implement the Rectangle class modifying

(i) getattr(ii) getattribute

4 Implement descriptor classes with the same behaviour ofclassmethod and staticmethod

Bibliography

Raymond Hettinger, How-to Guide for Descriptors

Michael Hudson, OOP in Python after 2.2 (or Fun with Descriptors),PythonUK (part of the ACCU Conference), 2004.

Alex Martelli, Python in a Nutshell (2nd Edition), O’Reilly Media Inc,2006.

Michele Simionato, The Python 2.3 Method Resolution Order

Guido Van Rossum, Unifying types and classes in Python 2.2

Guido Van Rossum, PEP-252: Making types look more like classes

Guido Van Rossum, PEP-253: Subtyping built-in types

License

Aquesta obra esta subjecta a una llicencia Reconeixement-Compartir ambla mateixa llicencia 2.5 Espanya de Creative Commons.Per veure’n una copia, visiteu

http://creativecommons.org/licenses/by-sa/2.5/es/

o envieu una carta a

Creative Commons559 Nathan Abbott WayStanfordCalifornia 94305USA

Object Oriented Programming in Python

Technology

Transcript of Object Oriented Programming in Python