Python - Basic Threading in Python

Basic Threading in Python(2005-04-04) - Contributed by Peyton McCullough

If you want your application to perform several tasks at once, you can use threads. Python can handle threads, but manydevelopers find thread programming to be very tricky. Among other points, Peyton McCullough covers how to spawn andkill threads in this popular language.

Introduction

Threads allow applications to perform multiple tasks at once. Multi-threading is important in many applications, fromprimitive servers to today's complex and hardware-demanding games, so, naturally, many programming languages sportthe ability to deal with threads. This includes Python.

However, Python's support for multi-threading is not without limitations and consequences, as Guido van Rossum writes:

"Unfortunately, for most mortals, thread programming is just Too Hard to get right.... Even in Python -- every timesomeone gets into serious thread programming, they send me tons of bug reports, and half of them are subtle bugs inthe Python interpreter, half of them are subtle problems in their own understanding of the consequences of multiplethreads...."

Before we begin to look at the code that makes threading work, the most important feature we must look at is Python'sglobal interpreter lock. If two or more threads were to attempt to manipulate the same object at the same time, problemswould inevitably pop up. The global interpreter lock fixes this. Only one thread can perform an action at any given time.Python automatically switches between threads when it is needed.

{mospagebreak title=Using the Threading Module}

The threading module provides an easy way to work with threads. Its Thread class may be subclassed to create a threador threads. The run method should contain the code you wish to be executed when the thread is executed. This soundsimple, right? Well, it is:

import threading

class MyThread ( threading.Thread ):

def run ( self ):

print 'Insert some thread stuff here.' print 'It'll be executed...yeah....' print 'There's not much to it.'

Executing the thread is also simple. All we have to do is create an instance of our thread class and then call its startmethod:

import threading

class MyThread ( threading.thread ):

def run ( self ):

http://www.devshed.com - Dev Shed Generated: 8 January, 2007, 02:46

print 'You called my start method, yeah.' print 'Were you expecting something amazing?'

MyThread().start()

Of course, it's no fun having just one thread. Just like us humans, threads get lonely after a while. Let's create a group ofthreads:

import threading

theVar = 1

class MyThread ( threading.Thread ):

def run ( self ):

global theVar print 'This is thread ' + str ( theVar ) + ' speaking.' print 'Hello and good bye.' theVar = theVar + 1

for x in xrange ( 20 ): MyThread().start()

Now let's actually do something semi-useful with the threading module. Servers often use threads to handle multipleclients at once. Let's build a simple but extendable server. When a client opens a connection with the server, the serverwill create a new thread to handle the client. To send the client's data to the thread, we will need to override the Threadclass's __init__ method to accept parameters. The server will now send the thread on its way and then wait for newclients. Each thread will send a pickled object to the appropriate client and then print no more than ten strings receivedfrom the client. (A pickled object is basically an object that has been reduced to a few characters. This is useful forstoring objects for later use and for sending objects over a network).

import pickleimport socketimport threading

# We'll pickle a list of numbers:someList = [ 1, 2, 7, 9, 0 ]pickledList = pickle.dumps ( someList )

# Our thread class:class ClientThread ( threading.Thread ):

# Override Thread's __init__ method to accept the parameters needed: def __init__ ( self, channel, details ):

self.channel = channel self.details = details threading.Thread.__init__ ( self )

def run ( self ):

print 'Received connection:', self.details [ 0 ] self.channel.send ( pickledList ) for x in xrange ( 10 ): print self.channel.recv( 1024 ) self.channel.close() print 'Closed connection:', self.details [ 0 ]

# Set up the server:server = socket.socket ( socket.AF_INET, socket.SOCK_STREAM )server.bind ( ( '', 2727 ))server.listen ( 5 )


# Have the server serve "forever":while True: channel, details = server.accept() ClientThread ( channel, details ).start()

Now we need to build a client that connects to the server, retrieves the pickled object, reconstructs the pickled object andfinally sends ten messages, closing the connection:

import pickleimport socket

# Connect to the server:client = socket.socket ( socket.AF_INET, socket.SOCK_STREAM )client.connect ( ( 'localhost',2727 ) )

# Retrieve and unpickle the list object:print pickle.loads ( client.recv ( 1024 ) )

# Send some messages:for x in xrange ( 10 ): client.send ( 'Hey. ' + str ( x ) + '\n' )

# Close the connectionclient.close()

Of course, the above client doesn't take advantage of our server's multi-threading capabilities. Only one thread isspawned, which really defeats the purpose of multi-threading. Let's thread the client to make things a bit moreinteresting. Each thread will connect to the server and execute the code above:

import pickleimport socketimport threading

# Here's our thread:class ConnectionThread ( threading.Thread ):

def run ( self ):

# Connect to the server: client = socket.socket ( socket.AF_INET, socket.SOCK_STREAM ) client.connect ( ( 'localhost',2727 ) )

# Retrieve and unpickle the list object: print pickle.loads ( client.recv ( 1024 ) )

# Send some messages: for x in xrange ( 10 ): client.send ( 'Hey. ' + str ( x ) + '\n' )

# Close the connection client.close()

# Let's spawn a few threads:for x in xrange ( 5 ): ConnectionThread().start()

{mospagebreak title=Pooling Threads}

It's important to remember that threads don't start up instantly. Creating too many of them can slow down yourapplication. It takes time to spawn and later kill threads. Threads can also eat up valuable system resources in largeapplications. This problem is easily solved by creating a set number of threads (a thread pool) and assigning them newtasks, basically recycling them. Connections would be accepted and then pushed to a thread as soon as it finished withthe previous client.


If you still don't understand, compare it to a doctor's office. Let's say that there are five doctors. These are our threads.Patients (clients) walk into the office, and if the doctors are busy, they are seated in the waiting room.

Obviously, we'll need something that can transfer client data to our threads without running into problems (it will need tobe “thread safe”). Python's Queue module does this for us. Client information is stored in a Queue object, where threadscan pull them out when needed.

Let's recreate our server to take advantage of a pool of threads:

import pickleimport Queueimport socketimport threading

# We'll pickle a list of numbers, yet again:someList = [ 1, 2, 7, 9, 0 ]pickledList = pickle.dumps ( someList )

# A revised version of our thread class:class ClientThread ( threading.Thread ):

# Note that we do not override Thread's __init__ method. # The Queue module makes this not necessary.

def run ( self ):

# Have our thread serve "forever": while True:

# Get a client out of the queue client = clientPool.get()

# Check if we actually have an actual client in the client variable: if client != None:

print 'Received connection:', client [ 1 ] [ 0 ] client [ 0 ].send ( pickledList ) for x in xrange ( 10 ): print client [ 0 ].recv (1024 ) client [ 0 ].close() print 'Closed connection:', client [ 1 ] [ 0 ]

# Create our Queue:clientPool = Queue.Queue ( 0 )

# Start two threads:for x in xrange ( 2 ): ClientThread().start()

# Set up the server:server = socket.socket ( socket.AF_INET, socket.SOCK_STREAM )server.bind ( ( '', 2727 ))server.listen ( 5 )

# Have the server serve "forever":while True: clientPool.put ( server.accept() )

As you can see, it's a little more complex than our previous server, but it's not amazingly complicated. The client wecreated in the previous section can be used to test this server, just like the previous server.

{mospagebreak title=More Thread Tricks}

There is more to threads than just spawning them and sending them on their way. The threading module's Threadmodule contains a few more methods that you should be aware of. The first two deal with naming threads. The methodsetName sets a thread's name, and the method getName retrieves a thread's name: http://www.devshed.com - Dev Shed Generated: 8 January, 2007, 02:46

import threading

class TestThread ( threading.Thread ):

def run ( self ):

print 'Hello, my name is', self.getName()

cazaril = TestThread()cazaril.setName ( 'Cazaril' )cazaril.start()

ista = TestThread()ista.setName ( 'Ista' )ista.start()

TestThread().start()

No suprises there. Also, as you can see, threads have names even if you don't specify them.

We can also check whether a thread is “alive” using the isAlive method. If the thread hasn't finished executing whatever isin its run method, then it's classified as being alive:

import threadingimport time

class TestThread ( threading.Thread ):

def run ( self ):

print 'Patient: Doctor, am I going to die?'

class AnotherThread ( TestThread ):

def run ( self ):

TestThread.run( self ) time.sleep ( 10 )

dying = TestThread()dying.start()if dying.isAlive(): print 'Doctor: No.'else: print 'Doctor: Next!'

living = AnotherThread()living.start()if living.isAlive(): print 'Doctor: No.'else: print 'Doctor: Next!'

The second thread remains alive because we force it to wait using the time module's sleep method.

If we want to make a thread wait for another thread to terminate itself, we can use the join method:



class ThreadOne ( threading.Thread ):

def run ( self ):

print 'Thread', self.getName(), 'started.' time.sleep ( 5 ) print 'Thread', self.getName(), 'ended.'

class ThreadTwo ( threading.Thread ):

def run ( self ):

print 'Thread', self.getName(), 'started.' thingOne.join() print 'Thread', self.getName(), 'ended.'

thingOne = ThreadOne()thingOne.start()thingTwo = ThreadTwo()thingTwo.start()

We can use the setDaemon method, too. If a True value is passed with this method and all other threads have finishedexecuting, the Python program will exit, leaving the thread by itself:


class DaemonThread ( threading.Thread ):

def run ( self ):

self.setDaemon ( True ) time.sleep ( 10 )

DaemonThread().start()print 'Leaving.'

Python also contains a thread module that deals with lower level threading. The only feature I would like to point out isthe start_new_thread function it contains. Using this, we can turn an ordinary function into a thread:

import thread

def thread( stuff ): print "I'm a real boy!" print stuff

thread.start_new_thread ( thread, ( 'Argument' ) )

Conclusion

There's much more to multi-threading than I explained in this article, but I will not bore you by extending the scope of thisarticle to include everything. Moreover, as Guido van Rossum mentioned, the advantage gained by complex multi-threading in Python may be outweighed by the consequences. A small dose of common sense can eliminate much of theproblems in simple multi-threading, however. http://www.devshed.com - Dev Shed Generated: 8 January, 2007, 02:46

Threading is very important when dealing with computer applications, and, as I mentioned earlier, Python isn't excluded.When used properly, threads can be very beneficial and often even crucial, as I've outlined in this article.


Python - Basic Threading in Python

Documents

Transcript of Python - Basic Threading in Python