2.3 interprocess 2.3 interprocess communcation (IPC)communcation (IPC)

(especially via shared (especially via shared memory & controlling access memory & controlling access

to it)to it)

Race conditionsRace conditions

►An error whereAn error where one process may wait foreverone process may wait forever or other inconsistencies may resultor other inconsistencies may result

►Occurs when two or more processes are Occurs when two or more processes are reading or writing some shared datareading or writing some shared data

►Applies to threads as wellApplies to threads as well►The final result depends on who runs The final result depends on who runs

precisely whenprecisely when►Difficult to debug and reproduceDifficult to debug and reproduce

Critical region/sectionCritical region/section

►Part of program where shared memory Part of program where shared memory is accessedis accessed Must be identifiedMust be identified

► mutual exclusion – method to exclude mutual exclusion – method to exclude other processes from using a shared other processes from using a shared variable until our process is finished variable until our process is finished with itwith it

Process cooperation rules:Process cooperation rules:

1.1. No two processes can be in their critical No two processes can be in their critical sections at the same time.sections at the same time.

2.2. Make no timing assumptions.Make no timing assumptions.► My code is faster/shorter; my processor is faster.My code is faster/shorter; my processor is faster.► My priority is higher.My priority is higher.► The probability is small for us both processes to do The probability is small for us both processes to do

this at the same time.this at the same time.

3.3. A process should not be blocked from A process should not be blocked from entering a critical region if all other entering a critical region if all other processes are outside the critical region.processes are outside the critical region.

4.4. No process should have to wait forever to No process should have to wait forever to get into its critical region.get into its critical region.

Mutual exclusion w/ busy Mutual exclusion w/ busy waitingwaiting

Methods to implement mutex:Methods to implement mutex:

1.1. Disable interruptsDisable interrupts

2.2. Lock variablesLock variables

3.3. Strict alternationStrict alternation

4.4. Peterson’s solutionPeterson’s solution

5.5. TSL instructionTSL instruction

Mutex method: disable Mutex method: disable interruptsinterrupts

►OK for (and used by) OSOK for (and used by) OS Consideration with MP systemsConsideration with MP systems

►NOT OK for appsNOT OK for apps

Mutex method: lock varsMutex method: lock vars

► Software methodSoftware method► Single, shared lock variable initially = 0Single, shared lock variable initially = 0► Busy wait Busy wait spin lock spin lock

shared int x=0;shared int x=0;//wait for lock//wait for lockwhile (x!=0) ; //while (x!=0) ; // note the empty statement note the empty statementx=1; //get lockx=1; //get lock//critical section//critical section……//end critical section//end critical sectionx=0; //release lockx=0; //release lock

►Doesn’t work (w/out hardware support).Doesn’t work (w/out hardware support).

Mutex method: strict Mutex method: strict alternationalternation

Mutex method: strict Mutex method: strict alternationalternation

►SoftwareSoftware►Problem: Violates #3. A process can Problem: Violates #3. A process can

be blocked from entering its C.S. by a be blocked from entering its C.S. by a process NOT in its C.S.process NOT in its C.S.

► In general, a process can’t be in it’s In general, a process can’t be in it’s C.S. 2x in a row.C.S. 2x in a row.

►The 2 processes must be running at The 2 processes must be running at about the same speed.about the same speed.

Mutex method: Peterson’s soln. Mutex method: Peterson’s soln. (software)(software)

Mutex method: TSL Mutex method: TSL instructioninstruction

► TSL RX, LOCKTSL RX, LOCK RX = register; LOCK = memory locationRX = register; LOCK = memory location Step 1: read contents of lock into RXStep 1: read contents of lock into RX Step 2: sets LOCK to 1Step 2: sets LOCK to 1 Indivisible instruction (non interruptible)Indivisible instruction (non interruptible) Memory, not cacheMemory, not cache Locks memory bus (so other processors can Locks memory bus (so other processors can

access/change LOCK)access/change LOCK)

► IA32 xchg and lock instructionsIA32 xchg and lock instructions

Mutex method: TSL Mutex method: TSL instructioninstruction

Priority inversion problemPriority inversion problem

►Unexpected consequence of busy waitUnexpected consequence of busy wait►Given H (a high priority job) and L (low Given H (a high priority job) and L (low

priority job)priority job)►Scheduling: whenever H is ready to Scheduling: whenever H is ready to

run, L is preempted and H is run.run, L is preempted and H is run.

Priority inversion problemPriority inversion problem

H runs…H runs…

H blocks on I/OH blocks on I/O

I/O completesI/O completes

H runs…H runs…

H attempts to enter C.S.H attempts to enter C.S.

H busy waits foreverH busy waits forever

L is ready to runL is ready to run

L runs…L runs…

L enters C.S….L enters C.S….

……

L is preemptedL is preempted

Using mutex (provided by Using mutex (provided by OS)OS)

►Simpler than semaphoreSimpler than semaphore

►Two states: locked or unlockedTwo states: locked or unlocked

►Functions:Functions: Declare mutex variableDeclare mutex variable Initialize mutex variable (just once)Initialize mutex variable (just once) Lock, C.S., unlockLock, C.S., unlock

#include <errno.h>#include <errno.h>#include <pthread.h>#include <pthread.h>……pthread_mutex_t mutex; //declare globalpthread_mutex_t mutex; //declare global……

//perform this one-time initialization (usually in main)//perform this one-time initialization (usually in main)int ret = pthread_mutex_init( &::mutex, NULL );int ret = pthread_mutex_init( &::mutex, NULL );if (ret) { perror( "main: mutex init error" ); exit(-1); }if (ret) { perror( "main: mutex init error" ); exit(-1); }……

//lock in thread code//lock in thread coderet = pthread_mutex_lock( &::mutex );ret = pthread_mutex_lock( &::mutex );if (ret) { printf( "%d: mutex lock error \n", tp-if (ret) { printf( "%d: mutex lock error \n", tp-

>whoAmI ); }>whoAmI ); } //critical section here//critical section here//unlock in thread code//unlock in thread codepthread_mutex_unlock( &::mutex );pthread_mutex_unlock( &::mutex );

Problem:Problem:

►Modify filter program to also Modify filter program to also determine overall min and max of determine overall min and max of input data.input data.

►Why do we need a mutex do to this Why do we need a mutex do to this (correctly in a multithreaded app)?(correctly in a multithreaded app)?

(Bounded(Bounded) )

producer-producer-consumer consumer problemproblem

(Bounded) Producer-consumer (Bounded) Producer-consumer problem problemsproblem problems

► Buffer is emptyBuffer is empty► Consumer checks count. It’s 0.Consumer checks count. It’s 0.► Scheduler interrupts consumer (puts Scheduler interrupts consumer (puts

comsumer on ready queue).comsumer on ready queue).► Producer runs.Producer runs.

Insert data into buffer.Insert data into buffer. Count is 1 so producer wakes up consumer.Count is 1 so producer wakes up consumer. But consumer is not asleep (yet)!But consumer is not asleep (yet)! Producer keeps inserting data into buffer until it’s Producer keeps inserting data into buffer until it’s

full. Then producer goes to sleep!full. Then producer goes to sleep!► Scheduler runs consumer. Consumer thinks Scheduler runs consumer. Consumer thinks

count=0 so it goes to sleep!count=0 so it goes to sleep!► Both sleep forever!Both sleep forever!

SemaphoresSemaphores

►Types:Types: POSIXPOSIX

►Shared only among threads only.Shared only among threads only.

System VSystem V►Can be shared according to user-group-other.Can be shared according to user-group-other.

Two basic operations:Two basic operations:►Up – increment the value of the semaphoreUp – increment the value of the semaphore►Down – decrement the value of the semaphoreDown – decrement the value of the semaphore

Binary semaphores (mutex)Binary semaphores (mutex)

►Create semaphore and initialize it to 1Create semaphore and initialize it to 1►ThenThen

downdown c.s.c.s. upup

POSIX SemaphoresPOSIX Semaphores

#include <semaphore.h>#include <semaphore.h>

int sem_init ( sem_t *sem, int pshared, int sem_init ( sem_t *sem, int pshared, unsigned int value );unsigned int value );

int sem_wait ( sem_t * sem );int sem_wait ( sem_t * sem );int sem_trywait ( sem_t * sem );int sem_trywait ( sem_t * sem );int sem_post ( sem_t * sem );int sem_post ( sem_t * sem );int sem_getvalue ( sem_t * sem, int * sval );int sem_getvalue ( sem_t * sem, int * sval );int sem_destroy ( sem_t * sem );int sem_destroy ( sem_t * sem );

System V SemaphoresSystem V Semaphores

► #include <sys/types.h>#include <sys/types.h>► #include <sys/ipc.h>#include <sys/ipc.h>► #include <sys/sem.h>#include <sys/sem.h>

► int semget ( key_t key, int nsems, int semflg );int semget ( key_t key, int nsems, int semflg ); create/access existingcreate/access existing

► int semctl ( int semid, int semnum, int int semctl ( int semid, int semnum, int cmd, ... );cmd, ... ); delete from systemdelete from system

► int semop ( int semid, struct sembuf *sops, int semop ( int semid, struct sembuf *sops, unsigned nsops );unsigned nsops ); used for used for upup and and downdown

Create/access existingCreate/access existing

//using the key, get the semaphore id//using the key, get the semaphore idconst int sid = semget( mySemKey, 1, IPC_CREAT | const int sid = semget( mySemKey, 1, IPC_CREAT |

0700 );0700 );

if (sid==-1) {if (sid==-1) { perror( "semget: " );perror( "semget: " ); exit( -1 );exit( -1 );}}printf( "sem id=%d \n", sid );printf( "sem id=%d \n", sid );

create if necessary

system-wide permissions

system-wide unique number

Access and deleteAccess and delete

//using the key, get the semaphore id//using the key, get the semaphore idconst int sid = semget( mySemKey, 1, 0700 );const int sid = semget( mySemKey, 1, 0700 );

if (sid==-1) {if (sid==-1) { perror( "semget: " );perror( "semget: " ); exit( -1 );exit( -1 );}}printf( "sem id=%d \n", sid );printf( "sem id=%d \n", sid );

//delete the semaphore//delete the semaphoresemctl(sid, 0, IPC_RMID, 0);semctl(sid, 0, IPC_RMID, 0);

Down functionDown function

static void down ( const int whichSid ) {static void down ( const int whichSid ) { struct sembuf sem_lock;struct sembuf sem_lock;

sem_lock.sem_num = 0; //semaphore number: 0 = sem_lock.sem_num = 0; //semaphore number: 0 = firstfirst

sem_lock.sem_op = -1; //semaphore operationsem_lock.sem_op = -1; //semaphore operation sem_lock.sem_flg = 0; //operation flagssem_lock.sem_flg = 0; //operation flags if (semop(whichSid, &sem_lock, 1) == -1) {if (semop(whichSid, &sem_lock, 1) == -1) { perror("semop");perror("semop"); exit(-1);exit(-1); }}}}

Up functionUp function

static void up ( const int whichSid ) {static void up ( const int whichSid ) { struct sembuf sem_unlock;struct sembuf sem_unlock;

sem_unlock.sem_num = 0; //semaphore number: 0 = sem_unlock.sem_num = 0; //semaphore number: 0 = firstfirst

sem_unlock.sem_op = 1; //semaphore operationsem_unlock.sem_op = 1; //semaphore operation sem_unlock.sem_flg = 0; //operation flagssem_unlock.sem_flg = 0; //operation flags if (semop(whichSid, &sem_unlock, 1) == -1) {if (semop(whichSid, &sem_unlock, 1) == -1) { perror("semop");perror("semop"); exit(-1);exit(-1); }}}}

Solution to (bounded) producer-consumer problem using semaphores.

Shared memory, pipes, and Shared memory, pipes, and named pipesnamed pipes

► Shared memoryShared memory shmgetshmget shmopshmop shmctlshmctl shmatshmat shmdtshmdt

► PipesPipes pipepipe popenpopen

► Named pipesNamed pipes MkfifoMkfifo

► Message queuesMessage queues

Other IPC mechanisms:Other IPC mechanisms:

►MonitorsMonitors►Message passing (MPI)Message passing (MPI)►BarriersBarriers