Multi-threading in Java

Jeff HUANG

Software Engineering Group @HKUSTSoftware Engineering Group @HKUST

Do you use them?

2

Do u know their internals?

3

Let’s see

File

DBDBHow can they service so many li i l lclients simultaneously?

4

Multi-threading

A thread is a single sequential flow of controlIn a single process, there could be multiple threadsbe multiple threadsThreads share memory

• The server program spawns multiple threads• Each thread handles a client request

5

ExampleA single core CPU and a task

compute readfile compute readfile compute writefile compute

0 1 2 3 4 5 6 7 sec

2• 2 such tasks6

A simple accountingSingle thread

T=14T=14

Two threads T=8Two threads T=8

7

What’s moreMulti-core & multi-processor

4096 processor Cray 32 processor Dual-core AMD p yX1

pPentium XeonAthlon X2

8

More accountingNumber of cores = 1,000,000 = 1M

Multi-threading can do such tasks in 82MMulti-threading can do such tasks in 8 seconds!Wh t b t th i l th d d i ?

2M

What about the single-threaded version?

9

More benefits

PerformanceUser responsivenesspResource utilizationSi li it f d liSimplicity of modelingSimplified handling of asynchronous eventsp g y…

10

Multi-threads are everywhere

Server programs (web/database server)Client programs (Downloading tools/Browser)p g ( g )Editors (Word/Photoshop/Adobe Reader)S h iSearch engineIDE (Matlab/Eclipse/Visual studio)( p )JVMSwing and AWTSwing and AWT…

11

Where are we now?

How to program?p g

12

Various ways of multithreadingThreadRunnableE F kExectutor FrameworkTimerTimerFutureTask…

13

Two major waysThreadRunnableE F kExectutor FrameworkTimerTimerFutureTask…

14

Creating threads in JavaThread classpublic class Thread{

public void run(){…}; }

Runnable interfacepublic interface Runnable {public interface Runnable {

public void run(); // work ⇒ thread}}

15

Thread Class

public class Thread extends Object implements Runnable {

public Thread();public Thread();public Thread(String name); // Thread namepublic Thread(Runnable R); // Thread ⇒ R.run() p ( ); ()public Thread(Runnable R, String name);

bli id () // if R k f h dpublic void run(); // if no R, work for threadpublic void start(); // begin thread execution...

}

16

More Thread Class Methods

public class Thread extends Object i l t R bl {implements Runnable {

…public static Thread currentThread()public static Thread currentThread()public String getName()public void interrupt()p p ()public boolean isAlive()public void join()

bli id tD ()public void setDaemon()public void setName()public void setPriority()public void setPriority()public static void sleep()public static void yield()p y ()

}17

Creating Threads in Java

1. Thread classExtend Thread class and override the run method

Examplepublic class MyThread extends Thread {public class MyThread extends Thread {

public void run() {… // work for thread

}}M Th d T M Th d () // t th dMyThread T = new MyThread () ; // create thread

T.start(); // begin running threadT.start(); // begin running thread… // thread executing in parallel

18

Creating Threads in Java

2. Runnable interfaceCreate object implementing Runnable interfacePass it to Thread object via Thread constructor

E lExamplepublic class MyRunnable implements Runnable {

public void run() {public void run() {… // work for thread

}}}MyRunnable myR = new MyRunnable(); // create runnable object

Thread T = new Thread(myR); // create thread

T.start(); // begin running thread… // thread executing in parallel 19

Producer - Consumerclass Producer extends Thread {

class Consumer extends Thread {{

private final BlockingQueue queue;Producer(BlockingQueue q){queue = q;}public void run()

{ private final BlockingQueue queue;Consumer(BlockingQueue q) { queue = q; }public void run()

public static void main (String[] args) {

{while(true) {

queue put(produce());

{while(true) {

consume(queue take());

BlockingQueue q = new SomeQueue(); Producer p = new Producer(q);

queue.put(produce()); }

}private Message produce()

consume(queue.take()); }

}private void consume(Message mes)

Consumer c = new Consumer(q); p.start();

{ return new Message();

}}

{…

}}

c.start(); }

} }

Example 120

Example 1

Simple WebServerpublic class MyRunnable implements Runnable {

public void run() {public void run() {handleRequest(connection);

}}

Process request

public class SimpleWebServer{

}

public static void main (String[] args) throws IOException {ServerSocket socket = new ServerSocket(80);while (true) {

fi l S k t ti k t t()Accept client request

final Socket connection = socket.accept();Runnable task = new MyRunnable(connection);Thread t = new Thread(task);t start(); t t th d

Create threadCreate Runnable

t.start();}

}}

start thread

}21Example 2

Let’s look a bit deeperThread statesThread schedulerDiffi l i i M l i h d dDifficulties in Multithreaded programmingp g g

22

Thread StatesJava thread can be in one of these states

New – thread allocated & waiting for start()Runnable – thread can begin executionRunning – thread currently executingBlocked – thread waiting for event (I/O, etc.)Dead – thread finished

Transitions between states caused byyInvoking methods in class Thread

new(), start(), yield(), sleep(), wait(), notify()…(), (), y (), p(), (), y()

Other (external) eventsScheduler, I/O, returning from run()…, , g ()

23

Thread States

runnablenewnew start

notify notifyAllrunnable

scheduler

new

yield,

notify, notifyAll,IO complete, sleep expired

running blocked

ytime slice

running blocked

terminateIO, sleep,wait, join

dead

24

Thread Scheduling

SchedulerDetermines which runnable threads to runCan be based on thread priorityCan be based on thread priorityPart of OS or Java Virtual Machine (JVM)

Preempted byScheduling policy

Non-preemptive (cooperative) scheduling

Preempted by scheduler

Non preemptive (cooperative) schedulingPreemptive scheduling

25

Let’s do a bit analysisTwo threads A and B

They both do the following workprint 1, print 2, print 3

The main thread first starts A, then starts B

main (): thread A, thread B

thread A: println 1, println 2, println 3

thread B: println 1, println 2, println 3

26

Let’s do a bit analysisNon-preemptive Preemptive scheduler scheduler

Single-core ???1, 2, 3, 1, 2, 3, , , , ,Multi-core ??? ???

main (): thread A, thread B

thread A: println 1, println 2, println 3

thread B: println 1, println 2, println 3

27

Java Thread Examplepublic class ThreadExample extends Thread {

public void run() {for (int i = 1; i <= 3; i++)

System.out.println(i);y p ( );try {

sleep((int)(Math.random() * 1000));} catch (InterruptedException e) { }} catch (InterruptedException e) { }

}

public static void main(String[] args) {public static void main(String[] args) {Thread A = new ThreadExample();Thread B = new ThreadExample();A t t()A.start();B.start();

System.out.println("Done");}

} 28

Java Thread Example

Possible outputsPossible outputs1,1,Done,2,2,3,3 1 Done 1 2 2 3 31,Done,1,2,2,3,31,1,Done,2,3,2,3 Done 1 1 2 2 3 3Done,1,1, 2,2,3,3…

Multi-threads can interleave their executions!!!

29

Another examplepublic class Worker extends Thread {

static int x = 0;static int x 0;public void run() {

for (int i = 0; i < 1000; i++) {x = x + 1; First increase x by 1x = x + 1;x = x – 1;

}}

First increase x by 1Then decrease x by 1

}public static void main (String[] args) {

x = 0;for (int i = 0; i < 1000; i++)

new Worker().start(); Start 1000 threads

… // wait for all threads exit

System.out.println(x);System.out.println(x);}

} 30

Data Racepublic class Worker extends Thread {

static int x = 0;static int x 0;public void run() {

for (int i = 0; i < 1000; i++) {x = x + 1; First increase x by 1x = x + 1;x = x – 1;

}}

First increase x by 1Then decrease x by 1

}public static void main (String[] args) {

x = 0;for (int i = 0; i < 1000; i++)

new Worker().start(); Start 1000 threads

… // wait for all threads exit

System.out.println(x); X is not always 0 !!System.out.println(x);}

} 31

X is not always 0 !!

Quiz time

32

Answer: Yes!

33

How Can This Happen?

Compiler can reorder statementsOr keep values in registersp g

The Java memory model is designed to allow aggressive optimizationaggressive optimizationOn multi-processor, values not synchronized to global memoryGood for performancep

34

Observations

Multithread execution is non-deterministicDepends on scheduler & single or multi-core

Thread scheduling may cause data racesThread scheduling may cause data racesModifying same data from multiple threadsResult depends on thread execution orderResult depends on thread execution order

Complier can reorder statementsThe memory model is designed to allow aggressive optimization

35

How to deal with it?

We need protocol to make sure the shared pdata is manipulated safelyWe need to force the compiler to ensure thatWe need to force the compiler to ensure that the modified shared data visible to other th dthreadsSynchronization

LockSemaphore Java synchronizationpMonitor

y

36

Java synchronization

Java uses the concept of monitorsJava uses the concept of monitorsJava uses the concept that every object is

i t d ith l kassociated with a lockuse synchronized keywordTwo ways

Synchronized methodsSynchronized methodsSynchronized blocks

37

Synchronized methodCalling a synchronized method attempts to possess the lock

If no one owns the lock, then this thread has/owns the lock and proceeds.Otherwise, it has to wait until the lock is released by some other thread

LockFor static method, it is the lock associated with the Class object to which the static method belongs toFor instance method, it is the lock associated with th bj t hi h th th d ll i b i dthe object on which the method call is being made

38

Synchronized blocks

Very similar to synchronized method but it isVery similar to synchronized method, but it is Blocks of code, rather than entire methods

public void someMethod() {{

// non-critical sectionsynchronized(someobject) synchronized(this){

// critical section}}// non-critical section

}}

39

Data Racepublic class Worker extends Thread {

static int x = 0;static int x 0;public void run() {

for (int i = 0; i < 1000; i++) {x = x + 1; First increase x by 1x = x + 1;x = x – 1;

}}

First increase x by 1Then decrease x by 1

}public static void main (String[] args) {

x = 0;for (int i = 0; i < 1000; i++)

new Worker().start(); Start 1000 threads

… // wait for all threads exit

System.out.println(x); X is not always 0 !!System.out.println(x);}

} 40

X is not always 0 !!

Using synchronization

public class Worker extends Thread {public class Worker extends Thread {static int x = 0;static Object lock = new Object();j jpublic void run() {

synchronized(lock) {f (i t i 0 i < 1000 i++) {for (int i = 0; i < 1000; i++) {

x = x + 1;x = x – 1;x x 1;

}}

}}

41

QuestionsWhat would happen if the lock field were not static?Why don’t we just make the run methodWhy don’t we just make the run method synchronized?Why don’t we just synchronize on x?

42

But, it is hard in practice

Holding locks is a global propertyHolding locks is a global propertyaffects entire program, cannot be hidden behind an abstract interfacean abstract interface

Results in lack of modularitycallers cannot ignore what locks their calleesacquire or what locations they access

f id b t l f l b lnecessary for race avoidance, but also for global ordering to avoid deadlock part of a method’s protocol which lock needs topart of a method’s protocol which lock needs to be held when called, which locks it acquires

43

Bank Account Example

l A t {

Atomicity violationclass Account {private int balance = 0;

public void deposit(int n) {int r read();public read() {

int r;synchronized(this) {

int r = read();

synchronized(this) { b l

other threads can update balanceyr = balance;

}return r;

balance = r + n; }

}return r;}

}}

Race freedom is not sufficientRace-freedom is not sufficient44

Optimized Bank Account

l A t {class Account {private int balance = 0;

public void deposit(int n) {synchronized(this) { public read() {

return balance;}

synchronized(this) { int r = balance;balance = r + n;

}

}

}}

45

Another Account Exampleclass Account {

private int balance = 0;private int balance = 0;public synchronized int withdraw(int amt){…}public synchronized void deposit(int i) {…}

}}class Client1{

public synchronized void move(Account a1, Account a2){

a2.deposit(a1.withdraw(1000000));}

}class Client2 … // same as Client1

Client1: move(a1,a2);Client move(a ,a );Client2: move(a2,a1); 46

Deadlockclient1 holds lock for account a1, waits for account a2client2 holds lock for account a2, waits for ,account a1Both of them can not proceedBoth of them can not proceed

Cli 1

a1 a2Client1 Client2

a1 a2

47

Impair performance

Frequent lock acquire and lock releaseFrequent lock acquire and lock release operationsWe are sacrificing performance for dataWe are sacrificing performance for data safetyBut sometimes there is no conflictOn some extreme situations, performance isOn some extreme situations, performance is greatly impaired

48

Is it hard?

49

Let’s make a conclusion

Multithreading is quite usefulg qBut you need to be carefulBugs are everywhere

50

What we have learned

Benefits of multithreadsPerformanceUser responsivenessp…

How to make multithreaded Java programsHow to make multithreaded Java programsThread class/Runnable interface

Diffic lties in M ltithreadingDifficulties in MultithreadingStatements reorder/Data races/Atomicity i l ti /D dl kviolation/Deadlock

Java synchronizationSyncrhonized methods/blocks

51

QuestionsWhat would happen if the lock field were not static?Why don’t we just make the run methodWhy don’t we just make the run method synchronized?Why don’t we just synchronize on x?

52

Using synchronization

public class Worker extends Thread {public class Worker extends Thread {static int x = 0;static Object lock = new Object();j jpublic void run() {

synchronized(lock) {f (i t i 0 i < 1000 i++) {for (int i = 0; i < 1000; i++) {

x = x + 1;x = x – 1;x x 1;

}}

}}

53

See you next time

Backupsp

Daemon Threads

Java threads typesUserDaemon

Provide general services Typically never terminateCall setDaemon() before start()

Program termination1. All user threads finish2. Daemon threads are terminated by JVM3. Main program finishesp g

56

Atomicity violation

57

Thread flow graph

58

Non-preemptive Scheduling

Threads continue execution until Thread terminatesExecutes instruction causing wait (e.g., IO)Thread volunteering to stop (invoking yield or sleep)

59

Preemptive Scheduling

Threads continue execution untilSame reasons as non-preemptive schedulingPreempted by scheduler

60

Transactional Memory

Instead of using synchronizations, the runtime system ensures that the critical section executes t ti lltransactionallyYou can just go ahead when entering a critical

ti h iti th iti l ti thsection; when you are exiting the critical section, the runtime system will check the data safety for you

if th i fli t j t dif there is no conflict, just proceedOtherwise, rollback

The transaction memory system guaranteesThe transaction memory system guaranteesother threads cannot see intermediate values of the transaction (all-or-nothing)( g)the transaction cannot see values of other transactions in the middle of its execution 61

Hardware & software TM

Hardware transactional memory(HTM)Software transactional memory (STM)Software transactional memory (STM)

62

Advantages

No deadlock!we never refer to locks explicitly

Composability & modularityp y yno need to know what callees do w.r.t. synchronization

Performance could better in some situations

63