Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java...
-
Upload
epamsystemsbulgaria -
Category
Documents
-
view
58 -
download
1
Transcript of Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java...
![Page 1: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/1.jpg)
Avoiding and Diagnosing
DEADLOCKS
Danail Branekov
July 4, 2015
![Page 2: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/2.jpg)
2CONFIDENTIAL
CREDITS
Dr Heinz Kabutz
• Born in Cape Town, South Africa, now lives in Greece
• Has a PhD in Computer Science
• Created the Java Specialists’ Newsletter
• One of the first Sun Java Champions
• Carries out Java training courses all over the world
• Regular speaker at all the major Java conferences
This presentation is based on materials from Heinz’ Extreme Java – Concurrency and
Performance training with his explicit permission
![Page 3: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/3.jpg)
3CONFIDENTIAL
FROM THE CLASSIC
• Classic problem is that of the dining philosophers which we change to drinking
philosophers
• This is where the word symposium comes from:
• sym – together, such as in symphony
• poto - drink
• Ancient Greek philosophers used to get together to drink & think
• In our example a philosopher needs two glasses to drink
• First he takes the right one, then the left one
• When he finishes drinking he returns them and carries on thinking
![Page 4: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/4.jpg)
4CONFIDENTIAL
WORST CASE SCENARIO
Table is ready, all philosophers are thinking
1
5 2
4 3
![Page 5: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/5.jpg)
5CONFIDENTIAL
WORST CASE SCENARIO
Philosopher 5 wants to drink, takes right cup
1
5 2
4 3
![Page 6: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/6.jpg)
6CONFIDENTIAL
WORST CASE SCENARIO
Philosopher 1 wants to drink, takes right cup
5 2
4 3
1
![Page 7: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/7.jpg)
7CONFIDENTIAL
2
WORST CASE SCENARIO
Philosopher 2 wants to drink, takes right cup
5
4 3
1
![Page 8: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/8.jpg)
8CONFIDENTIAL
3
2
WORST CASE SCENARIO
Philosopher 3 wants to drink, takes right cup
5
4
1
![Page 9: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/9.jpg)
9CONFIDENTIAL
4 3
2
WORST CASE SCENARIO
Philosopher 4 wants to drink, takes right cup
5
1
![Page 10: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/10.jpg)
10CONFIDENTIAL
4 3
2
WORST CASE SCENARIO
DEADLOCK
• All philosophers are waiting
for their left cups but they
will never become available
5
1
![Page 11: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/11.jpg)
11CONFIDENTIAL
GLOBAL ORDER WITH BOOZING
PHILOSOPHERS
We can solve the deadlock with the drinking philosophers by requiring that locks are
always acquired in a set order
• For example, we can make a rule that philosophers always first take the cup with the
largest number
• And return the cup with the lowest number first
![Page 12: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/12.jpg)
12CONFIDENTIAL
GLOBAL LOCK ORDERING
Table is ready, all philosophers are thinking
1
5 2
4 3
1 2
3
4
5
![Page 13: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/13.jpg)
13CONFIDENTIAL
5
GLOBAL LOCK ORDERING
Philosopher 5 takes cup 5
• remember our rule!1
2
4 3
1 2
3
4
5
![Page 14: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/14.jpg)
14CONFIDENTIAL
1
5
GLOBAL LOCK ORDERING
Philosopher 1 takes cup 2
• remember our rule!
2
4 3
1
3
4
5
2
![Page 15: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/15.jpg)
15CONFIDENTIAL
2
1
5
GLOBAL LOCK ORDERING
Philosopher 2 takes cup 3
4 3
1
4
5
2
3
![Page 16: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/16.jpg)
16CONFIDENTIAL
3
2
1
5
GLOBAL LOCK ORDERING
Philosopher 3 takes cup 4
4
1
5
2
3
4
![Page 17: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/17.jpg)
17CONFIDENTIAL
3
2
1
5
GLOBAL LOCK ORDERING
Philosopher 1 takes cup 1 - Drinking
4
5
2
3
4
1
![Page 18: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/18.jpg)
18CONFIDENTIAL
1
3
25
GLOBAL LOCK ORDERING
Philosopher 1 finished drinking, returns cup 1
• Cups are returned in the opposite
order to what they are
acquired
4
5
2
3
4
1
![Page 19: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/19.jpg)
19CONFIDENTIAL
1
3
25
GLOBAL LOCK ORDERING
Philosopher 5 takes cup 1 - Drinking
4
5
2
3
4
1
![Page 20: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/20.jpg)
20CONFIDENTIAL
5
1
3
2
GLOBAL LOCK ORDERING
Philosopher 5 returns cup 1
4
5
2
3
4
1
![Page 21: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/21.jpg)
21CONFIDENTIAL
5
1
3
2
GLOBAL LOCK ORDERING
Philosopher 1 returns cup 2
4
5
2
3
4
1
![Page 22: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/22.jpg)
22CONFIDENTIAL
5
1
3
2
GLOBAL LOCK ORDERING
Philosopher 2 takes cup 2 - Drinking
4
5
2
3
4
1
![Page 23: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/23.jpg)
23CONFIDENTIAL
5
1
3
2
GLOBAL LOCK ORDERING
Philosopher 5 returns cup 5
4
5
2
3
4
1
![Page 24: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/24.jpg)
24CONFIDENTIAL
4
5
1
3
2
GLOBAL LOCK ORDERING
Philosopher 4 takes cup 5
5
2
3
4
1
![Page 25: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/25.jpg)
25CONFIDENTIAL
2
4
5
1
3
GLOBAL LOCK ORDERING
Philosopher 2 returns cup 2
5
2
3
4
1
![Page 26: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/26.jpg)
26CONFIDENTIAL
2
4
5
1
3
GLOBAL LOCK ORDERING
Philosopher 2 returns cup 3
5
2
3
4
1
![Page 27: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/27.jpg)
27CONFIDENTIAL
2
4
5
1
3
GLOBAL LOCK ORDERING
Philosopher 3 takes cup 3 - Drinking
5
2
3
4
1
![Page 28: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/28.jpg)
28CONFIDENTIAL
3
2
4
5
1
GLOBAL LOCK ORDERING
Philosopher 3 returns cup 3
5
2
3
4
1
![Page 29: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/29.jpg)
29CONFIDENTIAL
3
2
4
5
1
GLOBAL LOCK ORDERING
Philosopher 3 returns cup 4
5
2
3
4
1
![Page 30: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/30.jpg)
30CONFIDENTIAL
3
2
4
5
1
GLOBAL LOCK ORDERING
Philosopher 4 takes cup 4 - Drinking
5
2
3
4
1
![Page 31: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/31.jpg)
31CONFIDENTIAL
4 3
25
1
GLOBAL LOCK ORDERING
Philosopher 4 returns cup 4
5
2
3
4
1
![Page 32: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/32.jpg)
32CONFIDENTIAL
4 3
25
1
GLOBAL LOCK ORDERING
Philosopher 4 returns cup 5
5
2
3
4
1
![Page 33: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/33.jpg)
33CONFIDENTIAL
LOCK ORDERING AS JAVA CODE
public class Symposium {
private final Cup[] cups;
private final Thinker[] thinkers;
public Symposium(int delegates) {
cups = new Cup[delegates];
thinkers = new Thinker[delegates];
for (int i = 0; i < cups.length; i++) {
cups[i] = new Cup();
}
for (int i = 0; i < delegates; i++) {
Cup right = cups[i];
Cup left = cups[(i + 1) % delegates];
thinkers[i] = new Thinker(i, left, right);
}
}
![Page 34: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/34.jpg)
34CONFIDENTIAL
LOCK ORDERING AS JAVA CODE
public class Symposium {
. . .
public ThinkerStatus run() throws InterruptedException {
ExecutorService exec = Executors.newCachedThreadPool();
for (Thinker thinker : thinkers) {
exec.submit(thinker);
}
. . .
}
}
![Page 35: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/35.jpg)
35CONFIDENTIAL
LOCK ORDERING AS JAVA CODE
public class Thinker implements Callable<ThinkerStatus> {
private final int id;
private final Cup smaller, bigger;
private int drinks = 0;
public Thinker(int id, Cup right, Cup left) {
this.id = id;
this.bigger = right.compareTo(left) > 0 ?
right : left;
this.smaller = this.bigger == right ?
left : right;}
public ThinkerStatus call() throws Exception {
for (int i = 0; i < 1000; i++) {drink(); think();}return drinks == 1000 ? ThinkerStatus.HAPPY_THINKER :
ThinkerStatus.UNHAPPY_THINKER;
}
. . .
public void think() { . . . }
}
![Page 36: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/36.jpg)
36CONFIDENTIAL
LOCK ORDERING AS JAVA CODE
public class Thinker implements Callable<ThinkerStatus> {
private final Cup smaller, bigger;
. . .
public Thinker(int id, Cup right, Cup left) {
this.bigger = right.compareTo(left) > 0 ?
right : left;
this.smaller = this.bigger == right ?
left : right;
}
public void drink() {
synchronized (bigger) {
synchronized (smaller) {
drinking();
}
}
}
private void drinking() { drinks++; }
}
![Page 37: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/37.jpg)
37CONFIDENTIAL
LOCK ORDERING AS JAVA CODE (VER 2)
public class Thinker implements Callable<ThinkerStatus> {
private static final Object TIE_LOCK = new Object();
. . .
public void drink() {
long leftHashCode = System.identityHashCode(left);
long rightHashCode = System.identityHashCode(right);
if (leftHashCode > rightHashCode) {
synchronized (left) {
synchronized (right) { drinking(); }
}
} else if (leftHashCode < rightHashCode) {
synchronized (right) {
synchronized (left) { drinking(); }
}
} else {
synchronized (TIE_LOCK) {
synchronized (right) {
synchronized (left) { drinking(); }
}
}
}
![Page 38: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/38.jpg)
38CONFIDENTIAL
DEADLOCKING COOPERATING OBJECTS
In this example, the deadlock is more subtle
• Taxi is an individual taxi with a location and destination
• Dispatcher represents a fleet of taxis, capable of providing an image representing the
current location of taxis
Spot the deadlock
![Page 39: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/39.jpg)
39CONFIDENTIAL
TAXI, REPRESENTING AN INDIVIDUAL VEHICLE
public class Taxi {
private Point location, destination;
private Dispatcher dispatcher;
public void setDispatcher(Dispatcher dispatcher) {
this.dispatcher = dispatcher;
}
public void setDestination(Point destination) {
this.destination = destination;
}
public synchronized Point getLocation() {
return location;
}
public synchronized void setLocation(Point location) {
this.location = location;
if (location.equals(destination))
dispatcher.notifyAvailable(this);
}
}
}
![Page 40: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/40.jpg)
40CONFIDENTIAL
DISPATCHER, MANAGING A FLEET OF TAXIS
public class Dispatcher {
private final Set<Taxi> taxis;
private final Set<Taxi> availableTaxis;
public Dispatcher(Taxi... taxis) {
this.taxis = new HashSet<Taxi>(Arrays.asList(taxis));
this.availableTaxis = new
HashSet<Taxi>(Arrays.asList(taxis));
}
public synchronized void notifyAvailable(Taxi taxi) {
availableTaxis.add(taxi);
}
public synchronized Image getImage() {
Image image = new Image();
taxis.forEach(taxi -> image.drawMarker(taxi.getLocation()));
return image;
}
}
![Page 41: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/41.jpg)
41CONFIDENTIAL
HOW TO DEADLOCK THE TAXI INDUSTRY
AsetLocation()
synchronized(taxi)
dispatcher.notifyAvailable(taxi)
synchronized(dispatcher)
BgetImage()
synchronized(dispatcher)
taxi.getLocation()
synchronized(taxi)
BLOCKED
BLOCKED
![Page 42: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/42.jpg)
42CONFIDENTIAL
OPEN CALLS
• Calling an alien method with a lock held is difficult to analyze and therefore risky
• Both Taxi and Dispatcher violate this rule
• Calling a method with no locks held is called an open call
• Makes it much easier to reason about liveness
![Page 43: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/43.jpg)
43CONFIDENTIAL
REFACTORED Taxi.setLocation()
• We should not call alien methods whilst holding locks
• Here we split the method up into parts that need the lock and those that call alien
methods
public void setLocation(Point location) {
boolean reachedDestination;
synchronized (this) {
this.location = location;
reachedDestination = location.equals(destination);
}
if (reachedDestination) {
dispatcher.notifyAvailable(this);
}
}
![Page 44: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/44.jpg)
44CONFIDENTIAL
REFACTORED Dispatcher.getImage()
• We make a copy of the taxis set to prevent race conditions
public Image getImage() {
Set<Taxi> copy;
synchronized (this) {
copy = new HashSet<>(taxis);
}
Image image = new Image();
copy.forEach(taxi -> image.drawMarker(taxi.getLocation()));
return image;
}
![Page 45: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/45.jpg)
45CONFIDENTIAL
DEADLOCK SOLVED
AsetLocation()
synchronized(taxi)
dispatcher.notifyAvailable(taxi)
synchronized(dispatcher)
BgetImage()
synchronized(dispatcher)
taxi.getLocation()
synchronized(taxi)
![Page 46: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/46.jpg)
46CONFIDENTIAL
ANOTHER SOLUTION
• An alternative solution would be to make the Taxi.location field volatile which allows
making the getLocation method not synchronized
public class Taxi {
private volatile Point location;
. . .
public Point getLocation() {
return location;
}
}
AsetLocation()
synchronized(taxi)
dispatcher.notifyAvailable(taxi)
synchronized(dispatcher)
BgetImage()
synchronized(dispatcher)
taxi.getLocation()
synchronized(taxi)
![Page 47: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/47.jpg)
47CONFIDENTIAL
BENEFITS OF OPEN CALLS
• Strive to use open calls throughout your program
• Programs that rely on open calls are far easier to analyze for deadlock-freedom than
those that allow calls to alien methods with locks held
• Alien method calls with lock held are probably the biggest cause of deadlocks in the
field
![Page 48: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/48.jpg)
48CONFIDENTIAL
OPEN CALL IN Vector
• In Sun Java 6 Vector.writeObject() is synchronized
private synchronized void writeObject(java.io.ObjectOutputStream s)
throws java.io.IOException {
s.defaultWriteObject();
}
• The reason to have the method synchronized is to provide thread safety during writing
• However, since it calls the alien s.defaultWriteObject() it can deadlock
• http://www.javaspecialists.eu/archive/Issue184.html
![Page 49: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/49.jpg)
49CONFIDENTIAL
OPEN CALL IN Vector
IBM avoids this problem with an open call
private void writeObject(ObjectOutputStream stream) throws IOException {
Vector<E> cloned = null;
// this specially fix is for a special dead-lock in customer
// program: two vectors refer each other may meet dead-lock in
// synchronized serialization. Refer CMVC-103316.1
synchronized (this) {
try {
cloned = (Vector<E>) super.clone();
cloned.elementData = elementData.clone();
} catch (CloneNotSupportedException e) {
// no deep clone, ignore the exception
}
}
cloned.writeObjectImpl(stream);
}
private void writeObjectImpl(ObjectOutputStream stream) throws IOException {
stream.defaultWriteObject();}
![Page 50: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/50.jpg)
50CONFIDENTIAL
OPEN CALL IN Vector
OpenJDK 7 also uses an open call
private void writeObject(ObjectOutputStream s) throws IOException {
final ObjectOutputStream.PutField fields = s.putFields();
final Object[] data;
synchronized (this) {
fields.put("capacityIncrement", capacityIncrement);
fields.put("elementCount", elementCount);
data = elementData.clone();
}
fields.put("elementData", data);
s.writeFields();
}
![Page 51: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/51.jpg)
51CONFIDENTIAL
RESOURCE DEADLOCKS
• We can also cause deadlocks waiting for resources
• For example, you may have two DB connection pools (D1 and D2)
• Some tasks might require connection to both databases
• Thus thread A might hold a semaphore for D1 and wait for D2, while thread B
might hold a semaphore for D1 and be waiting for D2
public class DatabasePool {
private final Semaphore connections;
public DatabasePool(int connections) {
this.connections = new Semaphore(connections);
}
public void connect() {
connections.acquireUninterruptibly();System.out.println("connect");
}
public void disconnect() {
System.out.println("disconnect");connections.release();
}
}
• NOTE: Thread dump and ThreadMXBean do not show this as deadlock!
![Page 52: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/52.jpg)
52CONFIDENTIAL
ANTI-DEADLOCK BEST PRACTICES
• If you only ever acquire one lock, you cannot get a lock-ordering lock
• This is the easiest way to avoid deadlocks but it is not always practical
• If you need to acquire multiple locks, include lock ordering in your design
• It is important to specify and document possible lock sequences
• Identify where multiple locks could be acquired
• Do a global analysis to ensure that lock ordering is consistent (could be
extremely difficult on large programs)
• Use open calls when possible
• Do not call alien methods whilst holding a lock
![Page 53: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/53.jpg)
53CONFIDENTIAL
UNIT TESTING FOR LOCK ORDERING
DEADLOCKS
• Code typically has to be called many times before a deadlock occurs
• How many times do you need to call it to prove that there is no deadlock?
• Nondeterministic unit tests are bad, a unit test should either always fail, or always
pass
![Page 54: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/54.jpg)
54CONFIDENTIAL
UNIT TESTING FOR LOCK ORDERING
DEADLOCKS
Consider the following example
public class Bank {
public boolean transferMoney(Account from,
Account to, DollarAmount amount) {
synchronized (from) {
synchronized (to) {
return doActualTransfer(from, to, amount);
}
}
}
}
In the transferMoney method a deadlock occurs if after the first lock (from) is granted,
the first thread is swapped out and another thread requests the second lock (to)
![Page 55: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/55.jpg)
55CONFIDENTIAL
UNIT TESTING FOR LOCK ORDERING
DEADLOCKS
• We can force the deadlock by sleeping a short while after requesting the first lock
public class Bank {
public boolean transferMoney(Account from,
Account to, DollarAmount amount) {
synchronized (from) {
sleepAWhileForTesting();
synchronized (to) {
return doActualTransfer(from, to, amount);
}
}
}
protected void sleepAWhileForTesting() {}
}
• The empty sleepAWhileForTesting method will be optimized away by the HotSpot
compiler
![Page 56: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/56.jpg)
56CONFIDENTIAL
UNIT TESTING FOR LOCK ORDERING
DEADLOCKS
• In our unit test we extend the class and use it instead
public class SlowBank extends Bank {
private final long timeout;
private final TimeUnit unit;
public SlowBank(long timeout, TimeUnit unit) {
this.timeout = timeout; this.unit = unit;
}
@Override
protected void sleepAWhileForTesting() {
try {
unit.sleep(timeout);
} catch (InterruptedException e) { . . . }
}
}
• We can then implement a parallel task which checks for thread deadlocks and fail the
test in case of one
![Page 57: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/57.jpg)
57CONFIDENTIAL
FINDING DEADLOCKED THREADS
• ThreadMXBean has two methods for finding deadlocks
• findMonitorDeadlockedThreads()
• Include only “monitor” locks, i.e. synchronized
• The only way to find deadlock in Java 5
• findDeadlockedThreads()
• Include “monitor” and “owned” (Java 5) locks
• Preferred method to test for deadlocks
• Does not find deadlocks between semaphores
• Both methods are designed for troubleshooting use but not for synchronization control
• Finding deadlocked threads might be an expensive operation
• http://www.javaspecialists.eu/archive/Issue130.html
![Page 58: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/58.jpg)
58CONFIDENTIAL
FINDING DEADLOCKED THREADS
private static final ThreadMXBean tmb =
ManagementFactory.getThreadMXBean();
public Collection<Thread> findDeadlockedThreads() {
long[] ids = tmb.findDeadlockedThreads();
if (ids == null) { return Collections.emptyList();}
return findThreadsByIds(ids);
}
. . .
private Thread findThreadById(long threadId) {
for (Thread thread : Thread.getAllStackTraces().keySet()) {
if (thread.getId() == threadId)
return thread;
}
return null;
}
![Page 59: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/59.jpg)
59CONFIDENTIAL
DEADLOCK ANALYSIS WITH THREAD DUMPS
• We can also analyze deadlocks via looking at the thread dump
• Thread dump can be caused in many ways:
• Ctrl-Break on Windows or Ctrl-\ on Unix
• Invoking kill -3 on the process id
• Calling jstack on the process id (only shows deadlocks since Java 6)
• It is useful to have unique thread names
![Page 60: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/60.jpg)
60CONFIDENTIAL
DEADLOCK ANALYSIS WITH THREAD DUMPS
Found one Java-level deadlock:
=============================
"pool-1-thread-2":
waiting to lock monitor 0x0000000002ae5bb8 (object 0x000000076b814b08, a
taxi.deadlock.Dispatcher),
which is held by "pool-1-thread-1"
"pool-1-thread-1":
waiting to lock monitor 0x000000001e08c458 (object 0x000000076b812b88, a
taxi.deadlock.Taxi),
which is held by "pool-1-thread-2"
![Page 61: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/61.jpg)
61CONFIDENTIAL
DEADLOCK ANALYSIS WITH THREAD DUMPS
Java stack information for the threads listed above:
===================================================
"pool-1-thread-2":
at taxi.deadlock.Dispatcher.notifyAvailable(Dispatcher.java:17)
- waiting to lock <0x000000076b814b08> (a taxi.deadlock.Dispatcher)
at taxi.deadlock.Taxi.setLocation(Taxi.java:30)
- locked <0x000000076b812b88> (a taxi.deadlock.Taxi)
at taxi.deadlock.TaxiDriver.run(TaxiDriver.java:25)
...
"pool-1-thread-1":
at taxi.deadlock.Taxi.getLocation(Taxi.java:19)
- waiting to lock <0x000000076b812b88> (a taxi.deadlock.Taxi)
at taxi.deadlock.Dispatcher.lambda$0(Dispatcher.java:22)
at taxi.deadlock.Dispatcher$$Lambda$1/710209934.accept(Unknown Source)
at java.lang.Iterable.forEach(Iterable.java:75)
at taxi.deadlock.Dispatcher.getImage(Dispatcher.java:22)
- locked <0x000000076b814b08> (a taxi.deadlock.Dispatcher)
at taxi.deadlock.TaxiObserver.run(TaxiObserver.java:14)
...
Found 1 deadlock.
![Page 62: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/62.jpg)
62CONFIDENTIAL
DEADLOCK ANALYSIS WITH THREAD DUMPS
ThreadMXBean (and JConsole) do not detect lock order deadlocks caused by acquiring
semaphoresSemaphore s1 = new Semaphore(1);
Semaphore s2 = new Semaphore(1);
ExecutorService execService = Executors.newCachedThreadPool();
execService.submit(new TestTask(s1, s2));
execService.submit(new TestTask(s2, s1));
static class TestTask implements Runnable {
public TestTask(Semaphore first, Semaphore second) {
this.first = first;
this.second = second;
}
public void run() {
try {
first.acquire();
second.acquire();
. . .
second.release();
first.release();
} catch (InterruptedException e) { . . . }
}
}
![Page 63: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/63.jpg)
63CONFIDENTIAL
DEADLOCK ANALYSIS WITH THREAD DUMPS
ThreadMXBean (and JConsole) do not detect lock order deadlocks caused by acquiring
semaphores
"pool-1-thread-1" #15 prio=5 os_prio=0 tid=0x000000001ddb2800 nid=0x1f78 waiting on
condition [0x000000001e69e000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x000000076b9637a8> (a
java.util.concurrent.Semaphore$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Abstract
QueuedSynchronizer.java:836)
. . .
![Page 64: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/64.jpg)
64CONFIDENTIAL
DEADLOCK RECOVERY
Can I recover from a deadlock?
• DISCLAIMER: Always prefer deadlock prevention via implementation best practices
over recovering from deadlocks which already took place. Recovering from deadlocks
should be only used as an extreme measure
• NO, if you deadlocked with synchronized
• synchronized goes into BLOCKED state
• YES, with ReentrantLock
• ReentrantLock.lock() and ReentrantLock.lockInterruptibly() go into WAITING
state
• Deadlocked threads must make sure that they release locks held when
interrupted or stopped
• We can implement a DeadlockArbitrator which interrupts or stops one of the
threads
![Page 65: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/65.jpg)
65CONFIDENTIAL
DEADLOCK RECOVERY VIA INTERRUPT
ReentrantLock first = new ReentrantLock();
ReentrantLock second = new ReentrantLock();
private class TestTask implements Runnable {
public void run() {
try {
first.lockInterruptibly();try {
// do some work
second.lockInterruptibly();try {
// do more work
} finally {
second.unlock();
}
} finally {
first.unlock();
}
} catch (InterruptedException e) { . . . }
}
![Page 66: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/66.jpg)
66CONFIDENTIAL
DEADLOCK RECOVERY VIA INTERRUPT
public class ThreadInterruptingDeadlockArbitrator {
public void tryResolveDeadlock() throws InterruptedException {
Thread t = findDeadlockedThread(); // via ThreadMXBean
if (t != null) {
t.interrupt();
}
}
}
![Page 67: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/67.jpg)
67CONFIDENTIAL
DEADLOCK RECOVERY VIA STOP
ReentrantLock first = new ReentrantLock();
ReentrantLock second = new ReentrantLock();
private class TestTask implements Runnable {
public void run() {
first.lock();try {
// do some work
second.lock();try {
// do more work
} finally {
second.unlock();
}
} finally {
first.unlock();
}
}
}
![Page 68: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/68.jpg)
68CONFIDENTIAL
DEADLOCK RECOVERY VIA STOP
public class ThreadInterruptingDeadlockArbitrator {
public void tryResolveDeadlock() throws InterruptedException {
Thread t = findDeadlockedThread(); // via ThreadMXBean
if (t != null) {
t.stop();
}
}
}
![Page 69: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/69.jpg)
69CONFIDENTIAL
LINKS
• Java Concurrency in Practice by Brian Goetz
• The Java Specialists’ Newsletter
• Sample source code
![Page 70: Tech Talks_04.07.15_Session 2_Danail Branekov_Avoiding And Diagnosing Deadlocks In Java Multithreaded Applications](https://reader034.fdocuments.in/reader034/viewer/2022052602/55c54b70bb61eb667c8b461c/html5/thumbnails/70.jpg)
70CONFIDENTIAL
THANK YOU!