Thread Blocking Is Evil

Game Network Component Team

Kim, J.H.

CONCURRENCY

Why is Thread Blocking

a Concern?

“The Free Lunch is OVER.”

Herb SutterDecember 2004

Clock speedExecution optimizationCache

Cache

HyperthreadingMulticoreCache

Write Multithreaded Application

CONCURRENCY

Waiting in Line

Thread Blocking

Starvation

Deadlock

Starvation

Task A Task B

1: Lock 1: Sleep 0 msec

2: Sleep 1msec

3: Unlock

Pro

cess

ti

me(m

s)

# of Thread

1 2 3 4 5 6 7 8 9984986988990992994996998

1000

Task BTask A

DeadlockHold & Wait

Thread A Thread B

1A: Lock 1B: Lock

2A: Request 2B: Send Response

3A: Wait for Response 3B: Unlock

4A: Do something with Response

5A: Unlock

Thread A Thread B

1A: Lock 1B: Lock

2A: Request 2B: Send Response

3A: Wait for Response 3B: Unlock

4A: Do something with Response

5A: Unlock

My Prob-lem

Framework for Lobby-Room style

Game

Remote Procedure Call

Easy to write code be-cause it’s all contained

inside one function.

Game Client

LobbyRoom‘Enter Room’

RPC‘Authentication’

RPC

Game Client

LobbyRoom

‘Enter Room’

RPC func-tion body

‘Authenti-cation RPC

‘Function

body

Call ‘Enter Room’ RPC

Call ‘Authentica-tion’ RPC

Return ‘Authentication’ RPC

Return ‘Enter Room’ RPC

Game Client

LobbyRoom

Call ‘Enter Room’ RPC

Call ‘Authentica-tion’ RPC

Return ‘Authentication’ RPC

Return ‘Enter Room’ RPC

‘Enter Room’

RPC func-tion body

‘Authenti-cation RPC

‘Function

body

Game Client

LobbyRoom

‘Enter Room’ RPC

function body

‘Authenti-cation RPC

‘Function

body

Call ‘Enter Room’ RPC

Call ‘Authentica-tion’ RPC

Return ‘Authentication’ RPC

Return ‘Enter Room’ RPC

Response DelayFrom Lobby Server

Game Client

LobbyRoom

‘Enter Room’ RPC

function body

‘Authenti-cation RPC

‘Function

body

Call ‘Enter Room’ RPC

Call ‘Authentica-tion’ RPC

Return ‘Authentication’ RPC

Return ‘Enter Room’ RPC

Room Serverthroughput Decline

Response DelayFrom Lobby Server

What is theAxis of Evil?

Thread Block-

ing

Game Client

LobbyRoom

Call ‘Enter Room’ RPC

Call ‘Authentica-tion’ RPC

Return ‘Authentication’ RPC

Return ‘Enter Room’ RPC

Inherent Limitation of RPC

Easy to write code be-cause it’s all contained

inside one function.

Can not split code be-cause it’s all contained

inside one function.

RPC::Result* JoinRoom(…){ … RPC::Result answer = RPC::SyncCall(“VerifyLobbyUserToken”, …); …. return result;}

What We Want

RPC Function Body

Non-Blocking

What is the Essence of Problems?

Request & Wait

Solution

Asynchronous

Programming

Request(callback, …)

Pros:Non-Blocking

Cons:Can’t Write Sequential CodeCan’t Use Stack VariableCan’t Split Programming Construct…

Too Difficult!!

Coroutine

Similar to Thread

Line of Execution

Stack&

Local Variable

But

Non-preemptive

Instruction:YieldYield BreakResume

Who does provide Coroutine?

C#ErlangHaskell

JavaScript(since 1.7)LuaPerl

Python(since 2.5)Ruby

…

What about C++?

On Your Own!

Windows: Fiber

Linux: getcontext/setcontextmakecontextswapcontext

Asynchronous Pro-gramming

+Coroutine

Non-Blocking&

Writing Sequential Code

Solution for Starva-

tion

Task A Task B

1: Lock 1: Sleep 0msec

2: Sleep 2msec

3: Unlock

Before

Task A Task B

Thread A 1: Sleep 0msec

1: Create & Resume Coroutine

2: Request Lock

3: Yield

Thread A’

4: Resume

5: Sleep 2msec

6: Yield Break

7: Unlock

Non-Blocking

Sequential Code

After

BeforePro

cess

ti

me(m

s)

# of Thread

1 2 3 4 5 6 7 8 9984986988990992994996998

1000

Task BTask A

AfterPro

cess

ti

me(m

s)

1 2 3 4 5 6 7 8 90

200

400

600

800

1000

1200

Task BTask A

# of Thread

Solution for

DeadlockHold & Wait

Thread A Thread B

1A: Lock 1B: Lock

2A: Request 2B: Send Response

3A: Wait for Response 3B: Unlock

4A: Do something with Response

5A: Unlock

Before

Thread A Thread B

1A: Lock 1B: Lock

2A: Create & Resume Coroutine 2B: Send Response

3A: Request 3B: Unlock

4A: Yield

5A: Unlock

Thread A’ (invoked by response)

6A’: Lock

7A’: Resume

8A’: Do something with response

9A’: Yield Break

10A’: Unlock

Non-Blocking

Sequential Code

After

Solutionfor

My Problem

Thread Block-

ing

Game Client

LobbyRoom

Call ‘Enter Room’ RPC

Call ‘Authentica-tion’ RPC

Return ‘Authentication’ RPC

Return ‘Enter Room’ RPC

Room Thread Lobby Service

1R: Call ‘Authentication’ RPC function 1L: Process the request

2R: Wait for the response 2L: Send back the response

3R: Process

Before

Room Thread Lobby Service

1R: Create & Resume Coroutine 1L: Process the request

2R: Request 2L: Send back the response

3R: Yield

Room Thread’(invoked by response)

4R’: Resume

5R’: Process

6R’: Yield Break

Non-Blocking

Sequential Code

After

Request & Wait

Request & Yield

Non-Blocking&

Writing Sequential Code

Coroutine(Fiber)

Thread

CPU CPU CPU CPU

User-mode SchedulingKernel-mode Scheduling

Corou-tine Pool

Network I/O Thread

Pool

RPC Task Queue

RPC Exe-cution

Thread Pool

RPC Task Queue

RPC Execu-tion Thread

Network I/O Thread

enqueuetask

dequeuetask

acquire corou-tine

enqueue

re-sumetask

dequeue

re-sumetask

resume

yield break

release corou-tine

Network Event

Network Event

Coroutine Pool

pool-ing

corou-tine

RPC Execu-tion

yield return

start

RPC Task Queue

RPC Execu-tion Thread

1

Coroutine Pool

RPC Execu-tion

enqueuetask

dequeuetask

acquire corou-tine corou-

tine

yield return

enqueue

re-sumetask

dequeue

re-sumetask

release corou-tine

Network Event

Network Event

pool-ing

RPC Execu-tion Thread

2

resume

yield break

resume

Network I/O Thread

RPC::Result* JoinRoom(…){ … RPC::Result answer = RPC::SyncCall(“VerifyLobbyUserToken”, …); …. return result;}

RPC::Result* JoinRoom(…){ … RPC::Result answer = RPC::SyncCallYield(“VerifyLobbyUserToken”, …); …. return result;}

TPS

# of Thread

Blocking

Yield

Wrap-up

“The free lunch is OVER.”Write Multithreaded

Application

CONCURRENCY

Thread Blocking is Evil

Starvation Deadlock

Asynchronous Programming

+Coroutine

Non-Blocking+

Writing Sequen-tial Code

Request & Wait

Request & Yield

Inspired by Jeffrey Richter

‘Simplified APM With The Asyn-cEnumerator’

Pictures fromwww.Istockphoto.comwww.Flickr.com