By: Rob von Behren, Jeremy Condit and Eric Brewer 2003

Presenter: Farnoosh MoshirFatemiJan-25-2012

Why Events Are A Bad Idea (for high-concurrency servers)

2

Contents Background Criticisms about threads Advantages of Threads Compiler support for threads Evaluation Conclusion

3

Background-Event based models

• Small, static number of processes• Explicit message system for communication

and synchronization• A message-oriented system• Pre-defined static communication mechanism• One execution stream, no CPU concurrency• No preemption of event handlers • Blocked I/O is complex to handle. Stack

ripping

4

Background-Thread-based models

• Large, rapidly changing processes• Synchronization mechanism is based on

shared data• A procedure-oriented system through fork

and join• System recourses encoded as global data

structures and shared through locks• Blocked I/O is a blocked thread

5

Background-Three Opinions

Events are better:

• There is no stack: managing state has lower overhead

• Cooperative multitasking: free synchronization (no need to mutexes and handle wait queues)

• Application level information: better scheduling, perform optimizations, and locality

• More flexible control flow

6

Background-Three Opinions

Events and threads are equal• Based on Lauer & Needham study• Message-oriented and procedure-oriented systems are duals• The choice should be made based on the nature of the machine

architecture upon which the system is going to be built

What they missed:

• Most modern event systems use cooperative scheduling• Most event systems use shared memory and global data

structure, which are assumed atypical in their study• SEDA is an exception

7

Background-Three Opinions

Threads are better:

• Threads can have the same properties as events

• Threads can have some advantages over events for high concurrent systems

• Compiler improvements can eliminate historical drawbacks with threads

Contents Background Criticisms about threads Advantages of Threads Compiler support for threads Evaluation Conclusion

8

9

Criticisms about threads

1- Performance: have not performed well for high concurrency

Respond:• Poor thread implementations (thread packages were

designed for both high concurrency and blocking operations)• Operations with O(n) in the number of threads• The context switch overhead and kernel crossing

Have modified the GNU Pth user-level threading package Remove O(n) operations from the scheduler Repeated the SEDA threaded server benchmark

Performance matches the event-based server

10

Criticisms about threads cont’d

2- Control Flow: restrictive control flow. - Encourage the programmer to think too linearly

Respond: Complex patterns are not used (difficult and error prone)

Threads express common patterns more naturally:

- call/return - parallel calls - pipelines

11

Criticism about threads cont’d

3- Synchronization: heavyweight synchronization -Events have “free” synchronization

Respond: Adya et al show that this advantage is due to

cooperative multitasking , not events themselves

It is “free” only with uniprocessor• High-concurrency servers have multi processors

12

Criticisms about threads cont’d

4- State Management: Thread are not effective in managing live state• A tradeoff between risking stack overflow

and wasting address space on large stacks• Event systems unwind the thread stack after

each event handler

Respond: Have proposed a mechanism which allow

stack to grow dynamically. (later)

13

Criticisms about threads cont’d

5- Scheduling: can’t optimize scheduling decisions

• Runtime system is too generic in thread-based• Event systems can schedule at application level (better

scheduling, and code locality)

Respond: Lauer and Needham indicate that the cooperatively

scheduled threads can have the same scheduling tricks

Threads and events are equivalent in power

Contents Background Criticisms about threads Advantages of Threads Compiler support for threads Evaluation Conclusion

14

15

Advantages of Threads

Observations about modern servers :

1- The concurrency is because of the largely independent concurrent requests

2- The code that handles each code is usually sequential

Threads provide a better programming abstraction

for servers.

16

Advantages of Threads cont’dControl Flow:Event-based systems: tend to obfuscate the control flow1- Programmer has to manually match the call/return pairs2-Manually save and restore live state. “Stack Ripping”

Thread-based systems: express control flow in a more natural manner1- Group calls with returns. Ensuring a one-to-one relationship.2- live state encapsulate at stack. Much easier debugging

17

Advantages of Threads cont’dException Handling and state Lifetime-The thread stack tracks the live state. cleaning up task state is simple.

-In event systems, task state is typically heap allocated. (difficult to freeing it on time)

-Garbage collection is inappropriate for high performance systems.

18

Advantages of Threads cont’dExisting Systems:- The event-based systems, using thread in the complex parts of the code: Ninja system- Applications without a high concurrency also using threads for simplicity: FTP server in Harvest uses threads

Just Fix Events:- Fixing the problems with events is equal to switching to threads

Threads have advantages over events

Contents Background Criticisms about threads Advantages of Threads Compiler support for threads Evaluation Conclusion

19

20

Compiler support for threadsMinor modification can improve safety and

performance of the threaded systems

1-Dynamic Stack Growth:

• Analyze upper bound on the stack size when calling a function

• Determine which call sites may require stack growth

21

Compiler support for threads cont’d

2- Live State Management: Remove unnecessary state from the stack -temporary variables before calling subroutines -popped entire frame in tail call.

3- SynchronizationWith compiler analysis can: - Warn the programmer about data races (reduce the occurrence of bugs) - Determine which atomic sections are safe to run concurrently.

Contents Background Criticisms about threads Advantages of Threads Compiler support for threads Evaluation Conclusion

22

23

Evaluation• Designed and implemented a simple user-

level cooperative threading package for Linux (5000 line)

• used the package to write a test web server, Knot. (700 line)

• Compared the performance of Knot with Haboob (SEDA’s event-driven web server)

• Two different scheduling policies for Knot:1-favors processing of current connections2-favors processing of accepting connections

24

Web server bandwidth versus the number f simultaneous clients.

Contents Background Criticisms about threads Advantages of Threads Compiler support for threads Evaluation Conclusion

25

26

Conclusion

Threads have the similar or better performance than events in high concurrency systems

Improvement to compiler can eliminate threads’ problems

Thanks for your attention