Scaling real world applications using gevent
-
Upload
aalokthemagnificient -
Category
Technology
-
view
2.519 -
download
1
description
Transcript of Scaling real world applications using gevent
![Page 1: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/1.jpg)
Concurrency & GeventScaling Real World Applications
![Page 2: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/2.jpg)
Concurrency
handling a number of things at the same time
Examples: (Incoming) WebServers, Database Servers
(Outgoing) SSH Mux
![Page 3: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/3.jpg)
SSH Mux
● Execute a command on a remote
SSH server
● Handle concurrent SSH Clients
● Command execution time varies
from seconds to days
● Command execution happens on
remote servers, SSH mux is I/O
bound
![Page 4: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/4.jpg)
SSH Client
1. Init session
2. Authenticate
3. Get a channel
4. Issue command
5. Read output
![Page 5: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/5.jpg)
Need Concurrency?
● Process blocks on read()
● No new connections can be inititated
● Need ability to handle multiple clients at the same
time
![Page 6: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/6.jpg)
Multiprocessing
● One process is the master
● Master can spawn workers
● Each worker handles one request at a time
● Pre-forked pool of workers
![Page 7: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/7.jpg)
Concurrent SSH Clients
![Page 8: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/8.jpg)
SSH Mux Memory Usage
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 1000
100
200
300
400
500
600
ssh mux memory usage
Processes
# Concurrent Reqs
Me
mo
ry (
MB
)
![Page 9: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/9.jpg)
SSH Mux Performace
20 40 80 150 200 400 800 1200 1500 1800 21000
20
40
60
80
100
120
140
160
ssh mux performance sheet
Processes
# Concurrent Reqs
Tim
e(s
)
![Page 10: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/10.jpg)
Multiprocessing yay
● Easy to get started
● OS guaranteed process isolation & fairness
● Covers up for misbehaving workers
● Add more concurrency by adding more workers
● Convenient when numbers are smaller numbers
![Page 11: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/11.jpg)
Multiprocessing nay
● Concurrency limited by number of processes
● Memory heavy
● Implicit scheduling
● Synchronization is not trivial
![Page 12: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/12.jpg)
More Concurrency?
● Command execution is happening on remote servers, we are mostly blocked on I/O
● Handle multiple I/O in a single process?
![Page 13: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/13.jpg)
Gevent
gevent is a coroutine-based Python networking library that uses greenlet to provide a high-level synchronous API on
top of the libevent event loop.
![Page 14: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/14.jpg)
Greenlets
● Lightweight 'threads' - not OS threads
● Explicit scheduling - Cooperative
● Minimal stack
● Application decides execution flow
● Easy to synchronize/ Avoid locks
● All run inside one process
![Page 15: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/15.jpg)
Libevent
● Use fastest mechanism to poll (portable)
● Fast Event loop
● In Gevent, event loop runs in a greenlet (event hub)
● Instead of blocking, greenlets switch to event hub
● It's all done behind the scene
![Page 16: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/16.jpg)
Monkey Patching
Monkey patchingModifies behaviour of blocking calls such as select, sleep to non-blocking
Patches the python standard socket library
![Page 17: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/17.jpg)
Gevent
● Greenlet 1 is running
● Greenlet 2 and 3 are ready
![Page 18: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/18.jpg)
Gevent
● Greenlet 1 has to wait for read
● Greenlet 1 switches to Event hub
![Page 19: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/19.jpg)
Gevent
● Event hub switches to Greenlet 3
![Page 20: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/20.jpg)
Gevent
● Greenlet 2 runs
![Page 21: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/21.jpg)
Gevent
● Greenlet 2 wants to sleep
● Greenlet 2 switches to Event hub
![Page 22: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/22.jpg)
Gevent
● Greenlet 1 data has come, moved to ready state
● Eventhub switches to Greenlet 3
![Page 23: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/23.jpg)
Gevent
● Greenlet 3 runs
![Page 24: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/24.jpg)
Gevent
● When Greenlet 1 resumes, its from next instruction
● It's as if it were a blocking call
![Page 25: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/25.jpg)
Green SSH Client
1. Init session
2. Authenticate
3. Get a channel
4. Issue command
5. Read output
![Page 26: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/26.jpg)
A closer look
![Page 27: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/27.jpg)
Going Concurrent
Use pre-forked processes to use all cores
![Page 28: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/28.jpg)
Memory usage
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 1000
5
10
15
20
25
30
35
40
45
ssh mux memory usage
Gevent+Processes
# Concurrent Reqs
Me
mo
ry(M
B)
![Page 29: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/29.jpg)
SSH Mux Performace
20 40 80 150 200 400 800 1200 1500 1800 21000
10
20
30
40
50
60
70
ssh mux performace chart
Gevent+Processes
# Concurrent Reqs
Tim
e(s
)
![Page 30: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/30.jpg)
SSH Mux memory usage
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 1000
100
200
300
400
500
600
ssh mux memory usage
Processes
Gevent
# Concurrent Reqs
Me
mo
ry(M
B)
![Page 31: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/31.jpg)
SSH Mux Performance
20 40 80 150 200 400 800 1200 1500 1800 21000
20
40
60
80
100
120
140
160
ssh mux performance sheet
Processes
Gevent+Processes
# Concurrent Reqs
Tim
e(s
)
![Page 32: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/32.jpg)
Gevent yay!
● Untwist – write linear non blocking code
● Explicit scheduling, dictate the execution flow
● Timeouts
● Events, AsyncResults for Synchronization
● gevent.wsgi
● Pre-spawned pool of greenlets
![Page 33: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/33.jpg)
Gevent beware of
● No multicore support
● Not great for CPU bound applications
● Third party libs must be green (non blocking)
● Misbehaving workers can be lethal
● No fairness when it comes to scheduling
![Page 34: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/34.jpg)
Take Away
● Gevent lets you write asynchronous code in a
synchronous manner
● No multicore support, still need multiprocessing
● Not so great for CPU bound applications
● Split your application into CPU bound and IO bound
parts
● Be willing to contribute patches
● Code available at
[email protected]:aaloksood/pyexamples.git
![Page 35: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/35.jpg)
Thank you
That's all folks!
![Page 36: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/36.jpg)
Countdown Timer● Count down from 200000000● Split work among workers
![Page 37: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/37.jpg)
Threads
1 2 3 4 5 6 7 8 9 100
5
10
15
20
25
Multithreading wonder
1 Core
4 cores
# Workers
Tim
e(s
)
![Page 38: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/38.jpg)
One core
1 2 3 4 5 6 7 8 9 1010.5
11
11.5
12
12.5
13
13.5
14
14.5
Execution time One Core
Processes
1 Core
Gevent_1
Gevent_4
# Workers
Tim
e (
s)
![Page 39: Scaling real world applications using gevent](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bd271b4c905706a8b4d86/html5/thumbnails/39.jpg)
Four cores
1 2 3 4 5 6 7 8 9 100
5
10
15
20
25
Execution time 4 cores
Process
Threads
Gevent_1
Gevent_4
# Workers
Tim
e(s
)