Orbited

Scaling Bi-directional web applications

A presentation by

Michael CarterCarterMichael@gmail.com

Classic Web Navigation

● Browser Submits Request to server GET /project.html HTTP/1.1

● Server responds HTTP/1.1 200 OK Content-length: 25 Hello! This is my project.

Ajax

● XMLHttpRequest Behind the scenes Javascript Asynchronous

● Allows for incremental updates without navigation

Problem

● HTTP Pull model up until now● What about the other direction?● No server initiated transfers allowed● Consider a chat application

A First Solution: Polling

● Browser makes Ajax requests every 0.5 seconds

● Server returns all events since last request● Simulates bi-directional flow● Allows us to make a chat application

Polling Problems

● High Latency 500 ms immediate overhead Plus latency between the server and browser And latency from the event originator and the server

● Heavy Load 2/req sec per user 1000 users = 2000 req / sec Well beyond the capacity of most dynamic web

applications

A Better Solution: Comet

● Browser makes a request for an event● The Server accepts the request but does not

respond● Later, when an event occurs, the server sends

data via the original request● The Browser processes the data, then waits for

more data

Comet Advantages

● Eliminates Latency overhead of polling● Allows for true bi-directional flow● Not difficult to implement in javascript

Comet Problems

● Web Servers can't handle comet + concurrency Requests are expected to be short lived Most servers are thread based Each request goes to its own thread So generally 1000 users can be handled with just

10-20 threads. Now connections are long lasting and sit idle So 10-20 threads can handle 10-20 users

Comet Problems

● Scaling Comet is relatively straightforward if you have a

single server node But if your capacity exceeds one node, you must

add another Classically, server nodes can be treated identically But now its unclear which user is connected to

which server, waiting for events

Comet Problems

● Integration Difficult to add comet to your applications Must access low-level web server apis Must keep track of state somehow Most web servers make this clumsy at best, and

sometimes nearly impossible

Orbited: Solving Comet's problems

● Orbited is separate, runs alongside the web server Event-based. It can handle thousands of concurrent

requests due to the lack of threads. Simplistic. Keeps track of minimal state Scalable. Due to the simplicity it is easy to scale

Orbited nodes

A Round Trip (an Orbit)

● Our Browser asks Orbited for an event● Orbited accepts the requests but doesn't reply● Browser asks the server to encode some music● The server responds with 'Ok. Will notify you later.'● The server finishes its tasks● The server connects to Orbited and tells it to

respond to the user● Orbited answers the request from long ago: 'All

done!'

Orbited: Chat app

● Two functions Join 'User Michael has joined' Msg 'Michael: hello'

Orbited Client API

● To send a msg to a user, simply call orbit.event(user, session, location, msg)

● Or to send a msg to multiple users call orbit.event(recipients, msg)

Scaling Orbited

● Client uses a hash function to map users to nodes

● Javascript uses the same hash function● The nodes are oblivious – they share no state● To scale, simply add more nodes and update

the hash function● Scales Linearly

Orbited Caveats

● Due to Browser security, comet pages and content pages must have the same host name originator.

● Therefore you must do complex DNS server manipulations

● Orbited includes a development mode where it acts as a proxy to the web application server so you can avoid the DNS configuration

● Some firewalls shutdown idle HTTP connections after 60 seconds. So a new connection must constantly be remade.

Further Research

● Additional transports● Triangular Routing

Allows dynamic load balancing Eliminates Proxy overhead No need for complex dns configuration

● More complete javascript support

Conclusion

● Orbited is fast● Orbited is scalable● Orbited is easy to integrate● Orbited is a very feasible solution to the comet

problem