SocketStream

A web framework for single page apps

Created byOwen Barnes

@temporalwave !!

(my boss at a former company)

About me

Paul Jensen @paulbjensen

!!

(I’m the new lead developer)

Where to begin?

Why Single Page Apps?

With a traditional web app, the user has to refresh the

page to see new information

Time

Server

Client

Time

Server

Client

GET/football/live HTTP/1.1

Time

Server

Client

HTTP/1.1 200 OK

Time

Server

Client

20 seconds later…

Time

Server

Client

I wonder what the latest score is… Let’s reload the page

Time

Server

Client

GET/football/live HTTP/1.1

Time

Server

Client

HTTP/1.1 304 Not Modified

Time

Server

Client

Time

Server

Client

The user had to press F5 to get at any new information

Time

Server

Client

Even though there was no new information,

the server still had to serve the HTTP request

This is not a fun experience

How do we make this better?

Time

Server

Client

We could use AJAX to update the page

We’d save the user having to press the F5 key

What else can we do?

Time

Server

Client

Optimise the response

GZIP the response data, and …

Avoid sending data we already have on the client

We could also separate the HTML

from the data

Reuse the HTML on the client

…and use the server to provide you with just

data

And the web site becomes a client

Server

Web App Native App API User

The server is just an API

A beautiful separation of concerns

Overview

• The server becomes a REST API serving JSON

• HTML compilation is done on the client

• As a result, less processing & bandwidth is consumed by the server

Why Realtime?

Time

Server

Client

Polling the server every [n] seconds for new data is redundant

There has to be a better way

What if the server could send its client(s) new

data as soon as it came about?

We can, thanks to WebSockets

WebSockets allows data to be sent both ways

Time

Server

Client

Time

Server

Client

Goal

Time

Server

Client

Goal

The server sends a message to the client that an action has

occurred

We eliminate the need to poll the server for

new data

Overview

• We can replace AJAX polling with WebSockets, and provide a better user experience as a result

• We also remove the need to make redundant polling requests back to the server.

• We use WebSockets for sending/receiving JSON

Single Page Apps +

The Realtime Web

There are many ways to build this kind of

app

You could build it mostly from scratch, and use Express + Socket.io

Or alternatively, you could use a web

framework like Meteor or Firebase

SocketStream is somewhere in-between

these 2 approaches

It provides tools to help with building realtime

single page apps...

... Whilst trying not to restrict what

technologies you can use with your app

For example, we don't provide an ORM.

Instead, you choose the database & the ORM

Also, we don't mandate using a specific client-side framework

!

You can use BackBone, Angular, Ember, or something else, that is

entirely your choice.

What we focus on instead are these

things:

Building RPC APIs

Live ReloadWebSocket Management

Minifying CSS/JS for production use

Client-side code organisation

Building PubSub APIs

HTML / CSS / JS code preprocessing

Session Management Building custom APIs on top of WS

HTML Templates Web Workers Connect middleware compatibility

I'll run through each of these, 1-by-1. But first, let's look at how to use

SocketStream

Getting started

npm install -g socketstream ! socketstream new my_app

Getting started

Success! Created app 'my_app' with: ! ✓ Basic chat demo (-m for minimal install) ✓ Javascript example code (-c if you prefer CoffeeScript) ✓ Plain HTML for views (-j if you prefer Jade) Next, run the following commands: ! cd my_app [sudo] npm install ! To start your app: ! node app.js

Here's what the initial file/folder structure looks like

Building RPC APIs








Client code is organised into 5 sub-folders

Client side code organisation• CODE stores client side JavaScript files and libraries

• CSS stores CSS files

• STATIC stores public files like images, font files, and other assets

• TEMPLATES stores HTML templates for the single page app to render on the client

• VIEWS stores HTML files that are rendered from the server for the initial page

Those sub-folders have sub-folders, but are optional

This is how we load them

// My SocketStream 0.3 app !var http = require('http'), ss = require('socketstream'); !// Define a single-page client called 'main' ss.client.define('main', { view: 'app.html', css: ['libs/reset.css', 'app.styl'], code: ['libs/jquery.min.js', 'app'], tmpl: '*' }); !// Serve this client on the root URL ss.http.route('/', function(req, res){ res.serveClient('main'); });

// My SocketStream 0.3 app !var http = require('http'), ss = require('socketstream'); !// Define a single-page client called 'main' ss.client.define('main', { view: 'app.html', css: ['libs/reset.css', 'app.styl'], code: ['libs/jquery.min.js', 'app'], tmpl: '*' }); !// Serve this client on the root URL ss.http.route('/', function(req, res){ res.serveClient('main'); });

SocketStream uses Browserify to handle

requiring JS files

Browserify allows us to use a Node.js style of

requiring JS files

// This file automatically gets called first by SocketStream and must always exist !// Make 'ss' available to all modules and the browser console window.ss = require('socketstream'); !ss.server.on('disconnect', function(){ console.log('Connection down :-('); }); !ss.server.on('reconnect', function(){ console.log('Connection back up :-)'); }); !ss.server.on('ready', function(){ ! // Wait for the DOM to finish loading jQuery(function(){ // Load app require('/app'); ! }); !});

Building RPC APIs








Over the years, developers have come up with new languages to generate

HTML, CSS, and JavaScript

SocketStream allows developers to use these code preprocessors in

their apps

Adding a preprocessor is simple

// Code Formatters ss.client.formatters.add(require('ss-stylus'));

For Javascript

• SS-COFFEE - supports using CoffeeScript

• SS-GORILLA - supports using GorillaScript

For CSS

• SS-STYLUS - supports using Stylus

• SS-LESS - supports using Less

For HTML Views

• SS-JADE - supports using Jade

For HTML Templating

• SS-HOGAN - supports using Twitter's Hogan.js

• SS-COFFEEKUP - supports using CoffeeKup

Setting a Templating engine

// Use server-side compiled Hogan (Mustache) templates. Others engines available ss.client.templateEngine.use(require('ss-hogan'));

Building RPC APIs








Having to press F5 to reload the page in order to view

changes to HTML/CSS/JS...

... is not a fun experience

In development mode, SocketStream will watch the client files for changes, and reload the page when they

occur

In the case of CSS, SocketStream will apply

the changes without reloading the page

Building RPC APIs








Client-side HTML templates are made

available to the browser via the ss.tmpl object

Building RPC APIs








When you're building a single page app, you'll

have a lot of JS files, and maybe a few CSS files

But serving a HTML page with lots of these files can take time, and

is inefficient

SocketStream provides a way to concatenate, minify, and GZip these files into 1

JS and 1 CSS file

This saves bytes being transferred, as well as

reducing the number of HTTP requests you make

Also, you can tell SocketStream to load

these files from a CDN

Setting a Templating engine

// Minimize and pack assets if you type: SS_ENV=production node app.js if (ss.env === 'production') ss.client.packAssets();

Building RPC APIs








Web Workers are handy for intensive client-side

JS operations

SocketStream provides support for using Web

Workers in your app

First, create a folder

Next, put your web worker files in that folder

Then, load the worker in a client code file, and enjoy

Building RPC APIs








SocketStream uses Connect middleware to support HTTP features

SocketStream uses the following middleware by default:

• compress - for GZipping assets

• cookieParser - for handling user tracking

• favicon - for serving a favicon.ico file

• session - for handling sessions

• static - for serving static assets

SocketStream uses the following middleware by default:

• compress middleware is loaded first, before all other middleware

• static middleware is loaded last, after all other middleware

SocketStream provides a way to load custom middleware into the

connect stack

ss.http.middleware.prepend() ss.http.middleware.append()

This allows you to use all of the connect

middleware out there today, i.e. EveryAuth

Building RPC APIs








We use connect’s session middleware, so authentication

can be done with either EveryAuth, PassportJS, or you

can roll your own.

We also recommend using connect-redis

Both HTTP and WebSocket interfaces

can get/set the session data

Via HTTP

// app.js ss.http.router.on('/updateSession', function(req, res) { req.session.myVar = 4321; res.end('req.session.myVar has been updated to', req.session.myVar); });

Via WebSockets

Building RPC APIs








RPC is a common pattern for clients

requesting data from the server

SocketStream provides a way to construct RPC

APIs with flexibility

Building RPC APIs








PubSub is a great pattern for Single Page

Apps

SocketStream handles this in various ways:

1 - Publishing to everyone viewing the app right now

ss.publish.all('newMessage', message); // Broadcast the message to everyone

Server

// Listen out for newMessage events coming from the server ss.event.on('newMessage', function(message) { // do something with the message });

Client

2 - Sending to private channels

// in a /server/rpc file after calling req.use('session') middleware !req.session.channel.subscribe('disney') !req.session.channel.unsubscribe('kids') !req.session.channel.reset() // unsubscribes the session from every channel !req.session.channel.list() // shows what channels are subscribed to

Server (subscribe/unsubscribe the session )

2 - Sending to private channels

// in a /server/rpc file ss.publish.channel('disney', 'chatMessage', {from: 'jerry', message: 'Has anyone seen Tom?'});

Server (publish to channel)

// in a /client/code file ss.event.on('chatMessage', function(msg, channelName){ console.log('The following message was sent to the ' + channelName + ' channel:', msg); });

Client (receive channel message)

3 - Sending to users

// in a /server/rpc file ss.publish.user('fred', 'specialOffer', 'Here is a special offer just for you!');

Server

4 - Sending to a browser tab

// in a /server/rpc file ss.publish.socketId('254987654324567', 'justForMe', 'Just for one tab');

Server

Building RPC APIs








On top of RPC and PubSub, SocketStream provides you with a way to create custom

request responders

Request Response is basically a WebSocket

message handler

It allows you to write message handling for games, where every

byte matters

Building RPC APIs








WebSockets are not immortal…

They are mangled by mobile networks…

Blocked by firewalls…

Routed to dead ends by proxy servers

And severed by train tunnels

Also, browser support for WebSockets isn’t

guaranteed

You need a transport strategy

Originally, SocketStream used Socket.io

But Socket.io asserted that if a browser

supported WebSockets, then it would work

They learned from this, by building Engine.io

I created the transport wrapper for Engine.io in

SocketStream for Bechtel & Dashku

And designed it to reconnect the client

when severed

Months later, it made it’s way into SocketStream’s

core.

SocketStream let’s you use this, alongside

SockJS

…and that is SocketStream in a nutshell. Whew!

Let’s look at some SocketStream apps in

the wild

Hollowhollowdocumentary.com

Vmuxvmux.co

Dashkudashku.com

SocketStream plugins

SS-BACKBONE

SS-ANGULAR

SS-CUCUMBER

Tips for deploying SocketStream in

production

1 - Check your server’s ulimit configuration

(This can bite you hard)

I learned this when Dashku went #1 on Hacker News in 45min

2 - Use HTTPS, but handle it at the load balancer level rather than at the app level

HTTPS helps to improve the stability of WebSocket

connections, especially on mobile devices

But Node’s HTTPS implementation is

noticeably slower than using HAProxy or Nginx

Where is SocketStream going next?

We’re in the process of getting SocketStream’s

test coverage up

We’re also trying to close some ancient

bugs

We also need better documentation

We’re giving the web site an overhaul

And we want to document how

SocketStream’s internals function, to help build 0.4

but what about 0.4?

…0.4 is starting to look like these:

I promise you all, it’s coming in June 2014

Thank You

SocketStream

Technology

Transcript of SocketStream