A COMMAND PROMPT TALK

Node.js and postgresAn Introductory Glance

With Live Code, Later

2010 AURYNN SHAW, COMMAND PROMPT INC.

Hi!

What is Node.js

Fast

Pure event-driven

server-side

javascript programming environment

Ew, javascript

Built on Google’s V8

Runs on modern unix-likes (Linux, OSX, Freebsd, Cygwin)

Speaks network natively, and easily (Hello world is in http!)

Ends up Similar to Python, Perl, Ruby, ETc.

Did I mention fast?

Single thread can do many thousands of connections

standard library doesn’t block

Did I mention fast?

Single thread can do many thousands of connections

standard library doesn’t block

Keeps the slow tasks from blocking

Keeps the quick tasks quick

blocking?

# Standard synchronous processingfh = open(“somefile”,”r”)contents = fh.read()# what if somefile is huge, 1GB+ ?print contents

Every step waits

How is node different?

Did I mention fast?

Single thread can do many thousands of connections

standard library doesn’t block

Keeps the slow tasks from blocking

Keeps the quick tasks quick

Really good for anything I/O

While slow tasks run, node carries on

Non-blocking!

fs.open(“somefile”, function (fd) { // When the file descriptor gets returned. var buffer = null;

fs.read(fd, buffer, length, position, function (e, bytes) {

// bytes read are here.// buffer will be not-null.

});});

As always, there’s caveats to this

Some caveats

node is entirely single-threaded

Event-driven programming is *really* different

A maze of callbacks

fs.open(“/tmp/hello”, function (fd) { var buffer = null;

fs.read(fd, buffer, length, position, function (e, bytes) {

// bytes read are here.// buffer will be not-null.

});});

Some caveats

node is entirely single-threaded

Event-driven programming is *really* different

Javascript’s prototype model is also really different

The syntax has warts

Build a bridge out of her?

/* This allows you to kind of inherit properly. */var o = function () {};var p = function () { /* Because ‘this’ changes */

var self = this;o.call(this);

};p.prototype = o;p.prototype.constructor = p;

So why is this important?

Why is it important?

Javascript owns the frontend

Why is it important?

Javascript owns the frontend

More jquery and javascript programmers every day

Why is it important?

Javascript owns the frontend

More jquery and javascript programmers every day

Javascript is fast

Faster every day

And now, we can use postgres!

When I got here...

postgres.js worked.

When I got here...

postgres.js worked.

Used the simple query protocol

When I got here...

postgres.js worked.

Used the simple query protocol

Used SQL escape to prevent injection attacks.

whyyyy

SOURCE: http://www.flickr.com/photos/striatic/2192192956/

But it did work!

And so I wanted to make it BETTER

So, I had to ask,

How do I not allow this?

Well...

to prepare a query

postgres.js Didn’t use libpq.

SOURCE: http://www.flickr.com/photos/tanaka/2319325408/

to prepare a query

postgres.js Didn’t use libpq.

Speaks pure protocol

to prepare a query

postgres.js Didn’t use libpq.

Speaks pure protocol

...So I had to learn the protocol.

Wax on, wax offMONTAGE!

Sadly, reality was a lot less exciting

Complexity Abounds

Simple protocol sends one message to pg

Easy to queue, internally.

QUERY

QUERY

QUERY

QUERY

QUERY

QUERY

RESULTS

REALLY SIMPLE MESSAGE QUEUE

Really, though

Simple protocol sends one message to PG

Easy to queue, internally.

Prepared statements require at least 5

Parse, describe, execute, bind, flush

SYNC

EXECUTE

BIND

DESCRIBE

PARSE

QUERY

QUERY

THIS TYPE OF QUEUE DOESN’T WORK

And while all this is happening

You cannot block the main thread

And postgres expects a synchronous request/response

But it does give responses...

And while all this is happening

You cannot block the main thread

And while all this is happening

You cannot block the main thread

And postgres expects a synchronous request/response

And while all this is happening

You cannot block the main thread

And postgres expects a synchronous request/response

But it does give responses...

So, we buffer message sets

SIMPLE

PREPAREDGROUP

SIMPLE

SIMPLE

PREPARED GROUP

SIMPLE

SIMPLE

KIND OF LIKE THIS

So we never block waiting for the DB

the application keeps responding

But that’s all a bit on the technobabbly side

SO

Let’s look codE!

Using postgres.js

/* Node setup */var pg = require(“lib/postgres-pure.js”), sys = require(“sys”);

var db = new pg.connect("pgsql://test:12345@localhost:5432/template1");

/* We now have a working DB handle. Or, we have an error. Either way.*/

Using postgres.js

/* We now have a working DB handle. *//* From here, we do: */

db.query(“select 1::int;”, function (err, rs) {// Wait, a function?console.log(sys.inspect(rs));

});

Why a function?

What’s going on here?

/* We now have a working DB handle. *//* From here, we do: */

db.query(“select 1::int;”, function (err, rs) {// Wait, a function?console.log(sys.inspect(rs));});

What you get

13// A little while later2

It’s happening asynchronously!

The function argument is what’s going to happen when the DB comes back

db.query returns immediately

and your app keeps on going!

this isn’t the cool bit

Prepared queries

/* Prepared queries work the same way */

db.prepare(“select ?::int as foobar;”, function (sth) {sth.execute(1, function (err, rs) {

console.log(sys.inspect(rs));});sth.execute(2, function (err, rs) {

console.log(sys.inspect(rs));});

});

Prepared queries

/* which will return */[ { foobar: 1 } ][ { foobar: 2 } ]

Or this!

/* This becomes a prepared statement behind the scenes*/db.query(“select ?::int as foobar;”,

1, // A single argumentfunction (err, rs) {

console.log(sys.inspect(rs));});

It would return

[ { foobar: 1 } ]

Sub-ideal Aspects

Bad, bad, bad.

postgres.js is still very alpha.

Node API is in flux

Learning Node takes a while

Cool bits!

Things that are pleasing

clean, perl DBI-based API.

Official support from Command prompt

Performance!

Vroom

[ananke:~/src/postgres-js] aurynn> time node benchmark.js 40000[ { count: 40001 } ]node benchmark.js 40000 18.85s user 2.75s system 87% cpu 24.602 total

Just over 2122 queries/second.

Some Stuff we can’t wait to add

Shiny things!

Notify support

PG notifications

/* Event-driven frameworks are ideal for this, and the notification setup can be handled behind the scenes.*/

db.on(“<NOTIFYNAME>”, function (payload) {// Do something useful as a result of // the notification from PG

});

Payload is also arbitrary data

Shiny things!

Notify support

Stability

Shiny things!

Notify support

Stability

Better error handling

Shiny things!

Notify support

Stability

Better error handling

More test cases

Shiny things!

Notify support

Stability

Better error handling

More test cases

Better type coercion

But the most important thing we’d like?

Users.

You guys. People who care.

So that’s postgres.js.

Any questions?

Okay. LIVE CODE!Shout out ideas!

https://github.com/commandprompt/postgres-js/

https://github.com/aurynn/postgres-js/

UNSTABLE

NOT QUITE AS UNSTABLE

FORK IT, AND HELP!