NodeJS: the good parts? A skeptic’s view (devnexus2014)
-
Upload
chris-richardson -
Category
Technology
-
view
3.308 -
download
2
description
Transcript of NodeJS: the good parts? A skeptic’s view (devnexus2014)
@crichardson
NodeJS: the good parts? A skeptic’s view
Chris Richardson
Author of POJOs in ActionFounder of the original CloudFoundry.com
@[email protected] http://plainoldobjects.com
@crichardson
Presentation goal
How a grumpy, gray-haired server-side Java developer discovered an appreciation for NodeJS and JavaScript
@crichardson
VIEWER DISCRETION IS ADVISED
WARNING!
@crichardson
1982 1986
RPG 3 BCPLPascal
C
About Chris
1983
LispWorks
1980 1984 1985 1987 1988 19891981
Z806502Assembler
Basic
@crichardson
C++
EJB
1992 199619931990 1994 1995 1997 1998 19991991
@crichardson
CloudFoundry.com
2002 200620032000 2004 2005 2007 2008 20092001
@crichardson
?About Chris
2012 201620132010 2014 2015 2017 2018 20192011
@crichardson
Agenda
Overview of NodeJS
JavaScript: Warts and all
The Reactor pattern: an event-driven architecture
NodeJS: There is a module for that
Building a front-end server with NodeJS
@crichardson
What’s NodeJS?
Designed for DIRTy apps
@crichardson
Growing rapidly
Busy!
@crichardson
But it’s not everywhere...
Primarily “edge” services: web apps, etc
@crichardson
NodeJS Hello Worldapp.js
$ node app.js$ curl http://localhost:1337
http://nodejs.org/
Load a module
request handler
No complex configuration: simple!
@crichardson
NodeJS
JavaScript
Reactor pattern
Modules
@crichardson
NodeJS
JavaScript
Reactor pattern
Modules
@crichardson
Dynamic and weakly-typedDynamic:
Types are associated with values - not variables
Define new program elements at runtime
Weakly typed:
Leave out arguments to methods
Read non-existent object properties
Add new properties by simply setting them
@crichardson
JavaScript is object-oriented> var fred = {name: “Fred”, gender: “Male”};undefined> fred.name“Fred”> console.log("reading age=" + fred.age);reading age=undefinedundefined> fred.age = 99;99> fred{ name: 'Fred', gender: 'Male', age: 99 }> delete fred.agetrue> fred{ name: 'Fred', gender: 'Male' }
Unordered key-value pairs
Keys = properties
Add property
Delete property
@crichardson
overrides
JavaScript is a prototypal language
__proto__name “Chris”
__proto__sayHello function
... ...
inherited
Prototype
Person
Chris
“CER”nicknameobject specific
@crichardson
Prototypal code$ node> var person = { sayHello: function () { console.log("Hello " + this.name); }};[Function]> var chris = Object.create(person, {name: {value: "Chris"}});undefined> var sarah = Object.create(person, {name: {value: "Sarah"}});undefined> chris.sayHello();Hello Chrisundefined> sarah.sayHello();Hello Sarahundefined> chris.sayHello = function () { console.log("Hello mate: " + this.name); };[Function]> chris.sayHello();Hello mate: Chrisundefined
Not defined here
create using prototype properties
@crichardson
JavaScript is Functionalfunction makeGenerator(nextFunction) {
var value = 0;
return function() { var current = value; value = nextFunction(value); return current; };
}
var inc = makeGenerator(function (x) {return x + 1; });
> inc()0> inc()1
Pass function (literal)as an argument
Return a function closure
@crichardson
But JavaScript was created in a hurry
http://thequickword.wordpress.com/2014/02/16/james-irys-history-of-programming-languages-illustrated-with-pictures-and-large-fonts/
@crichardson
Lots of flawsThe ‘Java...’ name creates expectations that it can’t satisfy
Fake classes: Hides prototypes BUT still seems weird
global namespace
scope of vars is confusingMissing return statement = confusion
‘function’ is really verbose
‘this’ is dynamically scoped
Unexpected implicit conversions: 99 == “99”!
truthy and falsy values52-bit ints
Dynamic + weakly-typed (+ event-driven) code
+ misspelt property names
⇒
lots of time spent in the abyss
Essential: Use IDE integrated with JSLint/JSHint + tests
Dynamic + weakly-typed code ⇒
Refactoring is more difficultUnderstanding code/APIs is more difficult
@crichardson
Prototypal languages have benefits BUT
Developers really like classes
JavaScript prototypes lack the powerful features from the Self language
e.g. Multiple (and dynamic) inheritance
http://www.cs.ucsb.edu/~urs/oocsb/self/papers/papers.html
@crichardson
Verbose function syntax> var numbers = [1,2,3,4,5]> numbers.filter(function (n) { return n % 2 == 0; } ).map(function (n) { return n * n; })[ 4, 16 ]>
scala> val numbers = 1..5scala> numbers filter { _ % 2 == 0} map { n => n * n }Vector(4, 16)
VersusPrelude> let numbers = [1,2,3,4,5]Prelude> map (\n -> n * n) (filter (\n -> mod n 2 == 0) numbers)[4,16]
Or
@crichardson
Verbose DSLsdescribe('SomeEntity', function () {
beforeEach(function () { ... some initialization ... });
it('should do something', function () { ... expect(someExpression).toBe(someValue); });});
class SomeScalaTest ...{
before { ... some initialization ... }
it should "do something" in { ... someExpression should be(someValue)}
Versus
Jasmine
Scalatest
@crichardson
JavaScript is the language of the web
“You have to use the programming language you have, not the one that you
might want”
@crichardson
It works but the result is lost opportunities
and impeded progress
@crichardson
But if you think that this isn’t a problem then perhaps ....
“Stockholm syndrome ... is a psychological phenomenon in which hostages ... have
positive feelings toward their captors, sometimes to the point of defending them...”
http://en.wikipedia.org/wiki/Stockholm_syndrome
@crichardson
Martin Fowler once said:
"...I'm one of those who despairs that a language with such deep flaws plays such an
important role in computation. Still the consequence of this is that we must take
javascript seriously as a first-class language and concentrate on how to limit the damage
its flaws cause. ...."
http://martinfowler.com/bliki/gotoAarhus2012.html
@crichardson
Use just the good parts
http://www.crockford.com/
Douglas Crockford
@crichardson
Use a language that compiles to JavaScript
TypeScript
Classes and interfaces (dynamic structural typing)
Typed parameters and fields
Dart
Class-based OO
Optional static typing
Bidirectional binding with DOM elements
Less backwards compatibility with JavaScript
Also has it’s own VM
@crichardson
CoffeeScript Hello Worldhttp = require('http')
class HttpRequestHandler constructor: (@message) ->
handle: (req, res) => res.writeHead(200, {'Content-Type': 'text/plain'}) res.end(@message + '\n')
handler = new HttpRequestHandler "Hi There from CoffeeScript"
server = http.createServer(handler.handle)
server.listen(1338, '127.0.0.1')
console.log('Server running at http://127.0.0.1:1338/')
Classes :-)
Bound method
Concise
@crichardson
No escaping JavaScript
@crichardson
NodeJS
JavaScript
Reactor pattern
Modules
@crichardson
About the Reactor pattern
Defined by Doug Schmidt in 1995
Pattern for writing scalable servers
Alternative to thread-per-connection model
Single threaded event loop dispatches events on handles (e.g. sockets, file descriptors) to event handlers
@crichardson
Reactor pattern structure
Event Handlerhandle_event(type)get_handle()
Initiation Dispatcherhandle_events() register_handler(h)
select(handlers)for each h in handlers h.handle_event(type)end loop
handleSynchronous Event Demultiplexer
select()
owns
notifies
uses
handlers
Applicationregister_handler(h1)register_handler(h2)handle_events()
@crichardson
Benefits
Separation of concerns - event handlers separated from low-level mechanism
More efficient - no thread context switching
Simplified concurrency - single threaded = no possibility of concurrent access to shared state
@crichardson
DrawbacksNon-pre-emptive - handlers must not take a long time
Difficult to understand and debug:
Inverted flow of control
Can’t single step through code easily
Limited stack traces
No stack-based context, e.g. thread locals, exception handlers
How to enforce try {} finally {} behavior?
@crichardson
Application code
NodeJS app = layers of event handlers
NodeJS event loop
Basic networking/file-system/etc.
HTTP DB driver ...
Event listener
Callback function
One time events:async
operation completion
Recurring events from Event
Emitters
@crichardson
Async code = callback hell
Difficult to implement common scenarios:
Sequential: A ⇒ B ⇒ C
Scatter/Gather: A and B ⇒ C
Code quickly becomes very messy
@crichardson
Messy callback-based scatter/gather code
getProductDetails = (productId, callback) -> productId = req.params.productId result = {productId: productId} makeCallbackFor = (key) -> (error, x) -> if error
callback(error) else result[key] = x if (result.productInfo and result.recommendations and result.reviews) callback(undefined, result)
getProductInfo(productId, makeCallbackFor('productInfo')) getRecommendations(productId, makeCallbackFor('recommendations')) getReviews(makeCallbackFor('reviews'))
The result of getProductDetails
Gather
Scatter
Update result
Propagate error
@crichardson
Simplifying code with Promises (a.k.a. Futures)
Functions return a promise - no callback parameter
A promise represents an eventual outcome
Use a library of functions for transforming and composing promises
Promises/A+ specification - http://promises-aplus.github.io/promises-spec
when.js (part of cujo.js by SpringSource) is a popular implementation
Crockford’s RQ library is another option
@crichardson
Simpler promise-based code class ProductDetailsService getProductDetails: (productId) -> makeProductDetails = (productInfo, recommendations, reviews) -> productId: productId productDetails: productInfo.entity recommendations: recommendations.entity reviews: reviews.entity
responses = [getProductInfo(productId), getRecommendations(productId),
getReviews(productId)]
all(responses).spread(makeProductDetails)all(responses) spread(makeProductDetails)
responses = [getProductInfo(productId), getRecommendations(productId),
getReviews(productId)]
@crichardson
Not bad but lacks Scala’s syntactic sugar
class ProductDetailsService .... {
def getProductDetails(productId: Long) = {
for (((productInfo, recommendations), reviews) <- getProductInfo(productId) zip getRecommendations(productId) zip getReviews(productId)) yield ProductDetails(productInfo, recommendations, reviews) }
}
getProductInfo(productId) zip getRecommendations(productId) zip getReviews(productId)
yield ProductDetails(productInfo, recommendations, reviews)
for (((productInfo, recommendations), reviews) <-
@crichardson
Long running computations
Long running computation ⇒ blocks event loop for other requests
Need to run outside of main event loop
Options:
Community: web workers threads
Built-in: NodeJS child processes
@crichardson
Using child processesvar child = require('child_process').fork('child.js');
function sayHelloToChild() { child.send({hello: "child"});}
setTimeout(sayHelloToChild, 1000);
child.on('message', function(m) { console.log('parent received:', m);});
function kill() { child.kill();}
setTimeout(kill, 2000);
process.on('message', function (m) { console.log("child received message=", m); process.send({ihateyou: "you ruined my life"})});
parent.js
child.js
Create child process
Send message to child
@crichardson
Modern multi-core machines vs. single-threaded runtime
Many components of many applications
Don’t need the scalability of the Reactor pattern
Request-level thread-based parallelism works fine
There are other concurrency options
Actors, Software transactional memory, ...
Go goroutines, Erlang processes, ...
Imposing a single-threaded complexity tax on the entire application is questionable
@crichardson
NodeJS
JavaScript
Reactor pattern
Modules
Core built-in modules
Basic networking
HTTP(S)
Filesystem
Events
Timers
...
@crichardson
Thousands of community developed modules
https://npmjs.org/
web frameworks, SQL/NoSQL database drivers, messaging, utilities...
@crichardson
What’s a module?
One or more JavaScript files
Optional native code:
Compiled during installation
JavaScript != systems programming language
Package.json - metadata including dependencies
exports.sayHello = function () { console.log(“Hello”);}
foo.js
@crichardson
Easy to install
$ npm install package-name --save
@crichardson
Easy to use
var http = require(“http”)var server = http.createServer...
Core module ORPath to file ORmodule in node_modules
Module’s exports
@crichardson
Developing with NodeJS modules
Core modules
Community modules
Your modules
Application code
@crichardson
There is a module for that...
Modules + glue code =
rapid/easy application development
AWESOME!...
@crichardson
... BUTVariable quality
Multiple incomplete/competing modules, e.g. MySQL drivers without connection pooling!
Often abandoned
No notion of a Maven-style local repository/cache: repeated downloads for all transitive dependencies!
Modules sometimes use native code that must be compiled
...
@crichardson
To summarize
NodeJS
JavaScript
Reactor patternModules
Flawed and misunderstood
Scalable yet costly and
often unnecessary
Rich but variable quality
@crichardson
How will future history view NodeJS?
C++EJB
?
@crichardson
Agenda
Overview of NodeJS
JavaScript: Warts and all
The Reactor pattern: an event-driven architecture
NodeJS: There is a module for that
Building a front-end server with NodeJS
@crichardson
So why care about NodeJS?
Easy to write scalable network services
Easy to push events to the browser
Easy to get (small) stuff done
It has a role to play in modern application architecture
@crichardson
Evolving from a monolithic architecture....
WAR
ReviewService
Product InfoService
RecommendationService
StoreFrontUI
OrderService
@crichardson
... to a micro-service architecture
Store front application
reviews application
recommendations application
product info application
ReviewService
Product InfoService
RecommendationService
StoreFrontUI
OrderService
orders application
@crichardson
Browser
WAR
StoreFrontUI
Model
View Controller
Presentation layer evolution....
HTML / HTTP
+ JavaScript
@crichardson
Browser Web application
RESTfulEndpointsModel
View Controller
...Presentation layer evolution
JSON-REST
HTML 5/JavaScriptIOS/Android clients
Event publisher
Events
Static content
@crichardson
Directly connecting the front-end to the backend
Model
View Controller Product Infoservice
RecommendationService
Reviewservice
REST
REST
AMQP
Model
View Controller
Browser/Native App
Traditional web application
Chatty API
Web unfriendly protocols
@crichardson
NodeJS as an API gatewayBrowser
Model
View Controller
HTML 5 - JavaScript
Product Infoservice
RecommendationService
Reviewservice
REST
REST
AMQP
APIGateway
Native App
Model
View Controller
Single entry point
Optimized Client specific APIs
Protocol translation
RESTproxy
Event publishing
NodeJS
@crichardson
Alternatively: Server-side MVC with micro-web apps
Desktop/mobileBrowser
Product Infoservice
RecommendationService
Reviewservice
REST
REST
AMQP
Product Catalog web app
Account mgmt web app
Order mgmt web app
Small footprint, nodeJS is ideal for micro-web apps
Small, cohesive, independently deployable web apps
@crichardson
Serving static content with the Express web framework
var express = require('express') , http = require('http') , app = express() , server = http.createServer(app) ;
app.configure(function(){ ... app.use(express.static(__dirname + '/public'));});
server.listen(8081);
From public sub directory
@crichardson
Templating with Dust.js
app.get('/hello', function (req, res) { res.render('hello', { title: (req.query.title || 'DevNexus') } );});
http://localhost:8081/hello?title=DevNexus+2014
<html><body> <h1>Hello {title}</h1></body></html>
https://engineering.linkedin.com/frontend/leaving-jsps-dust-moving-linkedin-dustjs-client-side-templates
Browser + Server
templating!
@crichardson
RESTful web services
@crichardson
Proxying to backend serverexpress = require('express')request = require('request')
app = express.createServer()
proxyToBackend = (baseUrl) -> (req, res) -> callback = (error, response, body) -> console.log("error=", error) originRequest = request(baseUrl + req.url, callback) req.pipe(originRequest) originRequest.pipe(res)
app.get('/productinfo/*', proxyToBackend('http://productinfo....'))
app.get('/recommendations/*', proxyToBackend(''http://recommendations...'))
app.get('/reviews/*', proxyToBackend('http://reviews...'))
Returns a request handler that proxies to baseUrl
@crichardson
Implementing coarse-grained mobile API
var express = require('express'), ...;
app.get('/productdetails/:productId', function (req, res) { getProductDetails(req.params. productId).then( function (productDetails) { res.json(productDetails); }});
@crichardson
Delivering events to the browser
@crichardson
Socket.io server-sidevar express = require('express') , http = require('http') , amqp = require(‘amqp’) ....;
server.listen(8081);...var amqpCon = amqp.createConnection(...);
io.sockets.on('connection', function (socket) { function amqpMessageHandler(message, headers, deliveryInfo) { var m = JSON.parse(message.data.toString()); socket.emit(‘tick’, m); }; amqpCon.queue(“”, {}, function(queue) { queue.bind(“myExchange”, “”); queue.subscribe(amqpMessageHandler); });});
Handle socket.io
connection
Subscribe to AMQP queue
Republish as socket.io
event
https://github.com/cer/nodejs-clock
@crichardson
Socket.io - client side
var socket = io.connect(location.hostname);
function ClockModel() { self.ticker = ko.observable(1); socket.on('tick', function (data) { self.ticker(data); });};
ko.applyBindings(new ClockModel());
<html><body>
The event is <span data-bind="text: ticker"></span>
<script src="/socket.io/socket.io.js"></script><script src="/knockout-2.0.0.js"></script><script src="/clock.js"></script>
</body></html>
clock.js
Connect to socket.io server
Subscribe to tick event
Bind to model
Update model
@crichardson
NodeJS is also great for writing backend micro-services
“Network elements”
Simply ‘route, filter and transform packets’
Have minimal business logic
@crichardson
NodeJS-powered home security
Upload2S3 UploadQueueProcessor
SQS Queue DynamoDBS3
FTP ServerLog file
FTP ServerUpload directory
IpCamViewerWeb App
@crichardson
SummaryJavaScript is a very flawed language
The single-threaded, asynchronous model is often unnecessary; very constraining; and adds complexity
BUT despite those problems
Today, NodeJS is remarkably useful for building network-focussed components