C o m m u n i t y E x p e r i e n c e D i s t i l l e d

Design and implement effi cient RESTful solutions with this practical hands-on guide

RESTful Web API Design with Node.jsSecond Edition

Valentin Bojinov

RESTful Web API Design with Node.jsSecond Edition

In this era of cloud computing, every data provisioning solution is built in a scalable and fail-safe way. Thus, when building RESTful services, the right choice for the underlying platform is vital. Node.js, with its asynchronous, event-driven architecture, is exactly the right choice to build RESTful APIs.

Starting with the fundamentals of REST, you will understand why RESTful web services are better data provisioning solution than other technologies. You will start setting up a development environment by installing Node.js, Express.js, and other modules. Next, you will write a simple HTTP request handler and create and test Node.js modules using automated tests and mock objects. You will then have to choose the most appropriate data storage type, having options between a key/value or document data store, and also you will implement automated tests for it. This module will evolve chapter by chapter until it turns into a full-fl edged and secure Restful service.

Who this book is written for

The ideal target audience for this book is web developers who have some experience with RESTful services. Familiarity with basic JavaScript programming techniques is required. No prior experience with Node.js or Express.js is required.

$ 29.99 US£ 19.99 UK

Prices do not include local sales tax or VAT where applicable

Valentin Bojinov

What you will learn from this book

Install, develop, and test your own Node.js user modules

Comprehend the differences between an HTTP and a RESTful application

Optimize RESTful service URI routing with best practices

Eliminate third-party dependencies in your tests with mocking

Learn about NoSQL data stores and integrate MongoDB in your Node.js application with Mongoose

Secure your services with NoSQL database integration within Node.js applications

Enrich your development skills to create scalable, server-side, RESTful applications based on the Node.js platform

RESTful W

eb API D

esign with N

ode.js Second Edition

P U B L I S H I N GP U B L I S H I N G

community experience dist i l led

Visit www.PacktPub.com for books, eBooks, code, downloads, and PacktLib.

Free Sample

In this package, you will find: • The author biography • A preview chapter from the book, Chapter 1 'REST – What You Did

Not Know' • A synopsis of the book’s content • More information on RESTful Web API Design with Node.js Second Edition

About the AuthorValentin Bojinov studied computer programming at the Technological School of ElectronicSystems in Sofia, Bulgaria, a college within the Technical University of Sofia. He wasintroduced to programming there and realized that his career would be in research anddevelopment. He holds a BSc in telecommunication and information engineering. Then, hisinterest in data transmission grew, and he ventured into B2B (business-to-business)communication. He is currently pursuing his MSc in software development. Valentin is anexpert in Java, SOAP, RESTful web services, and B2B integration.

A few years after he started his career as a .NET developer, he realized that B2B and SOAwere his passion. He then moved to SAP, where he contributed to the development of theweb services stack of the SAP JEE platform. He currently works as a senior Java developerfor the Bulgarian branch of Seeburger AG, a leader in the B2B and MFT solutions market.There, he develops and maintains several B2B communication adapters, including webservices and SAP adapters.

PrefaceRESTful services have become the de facto standard data feed providers for social services,news feeds, and mobile devices. They deliver a large amount of data to millions of users;therefore, they need to address high-availability requirements such as reliability andscalability. This book will show you how to utilize the Node.js platform to implement arobust and performant data service. By the end of the book you will have learned how toimplement a real-life RESTful service, taking advantage of the modern NoSQL database forserving both JSON and binary content. Important topics such as correct URI structuring andsecurity features are also covered with detailed examples, showing you everything youneed to know to start implementing robust RESTful APIs serving content for yourapplications!

What this book coversChapter 1, REST – What You Did Not Know, gives you a brief introduction into the historyof REST and how it couples with the HTTP protocol.

Chapter 2, Getting Started with Node.js, teaches you how to install Node.js and how towork with its package manager to install modules, develop your first HTTP serverapplication, and write automated unit tests for the HTTP handler by using mock requestobjects.

Chapter 3, Building a Typical Web API, teaches you how to structure your application usinghuman-readable URL and URI parameters and develop a read-only RESTful serviceapplication using the filesystem as storage.

Chapter 4, Using NoSQL Databases, explains how to use LevelDB and MongoDB NoSQLdatabases, understand the difference between key/value and document data stores, andwrite automated tests for NoSQL user defined modules.

Chapter 5, Implementing a Full-Fledged RESTful Service, implements a production-readyRESTful service that uses NoSQL to store its data. You will also learn how to handle binarydata and how to version an API while it evolves.

Chapter 6, Keeping the Bad Guys Out, explains how to restrict access to your data bychoosing an appropriate authentication approach, protect data leakage with transport layersecurity.

1REST – What You Did Not

KnowNowadays, topics such as cloud computing and mobile device service feeds, as well asother data sources driven by cutting-edge, scalable, stateless, and modern technologies suchas RESTful web services, leave the impression that REST was invented recently. Well, to behonest, it definitely was not! In fact, REST has been here since the end of the 20th century.

This chapter will walk you through REST's fundamental principles, and it will explain howREST couples with the HTTP protocol. You will look into the five key principles that needto be considered while turning an HTTP application into a RESTful-service-enabledapplication. You will also look at the differences in describing RESTful and classic SOAP-based web services. Finally, you will learn how to utilize already existing infrastructure foryour benefit.

In this chapter, we will cover the following topics:

REST fundamentalsREST with HTTPEssential differences in the description and discovery of RESTful servicescompared to classical SOAP-based servicesTaking advantage of existing infrastructure

REST – What You Did Not Know

[ 8 ]

REST fundamentalsIt actually happened back in 1999, when a request for comments was submitted to theInternet Engineering Task Force (IETF: h t t p : / / w w w . i e t f . o r g /) via RFC 2616:“Hypertext Transfer Protocol-HTTP/1.1.” One of its authors, Roy Fielding, later defined aset of principles built around the HTTP and URI standards. This gave birth to REST as weknow it today.

These definitions were given in Chapter 5, Representational State Transfer(REST), of Fielding's dissertation called Architectural Styles and the Design ofNetwork-based Software Architectures. The dissertation is still available athttp://www.ics.uci.edu/~fielding/pubs/dissertation/rest_a

rch_style.htm.

Let's look at the key principles around the HTTP and URI standards, sticking to which willmake your HTTP application a RESTful-service-enabled application:

Everything is a resourceEach resource is identifiable by a unique identifier (URI)Use the standard HTTP methodsResources can have multiple representationsCommunicate statelessly

Principle 1 – everything is a resourceTo understand this principle, one must conceive the idea of representing data by a specificformat and not by a physical file. Each piece of data available on the Internet has a formatthat could be described by a content type. For example, JPEG images; MPEG videos; HTML,XML, and text documents; and binary data are all resources with the following contenttypes: image/jpeg, video/mpeg, text/html, text/xml, and application/octet-stream.

Chapter 1

[ 9 ]

Principle 2 – each resource is identifiable by aunique identifierSince the Internet contains so many different resources, they all should be accessible viaURIs and should be identified uniquely. Furthermore, the URIs can be in a human-readableformat, despite the fact that their consumers are more likely to be software programs ratherthan ordinary humans.

Human-readable URIs keep data self-descriptive and ease further development against it.This helps you to reduce the risk of logical errors in your programs to a minimum.

Here are a few sample examples of such URIs:

h t t p : / / w w w . m y d a t a s t o r e . c o m / i m a g e s / v a c a t i o n / 2 0 1 4 / s u m m e r

h t t p : / / w w w . m y d a t a s t o r e . c o m / v i d e o s / v a c a t i o n / 2 0 1 4 / w i n t e r

h t t p : / / w w w . m y d a t a s t o r e . c o m / d a t a / d o c u m e n t s / b a l a n c e ? f o r m a t = x m l

h t t p : / / w w w . m y d a t a s t o r e . c o m / d a t a / a r c h i v e s / 2 0 1 4

These human-readable URIs expose different types of resources in a straightforwardmanner. In the example, it is quite clear that the media types of these resources are asfollows:

ImagesVideosXML documentsSome kinds of binary archive documents

Principle 3 – use the standard HTTP methodsThe native HTTP protocol (RFC 2616) defines eight actions, also known as HTTP verbs:

GET

POST

PUT

DELETE

HEAD

OPTIONS

REST – What You Did Not Know

[ 10 ]

TRACE

CONNECT

The first four of them feel just natural in the context of resources, especially when definingactions for resource data manipulation. Let's make a parallel with relative SQL databaseswhere the native language for data manipulation is CRUD (short for Create, Read, Update,and Delete) originating from the different types of SQL statements: INSERT, SELECT,UPDATE, and DELETE, respectively. In the same manner, if you apply the REST principlescorrectly, the HTTP verbs should be used as shown here:

HTTPverb

Action Response status code

GET Requests an existingresource

“200 OK” if the resource exists, “404 Not Found” ifit does not exist, and “500 Internal ServerError” for other errors

PUT Updates a resource orcreates it as an identifierprovided from the client

“201 CREATED” if a new resource is created, “200OK” if updated, and “500 Internal ServerError” for other errors

POST Creates a resource withan identifier generated atserver side or updates aresource with an existingidentifier provided fromthe client

“201 CREATED” if a new resource is created,”200OK” if the resource has been updated successfully,“404 Not Found” if the resource to be updated doesnot exist, and “500 Internal Server Error” forother errors

DELETE Deletes a resource “200 OK“or “204 No Content” if the resource hasbeen deleted successfully, “404 Not Found” if theresource to be deleted does not exist, and “500Internal Server Error” for other errors

Note that a resource can be created by either of POST or PUT HTTP verbs. When a resourcehas to be created under a specific URI with an identifier provided by the client, then PUT isthe appropriate action:

PUT /data/documents/balance/22082014 HTTP/1.1Content-Type: text/xmlHost: www.mydatastore.com

<?xml version="1.0" encoding="utf-8"?><balance date="22082014">

Chapter 1

[ 11 ]

<Item>Sample item</Item><price currency="EUR">100</price></balance>

HTTP/1.1 201 CreatedContent-Type: text/xmlLocation: /data/documents/balance/22082014

However, in your application, you may want to leave it up to the server REST application todecide where to place the newly created resource, and thus create it under an appropriatebut still unknown or non-existing location.

For instance, in our example, we might want the server to create the date part of the URIbased on the current date. In such cases, it is perfectly fine to use the POST verb to the main resource URI and let the server respond with the location of the newly created resource:

POST /data/documents/balance HTTP/1.1Content-Type: text/xmlHost: www.mydatastore.com

<?xml version="1.0" encoding="utf-8"?><balance date="22082014"><Item>Sample item</Item><price currency="EUR">100</price></balance>

HTTP/1.1 201 CreatedContent-Type: text/xmlLocation: /data/documents/balance

Principle 4 – resources can have multiplerepresentationsA key feature of a resource is that it may be represented in a different form than the one it isstored. Thus, it can be requested or posted in different representations. As long as thespecified format is supported, the REST-enabled endpoint should use it. In the precedingexample, we posted an XML representation of a balance, but if the server supported theJSON format, the following request would have been valid as well:

POST /data/documents/balance HTTP/1.1Content-Type: application/jsonHost: www.mydatastore.com

{

REST – What You Did Not Know

[ 12 ]

"balance": { "date": ""22082014"", "Item": "Sample item", "price": { "-currency": "EUR", "#text": "100" } }}HTTP/1.1 201 CreatedContent-Type: application/jsonLocation: /data/documents/balance

Principle 5 – communicate statelesslyResource manipulation operations through HTTP requests should always be consideredatomic. All modifications of a resource should be carried out within an HTTP request inisolation. After the request execution, the resource is left in a final state, which implicitlymeans that partial resource updates are not supported. You should always send thecomplete state of the resource.

Back to the balance example, updating the price field of a given balance would meanposting a complete JSON document that contains all of the balance data, including theupdated price field. Posting only the updated price is not stateless, as it implies that theapplication is aware that the resource has a price field, that is, it knows its state.

Another requirement for your RESTful application is to be stateless; the fact that oncedeployed in a production environment, it is likely that incoming requests are served by aload balancer, ensuring scalability and high availability. Once exposed via a load balancer,the idea of keeping your application state at server side gets compromised. This doesn'tmean that you are not allowed to keep the state of your application. It just means that youshould keep it in a RESTful way. For example, keep a part of the state within the URI.

The statelessness of your RESTful API isolates the caller against changes at the server side.Thus, the caller is not expected to communicate with the same server in consecutiverequests. This allows easy application of changes within the server infrastructure, such asadding or removing nodes.

Remember that it is your responsibility to keep your RESTful APIsstateless, as the consumers of the API would expect it to be.

Chapter 1

[ 13 ]

Now that you know that REST is around 15 years old, a sensible question would be, “whyhas it become so popular just quite recently?” My answer to the question is that we asdevelopers usually reject simple, straightforward approaches, and most of the time, preferspending more time on turning already complex solutions into even more complex andsophisticated ones.

Take classical SOAP web services for example. Their various WS-* specifications are somany and sometimes so loosely defined, that in order to make different solutions fromdifferent vendors interoperable, a separate specification WS-Basic Profile has beenintroduced.It defines extra interoperability rules in order to ensure that all WS-*specifications in SOAP-based web services can work together.

When it comes to transporting binary data with classical web services over HTTP, thingsget even more complex—as SOAP-based web services provide different ways oftransporting binary data. Each way is defined in other sets of specifications such as SOAPwith Attachment References (SwaRef) and Message Transmission OptimizationMechanism (MTOM). All this complexity was caused mainly because the initial idea of theweb service was to execute business logic remotely, not to transport large amounts of data.

Well, I personally think that when it comes to data transfer, things should not be thatcomplex. This is where REST comes into play, by introducing the concept of resources and astandard means for manipulating them.

The REST goalsNow that we've covered the main REST principles, let's dive deeper into what can beachieved when they are followed:

Separation of the representation and the resourceVisibilityReliabilityScalabilityPerformance

REST – What You Did Not Know

[ 14 ]

Separation of the representation and the resourceA resource is just a set of information, and as defined by Principle 4, it can have multiplerepresentations; however, its state is atomic. It is up to the caller to specify the desiredmedia type with the accept header in the HTTP request, and then it is up to the serverapplication to handle the representation accordingly and return the appropriate contenttype of the resource and a relevant HTTP status code:

HTTP 200 OK in the case of successHTTP 400 Bad Request if an unsupported content type is requested or for anyother invalid request

HTTP 500 Internal Server Error when something unexpected happensduring the request processing

For instance, let's assume that at server side, we have balance resources stored in an XMLformat. We can have an API that allows a consumer to request the resource in variousformats, such as application/json, application/zip, application/octet-stream,and so on.

It would be up to the API itself to load the requested resource, transform it into therequested type (for example, JSON or XML), and either use ZIP to compress it or directlyflush it to the HTTP response output.

The caller can make use of the Accept HTTP header to specify the expected media type ofthe response data. So, if we want to request our balance data inserted in the previoussection in XML format, the following request should be executed:

GET /data/balance/22082014 HTTP/1.1Host: my-computer-hostnameAccept: text/xml

HTTP/1.1 200 OKContent-Type: text/xmlContent-Length: 140

<?xml version="1.0" encoding="utf-8"?><balance date="22082014"><Item>Sample item</Item><price currency="EUR">100</price></balance>

Chapter 1

[ 15 ]

To request the same balance in JSON format, the Accept header needs to be set toapplication/json:

GET /data/balance/22082014 HTTP/1.1Host: my-computer-hostnameAccept: application/json

HTTP/1.1 200 OKContent-Type: application/jsonContent-Length: 120

{"balance": {"date": "22082014","Item": "Sample item","price": {"-currency": "EUR","#text": "100" } }}

VisibilityREST is designed to be visible and simple. Visibility of the service means that every aspectof it should self-descriptive and follow the natural HTTP language according to principles3, 4, and 5.

Visibility in the context of the outer world would mean that monitoring applications wouldbe interested only in the HTTP communication between the REST service and the caller.Since the requests and responses are stateless and atomic, nothing more is needed to flowthe behavior of the application and to understand whether anything has gone wrong.

Remember that caching reduces the visibility of your restful applicationsand in general should be avoided, unless needed for serving resourcessubject to large amounts of callers. In such cases, caching may be anoption, after carefully evaluating the possible consequences of servingobsolete data.

REST – What You Did Not Know

[ 16 ]

ReliabilityBefore talking about reliability, we need to define which HTTP methods are safe and whichare idempotent in the REST context. So, let's first define what safe and idempotent methodsare:

An HTTP method is considered to be safe provided that when requested, it doesnot modify or cause any side effects on the state of the resourceAn HTTP method is considered to be idempotent if its response is always thesame, no matter how many times it is requested

The following table lists shows you which HTTP method is safe and which is idempotent:

HTTP Method Safe Idempotent

GET Yes Yes

POST No No

PUT No Yes

DELETE No Yes

Scalability and performanceSo far, I have often stressed the importance of having stateless implementation and statelessbehavior for a RESTful web application. The World Wide Web (WWW) is an enormousuniverse, containing a huge amount of data and a lot of users eager to get that data. Theevolution of the WWW has brought the requirement that applications should scale easily astheir load increases. Scaling applications that have a state is hardly possible, especiallywhen zero or close-to-zero downtime is needed.

That's why being stateless is crucial for any application that needs to scale. In the best-casescenario, scaling your application would require you to put another piece of hardware for aload balancer. There would be no need for the different nodes to sync between each other,as they should not care about the state at all. Scalability is all about serving all your clientsin an acceptable amount of time. Its main idea is to keep your application running and toprevent Denial of Service (DoS) caused by a huge amount of incoming requests.

Chapter 1

[ 17 ]

Scalability should not be confused with performance of an application. Performance ismeasured by the time needed for a single request to be processed, not by the total numberof requests that the application can handle. The asynchronous non-blocking architectureand event-driven design of Node.js make it a logical choice for implementing an applicationthat scales and performs well.

Working with WADLIf you are familiar with SOAP web services, you may have heard of the Web Service Definition Language (WSDL). It is an XML description of the interface of the service anddefines an endpoint URL for invocation. It is mandatory for a SOAP web service to bedescribed by such a WSDL definition.

Similar to SOAP web services, RESTful services can also make use of a descriptionlanguage, called WADL. WADL stands for Web Application Definition Language. Unlike WSDL for SOAP web services, a WADL description of a RESTful service is optional, that is,consuming the service has nothing to do with its description.

Here is a sample part of a WADL file that describes the GET operation of our balanceservice:

<application xmlns="http://wadl.dev.java.net/2009/02"xmlns:xsd="http://www.w3.org/2001/XMLSchema"xmlns:service="http://localhost:8080/data/balance"><grammer><include href="balance.xsd"/><include href="error.xsd"/></grammer><resources base="http://localhost:8080/data/balance/"><resource path="{date}"><method name="GET"><request><param name="date" type="xsd:string" style="template"/></request><response status="200"><representation mediaType="application/xml" element="service:balance"/><representation mediaType="application/json" /></response><response status="404"><representation mediaType="application/xml" element="service:balance"/></response></method>

REST – What You Did Not Know

[ 18 ]

</resource></resources></application>

This extract of a WADL file shows how application-exposing resources are described.Basically, each resource must be a part of an application. The resource provides the URIwhere it is located with the base attribute, and describes each of its supported HTTPmethods in a method. Additionally, an optional doc element can be used at resource andapplication to provide additional documentation about the service and its operations.

Though WADL is optional, it significantly reduces the efforts of discovering RESTfulservices.

Taking advantage of the existing infrastructureThe best part of developing and distributing RESTful applications is that the infrastructureneeded is already out there waiting restlessly for you. As RESTful applications use theexisting web space heavily, you need to do nothing more than follow the REST principleswhen developing. In addition, there are plenty of libraries available out there for anyplatform, and I do mean any given platform. This eases development of RESTfulapplications, so you just need to choose the preferred platform for you and startdeveloping.

SummaryIn this chapter, you learned about foundation of a REST, looking at the five key principlesthat turn a web application into a REST-enabled application. We made a slight comparisonbetween RESTful services and classical SOAP web services, and finally took a look at howRESTful services are described and how we can simplify the discovery of the services wedevelop.

Now that you know the basics, we are ready to dive into the Node.js way of implementingRESTful services. In the next chapter, you will learn about the essentials of Node.js and theaccompanying tools that are necessary to use and understand in order to build a real-lifefully-fledged web service.

Where to buy this book You can buy RESTful Web API Design with Node.js Second Edition from the Packt Publishing website. Alternatively, you can buy the book from Amazon, BN.com, Computer Manuals and most internet book retailers.

Click here for ordering and shipping details.

www.PacktPub.com

Stay Connected:

Get more information RESTful Web API Design with Node.js Second Edition