Informix REST API Tutorial

REST:

The Key to Driverless, Unified Access

to JSON, Relational and Spatio Temporal Data

Brian Hughes

May 2016Erika Von Bargen

[email protected]

Please Note:• IBM’s statements regarding its plans, directions, and intent are subject to change or

withdrawal without notice at IBM’s sole discretion.

• Information regarding potential future products is intended to outline our generalproduct direction and it should not be relied on in making a purchasing decision.

• The information mentioned regarding potential future products is not a commitment,promise, or legal obligation to deliver any material, code or functionality. Informationabout potential future products may not be incorporated into any contract.

• The development, release, and timing of any future features or functionalitydescribed for our products remains at our sole discretion.

• Performance is based on measurements and projections using standard IBM benchmarksin a

• controlled environment. The actual throughput or performance that any user willexperience will vary depending upon many factors, including considerations such as theamount of multiprogramming in the user’s job stream, the I/O configuration, the storageconfiguration, and the workload processed. Therefore, no assurance can be given that anindividual user will achieve results similar to those stated here.

2

3

Notices and Disclaimers

Copyright © 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written

permission from IBM.

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.

Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the

date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS

document is distributed "AS IS" without any warranty, either express or implied. In no event shall IBM be liable for any damage arising from the use of this

information, including but not limited to, loss of data, business interruption, loss of profit or loss of opportunity. IBM products and services are warranted according

to the terms and conditions of the agreements under which they are provided.

Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.

Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those

customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments

may vary.

References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all

countries in which IBM operates or does business.

Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All

materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any

individual participant or their specific situation.

It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and

interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply

with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law.

3

4

Notices and Disclaimers (con’t)

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources.

IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims

related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant

the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM expressly disclaims all

warranties, expressed or implied, including but not limited to, the implied warranties of merchantability and fitness for a particular purpose.

The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or

other intellectual property right.

• IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DB2® , DOORS®, Emptoris®, Enterprise Document

Management System™, FASP®, FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM

Social Business®, IMS™, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™,

PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®,

Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson,

WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions

worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at

"Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

4

http://www.ibm.com/legal/copytrade.shtml

Agenda

• Data access challenges• High Level Solution• What is REST?• Why REST?• REST with Infomix• New Features for REST

– Stored procedure calls– Compression– XML,BSON support

5

DATA ACCESS CHALLENGES

6

Data Access Challenges

• Databases born before the rise of Internet service APIs require a three-tier architecture to access• Client → Application Server → Database

• Three-tier architecture solutions typically use different languages at each layer• JavaScript → PHP/Node.js → SQL/SPL

• Using different languages at each layer:• Complicates development, debugging, deployment, & maintenance• Developer skills, libraries, and tools are duplicated at each layer

• This results in a higher initial cost, lengthened time to market, and increased long term costs.

7


• Client access to the database is generally through a proprietary, vendor-supplied driver.

• New web application development frameworks and programming languages regularly enter the market.

• Customer is dependent on the vendor to support new frameworks and languages.

• Standard database APIs such as ODBC and JDBC along with bridges help to mitigate this issue. However, bridges often expose only a subset of the full capabilities of the client and database.

8


• Timeseries/Geospatial/NoSQL structures • REST is the WWW. The latest languages,

platforms, and frameworks support REST (HTTP) on day one.

• JSON is ubiquitous on the Internet due to its compact representation and expressive structure.

9


• Data is fragmented across many single-purpose silos:– Service A for relational data, B for unstructured documents

(JSON/BSON), C for time series data, D for spatial data.– The most compelling applications rely on integrating multiple types of

data.– Asking questions that span multiple silos is difficult and time

consuming, both in terms of development and query processing.

• Different query languages and APIs for each type of database.– Relational Data: SQL– Unstructured Documents: NoSQL, MongoDB syntax, Xquery– Time Series Data: Specialized time series query language

10

Summary of Challenges

• Rapidly evolving development landscape.

• Pressure to release quickly and frequently to capitalize on opportunities.

• Applications that provide the highest value rely on data that is stored in different silos.

11

HIGH LEVEL SOLUTION

12

High Level Solution

• Support all required data types out-of-the-box.– Lowers time to market.

• Support SQL and NoSQL queries against required data types.– Lowers time to market. Simplifies access to legacy relational data.

Developers without SQL skills can be leveraged.

• Support client access using REST. – Solution meets customers where they are, rather than forcing them to rely

on the vendor.

13

High Level Solution Architecture

14

Detailed Solution Architecture

15

WHAT IS REST?

16

What is REST?

• REpresentational State Transfer

• A W3C Standard for data access as HTTP/1.1 and URI (vs URL)

• The World Wide Web is a REST service

• REST is a style of design.– It is not a technology.

– It is not a protocol.

17

Basic HTTP Methods

• Four basic methods– GET

A request to retrieve data. Idempotent.– POST

A request to create a new resource.– PUT

A request to update/modify the resource referred to by the URI.– DELETE

A request to delete the resource referred to by the URI.

• This solution does not make use of other HTTP methods at this time.

18

REST != SOAP

• Both REST and SOAP can be used with HTTP as the protocol.• With REST

– The components of the URI refer to objects, often in a hierarchy.– The HTTP method specifies the action to perform on the object.– Example URIs: /customer/device, /database/table– Most or all HTTP methods apply to any URL in the model.

• With SOAP– The components of the URL often refer to application-specific operations that supplant the

HTTP operations. The URL is the API to a class.– The HTTP method is almost an afterthought.– Example URLs: /device/getDeviceIds, /device/createDevice– /device is the class, createDevice is clearly a method– Running the DELETE method on /device/createDevice would be meaningless/ambiguous.– //en.wikipedia.org/wiki/Representational_state_transfer

19

Why REST?

• REST is ubiquitous. 70% of APIs are RESTful.

• Nearly every programming language, operating system, and device is capable of using REST (HTTP).

http://blog.programmableweb.com/2013/09/23/programmablewebs-directory-hits-10000-apis-and-counting

20

RESTful Data Access

• URL Structure

– http://<host>[:port]/<db>/<collection>

• 'http' is the name of the scheme, often called the protocol

• host is the domain/host name but can also be a literal IP address

• port is the optional port number

• db, the first component of the resource path, is the db/database name

• collection, the second component of the resource path, is the collection/table name

• The model presented by the REST API is:

– A host contains zero or more dbs (databases).

– A db contains zero or more collections (tables).

– The terms db and collection refer to the MongoDB representations. The terms database and table refer to the Informix representation. Due to character restrictions, the database name may be similar but not identical to the db name in some situations.

21

REST WITH INFORMIX

23

HTTP Method Overview

24

DB Management

• Four basic operations:

– List all databases

– Create a database

– Delete a database

– Delete all databases

25

List All DBs

• Method: GET /

• Response: A JSON array of strings, where each string is the name of a database

• Example Response:

• [ "stores_demo" , "test" ]

26

Create a DB

• Method: POST /• POST Data: A JSON document that specifies details about the db to be created. At

a minimum, this document must include the field name' to specify the name of the db. For example:

• {name: "mydb"}• {name: "mydb" , locale: "en_us.utf8", dbspace: "dbspace0"}• Response: A JSON document describing the status of the request.• Example Response:• {msg:"created database mydb", ok: true}• Explicit creation of a db is not necessary. If a request for a db that does not exist is

made, it will automatically be created using default values. This was done to simplify programming.

27

Collection Management

• Three basic operations:

– List all collections

– Create a collection (explicitly)

– Delete a collection

• Delete all collections

– To delete all collections, drop the database

28

List All Collections

• Method: GET /<db>• Response: A JSON array of strings, where each

string is the name of a collection in the db.• Example Request:• GET /demo• Example Response:• [“users",“inventory"]

29

Create a Collection

• Method: POST /<db>• POST Data: A JSON document that describes the collection (table) to be

created. At a minimum this document must contain the field 'name', which is the collection's name.

• Example Request:POST /demo{name:"bar"}

• Example Response:{"msg":"created collection demo.bar","ok":true}

• Explicit creation of a collection is not necessary. If a request for a collection that does not exist is made, it will automatically be created using reasonable default values. This was done to simplify programming.

30

Delete a Collection

• Method: DELETE /<db>/<collection>• Response: A JSON document describing the

namespace that was dropped and the status.• Example Request: DELETE /demo/bar• Example Response:{"msg":"dropped collection","ns":"demo.bar","ok":true}

31

Query Example

• Get all documents from the namespace test.people (db=test, collection=people)GET /test/people

• Example Result[{"_id":{"$oid":"533501be27784337861702c5"},"name":"lance"},

{"_id":{"$oid":"533501c227784337861702c6"},"name":"brian"},

{"_id":{"$oid":"533501c627784337861702c7"},"name":"erika"}]

• The result is a JSON array of documents.• In this example, each document has two fields: _id and name• The _id has a value of a complex type ObjectId, which is transmitted as a sub-

document with the field $oid• The name field has a simple JSON string as the value

32

Query Example

• Query with condition:– Get all documents from the namespace test.users (db=test, collection=users)

whose age > 30.– Also sort the results by age and filter the fields in the response

GET /test/users?query={age:{$gt:30}}&sort={age:1} &fields:{firstName:1, age:1, _id:0}

• Example Result[{"firstName":"Sherry","age":31},

{"firstName":"John","age":31},

{"firstName":"Bob","age":47},

{"firstName":"Larry","age":49}]

?sort={age:1}&fields={_id:0,lastName:0}

33

Insert Example

• Insert a single document– POST to collection namespace, providing your JSON document to insert in the

data of the HTTP request

POST /test/usersData:

{firstName:"John",lastName:"Doe",age:31}

Response:{"n":1, "ok":true}

• To insert multiple documents:– POST to collection namespace, providing a array of JSON documents to insert

in the data of the HTTP request. The response will indicate the number of documents inserted: {"n":4, "ok":true}


34

Update Example

• Update a document– Send a PUT to a collection namespace, providing your

query condition in the URL and the update operation in the data of the HTTP request

PUT /test/users?query={firstName:“Larry”}Data:

{"$set": {age : 25}}



35

Delete Example

• Delete a document

– Send a DELETE request to a collection namespace, providing your query condition in the URL

DELETE /test/users?query={name:“Larry”}



36

HTTP Query Parameters

• HTTP supports parameters in the URL• Parameters start after the '?' symbol and are are key=value pairs delimited by the

'&' symbol• Example:

GET /demo/people?a=b&c=d

• Basic HTTP Query Parameters– query– fields– sort

• Advanced HTTP Query Parameters– limit– skip– batchSize , cursorId (uses HTTP request/response header)

37

Basic HTTP Query Parameters

• query– The value assigned to this parameter should be a JSON document that defines the query to be

performed. The document uses the Mongo NoSQL syntax.

• fields– The value assigned to this parameter should be a JSON document that specifies the fields to

include in the documents returned. This is the NoSQL (Mongo) equivalent of the SQL projection clause. The fields of this document have values of 1 or 0 to indicate if the field should be included in the result or not, respectively. For example, to include the field name and exclude the field id, the document should be {_id:0,name:1}

• sort– The value assigned to this parameter should be a JSON document that specifies the field(s) to

sort the result with. The fields of this document have values of 1 or -1 to indicate if the field should be used to sort ascending or descending, respectively. For example to sort by age descending and name descending: {age:1,name:-1}

38

Advanced HTTP Query Parameters

• limit– The value assigned to this parameter should be a positive integer that

specifies the maximum number of documents to return.

• skip– The value assigned to this parameter should be a positive integer that

specifies the number of documents to skip at the start of the result set returned by the query.

• batchSize– The value assigned to this parameter should be a positive integer that

specifies the maximum number of documents to return in each request to page more data from a cursor.

39

Paging Data

• The REST API supports the use of cursors to page data in small chunks from the server to the client.

• By paging data in smaller chunks, the processor, memory and network resource utilization of a query for a large number of rows can be reduced.

• A cursor is created to help page data from the server to a client in the following scenarios:

• The number of rows in the response is greater than the batch size specified by the user when performing the query

• The number of rows in the response is greater than the maximum number of rows in a response specified in the listener configuration

• The size in bytes of the response is larger than the maximum response size specified in the listener configuration

40

Paging Data

• When a cursor is available, an additional header is present in the HTTP response to a query.• The name of the header is 'cursorId'• To get more data from the cursor, send a GET request that includes the cursorId from the response

in the request headers.• As long as the cursorId header is present in responses to get more data from a cursor, additional

requests to retrieve more data from the cursor can be sent.• When the cursorId is no longer present in the response, there is no more data available.• The value of the cursor ID is of type long.• This feature is particularly useful with frameworks such as dojo and jquery that support

asynchronous data sources.• The ability to repeatedly return to the same session using cookies is critical to the operation of

cursors. A cursor is owned by a session. Cursors cannot be shared betweens sessions.

41

Command Example

• You can also use the REST API to run commands– Run a GET request against the pseudo “$cmd” collection in your database and

specify the command document as the query condition in the URL.– Command documents are based on the Mongo DB command syntax

• Example: Get the collection statistics about the “users” collection.GET /test/$cmd?query={collStats:“users”}Response:

[{"ns":“store_demo.users", "count":1000,

"size":322065, "avgObjSize":322,

"storageSize":323584, "numExtents":158,

"nindexes":1, "lastExtentSize":2048,

"paddingFactor":0, "flags":1,

"indexSizes":{"_id_":49152},

"totalIndexSize":49152, "ok":1}]

42

• You are able to do transaction work through the REST API by using the transaction commands.

• To enable transactions for your session

GET /test/$cmd?query={transaction:“enable”}– This binds your session id to a specific JDBC connection that will be kept exclusively for your

session until you disable transactions in your session. The REST listener will maintain your session binding for you based on a cookie.

– Any operations that you run – POST, PUT, DELETE – will be in considered as a single transaction until you run a REST request to commit or rollback the transaction.

• To commit a transaction

GET /test/$cmd?query={transaction:“commit”}

• To rollback a transaction

GET /test/$cmd?query={transaction:“rollback”}

• To disable transaction for your session

GET /test/$cmd?query={transaction:“disable”}

Transaction Example

43

Unified Data Access

• Objective: Provide unified access to the following types of data:– Relational Tables

– BSON Document Collections

– Time Series data

– Spatial Data

• The flexibility of JSON enables us to represent all of the above as a JSON document collection.

44

Document Collection Access

• JSON documents are transformed into a BSON representation and are stored in a column of a relational table.

• Document collections accessed through the REST API have the same capabilities as those accessed through the MongoDB API.

• Indexes can be created on fields of the JSON/BSON document using Informix functional indexes.

45

Relational Data Access

• The columns of a relational table are mapped to the fields of a JSON document.

• MongoDB API calls can be run on relational tables as though the tables were document collections. For example, the same call to create an index on a field of a document would create an index on a column of the same name.

• All basic operations available with document collections (insert, update, delete, query) can be used without modification against a relational table.

– Story: We had a user, who was using a collection, create a relational table that had the same name as the collection and had columns with corresponding data types and names as the fields in the document. The application worked without modification.

• Informix row types are represented as sub-documents.

46

Timeseries Data Access

• Informix time series data is stored in a way that is similar to a column-store database.

• The present strategy is to use the Informix Virtual Table Interface (VTI) to make the TimeSeries data appear to be a relational table.

• This virtual table can then be accessed by the REST API.

47

Spatial Data Access

• Spatial data can be inserted into a document collection using the GeoJSONspecification.

• Informix functional indexes enable users to efficiently query spatial data.

{

"type": "Feature",

"geometry": {

"type": "Point",

"coordinates": [125.6, 10.1]

},

"properties": {

"name": "Dinagat Islands" }

}

48

NEW FEATURES IN REST

49

Stored Procedures via REST

• Coming in 12.10xC7• New commands supported in both REST and MongoDB protocols

– “runProcedure”, “runFunction”

• Takes in the procedure name and an array of arguments• Always returns a response for success/failure

– { “ok” : 1.0 } for success

• Can return data from the procedure– Stored in JSON as “returnValue”– Can include one or more rows/columns

50

Stored Procedures via RESTCall procedure with arguments and no return value:

GET http://localhost:27018/spdb/$cmd?query={"runProcedure":"increase_price",

"arguments":[101, 10]}

Response: [{"ok":1.0}]

Call function that returns a single value:

GET http://localhost:27018/spdb/$cmd?query={"runFunction":"add_values",

"arguments":[3,6]}

Response: [{"returnValue":9,"ok":1.0}]

Call procedure that returns multiple row (using WITH RESUME):

GET http://localhost:27018/spdb/$cmd?query={runProcedure:"colors_list"}

Response:

[{"returnValue":[{"color":"Red","hex":"FF0000"},{"color":"Blue","hex":"0000A0"},{"color":"White",

"hex":"FFFFFF"}],"ok":1.0}]

51

Compression in REST

• Wire Listener now supports compression HTTP Headers

– deflate, gzip as encoding types

– Can get 50% or better compression on JSON objects

52

Compression HTTP Headers

• Accept-Encoding– Sent by the client– Tells server what encoding it can accept

• Accept– Defines what types of content the client is able to accept

• Content-Encoding– Sent back by the server– Tells client what encoding the message is encoded with

• Content-Type– Defines what the type of content the client is sending the server

53

Data Conversion in REST

• Wire Listener supports additional options from JSON for sending/receiving data

– XML, BSON

• BSON can be used for smaller data transfers

– Content-Type: application/bson

• Avoids JSON/BSON conversion in Wire Listener

54

XML Data in REST

• Content-Type: application/xml

• Accept: application/xml

• Example: { a: [ {b:adam} , {c:25} ] }

55

Example REST Headers

56

Final Thoughts

• The fusion of:– JSON documents– Schema-free document collections– NoSQL query syntax– Unified data manipulation language– RESTful API

• Enables Informix to provide a hybrid database solution that is uniquely capable of simplifying application development, reducing the time to market, and increasing the opportunity to create compelling applications.

57

THANK YOU!Questions or Comments?

58

Informix REST API Tutorial

Software

Transcript of Informix REST API Tutorial