REST Drupal

47
Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal REST Drupal Talking to Drupal from Javascript, iOS and Android 1 Sunday, November 18, 12

description

Talk given at Drupalcamp Arad 2012 about ways of connecting to a REST Drupal service using iOS and Android.

Transcript of REST Drupal

Page 1: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal

REST DrupalTalking to Drupal from Javascript, iOS and Android

1

Sunday, November 18, 12

Page 2: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 2

Hi, I’m Alexandru Badiu.I’m a software engineer, amateur game developer and part time Mister T impersonator.

Drupal user for 9 years, board member in Drupal Romania.

I work for Demotix.

Twitter @voidbergWeb ctrlz.ro

I pity the foolwho doesn’t use

Drupal

About me

Sunday, November 18, 12

Page 3: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 3

REST Drupal

Mobile is pretty big now days

Drupal 8 Web services initiative

For Drupal 7 there’s the Services module

Sunday, November 18, 12

Page 4: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 4

YASS

Oh boy, yet another Services session!

Sunday, November 18, 12

Page 5: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 5

Nope

Sunday, November 18, 12

Page 6: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 6

In this sessionDoing “web services” without Services

Doing “web services” without Drupal’s slowness

Using Services

... With Javascript

... With Objective-C

... With Java

Uploading files and cursing at Apple and Google

Some memes

Sunday, November 18, 12

Page 7: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 7

Sorry :(

Sunday, November 18, 12

Page 8: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 8

1Services without

services

Sunday, November 18, 12

Page 9: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 9

Why would you do that?

One word: Drupal is slow

Sunday, November 18, 12

Page 10: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 10

Services without services

Sometimes you really don’t need services

Getting a list of news and showing some details about them.

Showing nearby businesses on a map.

Users submitting anonymous tips or comments.

Use JSON and don’t shoot yourself in the foot by using XML or plists.

Sunday, November 18, 12

Page 11: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 11

HoweverServices is not the slowest thing, Drupal is

Shameless plug: Drupal NoBootstrap

Small php library which allows you to be able to use some of Drupal's functions without bootstrapping Drupal.

Used it for Solr powered instant Google like map of businesses

Used it for lightning fast autocomplete

Used it for coupon claiming app

https://github.com/voidberg/Drupal-NoBootstrap

Sunday, November 18, 12

Page 12: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 12

2Services

Sunday, November 18, 12

Page 13: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 13

Services

A long time ago in a Drupalcamp far, far away

Last year I was complaining about Services

Clunky way of connecting (session, nonce, timestamp)

Inconsistent responses

Bugs in json_server

Sunday, November 18, 12

Page 14: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 14

Services 3

Services 3 is much much nicer

REST based

No more nonce and other crap

Session is kept in the HTTP calls

Consistent responses

Sunday, November 18, 12

Page 15: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 15

Services 3REST in a nutshell

Resource based

HTTP based

CRUD

GET /endpoint/resource/id

POST /endpoint/resource + (post data)

DELETE /endpoint/resource/id

PUT /endpoint/resource/id + (post data)

GET /endpoint/resource

GET /services/comment?parameters[nid]=123&parameters[timestamp]=123456:123600

Sunday, November 18, 12

Page 16: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 16

Services 3REST in a nutshell

Actions

Do not target a specific resource

POST /services/apachesolr/reindex

Targeted actions

Target a specific resource

POST /services/node/123/unpublish

Relationships

GET /services/node/123/comments

Sunday, November 18, 12

Page 17: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 17

3Drupanium

Sunday, November 18, 12

Page 18: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 18

Drupanium

Distribution targeted at building mobile apps

For Titanium Studio

Should be possible to use with Cordova

Looks abandoned

Simple js code to grab

Decent list of core services resources

http://drupanium.org/api

Sunday, November 18, 12

Page 19: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 19

4iOS

Sunday, November 18, 12

Page 20: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 20

Android

Drupal iOS SDK

The library formerly known as KBDrupalConnect

Uses AFNetworking

Implements all core service resources

Can use OAuth

Uses blocks

https://github.com/workhabitinc/drupal-ios-sdk

Sunday, November 18, 12

Page 21: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 21

Drupal iOS SDK

Copy AFNetworking code and Drupal iOS SDK code in project

Update Settings.m with url, endpoint etc

Profit!

Sunday, November 18, 12

Page 22: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 22

Drupal iOS SDKDIOSUser *user = [[DIOSUser alloc] init];

[user userLoginWithUsername:username andPassword:password success:^(AFHTTPRequestOperation *operation, id responseObject) {

// Login ok DIOSSession *session = [DIOSSession sharedSession]; [session setUser:[responseObject objectForKey:@"user"]]; } failure:^(AFHTTPRequestOperation *operation, NSError *error) { // Login failed [MBProgressHUD hideHUDForView:self.view animated:YES]; }];

Sunday, November 18, 12

Page 23: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 23

Drupal iOS SDKDIOSNode *node = [[DIOSNode alloc] init];

NSMutableDictionary *nodeData = [NSMutableDictionary new];[nodeData setValue:[@"Node title" text] forKey:@"title"];

NSDictionary *bodyValues = [NSDictionary dictionaryWithObjects:[NSArray arrayWithObjects:@"Node title", nil] forKeys:[NSArray arrayWithObjects:@"value", nil]];NSDictionary *languageDict = [NSDictionary dictionaryWithObject:[NSArray arrayWithObject:bodyValues] forKey:@"und"];[nodeData setValue:languageDict forKey:@"body"];

NSDictionary *fieldValue = [NSDictionary dictionaryWithObjects:[NSArray arrayWithObjects:@"Field value", nil] forKeys:[NSArray arrayWithObjects:@"value", nil]];NSDictionary *fieldLangDict = [NSDictionary dictionaryWithObject:[NSArray arrayWithObject:sessionValue] forKey:@"und"];[nodeData setValue:fieldLangDict forKey:@"field_foo_bar"];[nodeData setValue:@"nodetype" forKey:@"type"];[nodeData setValue:@"und" forKey:@"language"];[node nodeSave:nodeData success:^(AFHTTPRequestOperation *operation, id responseObject) { // Node was saved} failure:^(AFHTTPRequestOperation *operation, NSError *error) { // Node save failed}];

Sunday, November 18, 12

Page 24: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 24

Extending it@interface DemotixServices: NSObject- (void)userRegister:(NSDictionary *)user success:(void (^)(AFHTTPRequestOperation *operation, id responseObject)) success failure:(void (^)(AFHTTPRequestOperation *operation, NSError *error)) failure;@end

#import "DemotixServices.h"#import "DIOSSession.h"

@implementation DemotixServices

#pragma mark userRegister- (void)userRegister:(NSDictionary *)user success:(void (^)(AFHTTPRequestOperation *operation, id responseObject)) success failure:(void (^)(AFHTTPRequestOperation *operation, NSError *error)) failure { [[DIOSSession sharedSession] postPath:[NSString stringWithFormat:@"%@/%@", kDiosEndpoint, @"demotix_services_user/register"] parameters:user success:success failure:failure];}@end;

Sunday, November 18, 12

Page 25: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 25

Saving the session// After a loginNSHTTPCookieStorage *cookieStorage = [NSHTTPCookieStorage sharedHTTPCookieStorage]; NSArray *cookies = [cookieStorage cookies]; for (NSHTTPCookie *cookie in cookies) { if ([cookie.name hasPrefix:@"SESS"]) { [keychainS setObject:cookie.name forKey:(__bridge id)kSecAttrAccount]; [keychainS setObject:cookie.value forKey:(__bridge id)kSecValueData]; }}

// On next app startNSString *sessionName = [keychainS objectForKey:(__bridge id)kSecAttrAccount];NSString *sessionValue = [keychainS objectForKey:(__bridge id)kSecValueData];

if (![sessionName isEqualToString:@""] && ![sessionValue isEqualToString:@""]) { DIOSSession *session = [DIOSSession sharedSession]; [session setDefaultHeader:@"Cookie" value:[NSString stringWithFormat:@"%@=%@", sessionName, sessionValue]];

DIOSSystem *system = [[DIOSSystem alloc] init]; [system systemConnectwithSuccess: ^(AFHTTPRequestOperation *operation, id responseObject) { } failure:^(AFHTTPRequestOperation *operation, NSError *error) { }];}

Sunday, November 18, 12

Page 26: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 26

File uploadNSMutableURLRequest *request = [[DIOSSession sharedSession] multipartFormRequestWithMethod:@"POST" path:[NSString stringWithFormat:@"%@/%@", kDiosEndpoint, @"demotix_services_story/upload"] parameters:params constructingBodyWithBlock: ^(id <AFMultipartFormData> formData) { NSError *error; if (![formData appendPartWithFileURL:[NSURL fileURLWithPath:filePath] name:@"joined" error:&error]) { NSLog(@"An error occured: %@", error); } else { NSLog(@"No error"); }}];

AFHTTPRequestOperation *operation = [[AFHTTPRequestOperation alloc] initWithRequest:request];[operation setUploadProgressBlock:^(NSInteger bytesWritten, long long totalBytesWritten, long long totalBytesExpectedToWrite) { [delegate updateUploadProgress:totalBytesWritten from:totalBytesExpectedToWrite];}];

[operation setCompletionBlockWithSuccess:success failure:failure];[operation start];

Sunday, November 18, 12

Page 27: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 27

File upload

Sunday, November 18, 12

Page 28: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 28

File upload

AFNetworking uses the iOS API for http calls

The API has a bug since iOS 3

When uploading files on 3G you get a OOM error

Solution is to throttle the upload which the API has no support for

AFNetworking is still working on this

I ended up using ASIHttpRequest just for file upload with throttled upload on 3G

Sunday, November 18, 12

Page 29: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 29

File upload[ASIHTTPRequest setShouldThrottleBandwidthForWWAN:YES];[ASIHTTPRequest setMaxBandwidthPerSecond:ASIWWANBandwidthThrottleAmount*throttle_amount]; NSURL *url = [NSURL URLWithString:[NSString stringWithFormat:@"%@/%@/%@", kDiosBaseUrl, kDiosEndpoint, @"demotix_services_story/upload"]];__block ASIFormDataRequest *request = [ASIFormDataRequest requestWithURL:url];

[request setCompletionBlock:^{ success(responseString);}];[request setFailedBlock:^{ failure(responseString, error);}];

for (NSString * fkey in files) { NSDictionary *file = [files objectForKey:fkey]; NSString *fileName = [file valueForKey:@"name"]; [request setFile:[file valueForKey:@"fileUrl"] forKey:fileName];}[request setShouldContinueWhenAppEntersBackground:YES];[request setTimeOutSeconds:999];[request setUploadProgressDelegate:self];[request startAsynchronous];

Sunday, November 18, 12

Page 30: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 30

5Android

Sunday, November 18, 12

Page 31: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 31

Android

DrupalCloud is not working anymore

Hasn’t been updated to Services 3

Never really liked the architecture

Dandy might work, I was scared

No documentation at all

Source code seems over engineered

Not sure it works with Services 3 (issue queue says it doesn’t)

Sunday, November 18, 12

Page 32: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 32

Android

So I built my own

Android Drupal SDK

https://github.com/voidberg/Android-Drupal-Sdk

Sunday, November 18, 12

Page 33: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 33

First stepLet’s choose a HTTP Client

It’s a mess

HttpUrlConnection

Been since the dark ages

Had some serious bugs before Froyo

Still behind the apache lib

Apache HTTP Client

Best option so far IMHO

Has been deprecated

Sunday, November 18, 12

Page 34: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 34

We have a winnerAndroid Asynchronous Http Client

http://loopj.com/android-async-http/

You’ll be in select company - Instagram

HTTP requests happen outside the UI thread

Requests use a threadpool to cap concurrent resource usage

Multipart file uploads with no additional third party libraries

Automatic smart request retries optimized for spotty mobile connections

Automatic gzip response decoding support for super-fast requests

Built-in response parsing into JSON with JsonHttpResponseHandler

Persistent cookie store, saves cookies into your app’s SharedPreferences

Uses the Apache HTTP Client

Sunday, November 18, 12

Page 35: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 35

Using it

ServicesClient client;client = new ServicesClient("http://www.example.com", "api/mobile");

client.setBasicAuth("username","password/token");

PersistentCookieStore cookieStore;cookieStore = new PersistentCookieStore(this);client.setCookieStore(cookieStore);

Sunday, November 18, 12

Page 36: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 36

Using itSystemServices ss;ss = new SystemServices(client);

JsonHttpResponseHandler connectHandler = new JsonHttpResponseHandler() { @Override public void onSuccess(JSONObject response) {

JSONObject user = response.getJSONObject("user"); int uid = user.getInt("uid"); }

@Override public void onFailure(Throwable e, JSONObject response) { // System.Connect call failed }

@Override public void onFinish() { }};

ss.Connect(connectHandler);

Sunday, November 18, 12

Page 37: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 37

Using itUserServices us;

us = new UserServices(client);

JsonHttpResponseHandler loginHandler = new JsonHttpResponseHandler() { @Override public void onSuccess(JSONObject response) { boolean error = false; JSONObject user = response.getJSONObject("user"); }

@Override public void onFailure(Throwable e, JSONObject response) { // Username or password were incorrect }

@Override public void onFinish() { activity.hideProgressDialog(); }};

activity.showProgressDialog("Logging you in");us.Login("username", "password");

Sunday, November 18, 12

Page 38: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 38

Extending itpublic class DemotixServices { private ServicesClient client;

public DemotixServices(ServicesClient c) { client = c; }

public void Register(String fname, String lname, String name, String email, String password, AsyncHttpResponseHandler responseHandler) { JSONObject params = new JSONObject(); try { params.put("name", name); params.put("mail", email); params.put("pass", password); params.put("field_firstname", fname); params.put("field_lastname", lname); params.put("legal_accept", "1"); params.put("from_mobile", "1"); } catch (JSONException e) { e.printStackTrace(); }

client.post("demotix_services_user/register", params, responseHandler); }}

Sunday, November 18, 12

Page 39: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 39

File upload

InputStream myInputStream = blah;RequestParams params = new RequestParams();params.put("secret_passwords", myInputStream, "passwords.txt");

File myFile = new File("/path/to/file.png");RequestParams params = new RequestParams();try { params.put("profile_picture", myFile);} catch(FileNotFoundException e) {}

client.post(...)

Sunday, November 18, 12

Page 40: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 40

File upload

Sunday, November 18, 12

Page 41: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 41

File upload

Turns out that the library tries to read all data before uploading

For large files this is a problem as you’ll run out of memory immediately

Solution: go low level and build the multi part yourself by attaching FileInputStreams

Sunday, November 18, 12

Page 42: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 42

File uploadHttpParams httpParams = new BasicHttpParams();HttpConnectionParams.setConnectionTimeout(httpParams, 900 * 1000);HttpConnectionParams.setSoTimeout(httpParams, 900 * 1000);

HttpClient httpclient = ServicesClient.client.getHttpClient();HttpContext httpContext = ServicesClient.client.getHttpContext();

HttpPost httppost = new HttpPost("http://www.example.com/api/mobile/service/upload");httppost.setParams(httpParams);

MultipartEntity entity = new MultipartEntity();// Add parameters to the post bodyentity.addPart("param1", new StringBody(param1.toString(),"application/json", Charset.forName("UTF-8")));

// Add file to post bodyInputStream istream1 = new FileInputStream("file1.jpg");entity.addPart("file1", new InputStreamBody(istream1, "file1"));

httppost.setEntity(entity);HttpResponse response = httpclient.execute(httppost, httpContext);

Sunday, November 18, 12

Page 43: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 43

File upload

Sunday, November 18, 12

Page 44: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 44

File upload

Nope

There is no easy way of getting the progress of an upload

We fake it: use CustomMultiPartEntity instead of MultiPartEntity

This class notifies a listener of the bytes read so far

Calculate an approximation of the size of the post data

Sunday, November 18, 12

Page 45: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 45

File uploadCustomMultiPartEntity postEntity = new CustomMultiPartEntity(new CustomMultiPartEntity.ProgressListener() { @Override public void transferred(long num) { updateUploadProgress((int) ((num / (float) totalUploadSize) * 100)); }});

try { postEntity.addPart("param1", new StringBody(param1.toString(),"application/json", Charset.forName("UTF-8"))); totalUploadSize = postEntity.getContentLength(); for (Map.Entry<String, String> entry : files.entrySet()) { try { InputStream istream = new FileInputStream(entry.getValue()); postEntity.addPart(entry.getKey(), new InputStreamBody(istream, entry.getKey())); totalUploadSize += new File(entry.getValue()).length(); } catch (FileNotFoundException e) {} } httppost.setEntity(postEntity);} catch (UnsupportedEncodingException e) {}

HttpResponse response = httpclient.execute(httppost, httpContext);

Sunday, November 18, 12

Page 46: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 46

Another caveat

Varnish has issues by default

You have a rule that does “return post” for, well, POST calls

Upload will fail with a socket exception

Add another rule that does “return stream” for POST calls made to your upload url

Sunday, November 18, 12

Page 47: REST Drupal

Drupalcamp Arad 2012 - Alexandru Badiu - REST Drupal 47

Thanks!Questions?

Alexandru Badiu.Twitter @voidbergWeb http://ctrlz.ro

Email [email protected]

D.O http://drupal.org/user/8662

Sunday, November 18, 12