Abstracting your complicated back end data

with RhoConnect

Krishna Raja

Enterprise Software Architect

Agenda

• Overview

• Store Server Backend application

• Creating new rhoconnect application and models

• Handling authentication

• Writing source adapter CRUD method

• Creating Rhodes application

• Working of sync flow

• Introduction to rhoconnect settings (poll_interval, partition_type)

• Error handling

• Handling slow backend queries

• Working through timeouts

• Async processing

• Introducing rhoconnect push to notify the client to initiate sync

• Introducing NEWORM

• Bulk Sync

• Writing custom routes

Store Server - Backend

• Rails Application • Source - https://github.com/rhomobile/rhostore.git

• Starting server locally – rails server

• Resource access • http://localhost:3000/products

• http://localhost:3000/customers

Rhoconnect Application

• Create New Rhoconnect application

• rhoconnect app storeapp

• Creating new source adapters

• rhoconnect model product

• rhoconnect model customer

• Handling authentication

• Application Controller

• post "/login"

• Writing source adapters CRUD

• login, logoff

• query

• CUD – create, update, delete, validate

• store_blob

Rhodes application • Creating new Rhodes application

• Rhodes app storeapp

• Creating new models

• rhodes model Product name,brand,price,quantity,sku

• rhodes model Customer first,last,email,phone,address,city,state,zip

• Property Bag vs Fixed Schema

• http://docs.rhomobile.com/en/5.2.2/guide/rhom_ruby

• Model Properties

• Sync

• Sync_type

• Sync_priority

• Full_update

• Partition

• Model Schema Properties

• property :<property_name>, :<data_type>

Sync Flow

• Triggered by manual (doSync/doSyncSource) or periodic sync

• Sync Client changes (POST)

• Create, Update, Delete

• Sync Server changes (GET)

• Query

• One or many pages

• Link records

• Error records

• Sync Notifications

• In_progress

• Ok

• Complete

• error

Rhoconnect settings

• Client side • sync_poll_interval – Periodic polling

• Manual sync – Using doSync or doSyncSource

• Rho::RhoConnectClient.pageSize = 20 (default 2000)

• Server side • poll_interval – To throttle the number of requests hitting the backend.

Server will not poll the backend automatically • Maintained per source per user (in case of user partition)

• Maintained per source per app (in case of app partition)

• partition_type – To optimize the space required in the redis • app - If the data is shared across all the users

• user (default setting) - If the data is for a user

Error Handling

• Server side

• Raising exceptions

• http://docs.rhomobile.com/en/5.2.2/rhoconnect/source-adapters#handling-exceptions

• Sample

• def delete(delete_hash)

• rest_result = RestClient.delete("#{@base}/#{delete_hash['id']}")

• if rest_result.code != 200 raise Rhoconnect::Model::Exception.new("Error deleting record.")

• end

• end

• Other Exceptions

• Rhoconnect::Model::LoginException

• Rhoconnect::Model::LogoffException

• Rhoconnect::Model::ServerTimeoutException

• Rhoconnect::Model::ServerErrorException

• Rhoconnect::Model::ObjectConflictError

• Batch processing of records

• Error record is sent to the client

Error Handling – Client side def sync_notify status = @params['status'] ? @params['status'] : "" if status == "in_progress" # do nothing elsif status == "complete“ Rho::WebView.navigate Rho::RhoConfig.start_path if @params['sync_type'] != 'bulk' elsif status == "error“ if @params['server_errors'] && @params['server_errors']['create-error'] Shopper.on_sync_create_error(@params['server_errors']['create-error'].keys, :recreate ) #or use :delete # Rho::RhoConnectClient.on_sync_create_error(@params['source_name'], @params['server_errors']['create-error'].keys, :recreate ) end if @params['server_errors'] && @params['server_errors']['update-error'] # Rho::RhoConnectClient.on_sync_update_error(@params['source_name'], @params['server_errors']['update-error'], :retry) Rho::RhoConnectClient.on_sync_update_error(@params['source_name'], @params['server_errors']['update-error'], :rollback, @params['server_errors']['update-rollback'] ) if @params['server_errors'] && @params['server_errors'][‘delete-error'] Rho::RhoConnectClient.on_sync_update_error(@params['source_name'], @params['server_errors'][’delete-error'], :retry) if rho_error.unknown_client?( @params['error_message'] ) Rhom::Rhom.database_client_reset # something gone wrong in server side, database would have flushed Rho::RhoConnectClient.doSync elsif err_code == Rho::RhoError::ERR_UNATHORIZED Rho::WebView.navigate(url_for :action => :login,:query => {:msg => "Server credentials are expired"} ) elsif err_code != Rho::RhoError::ERR_CUSTOMSYNCSERVER Rho::WebView.navigate( url_for :action => :err_sync, :query => { :msg => @msg } ) end end end

CUD processing • All objects from a client are placed in the queue as a batch

• Any error stops the processing of that batch

• Objects that succeeded in this batch are sent to the client (as link records)

• Error object is sent to client.

• Client can do the error handling appropriately using on_sync_*_error methods

• Unprocessed objects are placed as a batch at the back of the queue in the rhoconnect server

• You can view the unprocessed records for source model Product and create queue in the rhoconnect console using document name -

source:application:Product:create

• Next time sync on that model (from any device) will trigger the CUD processing again

• Forcing push changes (when there are no records to sync for that source)

• model.push_changes

Timeout

• Rhoconnect server side

• Backend timeout

• Thin server timeout (/etc/thin/rhoapp.yml ) – default 30 sec

• Nginx reverse proxy settings (/opt/nginx/conf/conf.d/rhoconnect.conf)

• Proxy_connect_timeout

• Proxy_send_timeout

• Proxy_read_timeout

• Node timeout – For js adapters (settings.yml) rhoconnect 5.1 onwards) -

• :node_channel_timeout: 60 #default 30 sec

• Rhoconnect Client side

• net_timeout (default = 30 sec)

async

• settings.yml • queue: product • query_queue: product • cud_queue: product

• Starting the resque job • QUEUE=*; rake resque:work #single • QUEUE=*; COUNT=5 rake resque:workers #multiple

• Rhoconnect console • http://localhost:9292/resque/overview

• Notify device when the backend operation is done using push

Async using Rhoconnect push • Rhoconnect push server

• Available as part of the RMS suite (prerequisite – node)

• Configure rhoconnect server for push

• Settings.yml - :push_server: http://some_app_name@localhost:8675

• Write PingJob in the source adapter

• Resque.enqueue(PingJob, {“user_id” => current_user.login, “message” =>”new product”, “sources”=>[‘product’]})

• Install rhoconnect push service apk (for android) to the device

• Client changes

• Include rhoconnect-push extension in the build.yml

• Include push configuration in the rhoconfig.txt

• Push.rhoconnect.PushServer = http://localhost:8675

• Push.rhoconnect.pushAppName = ‘some_app_name’

• Configure push callback

Rho::Push.startNotification ‘/app/Settings/push_callback

• push callback method

def push_callback

Rho::RhoconnectClient.doSyncSource(‘Product’,false)

end

NewORM

• Why NewORM?

• Performance improvement – C++ implementation

• Common API implementation for Ruby and Javascript

• Backward compatibility

• Rhoconfig.txt - use_new_orm=0

• Still be able to access using Rho::ORM when use_new_orm=1

• NewORM and NewORMModel

• Ruby and Javascript wrapper available to migrate the existing application

NewORM – Ruby Differences Old ORM Rho::ORM

NewORM (Wrapper) Rho::NewORM

NewORM(Common API) comments

find find findObjects findObjectsPropertyBagByCondArray findObjectsPropertyBagByCondHash

Not all advanced queries can be done with new wrapper. find_by_sql can be used instead

deleteAll delete_all deleteObjects deleteObjectsPropertyBagByCondArray deleteObjectsPropertyBagByCondHash

save save saveObject

update update updateObject

metadata Not supported

Database import/export

Not supported

RhomSource Not supported

NewORM – Javascript changes Old New

Include rhoapi-modules.js Include rhoapi-modules-ORM.js Include rhoapi-modules-ORMHelper.js Include rhoapi-modules-Ruby-RunTime.js

Include rhoapi-modules.js

products = Rho.ORM.addModel(function(model){ model.modelName("Product"); model.enable("sync"); model.set("partition","user"); model.set("fixedSchema","true"); model.property("brand", "integer"); });

products = Rho.ORM.addModel("Product", function(model){ model.enable ("sync"); model.setProperty("sync_type", "incremental"); model.setProperty("fixedSchema","true"); model.setModelProperty("brand", "integer"); });

onSyncCreateError, onSyncUpdateError, onSyncDeleteError

databaseFullResetEx (Hash) databaseFullResetEx (Array of models, bool client_info, bool local_models)

export Not supported

Bulk Sync

• When large data to be synced for the first time • http://docs.rhomobile.com/en/5.2.2/rhoconnect/bulk-sync

• Use incremental for subsequent sync

• Resque backgroud job

• Calls all the source adapetr query method and creates the sqllite database. It zips and sends back to the client

• Client side setup • bulksync_state=0 in rhoconfig.txt • Rho::RhoConfig.bulksync_state = '0'

• Server side setup • Requires sqlite • Start the resque worker thread • bulk_sync_poll_interval=3600

Writing custom routes

• Server side controller

• Client side (using Network API)

class ShopperController < Rhoconnect::Controller::Base

register Rhoconnect::EndPoint

# register SYNC routes

register Rhoconnect::Handler::Sync

# add your custom routes here

post '/my_custom_route', :login_required => true do

puts 'came to custom route'

end

End

def do_custom_route

session_rec = Rho::RHO.get_user_db.execute_sql("select client_id, session from client_info")

session = session_rec[0]['session']

client_id = session_rec[0]['client_id']

puts "session is #{session}, client_id is #{client_id}"

Rho::Network.post({"url"=>"http://localhost:9292/app/v1/Shopper/my_custom_route", "body"=>"sample", "headers"=>{"Cookie"=>session, "x_rhoconnect_client_id"=>client_id}},

"/app/Settings/url_callback")

end

THANK YOU