Naked and afraid Offline mobile

32
1

Transcript of Naked and afraid Offline mobile

Page 1: Naked and afraid Offline mobile

1

Page 2: Naked and afraid Offline mobile

• Principal IT Specialist, Office of the Sergeant at Arms, United States Senate• Mostly work in Python and Django• Web Developer for Over 15 Years• Newbie when it comes to mobile development

2

Page 3: Naked and afraid Offline mobile

• Why you might want to consider doing offline storage of data on mobile devices• How to do it using the skills you probably already know, meaning HTML and JavaScript• A few technologies that may be new to some of you, specifically Cordova, as well as

PouchDB and CouchDB• There are of course a million ways to do this, but these are the technologies I

used• Caveats and challenges to developing applications this way• Possible future enhancements

3

Page 4: Naked and afraid Offline mobile

• Due to time constraints we won’t be covering:• The ins and outs of the mechanics of deploying to specific mobile platforms, app

stores, etc.• Platform-specific considerations for functionality and necessary visual

adjustments and tweaks (luckily Cordova makes this pretty easy)• The minutiae of every possible method for storing data on a mobile device• Comprehensive details on each of the tools involved• Some of the slick cool hybrid mobile frameworks like Ionic

• That said find me after the talk throughout the rest of the conference and I’m more than happy to go into details on any of this

4

Page 5: Naked and afraid Offline mobile

• We rely on mobile devices to the point where they’ve become an appendage• In addition to all the wholly unnecessary things we use our phones for, they’re also used

for mission critical purposes and to access important data• Mobile devices can be particularly handy in emergencies since we have them with us,

ahem, “everywhere we go”• And with this whole cloud thing you may have heard of, our important information is

available everywhere from any device• Between WiFi and mobile networks, access doesn’t tend to be an issue, BUT …

5

Page 6: Naked and afraid Offline mobile

• Yes, I’ll ask the question no one wants to ask: What if the Internet’s down?• And when I use the term “Internet” I’m speaking of network access in general,

meaning assume you have no access to either WiFi or mobile networks• The ironic thing is the time we might most need access to all that stuff we put in the

cloud is when the cloud itself, or our ability to access it, is down• Granted if the Internet’s down you can’t do things like call or text people, but you might

have important information up in the cloud such as:• Where you hid your zombie kit• The passcode to get into your bunker• The geographic coordinates of your stash of gold bullion, and• Your grandma’s top secret chocolate chip cookie recipe

6

Page 7: Naked and afraid Offline mobile

• If we don’t plan for the scenario we hope never happens, there’s not a lot you can do other than yell at the cloud.

• Luckily with just a little bit of foresight and planning, and a healthy mix of HTML, JavaScript, and some other handy tools, we can make sure that in the event of any flavor of apocalypse, you’ll still be able to make your grandma’s chocolate chip cookies (provided you can find an unlooted supermarket of course)

7

Page 8: Naked and afraid Offline mobile

• I am not a mobile application developer.• I was presented with a challenge at work and asked if I could come up with a solution,

so as developers do I of course said “yes” first and figured out how to do it after committing

• The resultant application was my first foray into anything specifically built for mobile devices

• Thankfully with the tools I’m going to discuss it turned out to not be too terribly difficult, but I want to make it clear that there may be be better/more preferable ways to do all this and if there are people here who ARE experienced mobile developers and I’ve done anything horribly wrong I’d love to know more about how to do all of this better

8

Page 9: Naked and afraid Offline mobile

• The state of tools is such that even if like me you don’t have a lot of experience with mobile development, it’s dangerously easy to build mobile apps

• On the client side of the app, of course the foundation of all this is HTML5, and specifically HTML5 local storage that allows us to easily store data locally on any device

• Add in CSS3 and Twitter Bootstrap 3 to get an easy mobile-first design layer for UX-challenged people such as myself

• JavaScript and jQuery of course are pretty much essential to any webapp you’re going to build these days, mobile or otherwise

• A tool you may not have heard of that I’ll talk about in more detail later is PouchDB, which is a JavaScript database that acts as an abstraction layer for HTML5 local storage

• PouchDB is also the secret sauce in this particular example application because as you might guess from the name it syncs seamlessly with CouchDB, making keeping the data between the mobile clients and the server in sync dead simple

• CouchDB, if you’re not familiar with it, is a NoSQL document database that’s completely built around HTTP, JavaScript, and JSON so it’s a natural fit for a webapp, and a particularly good fit here

• Apache Cordova, which you might also know by the name PhoneGap, is what builds the client code into a mobile app for each targeted platform, which in my case was iOS and –yes it’s true – BlackBerry

• On the server side of the equation, I have a really simple Python and Django application

9

Page 10: Naked and afraid Offline mobile

used for managing the data, but you could of course use any backend• Honestly if your needs are simple enough you could get away with just syncing to

CouchDB and be done with it, provided you let users manage the data from the mobile device

• In the case of the real world app I built following this basic model the mobile app is read-only by design, hence why we needed a way to manage the data

• Also in the real world app I build I chose to use Postgres instead of CouchDB since I had some other considerations but that didn’t negate the benefit of using PouchDB, it’s just a whole lot slicker with CouchDB in the mix which is why I decided to use it for the app I built for this presentation

9

Page 11: Naked and afraid Offline mobile

• When you first load the application on your phone it will try to connect to the cloud and get a data update

• If it can connect, and if it needs an update (we’ll get into how to control that later), it will pull the latest data and repopulate the local database

• If it can’t connect, it can simply continue to operate using the local database• You’ll notice I also have some connections directly between the database on the mobile

device and CouchDB on the server, and we’ll talk about all that in a bit as well

10

Page 12: Naked and afraid Offline mobile

• HTML5 local storage follows the one standard of all things interwebs, and that standard is that there are multiple incompatible implementations with varying degrees of support in various browsers

• If you’re not familiar with the evolution of HTML5 offline storage, the short story is that there have been three major standards: Web Storage, Web SQL, and IndexedDB

11

Page 13: Naked and afraid Offline mobile

• Web Storage is a W3C spec, reaching the recommendation stage in July of 2013• Made up of 2 parts: localStorage, and sessionStorage• localStorage was the early favorite solution for in-browser, offline storage• Was originally part of the HTML5 spec but it’s now its own spec• Available in all browsers early on• Simple key/value API• Downsides:

• It doesn’t look or work anything remotely like what people would consider to be a database

• Can only store strings• There’s no query language associated with it, no schemas, no transactional

safety• Didn’t scale well for large datasets since to find anything you basically had to

loop over the entire object store, has other performance issues• Still exists and is still widely supported, but isn’t a particularly viable solution for most

applications

12

Page 14: Naked and afraid Offline mobile

• WebSQL was a W3C standard for an API that allowed for storing data that could be queried using SQL

• Web SQL seemed like the direction things were going for a while since it makes logical sense and addresses the problems with localStorage

• Initial implementation in Webkit-based browsers like Chrome and Safari was to use SQLite in the browser

• SQLite is a mature, proven, solid solution even for high traffic and with large datasets• As the name implies, it has SQL!• But, it was only available in Chrome and Safari, it was never implemented in Firefox or

IE, and will likely never be implemented in either• The W3C abandoned the specification for Web SQL in 2010 since there was only one,

and would evidently only ever be one, implementation of the standard, which was SQLite

• There is still a movement behind revising Web SQL because as we’ll see in a second, the implementation of in-browser storage that won out has its quirks as well

13

Page 15: Naked and afraid Offline mobile

• IndexedDB is a W3C standard, reached recommendation stage in January of 2015• The easiest way to describe IndexedDB’s characteristics is to think of it as a NoSQL-style

database• Specifically it’s most closely related to document-based datastores such as CouchDB or

MongoDB• Just so it’s clear, unlike the abandoned Web SQL standard, you can’t use SQL with

IndexedDB• On the plus side, with IndexedDB you’re basically just throwing JavaScript objects at it as

they are and it’ll store it• Note that I said “as they are” which means, unlike localStorage, it is capable of

storing more than just strings• Objects are indexed and retrieved with a unique key

• The key can be created by you, or you can specify a field in the object to use as the key, or you can use auto-incrementing keys

• All changes to the database are transactional, which is a big advantage over localStorage• For pulling back multiple documents, you can use a cursor and loop over the objects in

the database• Also, as the name indicates, in IndexedDB you can create indexes on fields within your

documents and retrieve one of multiple documents via the indexes• Remember that you don’t have SQL capabilities so you can’t do something like SELECT *

14

Page 16: Naked and afraid Offline mobile

FROM MYDB WHERE FIRSTNAME = ‘Bob’ but with judicious use of indexes you can grab things and loop over the results to find what you need pretty easily

• IndexedDB has full support in Chrome (both desktop and Android) and Firefox• Partial support in IE starting with IE 10• Even works on BlackBerry as of BlackBerry OS 10• So you might be thinking my short answer to the question of which in-browser storage to

choose would that IndexedDB is the clear winner. And it is. Or it should be anyway.• If you were paying attention – and I trust you were – you noticed I failed to mention a

pretty big browser in the mobile world.

14

Page 17: Naked and afraid Offline mobile

• Apple waited a long time to implement IndexedDB in Safari• Didn’t exist at all on the desktop until 7.1• Wasn’t in iOS until version 8

• Good news is, it’s there• Bad news is, it’s broken. Like FUBAR broken. Like the primary keys don’t work right and

data is overwritten unexpectedly broken.• I have some links in the resources slide but Ray Camden as well as Nolan Lawson from

PouchDB have great blog posts covering the issues in detail if you’re interested• Short answer: don’t use IndexedDB on Safari.• So if we want an app to work on multiple platforms the picture looks pretty bleak! We’re

faced with using something:• Not very good – localStorage, or• An abandoned standard – WebSQL – that will work in Chrome and Safari, but

won’t work in Firefox and IE• So you can’t just use IndexedDB. Because Safari.

15

Page 18: Naked and afraid Offline mobile

• PouchDB is an in-browser database written in JavaScript that simplifies working with HTML5 local storage

• Simplifies how you ask?• As the old adage goes, “All problems in computer science can be solved by another level

of indirection”• What PouchDB does is provide a standard layer on top of the underlying browser-

specific implementations• This allows you to write your application code to use PouchDB, and then PouchDB

decides what is the best underlying storage engine to use for the browser on which the application is running

• You can tell PouchDB which engine to preferably use if it can, but for example with Safari what PouchDB will do is use IndexedDB on Android and BlackBerry, but on Safari it will use Web SQL instead since Safari’s implementation of IndexedDB is horribly broken

• So to you as a developer you only have to figure out how to work with PouchDB, and PouchDB handles the rest

• The other nice thing is PouchDB is, in my opinion, a better, simpler API than using IndexedDB directly, and it also gives you capabilities for querying that IndexedDB doesn’t have

• If you’re familiar with CouchDB, PouchDB lets you write views which is some seriously powerful stuff

16

Page 19: Naked and afraid Offline mobile

• So my true short answer to the question of which in-browser storage to use is “use PouchDB.”

• If for some crazy reason you don’t want to use PouchDB there are other options for putting a layer on top of in-browser storage

• Best one I’ve come across is called localForage – it’s a Mozilla project and is available on github

16

Page 20: Naked and afraid Offline mobile

• In addition to what I said earlier about PouchDB being a JavaScript in-browser database, it’s also a JavaScript implementation of CouchDB, which is a popular NoSQL document database

• CouchDB is extremely well-suited to web and mobile apps by virtue of the fact that it’s built entirely around HTTP and REST, but its major killer feature is that it was built from the ground up with amazing replication and synchronization capabilities

• PouchDB is great in and of itself, and you can choose to use it alone, or even as a JavaScript client that connects directly to a remote CouchDB database.

• It’s when you combine PouchDB on the mobile device with CouchDB on the server things get really interesting, particularly given the challenges of online/offline mobile development

• PouchDB can sync to anything that implements the CouchDB replication protocol• In practice that pretty much means it syncs to CouchDB and its commercial

variants Cloudant and Couchbase• If you want to sync to other datasources you can easily write scripts to keep

things updated between an RDBMS like Postgres and CouchDB• Or, as I said if you want to just have your mobile app grab data via API that fronts

a relational database, that’s fine as well• Due to the design of CouchDB you get extremely powerful online/offline sync and data

conflict detection and resolution “for free” so if you are in a situation where you need to

17

Page 21: Naked and afraid Offline mobile

build an app of this nature, it’s well worth considering

17

Page 22: Naked and afraid Offline mobile

• Great quote from Jason Smith, a CouchDB committer: “The way I like to think about CouchDB is this: CouchDB is bad at everything, except syncing. And it turns out that's the most important feature you could ever ask for, for many types of software.”

• If you’re in a situation where users need to be able to update data from their mobile devices, good sync capabilities are pretty important

• To do bi-directional syncing between PouchDB on the mobile device to CouchDB on the server, despite what Luke might think, it’s literally just that one line of JavaScript where localDB is a variable that represents your local database, and remoteDB is a URL pointing to a remote CouchDB server

• You can either call replication on demand, or set it to do continuous replication, which sounds great at first but has a lot of implications for an online/offline app

• You can also do unidirectional replication in either direction by calling replicate to and replicate from functions

• Replication is asynchronous and PouchDB’s sync function has on complete and on error callback functions

18

Page 23: Naked and afraid Offline mobile

19

Page 24: Naked and afraid Offline mobile

20

Page 25: Naked and afraid Offline mobile

• The immediate concern with synchronizing between multiple databases is data conflicts, particularly if people are able to change data when they’re offline

• CouchDB supports multi-master architecture and every document in the database is versioned

• Specifically this means every document has a revision key that is unique, and that has to be included if you’re going to update an existing document

• All this is a topic in and of itself, but the long and short of it is if you have two conflicting updates to the same document, CouchDB will handle the conflict, and either when an update to a document is rejected, or by using the conflicts feed to audit conflicting documents, you can handle this as you see fit in your app

• To give a specific example, when someone wants to update a document and they’re offline, you could let them do this locally and save it in PouchDB.

• Then when they’re online and sync to CouchDB, if someone else had changed the same document and synced before they did, CouchDB would reject the update and you could then alert the user that the update was rejected because someone else already updated that document.

• And if you wanted to take things a step further, you could even grab the newer version of the document and show them the changes so they could see what differences there are between the two versions and make a decision as to which version, or what combination of versions, should be sent to CouchDB.

21

Page 26: Naked and afraid Offline mobile

• If you want to see what’s going on in your local databases, the developer tools in the browsers all have really good support for inspecting and messing around with local databases of all flavors

• PouchDB also has an add-on for both Chrome and Firefox that’s very similar to the CouchDB Futon tool

22

Page 27: Naked and afraid Offline mobile

• Now with all the concepts in place, let’s see how they’re put into practice in our sample application

23

Page 28: Naked and afraid Offline mobile

• As I mentioned at the beginning I didn’t have any time to talk about the mechanics of deploying to the various platforms

• Long and short of it is use Cordova – you can specify target platforms and get a build specific to each platform

• From there you follow the procedure for each specific platform such as code signing, etc. to make it fully ready for the target platform

• Cordova also supports platform-specific tweaks such as having icons, CSS, etc. appropriate to each platform

• Very easy, very powerful tool that is a huge piece of how we web developer types can get away with building cool mobile apps using the skills and tools we already know

• Warrants further discussion but for the purposes of this talk know that Cordova pretty much just works and is an indispensable piece of this puzzle

24

Page 29: Naked and afraid Offline mobile

• Other stuff I didn’t have time to cover includes:• Application security! Remember, as a web-based app you have many of the same

concerns when you deploy to mobile as you do with webapps in general.• Data security! Once the data is on the device, how is it secured?• Frameworks like Ionic that make your life a whole lot easier and your apps a

whole lot better. I purposely did plain old HTML for this app so we didn’t get bogged down with the complexities of any particular hybrid mobile development framework.

• Platform-specific customizations, which thankfully most of the tools used in this type of development handle pretty well.

• App store deployment and all the rigamaroll around that.

25

Page 30: Naked and afraid Offline mobile

• I know I covered a lot of low-ish-level technical information but if there’s one thing I hope you take away from this presentation it’s that as a web developer, building pretty sophisticated mobile apps that make critical data available in an emergency is relatively easy.

• You can use the languages and tools you already know, including HTML5, JavaScript, and CSS

• Combine that with a few other key technologies like PouchDB to take away some of the pain of dealing with in-browser storage, CouchDB on the server for data synchronization and distribution, and Cordova to build your app for each of your necessary target platforms, and – just as I did – you can pretend that you’re a real mobile developer in no time!

26

Page 31: Naked and afraid Offline mobile

27

Page 32: Naked and afraid Offline mobile

28