Post on 01-Jul-2015
description
• 2011:
- Port of CouchDB to iOS/Android
• 2012:
- TouchDB investigation
• 2013:
- Couchbase Lite and Sync Gateway in development
• 2014:
- May: Couchbase Lite 1.0, Sync Gateway 1.0
- August: 1.0.1
- September: 1.0.2
The Story So Far
2
• ForestDB storage engine
• Enhanced query APIs
• Database encryption
Some Upcoming Couchbase Lite Features
5
Disclaimer
6
This presentation contains certain forward-looking information and statements pertaining to
unreleased (“vaporware”) software features. The use of any of the words “expect”, “promise”,
“really soon now” and similar expressions are intended to identify forward-looking information or
statements. The forward-looking information and statements included in this presentation are not
guarantees of future availability and should not be unduly relied upon. Such information and statements
involve known and unknown unknowns and other factors that may cause actual product features to differ
materially from those anticipated in such forward-looking information or statements. No other warranty
expressed or implied. Performance may settle during shipment and handling. Void where prohibited by law. Do
not use while operating a motor vehicle or heavy equipment. Color variations are part of the natural character of
this feature, not defects. Simulated picture. Professional developer on closed track. Do not code this at home.
Formatted to fit your screen. Some equipment shown is optional. Discontinue product development if burning or
itching occur. Prize not redeemable for cash value. No user-serviceable parts inside. Substantial penalty for early
withdrawal. To be used as a supplementary restraint system only. Always fasten your safety belt. Subject to change
without notice. Times approximate. If condition persists, consult your physician. Not rated by the Motion Picture Association
of America. Call for nutritional information. Any resemblance to real persons, living or dead is purely coincidental. Limited time
offer, call sales dept. now to ensure prompt delivery. No passes accepted for this engagement. This is not an offer to buy securities.
Processed at a facility that also processes peanuts, tree nuts & soy. Not returnable if seal is broken. Post office will not deliver
without postage. Not tested on laboratory animals. Do not remove tag under penalty of law. Employees and their families and
friends are not eligible. All models over 18 years of age. May be too intense for some viewers. Use only as directed; intentional
misuse by deliberately concentrating and inhaling APIs can be harmful or fatal. Fingers crossed behind back. Do not taunt Happy
Fun Feature.
• Couchbase Lite currently uses SQLite for storage
- Ubiquitously available, highly reliable
• But SQLite has overhead we don’t need
- SQL processing, query planner, data constraints…
- Document vs. relational impedance mismatch
Why ForestDB?
8
• New storage engine developed at Couchbase
- For use by server and mobile
• Flat, ordered key-value blob store
- Same family as Berkeley DB, Tokyo Cabinet, LevelDB…
• Highly Optimized
- Hierarchical B+tree-Based Trie index
- Append-only log-based file format
• Watch video of the ForestDB talk from this morning
- “Preview: A Next Generation Storage Engine For NoSQL Database Systems”
What Is ForestDB?
9
• Faster
- HB+-Trie index is more compact
- More efficient write-ahead log
- Document revisions are grouped together
• Smaller Footprint
- Buffer cache shared between threads
• Higher Availability
- Writers never block readers
- Automatic background compaction
Couchbase Lite On ForestDB
11
• CBForest is a layer between ForestDB and CBLite
- Cross-platform C++
- https://github.com/couchbaselabs/cbforest
• Higher-level OO API
• Implements some CBLite features:
- Document revision trees
- Map/reduce indexes
- JSON-compatible key collation
CBForest
12
• Implemented in CBLite for iOS/Mac
- ‘forestdb’ branch
- github.com/couchbase/couchbase-lite-ios/tree/forestdb
• 2x – 5x performance of 1.0
• Coming to other platforms too; no ship date yet
Current Status
13
Performance
14
0
1000
2000
3000
4000
Populating DB Indexing (compound keys) Query with grouping
Docum
ents
/sec
SQLite (CBL 1.0.1) With ForestDB
Data set: iTunes library, 12189 documents(higher is better)
• Map/Reduce is
- Flexible
- Efficient
- Scalable
• but not
- Intuitive
- Full-featured
Map/Reduce
17
• Couchbase is working on a new query language
• Still in development
• Initial implementation is server-focused
• Watch videos of earlier talks:
- “A N1QL For Every Query”
- “Native APIs For Querying Couchbase Server With N1QL”
N1QL: A New Query Language
18
• Cocoa — CoreData
- NSPredicate, NSSortDescriptor, etc…
• .NET — LINQ (Language-INtegrated Query)
- IQueryable, etc…
• My investigation focuses on Cocoa
Existing Platform Query APIs
19
• Picks subset of rows using arbitrary criteria
• Supports NSPredicate’s simple query language
• Evaluated on every row; O(n) performance
- Not a substitute for proper map/reduce design!
Filtering (WHERE)
20
CBLQuery* query = [view createQuery];query.endKey = @1950;query.postFilter = [NSPredicate predicateWithFormat:
@"value.rating >= 90 or(value.inColor and value.rating >= 70)"];
• Allows post-processing of order of rows
Sorting (ORDER BY)
21
query.sortDescriptors = @[[NSSortDescriptor sortDescriptorWithKey: @"value.rating"
ascending: NO],[NSSortDescriptor sortDescriptorWithKey: @"value.title"
ascending: NO],];
Key-paths relative
to CBLQueryRow
• These enhancements will be in version 1.0.3of Couchbase Lite for iOS
- Currently on branch release/1.0.3
- Release date Oct 17
Surprise!
22
• Automatically generates map function
• Automatically configures CBLQuery parameters
• Variables can be substituted at query time
Future Query Planner
23
CBLQueryPlanner Example
24
CBLQueryPlanner* planner = [[CBLQueryPlanner alloc]initWithView: view
select: @[@"title", @"body", @"author", @"date"]where: @"type == 'post' and title beginswith[c] $PREFIX
and tags contains $TAG"orderBy: @[[NSSortDescriptor
sortDescriptorWithKey: @"date" ascending: YES]]error: &error];
CBLQueryEnumerator* e = [query run: &error];for (CBLQueryRow* row in e) {
...}
CBLQuery* query = [planner createQueryWithContext: @{@"PREFIX": @"Announcing", @"TAG":@"mobile"}];
• Encrypts the entire local database file
- Using SQLCipher
- AES-256 — current best practice for file encryption
Database Encryption
32
• Does not provide network encryption
- SSL already takes care of that
• Does not provide end-to-end document encryption
Database Encryption
33
• If you are required to
- Healthcare (HIPAA), banking, etc.
• If you don’t trust the device’s security
- User may not have set a passcode
- Device may have an old, less-secure OS version
- User may be targeted by the NSA, SVR, Illuminati…
Why to use database encryption
34
• Poor user experience
- Users dislike typing long passphrases
- Lost passphrase may mean lost data, support calls
• (Slight) drop in database performance
- Around 5-15%
• (Slight) growth in app binary size
- Due to having to embed custom SQLite library, ~1MB
• Device may actually be secure enough already
- iOS Security White Paper
- Android Security Overview
Why not to use database encryption
35
• Create a key
- Ask the user for a passphrase
- Or generate 32 securely random bytes
- SecRandomCopyBytes
- java.security.SecureRandom
• Register the key with the Manager
Using Database Encryption
36
• Passphrase:
- User must remember it (or write it on a sticky note)
- and type it in at launch time
• Raw key:
- Store it “somewhere secure”
- i.e. more secure than the file itself
- A regular iOS Keychain item is not more secure
- …but a TouchID-protected Keychain item is
Storing Keys
37
Stuff I Ran Out Of Time To Talk About
Subsetting a server database
Remote (server-side) queries
Improved model objects
Check it out:
forestdb
feature/query
feature/encryptio
n
github.com/couchbase/couchbase-lite-ios/tree/{