Post on 20-Aug-2015
What’s New in Couchbase Server 3.0 Beta
Don PintoProduct Management
Couchbase Inc.
Journey so far…
Couchbase Server v1.8High Scale Key/Value Database
Couchbase Server v2.0Cross Data Center Replication
Map/Reduce Indexing and Flex Data
Couchbase Server v2.1 Improved XDCR and Storage Layer
Couchbase Server v2.2Improved Security & HA/DR
Couchbase Server v2.5Rack/Zone Awareness and Connection Management
Couchbase Mobile 1.0Offline Data Availability & Auto Synchronization
Couchbase Server v3.0Mission Critical Scale & Perf
Simplified AdministrationImproved HA/DR & Security
N1QL Dev PreviewSQL-like Queries
N1QL BetaSQL=like Queries
2010
Couchbase Server v.NextWith N1QLSQL=like Queries
Couchbase Mobile
4
Couchbase LiteOn-device,
lightweight, native embedded
JSON database
Sync GatewaySynchronize on-
device Couchbase Lite with Couchbase Server in the cloud
Couchbase ServerHigh performance, scalable, always-on
JSON database in the cloud.
Deep Dive intoCouchbase Server v3.0
Disclaimer: Couchbase Server v3.0 is still in development. The generally available version of Couchbase Server at release of v3.0 may have differences with what’s presented here.
6
DEVELOPER EMPOWERMEN
T
MISSION CRITICAL
SCALE
ULTRA HIGH AVAILABILITY
SECURE COMMUNICATI
ON
SIMPLIFIED ADMINISTRATI
ON
• Fully Integrated JSON Programmability
• Native Async and Reactive Programmability
• New Integration with Popular Frameworks
• N1QL Preview Support in SDKs
• Optimized Massive Databases with Tunable Memory
• Auto-Tuning Thread Pool for Extreme IO Throughput
• Faster View Consistency
• Low Latency, In-Memory XDCR
• New Incremental Backup option
• Optimized Disaster Recovery with Restore options
• Encryption for Data Access
• Encryption for Administrative Access
• XDCR Pause and Resume
• Faster, Resilient Rebalance
• Delta Node Recovery
• Improved Resource Governance
What’s new in Couchbase Server v3.0
Developer Empowerment
What’s New in SDK 2.0
Refactored into separate library to provide low-level operations
• Java Core Library – Reusable Foundation Completely Async & Reactive - Based on Netty, RxJava, & LMAX Disruptor
• Java client library – Built on Java Core Library Integrated native JSON data type programmability Idiomatic Java 8 API support N1QL Preview Support
What’s New in SDK 2.0
• Document-oriented programming model
• Improved connection management
• Queryability: Enhanced View Access and N1QL Preview Support
• Native .NET: Async & Await Support with Couchbase Operations Web.config and App.config support
What’s New in SDK 2.0
• Node.js, PHP Document-oriented programming model Core IO enhancements through updated C libcouchbase Better connection management
• Python, Ruby Core IO enhancements through updated C libcouchbase Better connection management
• Framework Integrations Spring, gevent, Twisted, Ottoman
gevent for python
OttomanCouchbase ODM framework
Mission Critical Scale
Optimization for Large Data SizeMetadata Ejection
• Optimized Memory Usage with Metadata Ejection Policy Reduce memory required for Massive Data Footprints Enable efficient management of rarely accessed data set
• Cache only keys and data for the working set
• Eject all other keys and values out of memory
Tunable Memory – Metadata In RAM
13Note: 50M docs with value size ~0.5KB
Couchbase Server 2.5.1 - 3GBCouchbase Server 3.0 - 0.08GBMetadata ejection
Say Goodbye To Metadata Warning
No more metadata warning with metadata ejection in 3.0
Optimized Throughput with Auto-Tuning Thread Pool
• Global Thread Pool Shared thread pool across buckets for request processing and IO
operations
• Auto-tuning thread count based on system resources
• Improved latency across the board: read/write operations, rebalance, node restart with warm-up and more…
Couchbase Server: Data Manager
storage interface
11210Memcapable 2.0
Moxi
11211Memcapable 1.0
Memcached – cached keys & documents
8092Query API
Que
ry E
ngin
e
Data Manager
Read-Write Requests
R W R R W
Global Thread Pool
Disk
33 2
Recap of write operation in Couchbase 2.5 2
Managed Cache
Dis
k Q
ueue
Disk
Replication Queue
App Server
Couchbase Server Node
Doc 1Doc 1
Doc 1
To other node
XDCRDoc 1
Stream-based Views
• In memory change detection Change detection based on the new Database Change Protocol (DCP).
No longer require persisting to disk for change detection
• More up-to-date views Mutation show up at the indexes faster. Improved stale=false behavior
• Parts of the View engine is re-written in C
33 2
Stream-based Views2
Managed Cache
Dis
k Q
ueue
Disk
Replication Queue
App Server
Couchbase Server Node
Doc 1
Doc 1
To other nodeDoc 1Doc 1
Ultra High Availability
Stream- based Cross Data Center Replication (XDCR)
• In memory change detection with DCP Purely in memory change detection based on the new DCP (database
change protocol) protocol. No longer require persisting to disk for change detection
• Lower latency == Smaller Data Loss Exposure Improved protection against regional and data center failures
33 2
Stream-based XDCR2
Managed Cache
Dis
k Q
ueue
Disk
Replication Queue
App Server
Couchbase Server Node
Doc 1
Doc 1
To other nodeXDCRDoc 1
Doc 1
Efficient Recovery with Incremental Backup & Restore
• Enhanced Disaster Recovery Minimize resource and time consumption during backups. Enables more frequent backups
• New Incremental Backups Back up data updated since the last backup. New CBBackup options
• Differential
• Cumulative
Restore can mix-and-match differential or cumulative backups
Secure Communications
Encrypted Access
SERVER 3SERVER 1 SERVER 2
Couchbase Server
• Encrypted client-server communications Clients can optionally communicate with Couchbase Server on secured
connection using SSL encryption Some buckets choose to use SSL encryption and for other non-SSL
Encrypted Access
• HTTPS support for Administration Console (18901) HTTPS support for REST API and Couchbase Server Web Console
https://couchbase_server:18091/index.html
• HTTPS for Views (18092)
https://couchbase_server:18092/…
Simplified Administration
Maintenance with XDCRPause and Resume for Efficient Data Transfer
• XDCR Pause and Resume Want to suspend replication? No longer need to drop replications
• Ability to pause XDCR traffic anytime
• Resume and incrementally catch-up
Improved RebalanceDelta Node Recovery
• Delta Node Recovery Do not delete existing vbucket files on the node Catch up from where you left off
Improved Resource Governance with Bucket Priority
• Priority Setting on Buckets Ability to assign high and low priority to buckets High priority buckets get higher IO priority and more throughput Low priority buckets can still be processed without starvation
Hundreds of other improvements…
Cluster-wide Diagnostics Tool
Improved Crash Reporting
Better Serviceability
Improved Monitoring for Warmup
Stats Enhancements
Improved Checkpointing with XDCR
Expended Options with Couchbase-cli Parallelized Warmups
Database-Change-Protocol (DCP)
Increased-Connection-LimitsExtended-Documentation
Enhanced Event Logging
Extended XDCR Resiliency
Better-failover-resiliency-with-DCP
Enhanced SSD Performance
Streamlined Build
Side by side support for DCP and TAP
Improved Error Reporting
Built-in OS Tuning for Linux Flavors
… and More
Improved Resume-ability with Intra Cluster Replication
View Performance Improvements
Client-side JSON validation
Web.Config & App.Config support with .NET
Unified New App Model with 2.0 SDK
Java SDK 2.0 built on top of RxJava
N1QL Beta Support in the 2.0 SDKs
Improved Warmup time under Metadata Ejection
Access log for monitoring port 8091
CRAM & MD5 Support in .Net SDK
Client Side Log4Net Integration in .Net SDK
New Cluster object for Cluster Operations in SDK 2.0
Replica Read in SDK 2.0 with .Net and PHP
Graceful Failover
PART III - Beyond 3.0
Disclaimer: Couchbase Server Roadmap evolves based on feedback and market conditions.
N1QL – Query Language for Rich Data
Rich Data Represented in JSON Complex Real World
JSON Flexibility and Transitive Closure Expressiveness of SQL N1NF – Non First Normal Form
http://query.couchbase.com
PersonSUSAN KATZ
Product
N1QL
N1QL
IPAD 64GCARRIER : ATT
SELECT count(*), stateFROM customerWHERE customer.ccInfo.cardType = “visa”GROUP BY customer.stateORDER BY customer.stateLIMIT 2 OFFSET 3
{ "count": 2, "state": ”AR" }, { "count": 7, "state": ”CA" }
To search for purchase orders with a particular Item purchased 5 times or more -
SELECT *FROM purchasesWHERE ANY item IN purchases.lineItemsSATISFIES item.count >= 5 END
{ "customerId": "customer776", "lineItems": [ { "count": 5, "product": "product843" },…
N1QL — Basic SELECTs to Rich Queries
Try the queries at - http://query.couchbase.com
BASIC SELECT STATEMENT
COLLECTION EXPRESSIONS
Participate in 3.0 Beta
Couchbase Server 3.0 Beta Bug Bash Download, Find, Report, Win
• DOWNLOAD Couchbase Server 3.0 Beta
• Run some workload, play with it, FIND a bug
• If you find a bug, REPORT it! All bugs count – anything from correctness, to performance, to
usability, to docs
• WIN Prizes!!
http://www.couchbase.com/bugbash Contest ends Aug 31st 2014
Feedback and Q&A