Oracle NoSQL Database Technical Introduction Presentation


Oracle NoSQL DatabaseTechnical Introduction

Apostolos GiannakidisSoftware Quality Engineer

Dublin NoSQL Meetup

TCube – December 16, 2013


Safe Harbor

The following is intended to provide an introduction to the Oracle NoSQL database. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality and should not be relied upon in functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Disclaimer

● Any views expressed in this presentation are my own and do not necessarily reflect the views of Oracle.

● This presentation will not discuss product's road map as I do not represent the Oracle NoSQL database development team.

● Some slides and presentation material have been reused from publicly available presentations of Dave Segleau, Charles Lamb and Abhi Dubey. Most of the reused slides have been edited by me in order to update them or customize them according to this presentation's focus.


Agenda

§ Oracle NoSQL Database Introduction§ Oracle NoSQL Database Architecture§ Oracle NoSQL Database Operations & API§ Q&A


What is Oracle NoSQL Database (ONDB)Enterprise-class Distributed Key-Value store

● High availability● Multi-TBs to PBs capacity● Simple data model● Transparent sharding● Transparent load balancing

● Elastic scalability● Predictable performance● ACID transactions● No-Single Point of Failure● Simple administration


Oracle NoSQL DB Building BlocksOracle Berkeley DB Java Edition

§ Robust and reliable storage engine for a distributed key/value Database● ACID transactions● High availability● High throughput● Simple administration

§ Already proven in● Amazon Dynamo● Voldemort (Linkedin)● GenieDB


Oracle NoSQL DatabaseBuilding upon Berkeley DB Java Edition

§ Dynamic Partitioning (aka Sharding)

§ Intelligent routing of requests

§ Load balancing

§ Predictable latency

§ Multi-Node backup

§ Monitoring

§ Major/Minor key paradigm (with String keys)

§ RMW capabilities


§ Supported OS - Linux and Solaris 10

§ Pure implementation in Java

§ Java and C APIs

§ Bindings available for Jython, Jruby,

Clojure, Groovy, Rhino

Compatibility


Enterprise Integration – Extensions Integration with Oracle Database - External Tables

RDF Graph for NoSQL - Support for SPARQL

Integration with Middleware● Oracle Coherence● Oracle Event Processing

Integration with:● Apache Hadoop● Apache Avro


Oracle NoSQL DB versions

Oracle NoSQL DB Community Edition (AGPL v3)● Open Source

Oracle NoSQL Enterprise Edition

● Extra features for enterprise use (External Tables, RDF/SPARQL, OEP, Coherence, JMX, SNMP)

● Enterprise Support

Oracle NoSQL DB Client Driver under the Apache 2.0 License - New


Agenda



Map of Keys to Values Major / Minor Key paradigm Keys are Java Strings

Data Model

Oracle NoSQL Database

Values are stored as arbitrary byte arrays (opaque values) Values can be stored in Avro format (recommended) Values could be as simple or as complex the developer wants

● Serialization and Deserialization is left up to the application

No restriction on size of the Value● Large Object support

Full Key = Major Key Path + Minor Key Path One or more Major Key components Zero or more Minor Key components

Value field


Data Model


§ Full Key representation format:

/MajorComponent1/MajorComponent2/-/MinorComponent1/MinorComponent2/


Storage Nodes are physical (or virtual) machines with their own local storage

Every Storage Node hosts one or more Replication Nodes● By default, Replication Nodes are 1:1 with Storage Nodes

Replication Nodes are organized into Replication Groups (aka shards)● One of the Replication Nodes is elected to be the shard's Master node

Number of nodes belonging to a shard is called its Replication Factor

KVStore is a collection of Storage Nodes

Storage Terminology



Data is spread evenly across all the shards in use by the KVStore

All Key Value pairs with the same Major Key Path are clustered on the same shard

Set of Replication Nodes for each shard provides:

● High Availability

● Read Scalability

Write operations on the Μaster Replication Node

Read operations on any Replication Node

Storage Fundamentals



Shard 1

Replicas

Master

Shard 2

Replicas

Master

Shard N

Replicas

Master

NoSQL DB Driver

Application

Writes

Reads

Elastic Shards (split, add, remove)

Replication Nodes

Oracle NoSQL Database – Logical Architecture


● Single-master multi-replica replication strategy● One or more read-only replicas● Uses TCP/IP and standard Java libraries● Resilient to partition master failures● Supports Heterogeneous platform hardware/OS/JVM ● Provides linear scale out for read ops/sec

High availability High read throughput

Features - Replication


Features – Transparent Load Balancing Reads from master or replicas

Writes are evenly distributed across shards

Intelligent Client Driver is network topology and latency aware● Routes operations to the proper node● Performs load balancing● Run operations on fastest appropriate node● Single network hop except when node fails or topology changes● Topology changes returned with results


§ There is an unbounded delay in propagating write ops made on the Master Node to all the other Replication Nodes of the shard

§ Master node → most up-to-date data§ Replicas → possibly slightly older versions (eventual consistency)§ System records version numbers for key/value pairs, but

maintains the single latest version in the store.§ No need for version reconciliation at the application level

Consistency Fundamentals

Eventual Consistency and Data Versioning


Configurable Durability per op

● Several durability policies

Configurable Consistency per op

● Several consistency policies

Sequence of operations as a single atomic unit

● Records must share same Major key

ACID by default

Transaction scope is single API call

Greater Flexibility

Features – Configurable ACID Transactions


Simplified version of Oracle NoSQL Database server Single-node single-shard store (no replication) Intended for use by application developers

● Explore ONDB API

● Unit Test applications

Not for production deployments or benchmarking

Quick deployment for developers

KVLite


Agenda



Data is accessed using Major and (optionally) Minor keys

● This enables “multi” operations

CRUD operations

● Read / Insert / Update / Delete

● Single and multi-record operations

● ACID transactions

● Compare And Set Operations (RMW)

Basic operations

Oracle NoSQL Database – Operations & API


§ Iteration

● Unordered iteration across keys

● Ordered iteration across sub keys within a key

● Range scans and full key space scans

Basic operations (cont'd)



● put(Key, Value, [Durability, timeout])

● putIfAbsent(K, V, [Durability, timeout])

● putIfPresent(K, V, [Durability, timeout])

● putIfVersion(K, V, Version, [Durability, timeout])

● get(Key, [Consistency, timeout])

● delete(Key, [Durability, timeout])

● deleteIfVersion(Key, Version, [Durability, timeout])

CRUD API



● multiGetIterator(Direction, int batchSize, Key parentKey, KeyRange, Depth)

→ Iterator<KeyValueVersion>

● storeIterator(Direction direction, int batchSize [Key parentKey, KeyRange, Depth])

→ Iterator<KeyValueVersion>

Iteration Operations API



public interface OperationFactory

● createPut, createPutIfPresent, createDelete, etc

→ Operation

execute(List<Operation> operations)

→ List<OperationResult>

Bulk Operations API



● multiGet(Key parentKey, KeyRange, Depth, [Consistency, timeout])

→ SortedMap<Key,ValueVersion>

● multiDelete(Key parentKey, KeyRange subRange, Depth, [Durability, timeout])

→ int

KeyRange “Multi” Operations API



● createKey(List<String> majorPath)

● createKey(List<String> majorPath, List<String> minorPath)

● createKey(List<String> majorPath, String minorComponent)

● createKey(String majorComponent)

● fromString(String pathString)

→ Key

Key management API



● Getting Started with Oracle NoSQL Database

● Oracle NoSQL Database Availability and Failover

● Administrator's Guide

● Oracle Technology Network

● Product Overviews

● White Papers / Presentations

● Online Tutorials / Videos

● Online Webinars

Where do I go for more information?



Q&A


Slides available at www.giannakidis.info

http://www.giannakidis.info/

Oracle NoSQL Database Technical Introduction Presentation

Documents

Transcript of Oracle NoSQL Database Technical Introduction Presentation