Oracle Coherence An introduction to in memory data grid

Yossi Yadgar

NoSQL Seminar – MTA 2012

Data Grid – What is it good for?

• Extreme increase in Access, Volume and Complexity of Data Use Problem

• Difficulty meeting SLA

• Managing infrastructure growth

• costs Challenge

• Reliable, Scalable Universal Data Access & Management Solution

Data Grid Requirement

• Data as a service

• Java, .net, C++

• Horizontal Scale

• Commodity Hardware

• Generic network infra.

• Data Integration is in the Data Service

• Not in the Database!

• Please, lie to me! – but, compute localy.

Oracle Coherence

• TMCP – Tangosol Clusterd Messaging Protocol

• Completely asynchronous yet ordered messaging buil on UDP multicast/unicast.

• Truly peer-to-peer: equal responsibility for both producing and consuming the services of the cluster.

• Self Healing – Quorum based diagnostics

• Linearly scalable mesh architecture

• TCP-like features

• Messaging throughput scales to the network infrastructure

Coherence Cluster

• Transparent , dynamic and automatic cluster membership management

• Clustered Consensus: All members in the cluster understand the topology of the entire grid at all times.

• One Cluster (like “singleton”)

• Under the cluster there any number of uniquely named services (such as caching service)

• Each caching service has any number of named caches

Data Management - topologies

• Partitioned Caching vs . Replicated topology

• Extreme Scalability

• Automatically, dynamically and transparently partitions the data set across the members of the grid

• Pros:

• Linear scalability of data capacity

• Processing power scales with data capacity

• Fixed Cost per data access (one hop to the data)

• Cons:

• Cost Per Access: High percentage chance that each data access will go across the wire

• Primary Use:

• Large in memory storage environment

• Parallel processing environment

Partitioned cache – get operations

Partitioned cache – put operations

Coherence Data Management: Partitioned Fault Tolerance • Automatically, dynamically and transparently manages the

fault tolerance of your data

• Backups are guaranteed to be on a separate physical machine as the primary

• Backup responsibilities for one node’s data is shared amongst the other nodes in the grid.

• And the best part – it’s not up to the programmer!

Partitioned cache – fault tolerance

Data Management Cache Client and Servers • Partitioning can be controlled on a member by member basis

• A member is either responsible for an equal partition of the data or not (“storage enabled” vs. “storage disabled”)

• Cache Client – typically the application instances

• Cache Servers – typically stand-alone JVMs responsible for storage and data processing

Data Management: Near Caching • Better scalability and performance – combination of replicated

and partitioned topologies

• Pros:

• The same as partitioned

• Faster access

• Cons

• Updates costs more

• Primary Use:

• Big imdb with continuous data access.

Near Caching (client-server)

Data Management - Data affinity • Associate objects across caches

• Child and Parents guaranteed to be on the same node

Caching RDBMS

• Why should I?

• Read Through and Write Through

• Access to the data source go through Data Grid

• Read and write operations are always managed by the node that owns the data within the data grid

• Concurrent accesses are combined, greatly reducing database load

• Write-Through keeps the in-memory data and the database in sync

• Write-Behind

• I’ll do it later approach

• Modification accepted from all grid and async written back to the data source – optionally after a specific delay

• All write-behind data is synchronously and redundantly managed (Fault tolerant)

Read Through Cache

Write Through Cache

Write Behind Cache

Integration With JPA

• Example

Integrating with JPA

• Using TopLink-Grid (Oracle Product) allows you to use your standard Object Access (i.e JPA) backed with Coherence Cache – just change the configuration of the coherence cache config:

Cache-config.xml: <cache-config> <caching-scheme-mapping> <cache-mapping> <cache-name>*</cache-name> <scheme-name>eclipselink-distributed-readwrite</scheme-name> </cache-mapping> </caching-scheme-mapping>

<caching-schemes> <distributed-scheme> <scheme-name>eclipselink-distributed-readwrite</scheme-name> <service-name>EclipseLinkJPAReadWrite</service-name>-

Integrating with JPA

• You can also work with read-write cache directly: Cache-config.xml: <cache-config> <caching-scheme-mapping> <cache-mapping> <!-- Set the name of the cache to be the entity name --> <cache-name>*</cache-name> <!-- Configure this cache to use the scheme defined below --> <scheme-name>jpa-distributed</scheme-name> </cache-mapping> <cache-mapping> <cache-name>Country</cache-name> <scheme-name>jpa-distributed</scheme-name> </cache-mapping> </caching-scheme-mapping> <caching-schemes> <distributed-scheme> <scheme-name>jpa-distributed</scheme-name> <service-name>JpaDistributedCache</service-name> <backing-map-scheme> <read-write-backing-map-scheme>

Events

• JavaBean event model

• Listen to all events for all keys

• ENTRY_DELETED

• ENTRY_INSERTED

• ENTRY_UPDATED

• Example:

NamedCache cache = CacheFactory.getCache(“Country”);

cach.addMapListener(listener);

Events

• Key Based Event Model

• Listen to changes to a specific key

• Example:

NamedCache cache = CacheFactory.getCache(“Country”);

cach.addMapListener(listener, key);

• Filter Based Event Model:

• Listen to changes to a data that match a specific criteria (Filter)

• Example:

NamedCache cache = CacheFactory.getCache(“Country”);

cach.addMapListener(listener, filter);

Queries

• Programmatic query mechanism

• Queries performed in parallel across the grid

• Standard indexes provided

• Standard Filter provided out of the box

(AND, ALL, EQUALS, etc.)

• Continuous Query - The set is updated automatically!

Query and Filters

// get the country cache

NamedCache countries = CacheFactory.getCache(“country”);

// create the filter query – id==countryID and getArea==area

Filter filter =

new AndFilter(new EqualsFilter(“getId”, countryID),

new EqualsFilter(“getArea”, area);

// perform the query

Set setCountry = countries.entryset(filter);

// And now – continues query:

NamedCache FilteredCounties = ContinousQyeryCache(countries, filter);

General - Key Opponents

• VMware Gemfire (Java)

• Oracle Coherence (Java)

• Alachisoft NCache (.Net)

• Gigaspaces XAP Elastic Caching Edition (Java)

• Hazelcast (Java)

• Scaleout StateServer (.Net)

• Oracle TimesTen?

Questions?