OpenSplice Cache
33
Angelo Corsaro, Ph.D. Product Strategy Manager OMG RTESS and DDS SIG Co-Chair [email protected] Delivering Performance, Openness, and Freedom Open Splice DDS The Design and Implementation of a Data Caching Extension for the OMG DDS
-
Upload
angelo-corsaro -
Category
Technology
-
view
1.435 -
download
1
description
This presentation introduces OpenSplice Cache, a thin layer over OpenSplice DDS that adds traditional caching functionalities.
Transcript of OpenSplice Cache
Angelo Corsaro, Ph.D.Product Strategy Manager
OMG RTESS and DDS SIG [email protected]
Del iver ing Performance, Openness, and Freedom
OpenSplice DDS
The Design and Implementation of a Data Caching Extension for the OMG DDS
Age
nda
Why Caching?
OpenSplice Cache Explained
Standardization
Concluding Remarks
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
ChallengesThree-Tier Systems
‣ The number of clients growing exponentially due to increased adoption of e-*
‣ Amount of data that needs to be accessed growing due increased digitalization
‣ Computing can be scaled-out adding cheap HW at the Logic-Tier, but data access does not scale-out!
Relevance
‣ SaaS,
‣ Cloud Computing
‣ XTP
‣ WWW
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
ChallengesSense and Respond Systems
‣ Relevant state is either stored in a DBMS, or replicated wherever needed
‣ The DBMS approach leads to inevitable bottlenecks
‣ The fully replicated data, if not dealt properly, might induce excessive resource usage, or worse waste of resources
Relevance
‣ C2, C4I Systems
‣ ATC, ATM
‣ Vetronics
‣ DCS
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
ChallengesHigh Performance Computing
‣ Huge amount of data (exceeding the capacity of a single machine) needed for carrying on a specific computation
‣ Certain Data-points could be re-computed to save memory, but this takes time, e.g., re-computing a financial instrument
‣ Accessing data all at once, or from a centralized data server does not scale, or is too expensive
Relevance
‣ Financial Services
‣ Simulations
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
The Key Idea & Why it WorksIdea
‣ Cache data close to where it is needed and keep it there for as long as it is needed, or as long as you can...
Why it works
‣ Locality of References. Programs tends to reuse data they have used recently
‣ Temporal Locality. Recently accessed data items are likely to be accessed again in the near future
‣ Spatial Locality. Items whose addresses are close to each other tend to be referenced close together in time
L1
L2
Main Memory
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
CacheWebster Definition (1976)
‣ a safe place for hiding or storing things.
Webster Definition (2008)
‣ (a): a hiding place especially for concealing and preserving provisions or implements (b): a secure place of storage.
‣ something hidden or stored in a cache.
‣ a computer memory with very short access time used for storage of frequently or recently used instructions or data —called also cache memory.
Etymology (1797)
‣ French, from cacher to press, hide, from Vulgar Latin coacticare to press together, from Latin coactare.
Age
nda
Why Caching?
OpenSplice Cache Explained
Standardization
Concluding Remarks
Disk
Key Value
Key Value
Key Value
Key Value
OpenSplice Cache
OpenSplice DDS
Disk
Disk
Disk
Disk
OpenSplice Cache
OpenSplice Cache
OpenSplice CacheOpenSplice Cache
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
OpenSplice Cache in Brief‣ Fully Distributed Caching
Technology
‣ Supports SQL Queries on local Cache
‣ Support for Continuous SQL Queries on the whole distributed Cache
‣ High-Performance, ultra-low-latency communication engine based on OpenSplice DDS
‣ Fully exploits Data Locality
Caching API
Ownership PersistenceContent
Subscription
Minimum Profile
Real-Time Pub/Sub(DCPS)
OpenSplice Cache
Networking Technology
Application
UDP/IP
Interoperable Wire Protocol (DDSI)
DD
SI v
2.1
DD
S v
1.2
Real-Time Networking
SOAPSecure Networking
DBMS
Connectors
PowerTools
MDE
Tuner
Connectors
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
OpenSplice Cache Architecture‣ Fully exploits the
power of OpenSplice DDS Real-Time Pub/Sub for maintaining cache coherence
‣ Exposes a simple API for storing and retrieving items to/from the cache
‣ Can be used by applications in conjunction with OpenSplice DDS
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
OpenSplice Cache Core API‣ Simple and concise core API that keeps simple most commonly used operations
‣ Extended API providing‣ Rendezvous
‣ Advanced Cache Configuration
‣ Cached Data QoS
‣ Cache Replacement Strategy Customization
Disk
OpenSplice Cache
Key Value
Key Value
Key Value
Key Value
+ add_miss_handler(handler)
+ add_insert_handler(handler)Not
ifica
tions
+ query(sql): view
+ add_continuous_query(sql, listener)
Queries
+ read(key): value+ take(key): value
Read
+ write(key, value)
Write
© 2008, PrismTech. All Rights Reserved
A Performance Driven Architecture
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Performances Driven Architecture
‣ Cache performances are summarized by the following equation:
Average Data Access = HitTime + MissRate x MissPenalty
‣ Thus to optimize the Cache performances it is necessary to‣ Reduce Hit Time‣ Reduce Miss Rate‣ Reduce Miss Penalty
OpenSplice Cache has been designed to ensure that all these factors are optimized
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Hit Time Minimization‣ The hit-time is the time taken to access data when it is in the cache.
‣ OpenSplice Cache has two different hit time, one local and one remote, depending on wether the data is locally available, or available somewhere on the network
Disk
OpenSplice Cache
Key Value
Key Value
Key Value
Key Value
+ read(key): value+ take(key): value
Read
Local Hit-Time‣ Accessing an object locally is a
matter of micro-secondsRemote Hit-Time‣ OpenSplice Cache, relies on
OpenSplice DDS to resolve remotely a hit.
‣ As such, this hit time, is close to a message roundtrip on OpenSplice DDS, which depending on the size is measured in tens of microseconds
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Local and Remote Hits
Disk
OpenSplice Cache
OpenSplice DDS
Disk
OpenSplice Cache
Disk
OpenSplice Cache
Disk
OpenSplice CacheDisk
OpenSplice Cache
‘a’ ‘alpha’
‘b’ ‘beta’
read(‘a’): ‘alpha’Local Hit
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Local and Remote Hits
Disk
OpenSplice Cache
OpenSplice DDS
Disk
OpenSplice Cache
Disk
OpenSplice Cache
Disk
OpenSplice CacheDisk
OpenSplice Cache
‘a’ ‘alpha’
‘b’ ‘beta’
read(‘a’): ‘alpha’
read(‘b’): ‘beta’
Local Hit
Remote Hit
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Local and Remote Hits
Disk
OpenSplice Cache
OpenSplice DDS
Disk
OpenSplice Cache
Disk
OpenSplice Cache
Disk
OpenSplice CacheDisk
OpenSplice Cache
‘a’ ‘alpha’
‘b’ ‘beta’
read(‘a’): ‘alpha’
read(‘b’): ‘beta’
Local Hit
Remote Hit
read(‘c’): miss handlerMiss
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Miss Rate MinimizationCache misses can be classified in:
‣ Compulsory. The first access to a data item not in the cache. These are also called Cold Start Misses or First Reference Misses.
‣ Capacity. If the cache cannot contain all the data needed by an application during its execution, capacity miss will occur because of blocks being discarded and later retrieved.
‣ Conflict. Depending on how the cache is organized internally, some data items might compete for resources, and lead to replacement
OpenSplice Cache minimize architecturally all of these misses.
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Pre-Fetching & ‣ Compulsory Misses can be reduced
by trying to guess what an application will use next
‣ Temporal and Spatial correlation among the components of a distributed system can help in predicting the use
‣ OpenSplice Cache, exploits unused space in a local cache instance to eagerly pre-load data items -- thus reducing compulsory misses
‣ An API is also provided to the application to influence and customize the pre-fetching
Disk
OpenSplice Cache
Key Value
Key Value
Key Value
Key Value
Working Set
Prefetched Data
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Pre-Fetching
Disk
OpenSplice Cache
OpenSplice DDS
Disk
OpenSplice Cache
Disk
OpenSplice Cache
Disk
OpenSplice CacheDisk
OpenSplice Cache
‘a’ ‘alpha’
‘b’ ‘beta’
write(‘s’): ‘sigma’
‘s’ ‘sigma’
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Pre-Fetching
Disk
OpenSplice Cache
OpenSplice DDS
Disk
OpenSplice Cache
Disk
OpenSplice Cache
Disk
OpenSplice CacheDisk
OpenSplice Cache
‘a’ ‘alpha’
‘b’ ‘beta’
write(‘s’): ‘sigma’‘s’ ‘sigma’
‘s’ ‘sigma’
‘s’ ‘sigma’
‘s’ ‘sigma’
‘s’ ‘sigma’
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Pre-Fetching
Disk
OpenSplice Cache
OpenSplice DDS
Disk
OpenSplice Cache
Disk
OpenSplice Cache
Disk
OpenSplice CacheDisk
OpenSplice Cache
‘a’ ‘alpha’
‘b’ ‘beta’
write(‘s’): ‘sigma’‘s’ ‘sigma’
‘s’ ‘sigma’
‘s’ ‘sigma’
‘s’ ‘sigma’
‘s’ ‘sigma’
read(‘s’): ‘sigma’Local Hit
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Remote Cache Hit & Capacity Miss
‣ We’ve seen that OpenSplice Cache supports the concept of local and remote hits
‣ When a resource constrained cannot allocate the whole memory needed for its working set locally, it can still exploit caching done in more powerful nodes to improve its over all performance
‣ Thus, OpenSplice Cache through exploiting resources available, somewhere else, it is practically possible to limit, if not eliminate the capacity miss
Disk
OpenSplice Cache
Key Value
Key Value
Key Value
Key Value
+ read(key): value+ take(key): value
Read
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Cache Organization & Conflict Misses
‣ By default OpenSplice Cache behaves as a fully associative Cache managed by an LRU replacement policy
‣ The default choice is motivated by the optimality of LRU as a replacement strategy
‣ An API is provided to customize the replacement strategy so to fit a specific application need/behavior
‣ This allows for OpenSplice Cache to always minimize the conflict misses, and ensure minimal cache miss rate -- thus best performance
Disk
OpenSplice Cache
Key Value
Key Value
Key Value
Key Value
Disk
Key Value
Key Value
Key Value
Key Value
OpenSplice Cache
OpenSplice DDS
Disk
OpenSplice Cache
Key Value
Key Value
Key Value
Key Value
Disk
OpenSplice Cache
Key Value
Key Value
Key Value
Key Value
Disk
OpenSplice Cache
Key Value
Key Value
Key Value
Key Value
Disk
OpenSplice Cache
Key Value
Key Value
Key Value
Key Value
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Putting it all Together
Working Set
Prefetched Data
LRU
LRU
Random
Custom
Assoc.
© 2008, PrismTech. All Rights Reserved
Architectural Styles
Disk
Key Value
Key Value
Key Value
Key Value
OpenSplice Cache
OpenSplice DDS
Disk
OpenSplice Cache
Key Value
Key Value
Key Value
Key Value
Disk
OpenSplice Cache
Key Value
Key Value
Key Value
Key Value
Disk
OpenSplice Cache
Key Value
Key Value
Key Value
Key Value
Disk
OpenSplice Cache
Key Value
Key Value
Key Value
Key Value
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Symmetric‣ All nodes have similar
resources
‣ Caches instances on all nodes have similar sizes
‣ The resulting system is really peer-to-peer, and every node, can potentially serve remote hits
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Asymmetric‣ Some nodes have very
limited resources
‣ As a consequence Cache instances have different sized on various nodes
‣ The system, will rely on a set of “super-caches” to resolved remote-hits
‣ Super-Caches can exploit partitioning as well as load-balancing to ensure scalability
Disk
Key Value
Key Value
Key Value
Key Value
OpenSplice Cache
OpenSplice DDSDisk
OpenSplice Cache
Key Value
Key Value
Disk
OpenSplice Cache
Key Value
Key Value
Key Value
Key Value
OpenSplice Cache
Disk
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Disk
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
OpenSplice Cache
OpenSplice Cache
Disk
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Key Value
Disk
OpenSplice Cache
Key Value
Key Value
Disk
OpenSplice Cache
Key Value
Key Value
Key Value
Key Value
Load-Balanced/PartitionedSuper Caches
Age
nda
Why Caching?
OpenSplice Cache Explained
Standardization
Concluding Remarks
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Standardization
‣ No formal Caching Technology standard exist to date
‣ JCache is a JSR, but there not seem to be real interested toward converging over a standard
‣ Caching functionalities extend DDS in a very useful way
‣ Possible Standardization venue:
‣Standardize a Caching API that builds over DDS
‣Extend the DDS QoS to be able to have “Cache Semantics” for Subscriber/Reader or Topics
Age
nda
Why Caching?
OpenSplice Cache Explained
Standardization
Concluding Remarks
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Concluding Remarks‣ OpenSplice Cache is an innovative caching technology whose
architecture ensures maximum performance by construction
‣ It leverages OpenSplice DDS high performance messaging middleware for implementing its distributed consistency protocol, and deliver ultra-low hit-times
‣ Architecturally supports equally well symmetric system deployments as well as highly asymmetric systems, facilitating the creation of load-balanced/partitioned super caches
OpenSplice Cache provides best performance and architectural flexibility!
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
Online Resources
http://www.opensplice.com/
emailto:[email protected]
http://www.youtube.com/OpenSpliceTube
http://dds4u.blogspot.com http://bit.ly/1Sreg
http://www.dds-forum.org
http://portals.omg.org/dds
http://twitter.com/acorsaro/
