In memorybtree
30
2013 © Trivadis BÂLE BERNE BRUGG LAUSANNE ZURICH DÜSSELDORF FRANCFORT s. M. FRIBOURG e. BR. HAMBOURG MUNICH STUTTGART VIENNE 2014 © Trivadis In Memory B*Tree, an alternative to In Memory Database Cache Jacques Kostic, Senior Consultant 30-march-2014 In Memory B*Tree, an alternative to In Memory Database Cache
-
Upload
jacques-kostic -
Category
Technology
-
view
18 -
download
0
Transcript of In memorybtree
2013 © Trivadis
BÂLE BERNE BRUGG LAUSANNE ZURICH DÜSSELDORF FRANCFORT s. M. FRIBOURG e. BR. HAMBOURG MUNICH STUTTGART VIENNE
2014 © Trivadis
In Memory B*Tree, an alternative to
In Memory Database Cache
Jacques Kostic, Senior Consultant
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
PROGRAMME
1.Presentation
2.Typical use cases
3.Concept of In Memory Database Cache
4.Proposed alternative: In Memory B*Tree
5.One step beyond
6.Questions
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
Presentation
2013 © Trivadis
Presentation
When several sessions are retrieving the same referential rows,
unnecessary I/O’s are performed many times causing significant impact on
core business queries.
Those rows are usually called application metadata or referential
data.
Usually simples queries retrieving less than hundred rows and
executed thousands of time.
Usually those query are not appearing as most critical queries but
are impacting globally the I/O throughput
The objective of this presentation is to propose a workaround to
developer's to address that
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
Typical use cases
2013 © Trivadis
Typical use cases
List of values fields
Dynamic parent/child dependency list of values
Business rules reference tables
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
Typical use cases
Globally when it’s required to frequently access
reference data, a solution has to be put in place in
order to:
Speed up accesses
To
Minimize I/O impact on the system
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
Typical use cases
Solution:
Simply eliminate ALL related I/Os
How?
Doing that in memory!
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
Concept of In Memory Database
Cache
2013 © Trivadis
Concept of In Memory Database Cache
Global cache for application tier to cache physical Database
data
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
Concept of In Memory Database Cache
The In Memory Database Cache can reside :
in the database server
Or
in a dedicate separate server
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
Concept of In Memory Database Cache
The main goal stay and will always to be:
->Speedup all accesses
They are many other functionalities available in In Memory
Database Cache like:•In-memory database cache grid
•Automatic parallel data synchronization with Oracle Database
•Preloaded and on-demand SQL read/write caching
•Online addition/removal of cache nodes
•Transactional parallel replication with automated failover
It has a cost
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
Proposed alternative: In Memory B*Tree
2013 © Trivadis
In Memory B*Tree
An example of a B*Tree
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
In Memory B*Tree
What is In Memory B*Tree?
A library developed in C (by me)
Available under linux or Windows
Already used long long time ago (from 1997)
Still performing very well!
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
In Memory B*Tree
To store what?Keys (simple or composite)
Optionally attach additional fields
Looks like an IOT!
To be stored where?In memory
To be accessed how?=,<,>,<=,>=Likebetween
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
In Memory B*Tree
Why not an Hash Table?
Hash Table is very quick for equality search
Range scan is not supported!
Many libraries already exist
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
In Memory B*Tree
Main features
Create Purge, Delete a B*Tree, create forest!
Load B*Tree
Add Key (+ optionally data)
Replace Data
Delete a Key or a set of key
Find Key
Get (First, Last) Key
Get Result Set (partial key search or range scan)
Get Sorted Result set (apply a specific sort)
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
In Memory B*Tree
How to use it?Shared library in c, some functions:AllocBT
DeleteBT
ReinitBT
AddItemInBT
ReplaceItemInBT
FindItemInBT
FindItemByPartOfKeyInBT
GetNextKeyInBT
GetPreviousKeyInBT
GetSortedList
GetFirstKeyInBT
GetNextKeyInBTWithoutKey
Demo
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
In Memory B*Tree
Sequential Table Test
Loading to 1,000,000 keys, Press any key
Load 1,000,000 keys Done in 0.063 seconds
Searching keys from 1,000 to 2,000 Press any key
1,000 key found in 35.396 seconds
B*Tree Test
Loading to 10,000,000 keys, Press any key
Load 10,000,000 keys Done in 6.459 seconds
Searching keys from 1,000,000 to 2,000,000 Press any key
1,000000 key found in 0.327 seconds
Searching keys from 4,000,000 to 8,000,000 Press any key
4,000,000 key found in 1.389 seconds
Searching all key starting with 3 Press any key
1,111,111 key starting with 3 in 0.032 seconds
Deleting all key starting with 4 Press any key
1,111,111 key deleted with 4 in 0.795 seconds30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
Possible evolutions
2013 © Trivadis
On Step Beyond
How about this query?
SELECT SUM (s.amount_sold) sales_amount
FROM sh.sales s, sh.times t, sh.customers c, sh.channels ch
WHERE s.time_id = t.time_id AND
s.cust_id = c.cust_id AND
s.channel_id = ch.channel_id AND
c.cust_state_province = 'CA' AND
ch.channel_desc in ('Internet','Catalog') AND
t.calendar_quarter_desc IN ('1999-01','1999-02','2000-03');
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
On Step Beyond
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
Sales
Times Customers
Channels
Typical star query
2013 © Trivadis
On Step Beyond
I would love to do that…
Select GetSalesAmount
(
'CA',
'Internet,Catalog',
'1999-01,1999-02,2000-03'
)
from dual;
Without any I/O’s!
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
On Step Beyond
Another example with a lookup subquery:
Select count (*) From sh.customers
Where cust_id not exist
(select cust_id from sh.sales);
COULD BE:
Select count(*) from sh.customers
Where GetKeyExist ('SALES;'||Cust_Id) = 0;
Without accessing fact table!
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
On Step Beyond
But it’s not possible because to slow!
SQL> Select count (*) From sh.customers c Where not exists (select 1 from sh.sales s where s.cust_id
= c.cust_id);
COUNT(*)
--------------
48441
Elapsed: 00:00:00.13
SQL> Select count(*) from sh.customers Where GetKeyExist ('SALES;'||Cust_Id) = 0;
COUNT(*)
--------------
48441
Elapsed: 00:00:07.89 7,89 sec / 55500 rows (calls) = 0.14 milliseconds per call
There is a non negligible overhead when calling external procedures!
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
On Step Beyond
But using the feature is performing well as long
as the processing is done in the external
procedure!
SQL> select testbtree from dual;
TESTBTREE
--------------------------------------------------------------------------------
Load 10000000 keys Done in 9.00 seconds
Searching keys from 1'000'000 to
2'000'000 Press any key
1000000 key found in 0.39 seconds
Searching keys
from 4'000'000 to 8'000'000 Press any key
4000000 key found in 1.71seconds
Elapsed: 00:00:00.89
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
On Step Beyond
Solution to bypass the problem is:
Use JAVA!
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
On Step Beyond
Is there possible usage in your code?
All Ideas are welcome!!
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
2013 © Trivadis
BÂLE BERNE BRUGG LAUSANNE ZURICH DÜSSELDORF FRANCFORT s. M. FRIBOURG e. BR. HAMBOURG MUNICH STUTTGART VIENNE
Questions...
2014 © Trivadis
Jacques Kostic, Senior Consultant [email protected]
30-march-2014
In Memory B*Tree, an alternative to In Memory Database Cache
