Multi-Dimensional, In-GPU-Memory Databaseson-demand.gputechconf.com/gtc/2015/presentation/S... · -...

Business Driven Intelligence, Analytics & Performance Management. www.jedox.com - @JedoxAG - @PSJedox - #gtc15 - #s5481

Multi-Dimensional, In-GPU-Memory Databases:

Streaming Conditional Calculations in Big Data Sets

Peter Strohm | GTC15 | San José | 03/17/2015


2002 Founded in Freiburg, Germany Today Offices in Freiburg, Frankfurt, Düsseldorf, Paris, Boston 120 Global Business Partners 125 Countries with Jedox Users 25 Language Versions

2

Jedox SA App


Big Data in real-time: Twitter


Twitter: Big Data in Real-time


In-Memory OLAP-Database

All data in main memory 1

Jan

Feb

Mar

Q

1

Ap

r

May

Jun

Jul

Q2

Au

g

Sep

Q3

O

ct

No

v

Dec

Q4

Yea

r

All regions Europe France

Italy UK

North America USA

Canada Mexico

Count TweetID Longitude

In-GPU-Memory OLAP-Database

Jan Feb Mar Apr May Jun Jul Aug Sep Dec Nov Oct

Q1 Q2 Q3 Q4

Year

Hierarchical structure of consolidated elements

2

All data in GPU memory


In-Memory OLAP-Database

In-GPU-Memory & “on-the-fly” i

In-GPU-Memory OLAP-Database

Store only non-zero values 4

Calculation “on-the-fly” 3

Jan

Feb

Mar

Q

1

Ap

r

May

Jun

Jul

Q2

Au

g

Sep

Q3

O

ct

No

v

Dec

Q4

Yea

r

All regions Europe France

Italy UK

North America USA

Canada Mexico

Count TweetID Longitude


Persistent Twitter Data

in OLAP ad

d

Update trending with

Web+ETL

update

Adding Twitter data in ETL

Jedox Social Analytics Workflow

filtering

Streaming via php in . Web

Public Twitter Stream

streaming

analyzing

www

Mobile

SA App

Web

ExcelPLUS


Jedox Social Analytics with GPU

System: 2 x K40 with 12 GB GPURAM

CPU with 128GB RAM

1

Data sets with > 500 million entries 2

Calculating “on-the-fly” on all data entries 3


Jedox Social Analytics Showcase


Social Media Analytics: HeatMap

over 500 Million database entries

over 250.000 grid points


Social Media Analytics: Trending

Top 100 Trending Factor 1

~ 28.000 dictionary words

> 80 Million DB entries

2

# current day

# all previous days ________________


Social Media Analytics: Trending

1 Assigning Trending Factor to virtual cells (e.g. [TF] = [B] / [C])

2 Trending Factor is calculated on the fly

3 Conditional Calculation: IF-Rule

IF ([count] >1000) THEN [count,currentDay] / [count,previousWeek]

ELSE 0


Conditional Rule with constants

Rule Processor

> Processor

Constant (0)

[TF] = IF ([B] > 1000) THEN [C] ELSE 0

Matching Processor

Constant (1000)

MatchingProcessor



Rule Processor

Constant (0)

[TF] = IF ([B] > 1000) THEN [C] ELSE 0

Matching Processor

>

1

1

0

1

0

…

…



Rule Processor

Constant (0)

[TF] = IF ([B] > 1000) THEN [C] ELSE 0

>

1

1

0

1

0

…

…

T

A,1

A,2

A,3

A,4

A,5

…

…


Conditional Rules with constants

Rule Processor

[TF] = IF ([B] > 1000) THEN [C] ELSE 0

>

1

1

0

1

0

…

…

T

A,1

A,2

A,3

A,4

A,5

…

…

T


Conditional Rules on GPU

Rule Processor

[TF] = IF ([B] > 1000) THEN [C] ELSE 0

>

1

1

0

1

0

…

…

T

A,1

A,2

A,3

A,6

A,8

…

…

FALSE

TRUE

T



Rule Processor

[TF] = IF ([B] > 1000) THEN [C] ELSE 0

>

1

1

0

1

0

…

…

T

A,1

A,2

A,3

A,6

A,8

…

…

FALSE

TRUE

complete set

T



GPU MEMORY

SPACE FOR CALCULATIONS


Conditional Rule on GPU

Rule Processor

> Processor

Constant (1)

[TF] = IF ([B] > 1000) THEN [C] ELSE 1

Matching Processor

Constant (1000)

MatchingProcessor

Streaming Streaming Streaming


Streaming in Conditional Rules

Rule Processor

> Processor

Matching Processor

[TF] = IF ([B] > 1000) THEN [C] ELSE [D]

Matching Processor

Constant (1000)

MatchingProcessor

Part 1

Part 2

Part 1

Part 2 Total



Rule Processor

> Processor

Matching Processor


Matching Processor

Constant (1000)

MatchingProcessor

Part 1

Part 2

Part 1

Part 2

Total

FALSE

TRUE



Rule Processor

> Processor

Matching Processor


Matching Processor

Constant (1000)

MatchingProcessor

Part 2

Part 1 Part 2 Sub-

Total



Rule Processor

> Processor

Matching Processor


Matching Processor

Constant (1000)

MatchingProcessor

Part 2

Part 2 1



Rule Processor

> Processor

Matching Processor


Matching Processor

Constant (1000)

MatchingProcessor

Part 2

Part 2 Total

1



Rule Processor

> Processor

Matching Processor


Matching Processor

Constant (1000)

MatchingProcessor

Part 2 Part 2 Total

1

FALSE

TRUE



Rule Processor

> Processor

Matching Processor


Matching Processor

Constant (1000)

MatchingProcessor

1

Part 2 Sub-Total



Rule Processor

> Processor

Matching Processor


Matching Processor

Constant (1000)

MatchingProcessor

1

2



Rule Processor


complete result set

> Processor

Matching Processor

Matching Processor

Constant (1000)

MatchingProcessor


Features: Performance with GPU

Top 100 1

Heat Map 2

Trending Factor 3

Speed-up up to 50x and more! !

! In-GPU-Memory database & calculations


Jedox Social Analytics Showcase

Jedox Social Analytics App download at: www.jedox.com/en/jedox-social-analytics

GPU- & Social Analytics App-Team:

Alexander Haberstroh

Steffen Wittmer

Leo Mehlig


THANK

YOU

Multi-Dimensional, In-GPU-Memory Databaseson-demand.gputechconf.com/gtc/2015/presentation/S... · -...

Documents

Transcript of Multi-Dimensional, In-GPU-Memory Databaseson-demand.gputechconf.com/gtc/2015/presentation/S... · -...