VectorWise The world’s fastest database GIUA, 13 September 2011.

VectorWiseThe world’s fastest database

GIUA, 13 September 2011

DBT-3 Database Schema


DBT-3 Data


What is VectorWise?

Started as an academic project– Centrum Wiskunde & Informatica (CWI)

• Python• MonetDB• X100 VectorWise

Adopted as an Ingres community project Joint venture company set up between CWI and

Ingres Corp. Now wholly owned by Ingres Corp.


What is VectorWise for?

Data warehousing Data marts Data mining Online Analytical Processing (OLAP) Business Intelligence


VectorWise Technology

On Chip ComputingT

ime

/ Cyc

les

to

Pro

cess

Data Processed

DISK

RAM

CHIP

10GB2-3GB40-400MB

2-20

15

0-2

50

Mil

lion

s

Vector Processing

Breakthrough technology


On Chip Computing

Processing in Chip Cache CPU cache access is more efficient than RAM cache

access

Tim

e / C

ycle

s t

o P

roce

ss

Data Processed

DISK

RAM

CHIP

10GB2-3GB40-100MB

2-2

01

50-

25

0M

illio

ns


Vector Processing

verses

=

1 x 1 = 1

2 x 2 = 4

3 x 3 = 9

4 x 4 = 16

5 x 5 = 25

6 x 6 = 36

7 x 7 = 49

8 x 8 = 64

.

.

.

n x n = n2

1 x 1

2 x 2

3 x 3

4 x 4

5 x 5

6 x 6

7 x 7

8 x 8

.

.

.

n x n

1

4

9

16

25

36

49

64

.

.

.

n2

SISDSingle

InstructionSingle

Dataprocessed

SIMDSingleInstructionMultipleDataprocessed


VectorWise Technology

Automatic Compression

Updateable Column Store

Automatic Storage Indexes

Minimize IO

Innovations on industry proven techniques


Updateable Column Store

Only access relevant data

Efficient incremental update enabled– Traditionally a weakness of column stores

Cust_Num Cust_surname Cust_first_name Cust_DOB Cust_Sex Cust_Add_1 Cust_Addr_2 Cust_City Cust_State

46328927956 Jones Steven 17-JAN-1971 M 333 StKilda Rd Melbourne Vic

98679975745 Smith Leonard 04-APR-1964 M 147 Trafalgar Road Birmingham England

52634346735 Rogers Cindy 11-MAR-1980 F Belmont Rail Service421 Station St

Belmont CA

346737347347 Andrews Jenny 14-SEP-1977 F 117 West 42nd St New York NY

88673477347 Cooper Sheldon 30-JUN-1980 M Ingres Corporation Level 2, 426 Argello St Redwood City CA

34673447568 Kollwitz Rolf 22-DEC-1975 M IBM Headquarters 123 Mount View Crs Atlantic City PN

99554443044 Wong Penny 13-NOV-1981 F Ming On Tower 1 177 Moa Tzu Tung Rd Beijing China


Automatic Compression

Vectorized compression– Compressed on disk– Decompression for data processing in CPU cache– Compressed in RAM

Column based compression with multiple algorithms– Automatically determined by VectorWise


Compression Methods

Run Length Encoding– Efficient if many duplicate adjacent tuple values are present– Such as in ordered columns with few unique values

Patched Frame Of Reference– Encodes values as a small difference from a page-wide base value– PFOR is effective on any data distribution with some value distribution locality

Delta encoding on top of PFOR– Integers are made smaller by considering the differences between subsequent

values– Highly effective on ordered data

PDICT dictionary encoding– Efficient in case the value distribution is dominated by a limited amount of very

frequent values– Is currently the only one that applies to character data types


Automatic Storage Indexes

Stores min/max value per data block Automatically created Automatically maintained Enables efficient identification of candidate data

blocks


authpass

IVWLOG

User Interface (SQL, ABF, OpenROAD,

JAVA, etc.)

DASServer (iigcd)

CommunicationsServer (iigcc)

Archiver (iiacp)

Recovery Server (iircp)

JournalsJournals

JournalsJournals

IngresTransaction

Log File

VectorWise Instance Architecture

Name Server (iigcn)

User Interface (SQL, ABF, OpenROAD,

etc.)

DBMS Server (iidbms)

Locks Log Buffers

iix100 Server VectorWiseData Store

IVWMemory

IVW LOCK

Databases

Databases

Databases


Operating System

Currently available on 64-bit Linux and Windows Runs on

– RedHat– Fedora– CentOS– SuSE11– Ubuntu– Works on other Linux flavours– Windows 2008– Windows 7


Hardware Requirements

Fast multi-core CPUs Memory

– 2 Gbytes for OS + IVW requirements + other apps– Minimum 8 Gbytes

Disk– Lots


And now live


VectorWise The world’s fastest database GIUA, 13 September 2011.

Documents

Transcript of VectorWise The world’s fastest database GIUA, 13 September 2011.