Anti-Caching in Main Memory Database Systems

Justin DeBrabantBrown University

debrabant@cs.brown.edu

A bit of history…

1974 – System R

• query optimization • recovery • transaction serialization– lots of locks

Application

Primary Storage

Buffer Pool

Change is Good

1970 1973 1976 1979 1982 1985 1988 1991 1994 1997 2000 2003 2006 2009 20121E+00

1E+02

1E+04

1E+06

1E+08

1E+10

USD

($)

source: http://www.archivebuilders.com/whitepapers/22045p.pdf

Great, that’s what the buffer pool is for...right?

OLTP Through the Looking Glass, and What We Found ThereSIGMOD ‘08, pp. 981-992, 2008.

31%

31%

26%

12%

Buffer Pool

Locking

Recovery

Real Work

What to do with all this memory?

Application

Consistent?Updates?Writes?

Parallel Main MemoryTransaction Processing System

H-Store: A High-Performance, DistributedMain Memory Transaction Processing SystemVLDB 2008.

YCSB, Update-Heavy, data < memory

<>

Anti-Caching in H-Store

• Memory becomes primary storage for “hot” data

• “Cold” data is evicted to disk in blocks, fetched when requested by a transaction

• Still no locks/latches

Application

Anti-Cache

Primary Storage

YCSB, Update-Heavy, data > memory

15x

+ Anti-Caching > + Buffer

Pool

The New Traditional Wisdom

Future Work

• Alternative eviction strategies• Larger-than-memory queries• New hardware –flash, persistent memory

The Team

StanZdonik(Brown)

MikeStonebraker

(MIT)

StephenTu

(MIT)

AndyPavlo

(Brown)

JustinDeBrabant

(Brown)

Anti-Caching: A New Approach toDatabase Management System ArchitectureIn Preparation.

Questions?

debrabant@cs.brown.eduhstore.cs.brown.edu