Aerie: Flexible File-System

Interfaces to Storage-Class Memory [APSys’1 , EuroSys’1 ]

Haris Volos†

Sanketh Nalli, Sankaralingam Panneerselvam,

Venkatanathan Varadarajan, Prashant Saxena,

Michael M. Swift

HP Labs †

Outline

• Overview

• Motivation: Interface flexibility

• Aerie: In-memory library file systems

• Evaluation

2

• Persistent

• Short access time

3

Software overhead matters

Storage-Class Memory (SCM)

Flash-backed DRAM

Flash

Phase-Change Memory

Spin Torque MRAM Resistive RAM

ns μs

Latency

4

Memory

Controller

SCM DRAM

Storage-Class Memory (SCM)

• Persistent

• Short access time

• Byte addressable

Accessible via loads/stores

No software

overhead

• Direct user-mode access

for fast access to data

– Moneta-D, PMFS, Quill,

NV-Heaps, Mnemosyne

• File system for sharing

– Shared namespace

– Protection

– Integrity

5

OS/

File System

Application

Accessing SCM today

+

Device Driver Device Driver

Generic Block Layer

I/O Scheduler

SCM FS

App

Virtual File System

App

Disk FS

I/O Bus

Does SCM need a kernel FS?

6

MMU protects

CPU access

DMA is not

protected

Variable

latency to disk

~ Constant

latency to SCM

Load/store

interface

No standard

interface

SCM Disk

• Enable implementation flexibility

– Optimize file-system interface semantics

– Optimize operations regarding metadata

Library file systems

7

APP

LibFS

API

[Exokernel (MIT),

Nemesis (Cambridge)]

Outline

• Overview

• Motivation: Interface flexibility

• Aerie: In-memory library file systems

• Evaluation

8

• Universal abstraction: Everything is a file

– Has generic-overhead cost

9

Application

POSIX File (Virtual File System)

Storage

File IPC

Network

Socket

POSIX File: Expensive abstraction

• Rigid interface and policies

– Has fixed components and costs

– Hinders application-specific customization

10

Application

UNIX concurrency

semantics

Hierarchical

names

Byte

streams

Permissions

POSIX File: Expensive abstraction

POSIX File: Expensive abstraction

• Generic-overhead costs

• Rigid interface and policies

Application

POSIX File

(Virtual File System)

open()

~ 2.5 μs ≈ 25x SCM latency

Customizing the file system today

• Modify the kernel

• Add a layer over existing kernel file system

• Use a user-mode framework such as FUSE

12

Need flexible interfaces

Outline

• Overview

• Motivation: Interface flexibility

• Aerie: In-memory library file systems (libFS)

• Evaluation

13

Kernel safely multiplexes SCM

14

allocation, protection, addressing

Ke

rne

l H

W

• Allocation: Allocates SCM regions

• Addressing: Memory-maps SCM regions

• Protection: Keeps track of region access rights

Library implements functionality

15

APP

LibFS

(layout, logic)

allocation, protection, addressing

Use

r K

ern

el

HW

API

APP

LibFS

(layout, logic)

API

APP

OtherLibFS

(layout, logic)

API

16

APP

libFS

Use

r H

W

APP

/

common bob alice

/

common bob alice

common alice bob /

Shared namespace

libFS

open

17

APP

libFS

Use

r H

W

APP

common alice bob /

Hardware-enforced permissions

libFS

/

common bob alice

/

common bob alice Memory protection

prevents Bob from

accessing Alice’s files

18

APP

libFS

Use

r H

W

APP

libFS

/

shared alice

alice bob /

Hardware protection cannot

guarantee integrity

common

/

common bob

foo

bar

Trusted

FS

Service

(TFS)

19

APP

LibFS

(layout, logic)

API

Use

r K

ern

el

HW

APP

LibFS

(layout, logic)

API

Integrity via Trusted File Service

allocation, protection, addressing

20

HW

SCM

APP

LibFS

API

File data Metadata

Read/

Write Read

Read/

Write

APP

LibFS

(layout, logic)

API

Use

r

Trusted

FS

Service

(TFS)

Metadata

Server (integrity)

Update

Decentralized architecture

RPC

Lease

Manager (sharing)

Outline

• Overview

• Motivation: Interface flexibility

• Aerie: In-memory library file systems (libFS)

• Evaluation

21

File Systems

Functionality: PXFS

• POSIX interface:

open/read/write/unlink

• Hierarchical namespace

• POSIX concurrency

semantics

• File byte streams

22

File Systems

Functionality: PXFS

• POSIX interface:

open/read/write/unlink

• Hierarchical namespace

• POSIX concurrency

semantics

• File byte streams

Optimization: FlatFS

• Key-value interface:

put/get/erase

• Flat namespace

• KV-store concurrency

semantics

• Short, immutable files

23

Application-workload performance

0

5

10

15

20

Fileserver Webserver Webproxy

Late

ncy

pe

r o

p

( μ

s )

RamFS

ext4

PXFS

FlatFS

24

• PXFS performs better than kernel-mode FS

• FlatFS exploits app semantics to improve performance

10%

9% 22%

53% (2.1x) 30%

Scalability: Webproxy

0

200

400

600

800

1000

1200

0 5 10

Th

rou

gh

pu

t (K

op

s/s)

# Threads

PXFS

RamFS

ext4

FlatFS

• FlatFS retains its benefits over kernel-mode file systems

• Software interface overheads handicap fast SCM

• Flexible interface is a must for fast SCM

• Aerie: Library file systems help remove generic

overheads for higher performance

– FlatFS improves performance by up to 110%

26

Conclusion

Thank you! Questions?