Operating System Support for Application-Specific Speculation

Benjamin WesterPeter Chen and Jason Flinn

University of Michigan

EuroSys 2011 2

Speculative Execution Sequential dependent tasks Predict results of Task A

to break dependence Execute Task B in parallel

Isolate all effects Correct prediction: commit Wrong prediction: abort

A

BB

TIM

E

Predict A

EuroSys 2011 3

Speculation Everywhere!

Discrete event simulation I/O prefetching Distributed shared memory Distributed file systems Deadlock detection Remote displays Web page pre-rendering

EuroSys 2011 4

Speculation as a Service to Apps

How is this system designed?In what ways can it be customized for an app?How can those customizations be specified?

EuroSys 2011 5

Outline

Introduction Designing Speculation as a Service Implementation Evaluation Conclusion

EuroSys 2011 6

Design 1: In-App Speculation+ Complete semantic info+ Predict arbitrary app

operations+ Safe operations allowed

‒ No reuse: significant development needed

‒ Scope is limited: unsafe operations blockApp 1

Data

App 2

Data

EuroSys 2011 7

OS

Design 2: Generic OS Speculation+ Apps need no modifications+ Wide scope: unsafe

operations taint

‒ Lacks semantic understanding of app

‒ Predict system calls only‒ Handle application

conservativelyApp 2

App 1

EuroSys 2011 8

Separate Mechanism and Policy

Mechanism implements isolationPolicy describes customizations

Best of both extremes Mechanism built in OS

o Common implementationo Wide scope

Policy specified in Applicationso Expose semantic information

EuroSys 2011 9

OS

Design 3: Expose Predictions+ Predict arbitrary app

operations+ Reuse OS mechanism

(with app assistance)+ Wide scope for taint

propagation

‒ Limited semantic info‒ Speculative external output

never allowed‒ Commit on identical results

App 2 App 1

EuroSys 2011 10

OS

Design 4: Expose Safety+ Predict arbitrary app

operations+ Reuse OS mechanism

(with app assistance)+ Wide scope for taint

propagation

+ More semantic info+ Allow safe output+ Commit on equivalent results

App 2 App 1

EuroSys 2011 11

Customizable Policy

CreationoWhat tasks are predictableo How to predict them

OutputoWhat output is safe to allow

CommitoWhich results are acceptable to commit

EuroSys 2011 12

Outline

Introduction Designing Speculation as an OS Service Implementation Evaluation Conclusion

EuroSys 2011 13

Implementation

Mechanism built in OSo Based on Speculator kernelo Checkpoints & logs processes, files, IPC, etc.

Policies expressed using system call API

EuroSys 2011 14

SpeculativeProcess

ControlProcess

spec_fork()TI

ME

spec_fork

predict A

B

commit

A

abort

BC

EuroSys 2011 15

API Exampleint main() {

int x;int prediction = get_prediction();if (spec_fork() == SPECULATIVE) {

x = prediction;} else {

x = slow_function();if (equiv(x, prediction))

commit();else

abort();}set_output_policy(stdout, ALLOW);printf(“%d”, x);

}

Output Policy

Creation Policy

Commit Policy

EuroSys 2011 16

Outline

Introduction Designing Speculation as an OS Service Implementation Evaluation Conclusion

EuroSys 2011 17

Evaluation

Can apps effectively use APIto increase parallelism?

Case studies1. Predictive application launching in Bash2. SSL certificate checks in Firefox3. Replicated service in PBFT-CS

EuroSys 2011 18

Check command

App 1: Predictive Launching in BashTI

ME Run program …finished

bwester $ ▌

…finishedbwester $ grep foo –r .▌

grep foo –r .

Creation Policy:Use machine learningto predict user input

Commit Policy:Normalize user input before comparisonOutput Policy:

Safe X11 Messages: Allow

EuroSys 2011 19

How Much Work Can Be Hidden?

LaTeX build

Bash m

ake

GIMP Lo

ad

OpenOffice Lo

ad0

1

2

3

4

5

6

Run

Tim

e (s

econ

ds)

TIM

E

Non-Speculative45

Speculative

Prompt

Get cmd

Spec.Execute

cmd

Execute

Prompt

Executecmd

Get cmd

EuroSys 2011 20

How Much Work Can Be Hidden?

LaTeX build

Bash m

ake

GIMP Lo

ad

OpenOffice Lo

ad0

1

2

3

4

5

6

Run

Tim

e (s

econ

ds)

TIM

E

SpeculativeNon-Speculative45Up to 86% hidden

Prompt

Get cmd

Spec.Execute

cmd

Execute

Prompt

Executecmd

Get cmd

EuroSys 2011 21

Done

Request page

App 2: Firefox SSL ConnectionsTI

ME

Web Server

Validation Server

Open https://

Check cert.

ValidateGet session key

GET /index.html?id=0123

Creation Policy:Predict certificate is valid

Output Policy:Alow SSL connection

Output Policy:Block private data

EuroSys 2011 22

Connection Latency Hidden?TI

ME

Non-Speculative Speculative

Google Acco

unts

Windows L

ive ID

Chase.co

m0

100

200

300

400

500

600

Conn

ect T

ime

(mill

isec

onds

)

https://

Check cert.

Session key

Done

Request

https://

Check cert.

Session key

Done

RequestValidate

EuroSys 2011 23

Connection Latency Hidden?TI

ME

Non-Speculative Speculative

Google Acco

unts

Windows L

ive ID

Chase.co

m0

100

200

300

400

500

600

Conn

ect T

ime

(mill

isec

onds

)

Avg 60ms hiddenhttps://

Check cert.

Session key

Done

Request

https://

Check cert.

Session key

Done

RequestValidate

EuroSys 2011 24

App 3: PBFT-CS ProtocolTI

ME

Send Request Distributed Replicated

Service(Reply, sig1)

Check reply

Check reply

Request

(Reply, sig2)

Work…

Creation Policy:Agree on 1st reply

Output Policy:PBFT-CS Messages: Allow

EuroSys 2011 25

Improved Client Throughput?TI

ME

Non-Speculative

0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

100200300400500600700800900

Latency between replicas (mil-liseconds)

Clie

nt O

ps p

er S

econ

d

Speculative

Request1

1st Reply

VerifyRequest2

1st Reply

Request1

1st Reply

Verify

Request2

1st Reply

Null requests

EuroSys 2011 26

Improved Client Throughput?TI

ME

Non-Speculative Speculative

0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

100200300400500600700800900

Latency between replicas (mil-liseconds)

Clie

nt O

ps p

er S

econ

d

1.9x Throughput

Request1

1st Reply

VerifyRequest2

1st Reply

Request1

1st Reply

Verify

Request2

1st Reply

Null requests

EuroSys 2011 27

Cost of Generic Mechanism

0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

100200300400500600700800900

Latency between replicas (mil-liseconds)

Clie

nt O

ps p

er S

econ

d

App-specific MechShared OS MechNon-speculative

Null requests

EuroSys 2011 28

Cost of Generic Mechanism

0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

100200300400500600700800900

Latency between replicas (mil-liseconds)

Clie

nt O

ps p

er S

econ

d

App-specific:8% faster

App-specific MechShared OS MechNon-speculative

Null requests

EuroSys 2011 29

Conclusion Mechanism

o Common: checkpoints, output buffering, taint propagationo Implemented in OS

Policyo App-specific: Controls creation, output, and commito Implemented in applications

Demonstrated with 3 case studieso Improved parallelismo Small overhead relative to app-specific mechanism

Questions?