Performance Annotations for Cloud Computing

Daniele Rogora∗ Steffen Smolka% Antonio Carzaniga∗ Amer Diwan$

Robert Soulé∗∼

presented by

Daniele Rogora

∗Università della Svizzera italiana %Cornell University $Google ∼Barefoot Networks

HotCloud 2017

1 / 27

Data Centers Are Everywhere...

Data centers provide◮ Services for end users

◮ Google, Facebook, Dropbox

◮ Services for companies and universities◮ AWS for SAP, cloud mail services

◮ Raw processing power (IaaS, PaaS)◮ Amazon EC2, Microsoft Azure

2 / 27

Data Centers Are Everywhere...

Data centers provide◮ Services for end users

◮ Google, Facebook, Dropbox

◮ Services for companies and universities◮ AWS for SAP, cloud mail services

◮ Raw processing power (IaaS, PaaS)◮ Amazon EC2, Microsoft Azure

Data centers are widespread◮ Microsoft has more than 100 data centers

◮ they account for more than 1M servers

◮ Amazon has more than 30 data centers◮ they account for more than 1.5M servers

◮ Google has 15 data centers scattered around the world◮ in 2013 they accounted for around 900k servers

2 / 27

... But They Are Complex

Machine 1

Machine 2

Machine 3

Machine 4

Machine 5

Switch 1

Machine 6

Machine 7

Machine 8

Machine 9

Machine 10

Switch 2

Machine 11

Machine 12

Machine 13

Machine 14

Machine 15

Switch 3

3 / 27

... But They Are Complex

Machine 1

Machine 2

Machine 3

Machine 4

Machine 5

Switch 1

Machine 6

Machine 7

Machine 8

Machine 9

Machine 10

Switch 2

Machine 11

Machine 12

Machine 13

Machine 14

Machine 15

Switch 3

CPU HDD NET

3 / 27

... But They Are Complex

Machine 1

Machine 2

Machine 3

Machine 4

Machine 5

Switch 1

Machine 6

Machine 7

Machine 8

Machine 9

Machine 10

Switch 2

Machine 11

Machine 12

Machine 13

Machine 14

Machine 15

Switch 3

CPU HDD NET

Operating System

3 / 27

... But They Are Complex

Machine 1

Machine 2

Machine 3

Machine 4

Machine 5

Switch 1

Machine 6

Machine 7

Machine 8

Machine 9

Machine 10

Switch 2

Machine 11

Machine 12

Machine 13

Machine 14

Machine 15

Switch 3

CPU HDD NET

Operating System

Virtual Network

3 / 27

... But They Are Complex

Machine 1

Machine 2

Machine 3

Machine 4

Machine 5

Switch 1

Machine 6

Machine 7

Machine 8

Machine 9

Machine 10

Switch 2

Machine 11

Machine 12

Machine 13

Machine 14

Machine 15

Switch 3

CPU HDD NET

Operating System

Virtual Network

Virtual Machines

3 / 27

... But They Are Complex

Machine 1

Machine 2

Machine 3

Machine 4

Machine 5

Switch 1

Machine 6

Machine 7

Machine 8

Machine 9

Machine 10

Switch 2

Machine 11

Machine 12

Machine 13

Machine 14

Machine 15

Switch 3

CPU HDD NET

Operating System

Virtual Network

Virtual Machines

Software

3 / 27

... But They Are Complex

Machine 1

Machine 2

Machine 3

Machine 4

Machine 5

Switch 1

Machine 6

Machine 7

Machine 8

Machine 9

Machine 10

Switch 2

Machine 11

Machine 12

Machine 13

Machine 14

Machine 15

Switch 3

CPU HDD NET

Operating System

Virtual Network

Virtual Machines

Software

CPU HDD NET

Operating System

Virtual Network

Virtual Machines

Software

3 / 27

... But They Are Complex

Machine 1

Machine 2

Machine 3

Machine 4

Machine 5

Switch 1

Machine 6

Machine 7

Machine 8

Machine 9

Machine 10

Switch 2

Machine 11

Machine 12

Machine 13

Machine 14

Machine 15

Switch 3

CPU HDD NET

Operating System

Virtual Network

Virtual Machines

Software

CPU HDD NET

Operating System

Virtual Network

Virtual Machines

Software

4 / 27

... But They Are Complex

Machine 1

Machine 2

Machine 3

Machine 4

Machine 5

Switch 1

Machine 6

Machine 7

Machine 8

Machine 9

Machine 10

Switch 2

Machine 11

Machine 12

Machine 13

Machine 14

Machine 15

Switch 3

CPU HDD NET

Operating System

Virtual Network

Virtual Machines

Web Server

CPU HDD NET

Operating System

Virtual Network

Virtual Machines

DBMS

4 / 27

... But They Are Complex

Machine 1

Machine 2

Machine 3

Machine 4

Machine 5

Switch 1

Machine 6

Machine 7

Machine 8

Machine 9

Machine 10

Switch 2

Machine 11

Machine 12

Machine 13

Machine 14

Machine 15

Switch 3

CPU HDD NET

Operating System

Virtual Network

Virtual Machines

AUTH Web Server

CPU HDD NET

Operating System

Virtual Network

Virtual Machines

DBMS

RPC

4 / 27

... But They Are Complex

Machine 1

Machine 2

Machine 3

Machine 4

Machine 5

Switch 1

Machine 6

Machine 7

Machine 8

Machine 9

Machine 10

Switch 2

Machine 11

Machine 12

Machine 13

Machine 14

Machine 15

Switch 3

CPU HDD NET

Operating System

Virtual Network

Virtual Machines

AUTH Web Server

CPU HDD NET

Operating System

Virtual Network

Virtual Machines

DBMS

RPC

4 / 27

Machine 1

Machine 2

Machine 3

Machine 4

Machine 5

Switch 1

Machine 1

Machine 2

Machine 3

Machine 4

Machine 5

Switch 2

Machine 11

Machine 12

Machine 13

Machine 14

Machine 15

Switch 3

Operating System

Virtual Network

Virtual MachinesAUTH

Web Server

Operating System

Virtual Network

Virtual Machines

DBMS

RPC

Understanding the performance of a data center is difficult

many layers

limited scope of tools and people’s knowledge

Machine 1

Machine 2

Machine 3

Machine 4

Machine 5

Switch 1

Machine 1

Machine 2

Machine 3

Machine 4

Machine 5

Switch 2

Machine 11

Machine 12

Machine 13

Machine 14

Machine 15

Switch 3

Operating System

Virtual Network

Virtual MachinesAUTH

Web Server

Operating System

Virtual Network

Virtual Machines

DBMS

RPC

Understanding the performance of a data center is difficult

many layers

limited scope of tools and people’s knowledge

3 real-world questions by a data center operator

How much load increase can we support with the current

setup? Where will the bottleneck be? What would break

first?

How much would it help to move the database server to

faster hardware, or directly on the metal?

Can we understand and explain unexpected behaviors?

5 / 27

Goal: Creating a New Model

We want to build a dynamic performance model for data centers

comprehensive

live

interactive

6 / 27

Goal: Creating a New Model

We want to build a dynamic performance model for data centers

comprehensive

live

interactive

Logs

Query

Model

Goal: Creating a New Model

We want to build a dynamic performance model for data centers

comprehensive

live

interactive

Logs

Query

Model

How is it performing now?

Performance

Goal: Creating a New Model

We want to build a dynamic performance model for data centers

comprehensive

live

interactive

Logs

Query

Model

What if the we improve the hardware?

Predictive result

6 / 27

Goal: Creating a New Model

We want to build a dynamic performance model for data centers

comprehensive

live

interactive

Logs

Query

Statistical Model

6 / 27

Performance Annotations

7 / 27

Example: a Web Service

Request Response

System

The response time grows quadratically

with the size of the body of the request

Example: a Web Service

Request Response

System

The response time grows quadratically

with the size of the body of the request

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

0 20 40 60 80 100

tim

e (

ms)

body size (MB)

Example: a Web Service

Request Response

System

The response time grows quadratically

with the size of the body of the request

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

0 20 40 60 80 100

tim

e (

ms)

body size (MB)@Time ∼ a∗ |in|2 +b ∗ |in|+c

Example: a Web Service

Request Response

System

The memory used grows linearly

with the size of the body of the request

10

20

30

40

50

60

70

80

90

0 20 40 60 80 100

mem

ory

usage (

kB

)

body size (kB)@Mem ∼ a∗ |in|+b

8 / 27

Example: a Web Service

Request Response

System

Machine 1 Machine 7

Example: a Web Service

Request Response

System

Machine 1 Machine 7VM PostgreSQL

Example: a Web Service

Request Response

System

Machine 1 Machine 7VM PostgreSQL

Apache WS

Example: a Web Service

Request Response

System

Machine 1 Machine 7VM PostgreSQL

Apache WSOwnCloud

Example: a Web Service

Request Response

System

Machine 1 Machine 7VM PostgreSQL

Apache WSOwnCloud

getFile(string f)

normalizePath(string p)

parseQuery(string q)

execute(query q)

8 / 27

Example: a Web Service

Request Response

System

Machine 1 Machine 7VM PostgreSQL

Apache WSOwnCloud

getFile(string f)

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

0 20 40 60 80 100

tim

e (

ms)

body size (MB)

normalizePath(string p)

10

20

30

40

50

60

70

80

90

0 20 40 60 80 100

me

mo

ry u

sa

ge

(kB

)

body size (kB)

parseQuery(string q)

0

1

2

3

4

5

10 20 30 40 50 60 70 80 90 100

Me

mo

ry (

kb

yte

s)

String length

execute(query q)

0

10

20

30

40

50

60

70

80

90

0 20 40 60 80 100

tim

e (

ms)

query size

8 / 27

3

4

5

6

7

8

9

0 10 20 30 40 50 60 70 80

Me

mo

ry (

kb

yte

s)

String length

9 / 27

3

4

5

6

7

8

9

0 10 20 30 40 50 60 70 80

Me

mo

ry (

kb

yte

s)

String length

@Mem∼Constant(8kB)

9 / 27

3

4

5

6

7

8

9

0 10 20 30 40 50 60 70 80

Me

mo

ry (

kb

yte

s)

String length

@Mem∼Constant(8kB) 0

5

10

15

20

25

30

35

40

0 2 4 6 8 10 12 14

Me

mo

ry (

kb

yte

s)

Path length

9 / 27

3

4

5

6

7

8

9

0 10 20 30 40 50 60 70 80

Me

mo

ry (

kb

yte

s)

String length

@Mem∼Constant(8kB) 0

5

10

15

20

25

30

35

40

0 2 4 6 8 10 12 14

Me

mo

ry (

kb

yte

s)

Path length

Line Fit

9 / 27

3

4

5

6

7

8

9

0 10 20 30 40 50 60 70 80

Me

mo

ry (

kb

yte

s)

String length

@Mem∼Constant(8kB) 0

5

10

15

20

25

30

35

40

0 2 4 6 8 10 12 14

Me

mo

ry (

kb

yte

s)

Path length

Line Fit

@Mem∼Norm(5ms*path_len(path), 5ms)

9 / 27

3

4

5

6

7

8

9

0 10 20 30 40 50 60 70 80

Me

mo

ry (

kb

yte

s)

String length

@Mem∼Constant(8kB) 0

5

10

15

20

25

30

35

40

0 2 4 6 8 10 12 14

Me

mo

ry (

kb

yte

s)

Path length

Line Fit

@Mem∼Norm(5ms*path_len(path), 5ms)

0

1

2

3

4

5

10 20 30 40 50 60 70 80 90 100

Me

mo

ry (

kb

yte

s)

String length 9 / 27

3

4

5

6

7

8

9

0 10 20 30 40 50 60 70 80

Me

mo

ry (

kb

yte

s)

String length

@Mem∼Constant(8kB) 0

5

10

15

20

25

30

35

40

0 2 4 6 8 10 12 14

Me

mo

ry (

kb

yte

s)

Path length

Line Fit

@Mem∼Norm(5ms*path_len(path), 5ms)

0

1

2

3

4

5

10 20 30 40 50 60 70 80 90 100

Me

mo

ry (

kb

yte

s)

String length

@Mem∼Constant(4.2kB)

∨

@Mem∼Constant(0.5kB)

9 / 27

3

4

5

6

7

8

9

0 10 20 30 40 50 60 70 80

Me

mo

ry (

kb

yte

s)

String length

@Mem∼Constant(8kB) 0

5

10

15

20

25

30

35

40

0 2 4 6 8 10 12 14

Me

mo

ry (

kb

yte

s)

Path length

Line Fit

@Mem∼Norm(5ms*path_len(path), 5ms)

0

1

2

3

4

5

10 20 30 40 50 60 70 80 90 100

Me

mo

ry (

kb

yte

s)

String length

@Mem∼Constant(4.2kB)

∨

@Mem∼Constant(0.5kB)

700

750

800

850

900

950

1000

10 20 30 40 50 60 70 80 90M

em

ory

(b

yte

s)

String length 9 / 27

3

4

5

6

7

8

9

0 10 20 30 40 50 60 70 80

Me

mo

ry (

kb

yte

s)

String length

@Mem∼Constant(8kB) 0

5

10

15

20

25

30

35

40

0 2 4 6 8 10 12 14

Me

mo

ry (

kb

yte

s)

Path length

Line Fit

@Mem∼Norm(5ms*path_len(path), 5ms)

0

1

2

3

4

5

10 20 30 40 50 60 70 80 90 100

Me

mo

ry (

kb

yte

s)

String length

@Mem∼Constant(4.2kB)

∨

@Mem∼Constant(0.5kB)

700

750

800

850

900

950

1000

10 20 30 40 50 60 70 80 90M

em

ory

(b

yte

s)

String length

@Mem∼?

9 / 27

How To Create Annotations

10 / 27

Instrumentation

11 / 27

Instrumentation

For every call of all the functions in the system, we need:

metrics of interest◮ execution time◮ dynamic memory allocation◮ locks holding time

relevant features of the input parameters◮ string length◮ collection size

11 / 27

Instrumentation

For every call of all the functions in the system, we need:

metrics of interest◮ execution time◮ dynamic memory allocation◮ locks holding time

relevant features of the input parameters◮ string length◮ collection size

Also, the instrumentation must be:

crosslayer

crossplatform

11 / 27

Automatic Annotation Inference

Automatic Annotation Inference

Logs

Function: correctFolderSize feature: collection size

pcc: 0.5610

0

5

10

15

20

25

0 1 2 3 4 5 6 7 8 9

Mem

ory

(kbyte

s)

Collection size

12 / 27

Automatic Annotation Inference

Logs

Function: correctFolderSize feature: string length

pcc: 0.78673

0

5

10

15

20

25

0 20 40 60 80 100 120

Mem

ory

(kbyte

s)

String length

12 / 27

Automatic Annotation Inference

Logs

Function: correctFolderSize feature: path length

pcc: 0.9473

0

5

10

15

20

25

0 1 2 3 4 5 6 7

Mem

ory

(kbyte

s)

Path length

12 / 27

Automatic Annotation Inference

Logs

Function: correctFolderSize feature: path length

pcc: 0.9473

0

5

10

15

20

25

0 1 2 3 4 5 6 7

Mem

ory

(kbyte

s)

Path length

Line fit

regression

12 / 27

Automatic Annotation Inference

Logs

Function: broadcastEvent feature: collection size

pcc: -0.1155

0

0.5

1

1.5

2

2.5

3

3.5

4

0 1 2 3 4 5

Tim

e (

ms)

Collection size

12 / 27

Automatic Annotation Inference

Logs

Function: broadcastEvent feature: string length

pcc: -0.2764

0

0.5

1

1.5

2

2.5

3

3.5

4

8 10 12 14 16 18 20

Tim

e (

ms)

String length

12 / 27

Automatic Annotation Inference

Logs

Function: broadcastEvent no feature

0

0.5

1

1.5

2

2.5

3

3.5

4

-1 -0.5 0 0.5 1

Tim

e (

ms)

Scalar value

clusters

12 / 27

Annotations Uses

13 / 27

Uses

14 / 27

Uses

Documentation◮ how do functions behave?

14 / 27

Uses

Documentation◮ how do functions behave?

Anomaly/failure detection◮ is the system behaving normally? Is there a performance regression?

14 / 27

Uses

Documentation◮ how do functions behave?

Anomaly/failure detection◮ is the system behaving normally? Is there a performance regression?

Extrapolation◮ can we scale up?

14 / 27

Uses

Documentation◮ how do functions behave?

Anomaly/failure detection◮ is the system behaving normally? Is there a performance regression?

Extrapolation◮ can we scale up?

Composition◮ can we infer the behavior of the caller from the annotations of the callees?

14 / 27

Case Study

15 / 27

Case Study

setup

15 / 27

The System: Application Level

Client

VM

HAProxy

VMnginx

OwnCloudVM

PostgreSQL

VM

NFS

WEBDAV WEBDAV

TCP

TCP

16 / 27

The System: Computing Resources

ETH

Host01 Host02

Host03 Host04 Host05 Host06 Host07

Host08 Host09 Host10 Host11 Host12

17 / 27

The System: Computing Resources

ETH + VLAN

Host01

Test

Host02

Openstack

Neutron

Host03 Host04 Host05 Host06 Host07

Host08 Host09 Host10 Host11 Host12

18 / 27

The System: Computing Resources

ETH + VLAN

STORAGE NODE

Host01

Test

Host02

Openstack

Neutron

Host03 Host04 Host05 Host06 Host07

Host08

Ceph OSD

Host09

Ceph OSD

Host10

Ceph OSD

Host11

Ceph OSD

Host12

Ceph OSD

18 / 27

The System: Computing Resources

ETH + VLAN

COMPUTE NODE

STORAGE NODE

Host01

Test

Host02

Openstack

Neutron

Host03

Ceph master

DBMS

Host04

Ceph master

NFS server

Host05

Ceph master

Web server

Host06

Sync server

DBMS

Host07

Load balancer

LDAP

Host08

Ceph OSD

Host09

Ceph OSD

Host10

Ceph OSD

Host11

Ceph OSD

Host12

Ceph OSD

18 / 27

PHP Instrumentation

OwnCloud (php)

function foo(){...}

19 / 27

PHP Instrumentation

OwnCloud (php)

function foo_inner() {...}

19 / 27

PHP Instrumentation

OwnCloud (php)

function foo_inner() {...}

foo_inner()

function foo() {

start = time()

end = time()

log(end - start)

}

19 / 27

Cross-Platform Instrumentation

OwnCloud (php)

function write_to_db() {

...

}

19 / 27

Cross-Platform Instrumentation

OwnCloud (php)

function write_to_db() {...

trace_id = rnd_string()...

}

19 / 27

Cross-Platform Instrumentation

OwnCloud (php)

function write_to_db() {...

trace_id = rnd_string()...

}

PostgreSQL (C)

function execute_query() {...

get(trace_id)...

}

19 / 27

Cross-Platform Instrumentation

OwnCloud (php)

function write_to_db() {...

trace_id = rnd_string()...

}

PostgreSQL (C)

function execute_query() {...

get(trace_id)...

}

Marshaller (C)

marshall() {...

put(trace_id)...

}

Unmarshaller (C)

unmarshall() {...

get(trace_id)...

}

19 / 27

Case Study

20 / 27

Case Study

annotations

20 / 27

It Works!

0

20

40

60

80

100

120

140

160

180

200

220

0 50 100 150 200 250 300

Me

mo

ry (

kb

yte

s)

Collection size

@Mem ∼ 721.362B ∗ |in|+8851.16B

21 / 27

It Works!

3

4

5

6

7

8

9

0 10 20 30 40 50 60 70 80

Me

mo

ry (

kb

yte

s)

String length

\OC\Files\View::getOwner()

@Mem ∼ 8.240kB

21 / 27

It Works!

0

0.1

0.2

0.3

0.4

0.5

0.6

0 50 100 150 200 250 300

Tim

e (

ms)

Collection size

\Sabre\DAV\Server::generateMultiStatus()

@Time ∼ 3.6e−03B ∗ |in|2 +3.8e−05B ∗ |in|+6.6e−06B

21 / 27

It Works!

0

100

200

300

400

500

600

0 20 40 60 80 100 120

Me

mo

ry (

byte

s)

String length

\OC\Files\Cache\Cache::normalize()

@Mem ∼ Norm(

411.9,15.35)

∨Norm(

435.3,24.41)

∨Norm(

448,0)

∨Norm(

459.6,19.48)

∨Norm(

477.6,23.18)

∨Norm(

502.0,18.58)

21 / 27

Case Study

22 / 27

Case Study

anomaly detection

22 / 27

Anomaly Injection

Client

VM

HAProxy

VMnginx

OwnCloudVM

PostgreSQL

VM

NFS

WEBDAV WEBDAV

TCP

TCP

23 / 27

Anomaly Injection

Client

VM

HAProxy

VMnginx

OwnCloudVM

PostgreSQL

VM

NFS

WEBDAV WEBDAV

TCP

TCP

Added Latency: 1-10ms23 / 27

Annotations Catch The Anomaly

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100

%

Annotation robustness (%)

no delay

24 / 27

Annotations Catch The Anomaly

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100

%

Annotation robustness (%)

no delay

1ms

24 / 27

Annotations Catch The Anomaly

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100

%

Annotation robustness (%)

no delay

1ms

2ms

24 / 27

Annotations Catch The Anomaly

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100

%

Annotation robustness (%)

no delay

1ms

2ms

5ms

24 / 27

Annotations Catch The Anomaly

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100

%

Annotation robustness (%)

no delay

1ms

2ms

5ms

10ms

24 / 27

Future

25 / 27

Ongoing Work - Discussion

Machine learning techniques fine tuning

26 / 27

Ongoing Work - Discussion

Machine learning techniques fine tuning

Annotations composition◮ stack analysis

26 / 27

Ongoing Work - Discussion

Machine learning techniques fine tuning

Annotations composition◮ stack analysis

Workload creation◮ can we forge workloads that expose specific behaviors?

26 / 27

Ongoing Work - Discussion

Machine learning techniques fine tuning

Annotations composition◮ stack analysis

Workload creation◮ can we forge workloads that expose specific behaviors?

Feature Selection◮ Is a heuristically built set of basic features enough?◮ Can we exploit programmers’ knowledge of the system?

26 / 27

Ongoing Work - Discussion

Machine learning techniques fine tuning

Annotations composition◮ stack analysis

Workload creation◮ can we forge workloads that expose specific behaviors?

Feature Selection◮ Is a heuristically built set of basic features enough?◮ Can we exploit programmers’ knowledge of the system?

Extensive testing◮ Java, DaCapo benchmarks

26 / 27

Performance Annotations for Cloud Computing

Daniele Rogora∗ Steffen Smolka% Antonio Carzaniga∗ Amer Diwan$

Robert Soulé∗∼

presented by

Daniele Rogora

∗Università della Svizzera italiana %Cornell University $Google ∼Barefoot Networks

HotCloud 2017

27 / 27