CloudSimDisk: Energy-Aware Storage Simulation in...

2015 Summer School on GREEN ICT organized by2015 Summer School on GREEN ICT organized by PERCCOM (co-funded by Erasmus+ Programme)

Nancy, June 16th - 2015

CloudSimDisk: Energy-Aware

Storage Simulation in CloudSim.Master Thesis

Author:Baptiste Louis

Supervisors:Professor Christer ÅhlundDoctor Karan MitraDoctor Saguna Saguna

Luleå University of Technology, Skellefteå (SWEDEN)

2015 Summer School on GREEN ICT organized by2015 Summer School on GREEN ICT organized by

Presentation

2015 Summer School on GREEN ICT organized by

3

AGENDA

CloudSimDisk: Energy-Aware Storage Simulation in CloudSim3

Context & Thesis Overview1

Research Challenges & Literature Review2

Simulation Results (a case study)3.2

Conclusion & Future Work4

Processing & Modeling3.1


4

Context

1 2 3 4

“Digital Revolution”(Began in 1950/1970)

“Information Age”(Present time)

8% of global electricity consumption in 2008[1]

14% in 2020

4.4ZB (2013)

“Digital Universe” by IDC [2]

2% of global C02 emissions[3]

15% of global emissions in 2020[4]

x2.7


5

Thesis Overview

Thesis topic: energy efficient data storage in data center.

Thesis objective: identify and develop a method to study

the energy efficiency of the data center storage.

Storage Workload Trace

Hardware Configuration

Storage Management Rules

Power Requirements

1 2 3 4

What is the energy consumed by the data center storage?


6

Research Challenges

Cloud Computing

Ubiquitous Computing

Data Growth

Research Questions: HOW TO

• Study of Energy Efficient Data Storage?

• Management of Large Scale System?

• Energy Aware System?

IoT Network

Software

1 2 3 4


7

Literature Review

Energy efficient storage:

HDD well-established technology, widely used in data center.

» OCP* states: HDD RPM** ÷ 2 = save roughly 3-5W per HDD[5] ;

» Efficient hardware (helium-filled drives)[6] ;

» Increase areal density (SMR)[7] ;

» Efficient storage management (RAID5 or RAID6 cache)[8] ;

» Long term data storage, with very low access frequency (cold storage)[9].

ReaderWriter

1 2 3 4

* OCP = Open Compute Project

** RPM = Revolutions Per Minute


8

Cloud Simulators:

» Analyzed 19 Cloud Simulators*

• MDCSim[10] (commercial)

• GreenCloud[11] (slow, unpleasant)

• CloudSim[12] (“most popular”, “most sophisticated event-based simulator”)**

* Analyzed summarized in the Master Thesis, table.1.

** References available in the Master Thesis, page 18.

Literature Review

Modeling and Simulation:

» control on all the layers (hardware, policies, virtualization, etc.)

» stable (scenarios can be repeated), scalable (easy to expand)

» Collaborative (similar environment, reproduce experiments)

» time-efficient (discrete-time simulator)

» cost-efficient (avoid expensive HW and SW + maintenance)

» energy-efficient (avoid real Cloud consumption) => ICT for greening

1 2 3 4


9

Literature Review

CloudSim v3.0.3:

1 2 3 4

*Cloudlets = requests


10

CloudSimDisk: Energy-Aware Storage

Simulation in CloudSim

1 2 3 4

…

>> 12 Packages, 124 Classes. >> 8 Packages, 27 Classes.


11

CloudSimDisk processing

CloudSimDisk

Data Center Storage

RequiredFiles(Files to retrieve)

Filename | FilesizeFile_a 1 MBFile_b 3 MBFile_c 7 MB… …

DataFiles(Files to store)

Filename | FilesizeFile_A 1 MBFile_B 3 MBFile_C 7 MB… …

DistributionTime0.311 s 0.321 s 0.356 s

… s

1 2 3.1 4

Storage Workload Trace


12

CloudSimDisk processing

Final Result Energy consumed by

the whole persistent storage (Joule)

Energy consumed in each power mode for each HDD (Joule)

Time spent in each mode for each HDD (second)

Idle intervals for each HDD (second)

Excel spreadsheet

For each operation:• Detailed transaction times• Energy consumed• Target HDD• Filename and File size

Pre-built graphs

Simulation logs• Initial persistent storage• Event-passing• HDD states + queue size• Read/Write transactions• Final result

Real-time evolution of the simulation

Final results

Text logs

IDE Console

1 2 3.1 4


13

CloudSimDisk modeling

INPUTsTime | Filename | Filesize1.0 s File_A 1 MB2.0 s File_B 3 MB3.5 s File_C 7 MB… … …

1 2 3.1 4

Cloudlets = requests


14

Simulation Results: a case study

Configuration

Wikipedia Workload

5000 requests (req)

1 file to store/req

File [1 - 10] MB

1 HDD

HGST UltrastarHUC109090CSS600

1 2 3.2 4

153reqs/0.05s ≈ 3000 reqs.s-1

C:/Users/baplou/Dropbox/cloudSim_Res1428929536729.xls


15

0123456789

1011

0.000

0.002

0.004

0.006

0.008

0.010

0.012

0.014

0.016

0.018

0.020

0.022

0.024

0.026

0.028

0.030

0.032

0.034

0.036

0.038

0.040

0.042

0.044

0.046

0.048

0.050

0.052

0.054

Fil

e S

ize

(M

B)

Transfer Time (s)

Transfer Times compare with file sizesFile size (MB)

Linear regression


SeekTime+ RotationLatency+ TransferTime----------------------------= Transaction Time

1 2

3

1

2

3

1 2 3.2 4

0.0040 second0.0030 second

y= 198x


16

0123456789

1011

0.000

0.002

0.004

0.006

0.008

0.010

0.012

0.014

0.016

0.018

0.020

0.022

0.024

0.026

0.028

0.030

0.032

0.034

0.036

0.038

0.040

0.042

0.044

0.046

0.048

0.050

0.052

0.054

Fil

e S

ize

(M

B)

Transfer Time (s)

Transfer Times compare with file sizesFile size (MB)

Linear regression


SeekTime+ RotationLatency+ TransferTime----------------------------= Transaction Time

1 2

3

1

2

3

1 2 3.2 4

0.0040 second0.0030 second

y= 198x


17


Data center storage management rules ?

1 2 3.2 4


18


1 2 3.2 4

Request


19

Conclusion and Future Work

Conclusion

» Proposed a method to study energy efficient data storage in data center

» Simulation results in accordance with the HDD models.

» Scalable module for future improvement

» Work presented to and

1 2 3 4

Future work

» Send a paper to UCC 2015 (Utility and Cloud Computing)

» Compare simulation results with real world measurements (Fortlax interested)

» Analyze CloudSim and CloudSimDisk with GREENSPECTOR.

» CloudSimDisk development (Discussions with Facebook, Fortlax and Rodrigo)

• Implement Storage cache

• Implement Disk fragmentation

• Use CloudSim Network package


20

REFERENCES

[1] Mario Pickavet et al. Worldwide energy needs for ict: The rise of power-

aware networking. In Advanced Networks and Telecommunication

Systems, 2008. ANTS'08. 2nd International Symposium on, pages 13.

IEEE, 2008.

[2] John Gantz and David Reinsel. Extracting value from chaos. IDC iview,

IDC(1142):910, 2011.

[3] Mingay Simon et al. Green IT: The New Industry Shockwave. Gartner

inc. April 2007.

[4] Global e Sustainability Initiative et al. Smart 2020 report: Global ict

solution case studies. The Climate Group, Tech. Rep, 2008.

[5] Eran. Saving data center power by reducing hdd spin speed.

http://www.opencompute.org/blog/saving-data-center-power-by-reducing-

hdd-spin-speed/, August 2011. Accessed: 2015.03.13


21

REFERENCES (continued)

[6] Aruga, Keiji, et al. "A study on positioning error caused by flow induced

vibration using helium-filled hard disk drives." Magnetics, IEEE

Transactions on 43.9 (2007): 3750-3755.

[7] Seagate. Breaking capacity barriers with seagate shingled magnetic

recording. http://www.seagate.com/gb/en/tech-insights/breaking-areal-

density-barriers-with-seagate-smr-master-ti/, May, 06 2015. Accessed:

2015.05.06.

[8] Jiguang Wan et al. A new high performance, energy-efficient replication

storage system with reliability guarantee. In Mass Storage Systems and

Technologies (MSST), 2012 IEEE 28th Symposium on, pages 16. IEEE,

2012.

[9] Arie Taal et al. Storage to energy: modeling the carbon emission of

storage task offloading between data centers. In Consumer

Communications and Networking Conference (CCNC), 2014 IEEE 11th,

pages 5055. IEEE, 2014.


22

REFERENCES (continued)

[10] Tiffany Trader. Blu-ray for cold storage, what about tape? http://

www.hpcwire.com/2014/02/13/blu-ray-cold-storage-tape/, February, 13

2014. Accessed: 2015.05.07.

[11] Seung-Hwan Lim et al. Mdcsim: A multi-tier data center simulation,

platform. In Cluster Computing and Workshops, 2009. CLUSTER'09. IEEE

International Conference on, pages 19. IEEE, 2009.

[12] Dzmitry Kliazovich et al. Greencloud: a packet-level simulator of

energy-aware cloud computing data centers. The Journal of

Supercomputing, 62(3):12631283, 2012.

[13] Rodrigo N Calheiros et al. Cloudsim: a toolkit for modeling and

simulation of cloud computing environments and evaluation of resource

provisioning algorithms. Software: Practice and Experience, 41(1):2350,

2011.


23

Thank you all for your attention.


24

APPENDIX


25

APPENDIX

Source: The POOG. What is your carbon footprint? July, 02 2012. Accessed: 2015.05.15. Link: http://thepoog.com/?p=2732

Ice core drilling

Vostok& EPICA

Context: environmental


26

Software Implementation


27

Software Implementation


28

APPENDIX

CloudSim storage projects


29

APPENDIX

Analytical model


30

APPENDIX

Simulation results compare with analytical calculation


31

APPENDIX

1 2

3 SeekTime+ RotationLatency+ TransferTime----------------------------= Transaction Time

1

2

3

Simulation Results: study case


32

APPENDIX

Console output example


33

APPENDIX

GitHub


34

APPENDIX

Cloud Simulators


35

APPENDIX

Event-based Simulation


36

APPENDIX

Flow processing


37

APPENDIX

Flow processing


38

APPENDIX

Flow processing


39

APPENDIX

Transaction Times


40

Reduce global emissions

APPENDIX

2

Context: ICT


41

APPENDIX

HDD characteristics


42

APPENDIX

Seek time distribution: [0 ; 3×Avg] with a average of Avg.

http://faculty.plattsburgh.edu/jan.plaza/teaching/papers/seektime.html

http://www2.cs.uh.edu/~paris/7360/PAPERS03/IEEEComputer.DiskModel.pdf


http://www2.cs.uh.edu/~paris/7360/PAPERS03/IEEEComputer.DiskModel.pdf


43

APPENDIX

Rotation Latency distribution: [0 ; 2×Avg] with a average of Avg.




44

APPENDIX

Zoned Bit Recording(ZBR): same size of sectors


45

APPENDIX

Eco-design:

» Event-base simulator (avoid inefficient loops when no changes

occur).

» Use of Integer limited because it affect the accuracy of the

parameters and/or increase the complexity.

» Avoid method call in loop condition because it is greener and more

clear in the code.


46

APPENDIX

Scalability:

» Storage Management Rules

» HDD characteristics

» Power mode

» Time Distributions

» Respect CloudSim architecture

CloudSimDisk: Energy-Aware Storage Simulation in...

Documents

Transcript of CloudSimDisk: Energy-Aware Storage Simulation in...