Hardware- and Network-Enhanced Software Systems for Cloud Computing, OW2 Open Cloud Forum at Cloud...
13
http://www.harness-project.eu/ The HARNESS Project: Hardware- and Network-Enhanced Software Systems for Cloud Computing Prof. Alexander Wolf Imperial College London (Project Coordinator)
-
Upload
ocean-project -
Category
Technology
-
view
229 -
download
1
description
Cloud platforms are missing out on the revolution in new hardware and network technologies for realising vastly richer computational, communication, and storage resources. Technologies such as field-programmable gate arrays (FPGA), general-purpose graphics processing units (GPGPU), network middleboxes, and solid-state disks promise increased performance, reduced energy consumption, and lower cost profiles. However, their heterogeneity and complexity makes integrating them into the standard cloud Platform as a Service (PaaS) framework a fundamental challenge. This talk introduces the HARNESS project, whose aim is to explore how to bring these innovative and heterogeneous resources into cloud platforms.
Transcript of Hardware- and Network-Enhanced Software Systems for Cloud Computing, OW2 Open Cloud Forum at Cloud...
http://www.harness-project.eu/
The HARNESS Project:
Hardware- and Network-EnhancedSoftware Systems for Cloud Computing
Prof. Alexander WolfImperial College London
(Project Coordinator)
http://www.harness-project.eu/
HA
RN
ES
SH
AR
NE
SS
Software as a ServiceSoftware as a Service
Cloud Market Strata
PaaS
SaaS
Infrastructure as a ServiceInfrastructure as a ServiceIaaS
Platform as a ServicePlatform as a Service
http://www.harness-project.eu/
Provider prospective– minimise ownership costs– maximise usage– maximise market growth
PaaS Design Drivers
Application perspective– minimise development costs– minimise operating costs– maximise performance
standardised APIs
optimal deployment
commoditised resources
virtualised resources
scale out / scale up
data-centre expansion
http://www.harness-project.eu/
Provider prospective– minimise ownership costs– maximise usage– maximise market growth
PaaS Design Drivers
Application perspective– minimise development costs– minimise operating costs– maximise performance
standardised APIs
optimal deployment
commoditised resources
virtualised resources
scale out / scale up
data-centre expansion
Application requirements ExamplesFast job completion time with interdependent “big data”
scientific computingtime-series analysis
Fresh results within seconds on-line information retrievalon-line data analytics
State-of-the-art: Optimised for horizontal scale-out
over homogeneous resources But insufficient for many applications
http://www.harness-project.eu/
SoftwareswitchesFPGAsGPUs ASICs SSDsNetwork
middleboxes . . .
Provider prospective– minimise ownership costs– maximise usage– maximise market growth
An Innovative Approach to PaaSthe HARNESS project premise
Application perspective– minimise development costs– minimise operating costs– maximise performance
standardised APIs
optimal deployment
commoditised resources
virtualised resources
scale out / scale up
data-centre expansion
specialised resources
specialised resources
http://www.harness-project.eu/
Goal: Programmable and Manageable
GPU-based parallel-thread
engines
FPGA-based shared dataflow
engines
Solid-statedisk drives
ASIC-based OpenFlow
switching fabric
Middleboxes forin-network aggregation and storage
http://www.harness-project.eu/
Approach: Enrich IaaS and PaaS
Provide an IaaS layer that can manage heterogeneous resources– computation, communication and storage– resource allocation and scheduling
Provide a PaaS layer that can exploit heterogeneous resources– multi-tenancy– application development– cross-resource allocation and scheduling
http://www.harness-project.eu/
Driving Use Casesbasis for demonstration and validation
shared memory
cache cache cache
CPUs CPUs CPUs…
I/O
Delta Merge for SAP HANAin-memory OLTP and OLAPqueries for “big data” analytics
Reverse Time Migration (RTM)scientific computation for thegeosciences
……
f1f1
fnfn
y E {−1,1} y E {−1,1}
predictpredict
updateupdate
…
f1
fn
y E {−1,1}
predict
update
…
Share state
Aggregate
Iterate
…
Parallelize
…
Preprocess AdPredictor Machine Learningopen-source “map/reduce”data-flow distributed computation
http://www.harness-project.eu/
Driving Use Casesbasis for demonstration and validation
shared memory
cache cache cache
CPUs CPUs CPUs…
I/O
Delta Merge for SAP HANAin-memory OLTP and OLAPqueries for “big data” analytics
Reverse Time Migration (RTM)scientific computation for thegeosciences
……
f1f1
fnfn
y E {−1,1} y E {−1,1}
predictpredict
updateupdate
…
f1
fn
y E {−1,1}
predict
update
…
Share state
Aggregate
Iterate
…
Parallelize
…
Preprocess AdPredictor Machine Learningopen-source “map/reduce”data-flow distributed computationO(109) entries in daily Web visit logO(109) entries in daily Web visit log
two weeks on 300 multi-core nodestwo weeks on 300 multi-core nodes
20% of cycles and 10s of seconds locking20% of cycles and 10s of seconds locking
http://www.harness-project.eu/
AdPredictor Training Process
http://www.harness-project.eu/
HetMR: Heterogeneous MapReduce
MapReduce deployment and execution system for hybrid CPU/accelerator environments
Map on GPGPUReduce on CPUMap on GPGPUReduce on CPU
Map on CPUReduce on FPGA
Map on CPUReduce on FPGA
http://www.harness-project.eu/
Research Focus Areas
Application design
Performance predictionand cross-resource mgmt.
Resource virtualisation
Individual-resource mgmt.
http://www.harness-project.eu/
Mike’s Storyhttp://www.harness-project.eu/
