Post on 03-Apr-2020
Energy Efficiency in Cloud Computing and Optical Networking
Rod Tucker, Kerry Hinton, Rob Ayre
Centre for Energy-Efficient TelecommunicationsUniversity of Melbourne
ECOC 2012 Tutorial 2
Outline
• Introduction and overview of energy consumption and
efficiency in communications networks
• Estimating energy consumption in ICT equipment
– Telecommunications
– The “cloud”
• Improving network energy efficiency
– Technologies
– Architectures
– Protocols
– The cloud
ECOC 2012 Tutorial 3
World’s technology capacity
Source: M. Hilbert, P. Lopez, Science, 2011
1E+18
1E+19
1E+20
1E+21
1E+22
1E+23
1E+24
1985 1990 1995 2000 2005 2010 2015
Storage (bits)
Telecommunications
(bits/year)
1024
1023
1022
1021
1020
1019
1018
Ca
pa
city (b
its, b
its/y
ea
r)
ECOC 2012 Tutorial 4
Internet traffic growth trends
Co
mp
ou
nd
Ave
rage
Gro
wth
Rat
e (%
)
Source: Kilper et al., JSTQE 2011
ECOC 2012 Tutorial 5
Source : Cisco Cloud Index 2011
0
1000
2000
3000
4000
5000
2010 2011 2012 2013 2014 2015
Exab
yte
s/an
nu
m
Year
Data Centre Traffic 2010-2015
Datacentre to user
Between Datacentres
Within Datacentres
Exab
ytes
/an
nu
m
Data Center to User
Between Data
Centers
Within Data
Centers
Projections of data centre traffic
ECOC 2012 Tutorial 6
Po
we
r C
on
su
mp
tio
n (
W)
Year
40% p.a. Data growth
10% p.a. Growth in user numbers
Power consumption of the global Internet
2010 2015 2020
109
1011
1010
1012
Global electricity supply
1013
Power Consumption of Internet
(Including servers)
1.5 billion users
ECOC 2012 Tutorial 7
Why does energy matter?
• If nothing is done to address the growing “efficiency gap”:
– ICT will consume ever larger proportion of global energy
– Energy consumption could become a barrier to network growth
• Economic and engineering imperatives:
– Energy is a growing component of OPEX
– Increased energy consumption increased footprint
ECOC 2012 Tutorial 8
Estimating ICT power consumption
• Inventory based estimates
– Look at what is out there
• Use sales and deployment data from vendors and surveys
• Accessing accurate data is problematic
• Network design and dimensioning based estimates
– Design a network that will satisfy current and projected demands
• Use typical network design rules
• Difficult to include network inefficiencies, overlays & legacies
• Transaction based estimates
– Look at services required and design a network to provide them
• Similar to network design approach
ECOC 2012 Tutorial 9
Summary of estimates
Author Year
% national
electricity
use
Country
PC’s,
office
equip. &
servers
Wireless
accessNotes
Huber 1999 13% USA Yes No Severe over estimate
Koomey 1999 2% USA Yes No Users & equipment estimate
Kawatomo 2001 3% USA Yes No Users & equipment estimate
Turk 2001 0.5 - 1.7% Germany Yes No Users & equipment estimate
Barthel 2001 0.9 – 1.5% Germany Yes Yes Users & equipment estimate
Roth 2002 < 2.3% USA Yes Yes Users & equipment estimate
Cremer 2003 7.1% Germany Yes Yes Users & equipment estimate
Baliga 2007 0.5% OECD No No Network design & dimensioning
Vereecken 2010 Not given Not given No No Network design & dimensioning
Lange 2010 Not given Not given No Yes Network design & dimensioning
Kilper 2011 Not given USA No Yes Transaction
Pickavet 2007
2012Global Yes Yes Users and equipment estimate
ECOC 2012 Tutorial 10
Design and dimensioning approach
1. Split network into
– Access
– Metro/Edge
– Core
– Data centres, content storage
2. Model with representative architecture and equipment
3. Dimension network to accommodate expected traffic
4. Calculate power consumption per customer for network
Baliga et al., 2007
ECOC 2012 Tutorial 11
Key model parameters
• Peak vs average access speed– Contention & aggregation
• Network dimensioning
– Traffic growth
• Deployed capacity > demanded capacity
– Equipment redundancy
• Multi-homing , back-up storage
– Service protection
• 1 + 1, 1:1, 1:N protection
• Router hops between source and destination
• Data centre Power Usage Effectiveness (PUE)
Time
Tra
ffic
Peak
Average
ECOC 2012 Tutorial 12
Ethernet
Switch
OLTSplitter
Metro/Edge NetworkCore Network
Edge Routers
FTTP
FiberCore Router
Content Distribution Network
Storage
Server
Server
Storage
Fiber
Access Network
DSL
DSLAMCu
OLT
ONU
Cabinet
FTTN
DSLAM
Cu
Broadband Network Gateways
Hot
spots
PON
Data Center
Network segmentation
ECOC 2012 Tutorial 13
Access network energy consumption30
Peak Access Rate (Mb/s)
Po
wer
Per
User
(W)
1 1000
0
FTTP (PON)
100
20
10
10
FTTN
Wireless
HFC
PON is “greenest”
Baliga et al., OFC 2009
ECOC 2012 Tutorial 14
Data centers and content servers
Racksof Servers
Aggregation Switches
Load-Balancing Switches
Border Routers
RacktopSwitches
80% of traffic stays in data center
5% of traffic to other data centers
15% of traffic to users
ECOC 2012 Tutorial 15
Power consumption of equipment
• Which metric(s) are appropriate?
– Power consumption per “throughput”
– Power consumption per “good put”
– Energy per bit, power per bit rate
– Energy per customer bit
– etc.
• Different metrics provide different optimal solutions to
energy efficiency
• Total power and energy/bit are widely used
ECOC 2012 Tutorial 16
Power consumption in routers
Sources: METI, 2006, Nordman, 2007
Router Throughput
Pow
er
consum
ption
(W)
10
100
1,000
10,000
100,000
1,000,000
P = C2/3
where P is in Watts
where C is in Mb/s
5 nJ/bit
100 nJ/bit
1
1 Pb/s
P ~ 10
1 Tb/s1 Gb/s1 Mb/s
?
ECOC 2012 Tutorial 17
Energy efficiency of equipmentE
nerg
y p
er
bit (
nJ)
0.01
0.1
1
10
100
MEMS
OXC
Optical
Amp
PIC
Tx/Rx
WDM
Tx/Rx
Ethernet
Switch
Core
Router
FEC
Chip
0.001
Sub-wavelength Wavelength
2010 Data
Tucker et al., 2010
ECOC 2012 Tutorial 18
Equipment Energy Consumption Trends
1
10
100
1000
10000
1985 1990 1995 2000 2005 2010 2015
nan
o-J
ou
les
pe
r b
it
Year
Router Energy Efficiency
Cisco AGS
Wellfleet BCN
Cisco GSR 12000
Cisco GSR 12000b
Avici TSR
Cisco CRS 1
Cisco CRS-3
ALU7750
Actual improvement
may be declining
Linear fit gives ~25% improvement pa
Nielsen, ECOC 2011
ECOC 2012 Tutorial 19
Trends in transport energy consumption
First Trans-Atlantic
Marconi
Trans-Atlantic
Fessenden
Trans-Atlantic
NY - Paris
Key West - Havana
TAT-1
TAT-3
TAT-5TAT-8
TAT-9
TAT-10
TAT-11
TAT-12/13
Newhaven - Azores
10-6
1840 1860 1880 1900 1920 1940 1960 1980 2000 2020
Year
En
erg
y/B
it/1
000 k
m (
mJ)
Wireless
Telegraphy
Coax
Optical + Regen
Optical + EDFA
10-4
104
10-2
102
106
108
1
~15% improvement p.a.Current
Tucker, JSTQE 2011
ECOC 2012 Tutorial 20
Putting it all together
• Design and dimensioning approach
• 40% p.a. growth in network traffic
• 10% p.a. growth in user numbers
• 15% p.a. improvement in all technologies
• Projections of data centre traffic
ECOC 2012 Tutorial 21
Power consumption of the global Internet
Year
15% p.a. technology improvement
Po
we
r C
on
su
mp
tio
n (
W)
109
1011
1010
108
1012
2010
2015
2020
Access (PON)
Global electricity supply (3% p.a.)
TotalTotal (2010 Technology)
ECOC 2012 Tutorial 22
Network energy per user bit
Year
En
erg
y p
er
Use
r b
it (m
J)
1.0
100
0.1
10
0.01
2010 2015 2020
Total
PON
Core and Metro
Switches/Routers
Optical Transport
0.1
1
10
100
Avera
ge A
ccess R
ate
(M
b/s
)
ECOC 2012 Tutorial 23
Gap between theory and practice
Tucker, JSTQE 2011
10-6
10-9
10-10
10-8
10-11
10-7
10-5
10-12
Current
Trends
Lower Bounds
x 104
Year2010 2015 2020 2025
Netw
ork
Energ
y p
er
Bit (
J)
Transport
Switches
Transport
Access
10-4
Global Network
X 103
X 102
ECOC 2012 Tutorial 24
Gap between theory and practice
Management and control overheads, interconnects,
power supplies, etc.
Protocols, device efficiency, interconnects, system penalties,
system margins, etc
Subsystem
PTotal
Function
Overhead
Poverhead
Pfunction
Inefficiency Traffic (b/s)
Pow
er
consum
ption
Traffic (b/s)
Energ
y p
er
bit
Ideal
Ideal
ECOC 2012 Tutorial 25
Improving network energy efficiency
• Many ideas for improving energy efficiency
– Insufficient time to cover all of them
• Key approaches
– Technologies
– Architectures
– Protocols
– The cloud
ECOC 2012 Tutorial 26
A. Technologies
• Fundamental physical technologies for telecommunications:
– Electronics: primarily CMOS for signal and data processing and
storage - Improvements by Moore’s Law
– Optics/photonics, primarily used to transport data
– More than 99% of network energy is consumed by electronics
• Advances are needed in
– Optical and electronic switch technologies
– Optical and electronic interconnects at all levels
ECOC 2012 Tutorial 27
B. Architectures
• Architectures that reduce the number of network hops
• Optical bypass
• Layer 2 rather than Layer 3 where possible
• Dedicated content-delivery networks
ECOC 2012 Tutorial 28
Bypass optionsWithout bypass:
• All traffic goes to IP layer for processing
• 10 nJ per bit
• Allows aggregation of incoming traffic flow
• Statistical multiplexing increases utilisation of paths
WDM
LinksWDM
Links
IP
Patch
panel
IPTime
Pa
cke
ts
Time
Pa
cke
ts
Time
Pa
cke
ts
Time
Pa
cke
ts
Peak rate
(channel
capacity)
ECOC 2012 Tutorial 29
With bypass:
• TDM Layer
– Some traffic streams processed at TDM level
– ~ 1 nJ per bit
• WDM layer
– Some traffic switched at WDM layer
– < 0.1 nJ per bit
WDM
OXC
IP
TDMTDM layer
bypass
WDM layer
bypass
TDM
WDM
OXC
Bypass options (cont’d)
ECOC 2012 Tutorial 30
C. Protocols
• Service transactions and protocols
• Efficiency of multi-layer protocol suite
• XGPON framing
• Sleep and standby states
• Energy-efficient Ethernet
• Dynamic rate adaption
ECOC 2012 Tutorial 31
Service transactions & protocolsService based energy model reflects how services are transported through the network
NIC
IP
TCPUDP...
Ap
p 1
Ap
p 2
Ap
p 3
NIC
IP
TCPUDP...
Ap
p 1
Ap
p 2
Ap
p 3
NIC
IP
TCPUDP...
Ap
p 1
Ap
p 2
Ap
p 3
NIC
IP
TCPUDP...
Ap
p 1
Ap
p 2
Ap
p 3
NIC
IP
TCPUDP...
Ap
p 1
Ap
p 2
Ap
p 3
NIC
IP
TCPUDP...
Ap
p 1
Ap
p 2
Ap
p 3
End users
Eth Eth
Aggregation Edge
Eth Eth
IP
IP
L2/L1
L2/L1
CoreNetwork
node
node
node
node
Source: Zhong et al., OFC 2012
ECOC 2012 Tutorial 32
Efficiency of multi-layer protocol suite
Combining boxes for multi-layer protocol suites can improve efficiency
Encapsulation of
small packets
Packet
fragmentation
Multiple
O/E/O
IP/Ethernet
IP/COE/WDM
(integrated box)
IP/COE/WDM
(separate boxes)
IP
Eth
WDMWDM
LinkWDM
Link
PPP
IP
SDH
WDMWDM
LinkWDM
Link
IP
COE
WDM
Link
WDM
Link
IP
WDM
OXC
COE
Efficiency =Customer payload IP layer energy
Total multi-protocol suite packet energy
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
20 200 2000 20000
Customer payload (Bytes)
Effic
ien
cy IP/PPP/SDH/WDM
Source: Zhong et al., OFC 2012
XGPON DS Framing Structure
Port-ID
Chow et al.,
ECOC 2012
XG-PON framing protocols
ONU 1
TxOLT-MAC
(FPGA) Rx
ONT-FPGA
UN
I
Line card10Gbit/s
ONU 2Bit-interleaved data
BIPONDS
BI
DS
ONU 1
TxOLT-MAC
(FPGA)Rx
ONT-FPGA
UN
I
Line card10Gbit/s
ONU 2Packet data
XGPONDS
XGPON
DS
Deser
Deser
Delta
Solution: BiPONBit-Interleaved PON
Conventional PON
~ 5 W /user
< 200 mW /user
Chow et al., ECOC 2012
ECOC 2012 Tutorial 35
Energy-efficient protocols• Sleep & standby states
– Network devices enter low power state when not in use
– Can apply to systems and sub-systems
– Need to ensure network presence is retained
• Use Network Connection Proxy with sleep protocol
– Need to account for state transition energy and time
– May have multiple lower energy states
• IEEE Energy Efficient Ethernet (802.3az)
– Low power idle mode when no packets are being sent
– Approved Sept. 2010
– Currently applies to copper interface only; not optical
ECOC 2012 Tutorial 36
PON burst-mode laser driverKoizumi et al., ECOC 2012
• Driver turned off between bursts
• Power reduced by 93%
ECOC 2012 Tutorial 37
Energy-efficient protocols• Dynamic rate adaptation
– Modify capacity of network devices in response to traffic demands
– Change clock frequency, processor voltage
– Slower speed to reduce power consumption
– Need to allow transition time between rates
2Power C Voltage Frequency
Power
Packetstime
time
time
No protocol
Sleep state
Adaptation
Both
Bolla et al., 2011
time
• Dynamic rate adaptation and standby states can be combined
ECOC 2012 Tutorial 38
• Cloud services widely promoted as greener than on-site facilities:
– Cloud Computing – The IT Solution for the 21st Century
• Carbon Disclosure Project Study 2011
– Salesforce.com & the Environment
• WSP Environment & Energy 2011
• Strong case for enterprise private cloud
• What about the public cloud?– Apple iCloud
– Google drive
– Microsoft sky drive
D. The cloud
ECOC 2012 Tutorial 39
0.01
0.1
1
10
100
0.01 0.1 1 10 100
Po
we
r C
on
su
mp
tio
n (W
)
Downloads per Hour
Transport
energy dominates
1 GB Cloud Storage
5 GB Cloud Storage
Local HDD
Example: Public storage as a service (SaaS)Storage of application & data “in the cloud” compared with storing on a local disk.
50 MBytes per download. Modern laptop-style HDD 20% read/write and 80% idle.
Baliga et al., Proc IEEE, 2011
ECOC 2012 Tutorial 40
Rethinking the “Green Cloud”
• Need to improve access energy efficiency
– Small wireless cells
– PON
• Keep some processing power in user’s device
• Reduce the number of router hops
– Avoid public Internet
– Use optical layer by-pass of routers
• Improve protocol efficiency
– Less overhead bytes
– Smart scheduling
ECOC 2012 Tutorial 41
Ethernet
Switch
OLTSplitter
Cabinet
Metro/Edge NetworkCore Network
Edge Routers
PON
Fiber
EDFA
Core Router
Distant
IPTV Server
Fiber
Broadband Network Gateways
“Local”
IPTV Server
Server
Storage
• Location of movies is a trade off
between energy for storage and
energy for delivery
• Each movie is replicated on R
servers throughout the network
Direct optical link to edge routerServer
Storage
IPTV over the public Internet
ECOC 2012 Tutorial 42
P2P
En
erg
y p
er
do
wn
load
(W
h)
(lo
g s
cale
)
10
100
104
Downloads per hour
10310210110010-110-2
R = 2000R = 200R = 20
R = 2
IPTV over the public Internet
Baliga et al., OFC, 2009
ECOC 2012 Tutorial 43
Total
Storage
Transport
R = 20
Po
wer
per
mo
vie
(W
)
100
103
10-1 101 10-2
Downloads per hour10-2
101
102
Video
Servers
100
IPTV over the public Internet
Baliga et al., OFC, 2009
ECOC 2012 Tutorial 44
Conclusions
• Energy consumption of the network is growing
• Access network energy dominates– Servers in data centres are likely to become dominant in ~2015
– Core and metro networking to overtake access in ~2020
– Optical transport is relatively “green”
– Beware “the cloud”
• Many opportunities for improving network energy efficiency– Technologies
– Architectures
– Protocols
• If you are inspired, join one of the networks or consortia:
www.greentouch.org, www.fp7-trend.eu