Optical Networking Interconnecting Disaggregated Compute Resources: An

enabler of the 5G Vision

AnnaTzanakaki(1),(2),Markos P.Anastasopoulos(1),DimitraSimeonidou(1)

(1) UniversityofBristol,UK(2)NationalandKapodistrian UniversityofAthens,DepartmentofPhysics,Greece

Outline

§ Motivation 5G vision

§ Proposed 5G architecture

§ Optical Networking Interconnecting Compute Resources

Inter- and Intra-DC connectivity

Disaggregation of Resources

Increase granularity in the control of resources and utilisation efficiency

§ Modelling results: power consumption

§ Conclusions

1

Motivation - 5G Vision• Newbusinessopportunitiesmeetingtherequirementsoflargevarietyofusecases

(enablingverticals)• 5GNetworkstobefutureproof(i.e.designtoevolveandchangeandnottobereplaced)

• bymeansofimplementingnetworkslicinginacostefficientway

– addressingbothenduserandoperationalservices

– supportingnativesoftwarization

– integratingcommunicationandcomputation

– integratingheterogeneoustechnologies(incl.fixedandwirelesstechnologies)

5G PPP View on 5G Architecture – (White Paper), https://5g-

ppp.eu/white-papers/

2

Supporting operational and end-user services

EPCMME/SGSN

P-SGW/GSN InternetMacro-cell

IndoorWireless

OpticalTransport

WDM-PON

SmallCells

LTE

BBUBBU

BBUBBU

BBUBBU

BBUBBU

MobileFronthaul MobileBackhaul

DataCenters

RRHRRHRRH

RRHRRHRRH

VM

vBBUvBBU Internet

Fronthaul

OpticalTransportDataCenters

BBU

RRHRRHRRH

BackhaulWirelessAccess

WiFi

SmallCells

• Early solutions: Separate Fronthaul and Backhaul networks• Fronthaul: Network between RRHs and dedicated BBUs

• Backhaul: Between the core network and the sub-networks at the edge

• C-RAN: Sharing of BBUs but Separate Fronthaul and Backhaul networks• Proposal: Adopt C-RAN and offer a common transport to support jointly

converged FH and BH functionalities.• high transport bandwidth

• Strict delay constraints• enhanced synchronisation capabilities

• flexibility and elasticity in bandwidth allocationA. Tzanakaki et al., Wireless-Optical Network Convergence: Enabling the 5G Architecture to Support Operational and End-User Services, IEEE Commun. Mag. 2017 (in press) 3

Flexible Functional Splits

RFtoBaseband

CyclePrefix&FFT

Resourcedemapping

Receiveprocessing

Decoding

MAC

(1)

(2)

(3)

(4)

(5)

Clou

dRA

NTrad

ition

alR

AN

• highnetworkbandwidth• IncreasedBBUsharing

• lownetworkbandwidth• LimitedBBUsharing

• FlexiblesplitprocessingtorelaxthestringentrequirementsofFHservicesfor:– transportcapacity– delay– synchronisation

• SplitOptions– Split(1):“CPRI”– Split(2):“load-dependentFH”– Split(5):“BHwithcentralizedscheduling”– Processingcanbealsoperformedatgeneralpurposeservers

A. Tzanakaki et al., "5G infrastructures supporting end-user and operational services: The 5G-XHaul architectural perspective," IEEE ICC, 2016.

4

Overall Infrastructure

• Key Challenge: • Converged Fronthaul and Backhaul Services• Converged Network and Compute Services• Disaggregation of Resources

EdgeNode

RFtoBaseband

CyclePrefix&FFT

Resourcedemapping

Receiveprocessing

Decoding

MAC

(3)

(4)

(5)

• highnetworkba

ndwidth

• IncreasedBB

Usharing

• lownetworkband

width

• Limite

dBB

Ush

aring

(2)

(1)

FastSwitch

moduleλ1

FastSwitch

moduleλ2

FastSwitch

moduleλΝ

MUX

MUXBV-WSS

BV-WSS

BV-WSS

FastSwitch

moduleλ1

FastSwitch

moduleλ2

FastSwitch

moduleλΝ

MUX

MUX

b)

c)

RU:RemoteUnit

ProcessingattheServerRU

ProcessingattheRU

BVT:BandwidthvariabletransponderBV-WSS:Bandwidthvariablewavelengthselectionswitch

CoreNode

OpticalLink

a)

RU1

RU3 Core

Node

CoreNode

EdgeNode

EdgeNode

SPP

SPP

GPP

GPP

ToR

ToR

BVT

λ1

λ2

λΝ

EdgeNode

SPPSPPSPP

SPP

ToR

SPP

CPUCPU

SPP

ToR ToR

RU2

(5)

(2)

ElasticOpticalNetwork DataCentre

Tx

SFP RUETHTx

I/QTx

SFP RUETHTx

I/QTx

SFP RUETHTx

I/QTx

SFP RUETHTx

I/QTx

SFP RUETHTx

I/QTx

BVT

BVT

λ1

λ2

λΝ

λ1

λ2

λΝ

SFPETHTx

I/QTxRU Sw

itch

SYNC

BV-WSS

Multi-protocolInterfa

ce

SwitchSYNC

Multi-protocolInterface

(1)CPUSPP SPP SPP

Switch Switch

(3) (4)

d

RU2

SPP

SPP

GPP

GPP

SPP

SPP

GPP

GPP

GPP:GeneralPurposeProcessorSPP:SpecificPurposeProcessor

FronthaulFlow

5

Physical Network Infrastructure

• Integratedopticalandwirelessnetwork infrastructurefortransportandaccess• Wirelessdomain:

– Denselayerofsmallcellscomplementedbymacrocellstoensure ubiquitouscoverage– Smallcellscan bebackhauledtothemacro-cellsiteeitherwirelessly usingacombinationofmm-

Waveandsub-6wirelesstechnologiesorusingahybridopticalnetworkplatform• OpticalDomain

– Adoptionofadynamicandflexible/elasticframebasedopticalnetworksolutioncombinedwithenhancedcapacityWDMPONs

• BBProcessing:RUsareconnectedtoremoteBB processing pools throughhighbandwidthtransportlinks

VM

vBBU2

DataCenters

SmallCells

RRH

RRH

RRH

RRH

eNB

RRH

RRHRRHRRH

EPCSGW/GSN

vBBU

vBBUvBBU

PDN-GW

vBBU1

Backhaul

Fronthaul

vBBU

vBBU1 vBBU2

VM

eNB

MacroCell

Internet

60GHz/Sub-6linksforFH/BHFHBH

HeNBFemtoCells

HeNBHeNBGW

HeNB

WiFiWLANGW

TSON

WDM-PON

5G-XhaulWirelessBackhaul/Fronthaul

WirelessAccess

EdgeNode

RFtoBaseband

CyclePrefix&FFT

Resourcedemapping

Receiveprocessing

Decoding

MAC

(3)

(4)

(5)

• highnetworkba

ndwidth

• IncreasedBB

Usharing

• lownetworkband

width

• Limite

dBB

Ush

aring

(2)

(1)

FastSwitch

moduleλ1

FastSwitch

moduleλ2

FastSwitch

moduleλΝ

MUX

MUXBV-WSS

BV-WSS

BV-WSS

FastSwitch

moduleλ1

FastSwitch

moduleλ2

FastSwitch

moduleλΝ

MUX

MUX

b)

c)

RU:RemoteUnit

ProcessingattheServerRU

ProcessingattheRU

BVT:BandwidthvariabletransponderBV-WSS:Bandwidthvariablewavelengthselectionswitch

CoreNode

OpticalLink

a)

RU1

RU3 Core

Node

CoreNode

EdgeNode

EdgeNode

SPP

SPP

GPP

GPP

ToR

ToR

BVT

λ1

λ2

λΝ

EdgeNode

SPPSPPSPP

SPP

ToR

SPP

CPUCPU

SPP

ToR ToR

RU2

(5)

(2)

ElasticOpticalNetwork DataCentre

Tx

SFP RUETHTx

I/QTx

SFP RUETHTx

I/QTx

SFP RUETHTx

I/QTx

SFP RUETHTx

I/QTx

SFP RUETHTx

I/QTx

BVT

BVT

λ1

λ2

λΝ

λ1

λ2

λΝ

SFPETHTx

I/QTxRU Sw

itch

SYNC

BV-WSS

Multi-protocolInterfa

ce

SwitchSYNC

Multi-protocolInterface

(1)CPUSPP SPP SPP

Switch Switch

(3) (4)

d

RU2

SPP

SPP

GPP

GPP

SPP

SPP

GPP

GPP

GPP:GeneralPurposeProcessorSPP:SpecificPurposeProcessor

FronthaulFlow

6

Elastic Optical Network

• Opticalnetworkelasticityinthefrequencydomain

• Addressesefficientlythegreatlyvaryingbandwidthrequirementsofthevarious5Gservices– E.g.heavyCPRIflows,lightersplit

options,enduserservices

serverETHTx

I/QTx

serverETHTx

I/QTx

serverETHTx

I/QTx

serverETHTx

I/QTx

serverETHTx

I/QTx

serverETHTx

I/QTx

EthernetSwitch

BVR

BVR

Tx

SFP RUETHTx

I/QTx

SFP RUETHTx

I/QTx

SFP RUETHTx

I/QTx

SFP RUETHTx

I/QTx

SFP RUETHTx

I/QTx

EthernetSwitch BVT

EdgeNode

BVT

λ1

λ2

λΝ

λ1

λ2

λΝ

Spectrum

HeavyCPRI

LightCPRI

LightEthernet

HeavyEthernet

Wastedresources

HeavyCPRI

LightCPRI

LightEthernet

HeavyEthernet

Spectrumsavings

Spectrumsavings

Spectrum

I) II)

III) IV)

SFPETHTx

I/QTxRU

ACGroup1{AC0,AC1}

ACGroup2{AC1,AC2}

BV-WSS

BV-WSS

FastSwitch

moduleλ1

FastSwitch

moduleλ2

FastSwitch

moduleλΝ

MUX

MUX

BV-WSS

BV-WSS

FastSwitch

moduleλ1

FastSwitch

moduleλ2

FastSwitch

moduleλΝ

MUX

MUX

I/QSwitch

SYNC

I/QSwitch

SYNC

c)

d)

7

Convergence of Networking and Computation

MegaDCs Internet

Metro

Core

Metro

RegionalDCs

Mid-DCs

Micro-DCs

WirelessAccessMacro-cells

ControllerResources

NFVI

VIM

ControllerResources

NFVI

ControllerResources

NFVI

ControllerResources

NFVI

VIM

VIM

VIM

VNF

VNF

SSD

RAMGPP SPPPNF

PNF

VNF SSD

RAMGPP SPP

PNF

PNF

VNF

VNF

VNF

Frontlhaul

VNF

LocalDCsWireless

VirtualizationPHYFunctions

RegionalDCsOpticalMetro RemoteDCsOpticalCore

VirtualizationPHYFunctions VirtualizationPHYFunctions

PHYInf

VirtualResources

VirtualResources

VirtualResources

PHYResources

PHYResources

PHYResources

PNF VNFPNF VNF VNF VNF

VirtualTransport

forwarding

vBBU(lowerlayers)

vBBU(upperlayers)

EndpointEndpoint

forwardingforwarding

OperationalNetworkServices

End-userservices

RFto

Baseb

and

Cycle

Prefix&FFT

Resource

demapping

Receive

processin

g

Decoding

MAC

Service

Man

agem

ent&

Orche

stratio

nInf.Man

agem

ent&

Co

ntrol

LocalDCs Remote/regionalDC

LocalDCsWireless

VirtualizationPHYFunctions

RegionalDCsOpticalMetro RemoteDCsOpticalCore

VirtualizationPHYFunctions VirtualizationPHYFunctions

PHYInf

VirtualResources

VirtualResources

VirtualResources

PHYResources

PHYResources

PHYResources

PNF VNFPNF VNF VNF VNF

VirtualTransport

forwarding

vBBU(lowerlayers)

vBBU(upperlayers)

EndpointEndpoint

forwardingforwarding

OperationalNetworkServices

End-userservices

RFto

Baseb

and

Cycle

Prefix&FFT

Resource

demapping

Receive

processin

g

Decoding

MAC

Service

Man

agem

ent&

Orche

stratio

nInf.Man

agem

ent&

Co

ntrol

LocalDCs Remote/regionalDCHWA GPP

• Fronthaul adoptingflexiblefunctionalsplits

• TransportNetworkandComputeResources

• Processingparallelisation

• FullGPPwithcommodityhardwareandhybridsolutions

• DisaggregationofNetworkandComputeresources

• Mixingandmatchingofresourcestoefficientlysupportservices

8

Processing Parallelization

SPP SPP

ToR

(5)SPP SPP

Switch Switch

(4)(3)

CoreNodeEdge

NodeRU2

EdgeNode

SPP SPP

ToR

(5)SPP SPP

Switch Switch

(4)(3)SPP SPP

ToR

(5)SPP SPP

Switch Switch

(4)(3)SPP SPP

ToR

(5)SPP SPP

Switch Switch

(4)

d)

(3)

c)b)a)

RFtoBaseband

CyclePrefix&FFT

Resourcedem

apping

Receiveprocessing

Decoding

MAC

(3)

(4)

(5)(2)

(1)

RU DCOpticalinterDCnetwork

Transportrequirements:Split2

RFtoBaseband

CyclePrefix&FFT

Resourcedem

apping

Receiveprocessing

Decoding

MAC

(3)

(4)

(5)(2)

(1)

RU SPP/GPPIntraDCnetwork

Transportrequirements:

Split3

RFtoBaseband

CyclePrefix&FFT

Resourcedem

apping

Receiveprocessing

Decoding

MAC

(3)

(4)

(5)(2)

(1)

RU SPP/GPPIntraDCnetworkTransport

requirements:Split4

SPP/GPPSPP/GPP

• BBUsub-frameprocessingtime<1ms• Toreduceprocessingdelay,BBprocessingishandledinparallelover

disaggregatedcomputeresources• Model1:parallel(orsequential)processingmode

– eachfunctionisdistributedacrossmultipleprocessingunits(1:N)

• Model2:Pipelining– eachprocessingunithandlesaspecificfunctionadopting1:1mapping

9

Use case: Joint optimization of Inter-intra DC network for FH services

Sub-problem1:TransportNetworkOptimization• Identificationofoptimalresourcestosupport

FHorBHservicesintermsofbothtopologyandresourcesincludes:– Ordering(SC) oftherelevantfunctions(VNFor

PNF)– Estimatingvirtualresourcesrequiredtosupport

SCandexecuting thecorresponding applications– Mapping virtualresourcestophysicalresources

MegaDCsInternet

Metro

Core

Metro

MobileEdgeDCs

RegionalDCs

WirelessAccessMacro-cells

SSD

RAMGPP SPP

SSD

RAMGPP SPP

SSD

RAMGPP GPP

SSD

RAMGPP SPP

SSD

RAMGPP SPP

SSD

RAMGPP SPP

SSD

RAMGPP SPP

SSD

RAMGPP SPP

RegionalDCs

Disaggregatedresources• RAM,GPP,SPP• HWAccelerators

M. P. Anastasopoulos, A. Tzanakaki and D. Simeonidou, "Energy-aware offloading in mobile cloud systems with delay considerations," 2014 IEEE Globecom Workshops (GC Wkshps), Austin, TX, 2014, pp. 42-47.

10

Sub-problem 1: Constraints

• BBUchaining:VNFsareprocessedintheorderdefinedbythecorrespondingSC

• Flowconservationaswellasmappingandaggregation/de-aggregationoftrafficbetweendifferentdomains

• End-to-enddelayguarantees• FlexibleFHprocessingsplits

– BBUProcessingconstraints– RUDemandConstraints

• Transportnetworkcapacityrequirements– Constraintsimposedbytheelastic

opticalnetwork

LargeDCs Internet

Metro

Core

Metro

MiniDCs

Mid-DCs

Micro-DCs

WirelessAccessMacro-cells

ControllerResources

NFVI

VIM

ControllerResources

NFVI

ControllerResources

NFVI

ControllerResources

NFVI

VIM

VIM

VIM

Frontlhaul

VNF

VNF

SSD

RAMGPP SPPPNF

PNF

VNF SSD

RAMGPP SPP

PNF

PNF

VNF

VNF

VNFBackhaul

vBBU vBBU

Frontlhaul

11

Sub-problem 2: Disaggregated DC Network Optimization

Sub-problem2:• IdentifiestheoptimalprocessingmoduleswherepartsoftheFHSChavetobe

allocated.• EstablishmentofapathinterconnectingedgeDCnodewiththeGPP/SPP

modulesthatwillprocesstheremainingFHfunctions• TheorderofFHfunctionsprocessingisdefinedbythecorrespondingSC

SPP SPP

ToR

(5)SPP SPP

Switch Switch

(4)(3)

CoreNodeEdge

NodeRU2

EdgeNode

SPP SPP

ToR

(5)SPP SPP

Switch Switch

(4)(3)SPP SPP

ToR

(5)SPP SPP

Switch Switch

(4)(3)SPP SPP

ToR

(5)SPP SPP

Switch Switch

(4)

d)

(3)

c)b)a)

RFtoBaseband

CyclePrefix&FFT

Resourcedem

apping

Receiveprocessing

Decoding

MAC

(3)

(4)

(5)(2)

(1)

RU DCOpticalinterDCnetwork

Transportrequirements:Split2

RFtoBaseband

CyclePrefix&FFT

Resourcedem

apping

Receiveprocessing

Decoding

MAC

(3)

(4)

(5)(2)

(1)

RU SPP/GPPIntraDCnetwork

Transportrequirements:

Split3

RFtoBaseband

CyclePrefix&FFT

Resourcedem

apping

Receiveprocessing

Decoding

MAC

(3)

(4)

(5)(2)

(1)

RU SPP/GPPIntraDCnetworkTransport

requirements:Split4

SPP/GPPSPP/GPP

Sub-problem 1

12

min𝐹% =' ℰ) ' 𝑝)+,-𝜎)+�

+12+' ℰ4 𝐶4

4∈𝐄)∈𝐑

Optimization framework• JointoptimizationofInter/IntraDCNetwork• 1st Sub-problem:Optimaltransportnetworkisidentifiedthroughthe

minimizationoftheenergycost:

subjecttocapacity,latencyanddemandconstraints.• 2nd Sub-problem:DisaggregatedDCNetworkOptimization

subjecttodemandprocessingandcapacityconstraints

FH energy, local processing FH energy, transport

min𝐹9 =' ℰ: ' ' ' 𝑝;𝑎;=�

;>𝐅𝐇𝒓𝒊𝒅+12)∈𝐑

+' ℰ4 𝐶44∈𝐄𝒅:∈𝐌𝒅

FH energy, DC processing FH energy, DC network

13

Numerical evaluation: Scenario

• BristolisOpen(BIO)smartcitytest-bedtopology• Snapshotofspatialtrafficloadandaveragetraffic/BSbasedonrealdatasets

0

5

10

0

5

10

20

40

60

80

Km

Aver

age

data

rate

(Mbp

s) 0 20 40 60 80

X (km)Y (km)

OpticalEdgeNodeDataCenterOpticalLink

14

Numerical results: Energy

• “Distributed-RAN(D-RAN)”RUsandBBUsareco-locatedandFHserviceprocessingiscarriedoutexclusivelybyspecificpurposehardware.

• “C-RANwithspecificpurpose(SP)BBUhardware(SP-C-RAN)”:SpecificpurposeBBUsareplacedatacentralizedlocationtobeshared.

• “C-RANwithvirtualBBUs(V-C-RAN)-”:AllowsBBUstobeinstantiatedasvirtualfunctionsandrunonGPPSenablingsharingofresourcesandon-demandresizingofcomputeresources.

• “Disaggregated-RAN(DA-RAN)”:CombinesthebenefitsofSP-C-RANandV-C-RANallowingFHfunctionstobeprocessedeitheratSPPorGPPbasedontheirspecificcharacteristics.

15

Functional Splits

16

Conclusions

§ Motivation 5G vision

§ Proposed 5G architecture

§ C-RAN

§ Flexible Functional Splits

§ Optical Networking Interconnecting Compute Resources enablingResource Disaggregation

Increased resource efficiency

§ Modelling results indicate significant benefits in power efficiencycompared to alternative approaches

17

Thankyou!

ACKNOWLEDGMENT

• ThisworkhasbeensupportedbytheEUHorizon20205GPPPproject5G-XHaul

Sub-problem 1: Constraints

• BBUchaining:VNFsareprocessedintheorderdefinedbythecorrespondingSC

• Flowconservationaswellasmappingandaggregation/de-aggregationoftrafficbetweendifferentdomains

• End-to-enddelayguarantees• FlexibleFHprocessingsplits

– BBUProcessingconstraints– RUDemandConstraints

• Transportnetworkcapacityrequirements– Constraintsimposedbytheelastic

opticalnetwork

LargeDCs Internet

Metro

Core

Metro

MiniDCs

Mid-DCs

Micro-DCs

WirelessAccessMacro-cells

ControllerResources

NFVI

VIM

ControllerResources

NFVI

ControllerResources

NFVI

ControllerResources

NFVI

VIM

VIM

VIM

Frontlhaul

VNF

VNF

SSD

RAMGPP SPPPNF

PNF

VNF SSD

RAMGPP SPP

PNF

PNF

VNF

VNF

VNFBackhaul

vBBU vBBU

Frontlhaul

Optical Networking: RU-BBU interconnection

EdgeNode

FastSwitch

moduleλ1

FastSwitch

moduleλ2

FastSwitch

moduleλΝ

MUX

MUXBV-WSS

BV-WSS

BV-WSS

FastSwitch

moduleλ1

FastSwitch

moduleλ2

FastSwitch

moduleλΝ

MUX

MUX

c)

CoreNode

RU1

RU3CoreNode

CoreNode

EdgeNode

SPPSPP

GPPGPPToR

EdgeNode

SPPSPPSPP

SPP

ToR

SPP

GPPGPP

SPP

ToR ToR

RU2

OpticalNetwork DataCentre

Tx

ETHTx

I/QTx

ETHTx

I/QTx

ETHTx

I/QTx

ETHTx

I/QTx

ETHTx

I/QTx

BVT

BVT

λ1

λ2

λΝ

λ1

λ2

λΝ

ETHTx

I/QTx

Switch

SYNC

BV-WSS

Multi-protocolInterfa

ce

GPPSPP SPP SPP

Switch Switch

d)

SPPSPP

GPPGPP

SPPSPP

GPPGPP

ToREdgeNode

BVT

λ1

λ2

λΝ

SwitchSYNC

Multi-protocolInterface

a)

Key Network Component

• Interfacingbetweendifferenttechnologydomains:– Scheduling– Aggregation/De-Aggregation– Trafficadaptation– QoS Mapping

Rx FIFO 1

Rx FIFO 2Buffer

Tx FIFO λ1

Tx FIFO λ2

Tx FIFO λ3

Tx FIFO λ4

TSON

FIFO #1

FIFO #1

….

FIFO #M

DC

Fixed Traffic

Mob

ile U

sers

Rx FIFO 3

Rx FIFO 4Scheduler

Buffer

Aggregator

FIFO #1

FIFO #1

….

FIFO #N

Wireless

Buffer

Rx FIFO 1

Rx FIFO 2

Rx FIFO 3

Rx FIFO 4De

Aggregation

Buffer