Dell EMC Network Edge Reference Architecture for VMware v ... · 10 Dell EMC Network Edge Reference...
Transcript of Dell EMC Network Edge Reference Architecture for VMware v ... · 10 Dell EMC Network Edge Reference...
1 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
Dell EMC Network Edge
Reference Architecture for
VMware v Cloud NFV
OpenStack Edition 3.1
Dell EMC Service Provider Solutions
2 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
Date Description
August 2019 Initial Release
The information in this publication is provided “as is.” Dell Inc. and its suppliers makes no representations
or warranties of any kind with respect to the information in this publication, and specifically disclaims
implied warranties of merchantability or fitness for a particular purpose.
Use, copying, and distribution of any software that is described in this publication requires an applicable
software license. Copyright © 2019 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, and other
trademarks are trademarks of Dell Inc. or its subsidiaries.
Other trademarks may be the property of their respective owners. Published in the USA. Dell believes
that the information in this document is accurate as of its publication date. The information is subject to
change without notice
3 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
Contents
1 Introduction........................................................................................................................................ 7
1.1 Overview .................................................................................................................................... 7
2 Network Edge Reference Architecture Overview .......................................................................... 9
2.1 Network Edge physical landscape ............................................................................................ 9
2.1.1 Core ......................................................................................................................................... 10
2.1.2 Near Edge ................................................................................................................................ 10
2.1.3 Far Edge .................................................................................................................................. 10
2.1.4 Device Edge ............................................................................................................................ 11
2.2 Network Edge Platform Overview ........................................................................................... 11
2.3 Hardware resources ................................................................................................................ 11
2.3.1 Compute .................................................................................................................................. 11
2.3.2 Networking ............................................................................................................................... 12
2.3.3 Storage .................................................................................................................................... 12
2.4 Virtualization Layer .................................................................................................................. 12
3 Dell EMC Edge Portfolio ................................................................................................................. 13
3.1 Dell PowerEdge R640 servers ................................................................................................ 13
3.2 Dell PowerEdge R740xd servers............................................................................................. 13
3.3 Dell EMC PowerEdge XR2 servers ......................................................................................... 14
3.4 Dell EMC Edge Modular Data Centers (MDC) ........................................................................ 14
4 Networking Designs for Edge Deployments ................................................................................ 15
4.1 Dell EMC Open Networking ..................................................................................................... 15
4.2 Network Traffic......................................................................................................................... 15
5 Near Edge Reference Architecture ................................................................................................ 17
5.1 Network Topology .................................................................................................................... 17
5.2 Logical Networking .................................................................................................................. 18
5.3 Multi-Tenancy .......................................................................................................................... 21
5.4 Near Edge Reference Components ........................................................................................ 22
5.4.1 Networking ............................................................................................................................... 22
5.4.2 Servers .................................................................................................................................... 22
5.5 Software requirements ............................................................................................................ 23
5.6 Recommended BIOS Settings ................................................................................................. 24
6 Edge virtualized functions – Use Cases ....................................................................................... 25
6.1 MEC ......................................................................................................................................... 25
4 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
6.2 vCDN and Content Acceleration ............................................................................................. 25
6.3 vRAN ....................................................................................................................................... 26
A.1 Bill of Materials ........................................................................................................................ 27
A.2 References .............................................................................................................................. 29
5 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
Glossary
AR Artificial Reality
CO Central Office
COTS Commercial Off the Shelf
CUPS Control and User Plane Separation
DC Data Center
ETSI European Telecommunications Standards Institute
FPGA Field Programmable Gate Arrays
GPU Graphics Processor Unit
IIoT Industrial Internet of Things
MEC Multi-Access Edge Computing (or known as Mobile Edge Computing)
NEBS Network Equipment Building System
NFV Network Function Virtualization
ONIE Open Network Install Environment
PoP Point of Presence
QoE Quality of Experience
QoS Quality of Services
SDN Software-Defined Network
SDS Software-Defined Storage
SLA Service Level Agreement
URLLC Ultra-Reliable Low-Latency Communication
vBBU Virtualized Baseband Unit
vCDN Virtualized Content Delivery Network
vCU Virtualized Central Unit
vDU Virtualized Distributed Unit
vEPC Virtualized Evolved Packet Core
6 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
VIM Virtual Infrastructure Manager
VNF Virtualized Network Functions
VR Virtual Reality
vRAN Virtualized Radio Access Network
7 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
1 Introduction The advent of network function virtualization (NFV), the move to open networking, and the
emergence of new edge-related standards and operating models for edge workloads have a
profound structural effect on the supply chain, ecosystems, and underlying architectures.
These positive developments are driving the industry towards openness, choices, and
hardware and software disaggregation.
The move to a cloud infrastructure using edge functionality (from the cloud to the core and
closer to the delivery point) represents an opportunity for Service Providers.
This distributed cloud platform permits massive automation with granular control, visibility,
and security across administrative domains spanning different organizations and
geographies.
Dell EMC believes the journey to network transformation starts with open standards-based
infrastructure optimized for the customer's workloads and implementation.
With open, standards-based compute infrastructure as a foundation, the next step is about
the workload – i.e. the various pieces of software running on the deployed edge nodes. The
principal goal is a differentiated workload execution environment that can reflect and enforce
service logic. The result is sophisticated composable functions, service graphs, and the
enforcement policies with granular workload visibility.
5G and mobile edge infrastructure modernization rollout complements the best of IT, service,
and workload management with distributed mobility capabilities together in one unified,
validated platform with extensive management and disaster recovery capabilities.
Dell EMC and VMware have been working jointly on validated solutions for the Network Edge
to deliver on a joint vision of new generation intelligent, programmable and automated edge
platform.
This reference architecture guides the design and creation of Network Function Virtualization
Infrastructure (NFVI) for distributed Telco Edge deployments using Dell EMC Infrastructure
and VMware vCloud NFV solutions. This version of the reference architecture extends the
vCloud NFV 3.1 (OpenStack Edition) functionality to distributed nodes.
1.1 Overview The number of devices connected to a Service Provider network has been growing
exponentially, resulting in a significant increase in Network bandwidth requirements. As
shown in Figure 1 , together with management complexity. Service Providers are struggling
to keep pace with this the ever-increasing need for bandwidth, which is outpacing service
providers’ network infrastructure capacity. Additionally, Service Providers are finding it harder
to support newer data-sensitive application services required by 5G, IoT devices, and next-
generation mobile devices that require real-time processing, ultra-low latency, and massive
scalability.
8 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
Figure 1 Growing number of network edge and connected devices
Edge computing can help service providers keep up with this ever-increasing need for
bandwidth and allows them to deliver newer services, faster. Edge computing allows for
moving data, content, applications, and services closer to the end-users or devices (delivery
point), reducing round-trip delays. At the same time, since data no longer needs to be sent to
the central location or cloud, it results in significant backhaul traffic reduction resulting in
OPEX and CAPEX savings for the Service Providers.
Dell EMC is working with the leading Service Providers on their journey to Network and Edge
transformations. Dell EMC is helping to address many service provider pain points by:
• Delivering newer infrastructure platforms to address Edge requirements.
• Delivering an optimized NFVI platform for the Edge to host Network Functions at that
location. (as a natural extension to existing core and cloud capabilities)
• Building a curated partner ecosystem to address specific emerging applications and
edge use cases such as vRAN, vCDN, IIoT.
• Contributing to several Open Source Consortia focused on Network Transformation
and the Edge.
9 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
2 Network Edge Reference Architecture
Overview This reference architecture provides an overview of the key aspects of edge requirements
and provides a reference implementation of network edge cloud platform to deploy distributed
network services in a cloud environment at the edge.
The focus of this reference architecture is Near-Edge, which is usually a regional/central
office location that aggregates all types of traffic from front-haul, including fixed-line, 4G, and
Wi-Fi to back-haul. Sufficient IT-datacenter capacity, power, cooling, and locations are
mandatory for Near Edge functionality. Site capacity range includes one to five full racks and
multiple VNFs workloads (local vEPC, vCDN, MEC) and aggregation of network services (x-
haul) operate at this site.
Key characteristics of all network edge environments:
Networking: high-bandwidth, network slicing, aggregation, load balancing
Performance/Reliability: real-time/near real-time, ultra-low latency/jitter, SLA/QoS/QoE
Geo-location: Central Office (CO), Region, Aggregation, Local Exchange, Point of
Presence (PoP)
Processing: Management, Orchestration, Analytics, Intelligence, CUPS
2.1 Network Edge physical landscape The distance from the core datacenter on the operator’s infrastructure and the services
hosted at that location determines the physical location of the Network Edge.
The Edge locations highlighted in Figure 2 Network Edge are Near edge, Far edge, and
Device edge or Customer Premises Edge.
10 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
Figure 2 Network Edge requirements
2.1.1 Core The core is the heart of CSP network services. It is a centralized location for the CSP’s
management plane and all federated control planes for the Near Edge regions. VNFs such as
EPC, IMS, PCRF, MANO, Analytics, and OSS/BSS, are part of this 4G/5G Core architecture.
2.1.2 Near Edge Near Edge (regional/central office) aggregates all types of traffic from front-haul including
fixed-line, 4G, Wi-Fi traffic to back-haul. Each Near Edge site manages multiple Far-Edge
sites for management, control, and data plane.
Usually, multiple racks of infrastructure are deployed at this site supporting various workloads
such as MEC, CDN, CORD, UPF, and Network Slicing application functions.
2.1.3 Far Edge Far Edge is synonymous with a micro-datacenter located at a cell tower or close to a
customer’s premises. This location is sometimes referred to as the “Last Mile” to the
subscribers. The Far Edge infrastructure has some specific physical requirements for
thermal/cooling, power, rack spacing, and front/rear I/O access. This location is the closest
to the user’s access and provides key services where latency, reliability, and experience are
the most critical factors to the users.
Network services deployed and utilized by Far Edge include vRAN (NG-RAN), Industrial IoT,
Private LTE, Connected Cars, AR/VR, etc. Most of these services require ultra-low latency,
high network bandwidth, and real-time synchronization.
11 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
2.1.4 Device Edge Device Edge refers to customer or enterprise premises where thousands or in some cases,
millions of devices are connected to the network and can act as sources and consumers of
data. These include mobile devices, IoT sensors, industrial equipment that connect either
through wireless or wireline connections.
2.2 Network Edge Platform Overview The Network Edge Platform architecture contains three layers:
1. Hardware resources
2. Virtualization layer
3. Edge virtualized functions
See Figure 3 Network Edge Platform layers
Figure 3 Network Edge Platform layers
2.3 Hardware resources
2.3.1 Compute The workloads running at the edge determine the requirements for the compute
infrastructure. Appropriate sizing based on the workloads’ requirements is needed to decide
the optimal compute configuration for the edge workload.
Sizing and other requirements determine the following configuration settings for the compute
infrastructure:
• CPUs, sockets, cores, and clock frequencies
12 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
• Max power consumption (AC/DC)
• Thermal limits & cooling requirements
• Climate tolerance
• Server depth and front/rear I/O access
• Compliance requirements such as NEBS
• Level of ruggedization and tolerance to various “environmentals”
Depending on the workload requirements, hardware components such as GPUs, SmartNICs,
and FPGAs may be required to provide optimal performance. (acceleration, offload, visibility
etc.)
2.3.2 Networking Networking at the edge needs to consider overlays/underlays usually provided by a Software-
Defined Network (SDN). Synchronization and time-sensitive integration to network
connections help devices to achieve accuracy of communication regardless of the location.
2.3.3 Storage Data access from the edge can be either stateless or stateful. Stateful network services
depend on storage access, and storage requirements increase with services like video
streaming, caching, and media transcoding. Edge Locations are usually space-constrained,
and hence external storage arrays are generally not recommended for such sites.
2.4 Virtualization Layer Virtualization is an essential element to host VM based network functions. The virtualization
layer combined with SDN and Software-Defined Storage (SDS) delivers a flexible, scalable
edge cloud environment.
13 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
3 Dell EMC Edge Portfolio Optimal Compute infrastructure serves a critical role in transforming the service provider
network to a highly efficient, flexible, and scalable disaggregated model. Dell EMC offers a
wide range of compute options which gives Service Providers the flexibility to design their
disaggregated Network architecture based on the appropriate workloads and the data center
location requirements.
Figure 4 Dell EMC Edge portfolio
3.1 Dell PowerEdge R640 servers The Dell EMC PowerEdge R640 server is a hyper-dense, two-socket, 1U rack server. The
PowerEdge R640 is the ideal dual-socket, 1U platform for dense scale-out cloud computing.
The scalable business architecture of the Dell EMC PowerEdge R640 is designed to
maximize application performance and provide the flexibility to optimize configurations based
on the application and use case.
With the Dell EMC PowerEdge R640 you can create an NVMe cache pool and use either 2.5”
or 3.5”drives for data storage. Combined with up to 24 DIMM’s, 12 of which can be
NVDIMM’s, you have the resources to create the optimum configuration to maximize
application performance in only a 1U chassis.
3.2 Dell PowerEdge R740xd servers The Dell EMC PowerEdge R740xd delivers a perfect balance between storage scalability and
performance. The 2U two-socket platform is ideal for software-defined storage. The R740xd
ability to mix any drive type to create the optimum configuration of SSD and HDD for either
performance, capacity, or both. The Dell EMC PowerEdge R740xd is the platform of choice
for software-defined storage such as VMware vSAN.
14 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
3.3 Dell EMC PowerEdge XR2 servers Dell EMC PowerEdge XR2 is built from the ground up for harsh environments and features
the latest Intel® Xeon® SP processors. The rackable, rugged 1U-short depth server with
certifications in shock, vibration, dust, humidity, EMI and maritime is ideal for computing in
space-constrained deployments.
3.4 Dell EMC Edge Modular Data Centers (MDC) Modular Data Centers (MDC) provide a self-contained design with cooling, power, and
dedicated enclosure to secure and performance capable deployment in a remote location.
MDCs are architected to be flexible and can host a variety of Dell EMC servers, networking,
and storages to accommodate diverse edge workloads. Micro-MDC is pre-integrated data
center that can quickly deploy at Near and Far Edge sites.
Figure 5 Dell EMC Edge Modular Data Centers (MDC)
15 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
4 Networking Designs for Edge Deployments Physical connectivity between servers within sites requires a sufficient number of switch ports
to be available along with the ability to scale bandwidth. A reference physical switch
configuration per site expects guaranteed bandwidth with no over-subscription and using a
non-blocking design. This reflects to network edge capacity and bandwidth aggregation. This
reference architecture is shown in Figure 6.
Near Edge DCStack ID
Reset
25Gb/sFar EdgeDCeNB
25Gb/s
25Gb/s
25Gb/sFar EdgeDC CU/DU
25Gb/s
25Gb/s
25Gb/s
Far EdgeDCCU
25Gb/s
25Gb/s
Stack ID
Reset
Fronthaul
Midhaul
Backhaul
DU
4G Radio
N * 40/100GbN * 100/400Gb
Figure 6 End-to-End Network Edge Topology
4.1 Dell EMC Open Networking Open Networking is a core element of Dell EMC’s networking strategy and mission. Open
Networking separates the hardware from the operating system, giving you the choice of
picking the operating system that best fits your unique network infrastructure needs. Open
Networking uses standards-based open-source building blocks.
In the Dell EMC Networking portfolio, any switch model with an “-ON” suffix, such as the Dell
EMC Networking Z9264F-ON and the Dell EMC Networking S5248F-ON switches, has Open
Network Install Environment (ONIE) enabled.
4.2 Network Traffic The network architecture employs a Virtual Link Trunking, (VLT) connection between the two
Top of Rack (ToR) switches. The inherent redundancy of a non-VLT environment requires
standby equipment, which increases infrastructure costs and risks. In a VLT environment, all
paths are active, adding immediate value and throughput while protecting against hardware
failures. VLT technology enables a server or bridge to uplink a physical trunk into more than
one Networking S5248-ON switch by treating the uplink as one logical trunk. A VLT-
connected pair of switches function as a single switch to a connecting bridge or server. The
major benefits of VLT technology are:
16 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
• Dual control plane for highly available resilient network services
• Full utilization of the active LAG interfaces
• Active/active design for seamless operations during maintenance events
The Dell EMC Networking S5248-ON switches each provides six 40/100 GbE uplink ports.
The VLT interconnect (VLTi) configuration in this architecture uses two 40/100 GbE ports
from each ToR switch to provide a 200 GB data path between the switches.
Figure 7 illustrates the Networking Z9264F-ON
Stack ID
Reset
Stack ID
Reset
VLTi
Figure 7 Z9264F-ON
Figure 8 illustrates the Networking S5248F-ON
Figure 8 S5248F-ON
17 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
5 Near Edge Reference Architecture Dell EMC vCloud NFV Edge Architecture is based on VMware vCloud NFV Edge 3-Pod
design that includes Management, Edge, and Resource pods as shown in Figure 9. Near
Edge architecture is based on VMware vCloud NFV OpenStack Edition 3.1 Edge Reference
Architecture with VMware Integrated OpenStack (VIO) as the VIM.
N
N
N4
32
43
2
43
2
QSFP+
29 31
30 32
25 27
26 28
21 23
22 24
17 19
18 20
13 15
14 16
9 11
10 12
5 7
6 8
1 3
2 4
Stack ID
C C D
QSFP+
29 31
30 32
25 27
26 28
21 23
22 24
17 19
18 20
13 15
14 16
9 11
10 12
5 7
6 8
1 3
2 4
Stack ID
QSFP+
29 31
30 32
25 27
26 28
21 23
22 24
17 19
18 20
13 15
14 16
9 11
10 12
5 7
6 8
1 3
2 4
Stack ID
33 34 35 3631 3229 3027 2825 26 45 46 47 4843 4441 4239 4037 389 10 11 127 85 63 41 2 21 22 23 2419 2017 1815 1613 14 50 52 54
49 51 53
Sta
ck
ID
Management Pod (PowerEdge R640)
VLTi
SPINE 1 SPINE 2
LEAF1 LEAF2Management
VLTi
QSFP+
29 31
30 32
25 27
26 28
21 23
22 24
17 19
18 20
13 15
14 16
9 11
10 12
5 7
6 8
1 3
2 4
Stack ID
Edge Pod (PowerEdge R640)
Resource Pod (PowerEdge R740XD)
Z9264F-ON Z9264F-ON
S5232F-ON/S5248F-ON S5232F-ON/S5248F-ONS4148T-ON
Stack ID
Reset
Stack ID
Reset
Stack ID Stack IDSta
ck ID
Figure 9 Near edge architecture
5.1 Network Topology Core datacenter to edge site connects via the provider's WAN edge router. Spine-leaf is the
recommended topology for a Near Edge datacenter.
18 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
Near Edge datacenter provides spine connectivity to Far Edge site’s leaf switches. Link-
aggregations with active-active, (LAG and LACP) is required for resilient connectivity
between spine and leaf switches. Bi-directional Forwarding Detections (BFD), prevents an
unresponsive link-state forward traffic, to configure all ToR switches.
Since the Near Edge is aggregation traffic from fronthaul to backhaul of core data center,
compute and network infrastructure requires sufficient bandwidth to handle various Edge use
cases, such as vCDN and vRAN. See Figure 10.
Far EdgeNear Edge
18 2014 1610 126 82 4 30 3226 2822 24
17 1913 159 115 71 3 29 3125 2721 23 33SFP+
34SFP+
Stack ID
EST
18 2014 1610 126 82 4 30 3226 2822 24
17 1913 159 115 71 3 29 3125 2721 23 33SFP+
34SFP+
Stack ID
EST
18 2014 1610 126 82 4 30 3226 2822 24
17 1913 159 115 71 3 29 3125 2721 23 33SFP+
34SFP+
Stack ID
EST
18 2014 1610 126 82 4 30 3226 2822 24
17 1913 159 115 71 3 29 3125 2721 23 33SFP+
34SFP+
Stack ID
EST
VLT
Spine1 Spine2
LACP LACP
Leaf1 Leaf2LACPLACP
BGP/BFD
Telco WAN
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Stack ID
Stack ID
Stack ID
Reset
Stack ID
Reset
Sta
ck I
D
VLT
18 2014 1610 126 82 4 30 3226 2822 24
17 1913 159 115 71 3 29 3125 2721 23 33SFP+
34SFP+
Stack ID
EST
18 2014 1610 126 82 4 30 3226 2822 24
17 1913 159 115 71 3 29 3125 2721 23 33SFP+
34SFP+
Stack ID
ESTVLTLeaf1 Leaf2
Telco WAN
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Stack ID
Stack ID
Sta
ck I
D
BGP/BFD
LACPLACP
Figure 10 Near edge to Far Edge topology
5.2 Logical Networking The Dell EMC network switching infrastructure uses VMware NSX network virtualization,
which is part of the VMware vCloud NFV infrastructure.
NSX-T technology enables the decoupling of network services from the physical
infrastructure i.e. logical networks created on top of a basic Layer 2 (switched) or Layer 3
(routed) physical infrastructure can be decoupled from the physical and virtual environments.
This enables agility and security in the virtual environment while allowing the physical
environment to focus on throughput.
The NSX-T platform also provides for network services in the logical space, including
switching, routing, firewalls, load balancing, and VPN services.
NSX-T provides:
• Simplified network service deployment, migration, and automation
• Reduced provisioning and deployment time
• Scalable multi-tenancy across one or more data centers
19 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
• Distributed routing and a distributed firewall at the hypervisor, to enable better east-
to-west traffic flow and an enhanced security model
• Providers solution for traditional networking problems, such as limited VLANs, MAC
addresses, and FIB and ARP entries
• Normalization of the underlying hardware, enabling more straightforward hardware
migration and interoperability
In addition, application requirements do not require modification to the physical network. See
Figure 11.
Figure 11 Near Edge Compute Networking
Near site has Edge nodes (VM or Bare-metal) operating its transport and networking
services. Each Near Edge has full NSX deployment, including NSX Manager, NSX Controller
and NSX Edge.
20 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
Each Near Edge communicates with and manages multiple Far Edge sites. Each edge site,
Near Edge or Far Edge, is connect with Tier-0 router to telco’s own Provider Edge router(s) in
Figure 12 Network Edge physical to logical topology.
Near Edge / Region / CO
Far Edge Site2Far Edge Site2Far Edge Site1
vCloud NFV
Management Pod
Stack IDStack ID
2x25G 2x25G1x1G
CPU2 x16CPU1 x16CPU1 x16
Edge / Resource Pod
CPU2 x16CPU1 x16CPU1 x16
Edge / Resource Pod
CPU2 x16CPU1 x16CPU1 x16
Edge / ResourcePod
CPU2 x16CPU1 x16CPU1 x16
Stack ID Stack ID
Stack ID Stack ID Stack ID Stack ID
Telco WANNSX-T Overlay/Underlay
eBGP
Tier-1
Tier-0
Tier-1
Tier-0
Tier-1
Tier-0
Resource Pod
Edge Pod
CPU2 x16CPU1 x16CPU1 x16
CPU1 x16 CPU2 x16
Stack ID Stack ID
Tier-0Tier-1
Figure 12 Network Edge physical to logical topology
Inside NSX configuration, two primary transport zones communicate between Near Edge and
Far Edge sites as shown in Figure 13 VMware Edge NSX-T topology. It also highlights
external VLAN-Based and Overlay Traffic.
21 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
Figure 13 VMware Edge NSX-T topology
5.3 Multi-Tenancy VMware Integrated OpenStack (VIO) at Near Edge site(s) manages local Resource Pod
which runs VNF workloads such as local EPC and CDN. Concurrently, VIO manages each
Far Edge site as compute resources per tenant and tenant availability zone as shown in
Figure 14 VMware vCloud NFV Edge topology.
22 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
Figure 14 VMware vCloud NFV Edge topology
5.4 Near Edge Reference Components
5.4.1 Networking Dell EMC Z9264-ON, S5232-ON/S5048-ON, S4048-T-ON/S4148T-ON (optional NEBS
models)
5.4.2 Servers Dell EMC PowerEdge R640, PowerEdge R740/740XD, PowerEdge XR2 (optional Carrier-
Grade models)
23 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
Model System info Firmware
R640/R740XD BIOS 2.1.8
iDRAC 9 3.32.32.32
BOSS 2.5.13.3020
HBA330 16.17.00.03
Intel X710 – QP – SFP+ rNDC 18.8.9
Intel XXV710 – DP- SFP+ 18.8.9
Table 1 Servers
S4148T-ON (ToR) OS10 10.4.3.1
Z9264-ON (Spine) OS10 10.4.3.1
S5232/S5248-ON (Leaf) OS10 10.4.3.1
Table 2 Switches
5.5 Software requirements vCloud NFV 3.1
ESXi 6.7u1 Build 11675023
vCenter 6.7u1 Build 10244745
NSX-T 2.3.0
VIO 5.1.0
vRealize Log Insight 4.7.0
Table 3 Software Requirements
24 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
5.6 Recommended BIOS Settings
Components BIOS Settings Proposed Settings
Processor Logical Processor Enabled
CPU Interconnect Speed Maximum Data Rate
Virtualization Technology Enabled
Adjacent Cache Line Prefetch Enabled
Hardware Prefetcher Enabled
DCU Streamer Prefetcher Enabled
DCU IP Prefetcher Enabled
Sub NUMA Cluster Disabled
UPI Prefetcher Enabled
Logical Processor Idling Disabled
x2APIC Mode Disabled
Dell Controlled Turbo Disabled
Number of Cores per Processor All
Memory Memory Operating Mode Optimizer Mode
Node Interleaving Disabled
Correctable Memory ECC SMI Enabled
Opportunistic Self-Refresh Disabled
Power Power Cap Disabled
Redundancy Policy A/B Grid Redundant
Hot Spare Enabled
Primary PSU PSU1
Power Factor Correction Disabled
Fans Thermal Profile Optimization Default Thermal Profile
Boot Mode BIOS
Boot Order Hard Drive, NIC
HDD Boot Order Internal SD: IDSDM
System Profile Performance
CPU Power Management Maximum Performance
Memory Frequency Maximum Performance
Turbo Boost Enabled
C1E Disabled
C States Disabled
Write Data CRC Disabled
Memory Patrol Scrub Standard
Memory Refresh Rate 1x
Uncore Frequency Maximum
Energy Efficient Policy Performance
Number of Turbo Boost Enabled Cores for Processor 1 All
Number of Turbo Boost Enabled Cores for Processor 2 All
Enabled
CPU Interconnect Bus Link Power Management Disabled
PCI ASPM L1 Link Power Management Disabled
SR-IOV Global Enable Enabled
Internal SD Card Port On
Internal SD Card Redundancy Mirror
Internal SD Primary Card SD Card1
Wake on LAN Disabled
25 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
6 Edge virtualized functions – Use Cases Disaggregation enables decoupling of Network Functions from traditional proprietary
equipment. Network functions like vBBU, vCU/vDU, vEPC deployed as a virtualized
deployment on standard x86 hardware permit CSPs to deploy network services promptly on a
distributed and highly scalable architecture.
6.1 MEC Multi-Access Edge Computing (MEC) is a framework defined by ETSI. MEC enables Service
Providers to deliver newer applications, services, and content to end-users and devices that
can take advantage of the real-time, high-bandwidth, and low-latency environment available
at the edge.
MEC allows a service provider to open and share Radio Access Network (RAN) APIs to
develop and deliver mobile or multi-access applications such as AR/VR, gaming,
autonomous vehicles, content caching and video streaming at the edge.
MEC is not limited to enable radio access usage, but also Wi-Fi and wired access
technologies. It is an enabler for 5G network service evolution.
The key benefits of MEC:
• Accelerate priority services to edge network for optimized performance (latency and
throughput) for LTE and 5G RAN
• Enable better media content services to customer’s QoE and edge caching reduce
transport backhaul cost
• Hosting IoT/M2M services
6.2 vCDN and Content Acceleration An increasing number of users are accessing their favorite content, including ultra-high-
definition (UHD) media over multiple devices from fixed-access to wireless. vCDN allows
Service Providers to bring the media content distribution model to the edge. The device can
then access the media content at the Network Edge in a low-latency environment and avoid
network congestion to the CSP’s backhaul.
To address vCDN datacenter requirements, operators or content providers are looking at the
bandwidth, media delivery efficiency, and resource capacity at the Near Edge and Far Edge
of the network.
For example, the network bandwidth calculation is the key to the design for vCDN to deliver
HD or UHD (4K or 8K) data. 4K UHD format requires 15 Mbit/s to 25 Mbits/s per stream while
8K UHD format requires 80 Mbit/s or higher per stream.
The content providers need to right-size and architect the network infrastructure to handle
multiple streams simultaneously. Each compute node requires enough bandwidth to handle
at least 1000 concurrent streams (UHD) to edge devices and caching contents from internet
26 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
or core content storages. The concurrent stream numbers can vary based on the location
and the populations.
Caching is also key to accelerate the content delivery to devices. A server equipped with
traditional hard drive disk (HDD), is not suitable for data caching purposes. Solid-state-
drives(SSD) are needed to meet the performance needs for caching.
However, for optimizing the caching performance, in-memory (RAM) type of storage is an
essential requirement for the vCDN’s caching. Each site hosting vCDN infrastructure should
have SSD and in-memory storage to process and prioritize the contents based on the
accessing frequency. Frequently accessed media should be stored in in-memory storage
and lesser-visited media in SSD/NVMe storage.
Compression is another key to optimizing vCDN deployments. Every vCDN deployment site
needs to reduce the size of the image or media by compression of the content files. Lossy
compression can be used to reduce the amount of data stored. Accelerator HW such as Intel
Quick Assist Adapter can be used to improve the compression/decompression performance.
6.3 vRAN NFV and SDN are changing the game and this new virtualized RAN architecture provides the
same benefits that the Operators have seen with the deployment of virtualized services in the
CORE (vEPC, vIMS, etc). Virtualization which is becoming very common in the core data
centers is now moving to Edge locations for NFVI based Cloud Solutions at the Edge.
Important Benefits of Virtualized RAN:
vRAN decomposes the traditional radio stacks and maps them into discrete elements.
Decomposition allows the hardware solutions to be decoupled from software
implementations, enabling an eco-system of vendors to emerge that will deliver the
infrastructure for more cost effective and efficient access networks.
A truly open virtualized RAN helps in several areas, including:
• Leveraging economical and ubiquitous Ethernet and/or IP based transport to support
lower
• cost, future-proof deployments
• Utilizing COTS hardware and NFV to provide elasticity of capacity
• Pooling capacity resources and licenses to cater to large sets of devices with diverse
needs
• Providing a flexible platform for Edge Computing essential for a new generation of
services
• Allowing dynamic orchestration of 5G slices
27 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
A.1 Bill of Materials
GMDY9LU Dell EMC Z9264F-ON Switch, 64x 100GbE QSFP28, PSU to IO air, 2x PSU, OS10
Table 4 Spine - S9264F-ON
G8LUBKE Dell EMC S5232F-ON Switch, 32x 100GbE QSFP28 ports, PSU to IO air, 2x PSU, OS10
Table 5 Leaf - S5232F-ON
G4S0GLF Dell EMC S5248F-ON Switch, 48x25GbE SFP28, 4x100GbE QSFP28, 2x100GbE QSFP-DD, PSU to IO, 2xPSU, OS10
Table 6 Leaf – S5248F-ON
R640 VSAN Ready Node (Management and Edge Pod)
5103690 960GB SSD SAS Mix Use 12Gbps 512n 2.5in Hot-plug Drive, PX05SV,3 DWPD,5256 TBW
5103865 900GB 15K RPM SAS 12Gbps 512n 2.5in Hot-plug Hard Drive
R640 PowerEdge R640 Server
NTPM No Trusted Platform Module
5099278 2666MT/s RDIMMs
5098888 16GB RDIMM, 2666MT/s, Dual Rank
5101341 Riser Config 4, 2x16 LP
X710DP Intel X710 DP 10Gb DA/SFP+, + I350 DP 1Gb Ethernet, Network Daughter Card
X710FP Intel X710 Dual Port 10Gb Direct Attach, SFP+, Converged Network Adapter, Low Profile
5102436 IDSDM and Combo Card Reader
5100615 2x 16GB microSDHC/SDXC Card
5101051 2.5” Chassis with up to 10 Hard Drives and 3PCIe slots
5101090 Intel® Xeon® Gold 6132 2.6G,14C/28T,10.4GT/s, 19M Cache,Turbo,HT (140W) DDR4-2666
12GBRC HBA330 12Gbps SAS HBA Controller (NON-RAID), Minicard
5101091 Intel® Xeon® Gold 6132 2.6G,14C/28T,10.4GT/s, 19M Cache,Turbo,HT (140W) DDR4-2666
Table 7 R640 (Management and Edge Pod)
28 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
R740XD VSAN Ready Node (Resources Pod)
R740XD PowerEdge R740XD Server
NTPM No Trusted Platform Module
5099278 2666MT/s RDIMMs
24HD2P Chassis with Up to 24 x 2.5” Hard Drives for 2CPU
5098890 32GB RDIMM, 2666MT/s, Dual Rank
5101074 HBA330 Controller, 12Gbps Adapter, Low Profile
5103619 BOSS controller card + with 2 M.2 Sticks 240G (RAID 1),FH
X710SP Intel X710 Quad Port 10Gb DA/SFP+ Ethernet, Network Daughter Card
G2WY1IJ Intel XXV710 Dual Port 25GbE SFP28 PCIe Adapter, Full Height
5101685 Riser Config 4, 3x8, 4 x16 slots, Double-Wide GPU compatible
5106652 480GB SSD SATA Read Intensive 6Gbps 512e 2.5in Hot-plug Drive, S4500, 1 DWPD,876 TBW
5103923 1.8TB 10K RPM SAS 12Gbps 512e 2.5in Hot-plug Hard Drive
5101102 Intel® Xeon® Gold 6148 2.4G,20C/40T,10.4GT/s, 27M Cache,Turbo,HT (150W) DDR4-2666
5101124 Intel® Xeon® Gold 6148 2.4G,20C/40T,10.4GT/s, 27M Cache,Turbo,HT (150W) DDR4-2666
Table 8 R740XD (Resources Pod)
29 Dell EMC Network Edge Reference Architecture for VMware vCloud NFV 3.1
Internal Use - Confidential
A.2 References
https://docs.vmware.com/en/VMware-vCloud-NFV-OpenStack-Edition/3.1/vloud-nfv-edge-
reference-arch-31.pdf
https://www.dellemc.com/resources/en-us/asset/technical-guides-support-
information/solutions/bill_of_materials_guide_v3.1_for_vio_automated_deployment.pdf
https://www.dellemc.com/resources/en-us/asset/technical-guides-support-
information/solutions/hardware_guide_v3.1_for_vio_automated_deployment.pdf
https://www.dellemc.com/resources/en-us/asset/technical-guides-support-
information/solutions/architecture_guide_v3.1_for_vio_automated_deployment.pdf