GTP Load Balancing

How to do GTP Load Balancing with Cubro Products

27 Sep 2017

www.cubro.com

• GTP: GPRS Tunneling Protocol

• GTP is used to transport packet data from the eNodeB tothe internet via an IP tunnel.

GTP Load Balancing

GTP Load Balancing in L 4

GTP is an IP in IP Tunnel used on many mobile Interfaces like GN, S5, S8 and S1U. GTP consists of two types of packets GTP-C and GTP-U

GTP-U = is the user plan where the user traffic is transported

GTP-C = is the control plane of the protocol

GTP Load Balancing in L4

• IP address range is different between outer and inner IP

• Outer IP address range is from a few hundreds to thousands

• Inner IP address range is millions as each subscriber has a unique IP

• Typically a Probe needs GTP-U and GTP-C to produce useful meta data (this cannot be done with outer IP LB)

Outer IP Load Balancing

Outer IP address is in L4 which means any NPB can do load balancing based on a 5 tuple hash

If only Outer IP LB is done on GTP traffic, following issues will happen:

▪ Monitoring session for a user is interrupted when the customer moves to another location

▪ LB could be asymmetric due to the small amount which means the output ports can be overloaded leading to packet drop and bad monitoring quality

▪ Corresponding GTP-C traffic is not on the same port

▪ On LTE not on the same interface (S11)

LB Configuration in EX (outer tunnel)

Hash configuration (defines the field which is used for LB)

Configure output group (defines the amount and which ports are used for the LB)

Add a rule for the in ports to the LB group

(Binds the ports where the traffic is received to the load balancing group and the defined outputs to load balance the traffic based on the configured hash)

Typical Application

• For most of the Probes on the market this will not work because theInner IP traffic could be disintegrated - the traffic of the subscribersession can be distributed to different Probes.

• This application is usable for capture solution or in the first stage ofan inner IP load balancing solution (see next pages)

Inner IP LB with session correlation and control plan correlation

• LB on Outer IP to reduce the traffic from100 Gbit to multiple 10 Gbit links

• Sessionmaster now decodes the GTP-C or and S11 and S6a user plan to get all meta data to do a full subscriber based correlation needed because 2G - 3G - 4G PS Domain traffic has some differentiation and therefore different methods must be used to handle the traffic properly (especially in a fall back situations and when the customer is moving and the session jumps between network elements, it is not easy to forward the traffic always to the same probe)

• Each Sessionmaster is connected to each Probe and also between each other to perform this challenging application

• In the case of more then 3 – 5 Probes an additional Packetmaster is used to steer the traffic (see next page)

GTP LB Application for 1-1,2 TB

Mixed traffic 2G/3G/4G (including S1-MME/S11/S6a/S1-U)

Input aggregation & first outer tunnel load balancing, session and endpoint out learning meta data provider, l7 traffic tagging subscriber based, output load balancing based on L7 tagging to multiple probes.

(The first and the last units can be one unit depending on the amount of ports used)

Simple DPI Integration – Inner IP LB

Mixed traffic (including S1-MME/S11/S6a/S1-U)

Functions:EX48(1): Auto study for eNB/xGW IP address from S1MME, LB traffic to EX48DEX48D: Find user IP from inner layer, by matching eNB/xGW IP combination, then LB the traffic by inner layer IPEX48(2): Aggregate output traffic from EX40D by VLAN (same user has same VLAN tag)

Performance:EX40D:80Gbps/U (40G per CPU, 2x)

(EXA40D can also be upgraded to EXA24160, to get better performance per unit)

Simple DPI Integration – IMSI Based LB

Mixed traffic (including S1-MME/S11/S6a/S1-U)

Functions:EXA40(1): Auto Study for eNB/xGW IP address from S1MME, LB traffic to EXA40DEXA40D(1): Processing control plane traffic and get correlation meta data, then send meta data to EXA40(2), and output control plane traffic by IMSIEXA40D(2): Correlation user plane and control plane meta data, then output user plane traffic by IMSIEXA40(2): Aggregate output traffic from EXA40D(1)/(2) by IMSI based VLAN (same IMSI has same VLAN tag)

Performance:EXA40D(1):80Gbps/U (40G per CPU, 2x)EXA40(2): 5~6Gbps/U (Control plane decoding only)

Review of the two DPI Integration

• Auto study of information of eNB/xGW from traffic - no manual configuration work on thousands of IP addresses

• Have enough access ports (up to 96 10GE), and the port number can be extended by adding simple aggregation switches

• Have the same performance on user plane processing, i.e. 80G bps per 2xCPU unit

Since complex solution needs correlation, specialprocessing on control plane traffic by decoding is needed,which is a performance consuming task. Each 2xCPU unitscan handle 5Gbps signaling, i.e. 5% of 100G all-mixtures-traffic. The user needs extra 3~4 units to do correlation.

Scale-out possibility

Mixed traffic (including S1-MME/S11/S6a/S1-U)

When the future traffic to be load balanced will in the line of 100GE, we can upgrade:

▪ EXA48 (Position 1) to EX32100 (32x100G)

▪ EXA40D (Position 1 and 2) to EXA24160 (each unit can handle 120G+ user plane traffic and 10Gcontrol plane traffic)

▪ By EX32100, we can connect up to 32 EXA24160 nodes for 2.4T traffic and have 20xEXA24160 (toreplace EXA40D (2)) to process user plane and 12xEXA24160 (to replace EXA40D (1)) to processcontrol plane traffic.

▪ The final output traffic can use EX32100/EX48600 combination to many 10G/40G end points

Cubro Product Portfolio for GTP LB

EX32 (G4 Series)

EX48600

EX32100

EXA40/40D

EXA24160

Upto L4 Packet Oriented Up to L7 Session Oriented

EX20400

Packetmaster EX32100

✓ MPLS tag/detag✓ VLAN tag/detag / Q in Q✓ Header modification Layer 4✓ Load balancing Layer 4✓ GRE de/encapsulation✓ VXLAN de/encapsulation✓ All ports activated✓ All software activated✓ Low power design✓ Jumbo Frames 12000 Bytes

(Management ports on the back)

Packet load 3,2 Tbps

Ports 40 Gbit 32 QSFP

Ports 100 Gbit 32 QSFP28

GUI WEB/CLI/GUI

Packet buffer YES 24 MB

Delay < 700 ns

Dual Power YES

112 x 10 Gbit (with breakout cable) + 4 x 100 Gbit128 x 10 Gbit (with breakout cable)

1.2 Bpps forwarding performance:

L2 FDB/IPV4 Host /IPV6 Host/MPLS Labels (unified) (IP accurate) 320K/160K/80K/160KIPV4 Routes (LPM) / IPV6 LPM 64K/16K (IP with mask) iPACL/iVACL/iRACL – 64b keys (5-tuple with wildcard) eACL 6K per pipe (5-tuple with wildcard)

Sessionmaster EXA40 / EXA40D

✓ 1 Octeon III CPU (28 cores each)

✓ 1 x 64 GB DDR Memory

✓ 1 x 80 Billion instructions per sec

✓ Up to 40 Gbps L7 filtering performance (forwarding only)

✓ Up to 20 Gbps L7 correlation performance

✓ Up to 10 Gbps L7 metadata processing

✓ The D Model comes with two CPU and double the performance

Sessionmaster EXA24160

✓ 2 x Octeon III CN7890 CPU (48 cores each)

✓ 2 x 64 GB DDR Memory

✓ 2 x 240 Billion instructions per sec

✓ Up to 160 Gbps L7 filtering performance (forwarding only)

✓ Up to 120 Gbps L7 correlation performance

✓ Up to 60 Gbps L7 metadata processing

Packetmasters – L 7 Filtering

Packetmaster EXA48600

Packetmaster EXA32100

These advanced units are a mixture between Sessionmaster and Packetmaster

• L7 filtering in line rate up to multiple 100 Gbit

• But cannot handle session like the Sessionmaster

EXA32100

• EX32100 / EXA48600 can do LB on the inner IP, andremove the GTP tunnel if needed

• EXA32100 can help to offload the Sessionmaster and the amount of units.

Next generation GTP LB

Aggregation and filtering stage (EX32100)

GTP-U load balancing (user plan) (EXA32100) GTP-C load balancing control plan) (EXA24160)

This BOM represents a solution for 400 – 500 Gbps fullcorrelated LB for 2G/3G/4G, included signaling correlation

The complete picture

Thank you

For more information contact us at: support@cubro.com