Technology Integration: RSerPool & Server Load-balancing
description
Transcript of Technology Integration: RSerPool & Server Load-balancing
Technology Integration: Technology Integration: RSerPool & Server RSerPool & Server
Load-balancingLoad-balancing
Curt Kersey, Cisco SystemsCurt Kersey, Cisco Systems
Aron Silverton, Motorola LabsAron Silverton, Motorola Labs
ContentsContents
MotivationMotivation Background:Background:
Server Load-balancingServer Load-balancing Server FeedbackServer Feedback RSerPoolRSerPool
Unified approach:Unified approach: DescriptionDescription Sample FlowsSample Flows
Work ItemsWork Items
Assumptions / TerminologyAssumptions / Terminology
All load-balancing examples will use TCP/IP All load-balancing examples will use TCP/IP as the transport protocol. This could easily as the transport protocol. This could easily be any other protocol (e.g., SCTP).be any other protocol (e.g., SCTP).
SLB = Server Load-Balancer.SLB = Server Load-Balancer. Virtual Server = Virtual instance of Virtual Server = Virtual instance of
application running on SLB device.application running on SLB device. Real Server = physical machine with Real Server = physical machine with
application instances.application instances.
MotivationMotivation
Highly redundant SLB.Highly redundant SLB. More accurate server pooling.More accurate server pooling.
Server Load-balancingServer Load-balancing
What does a SLB do?What does a SLB do?
Gets user to needed resource:Gets user to needed resource: Server must be availableServer must be available User’s “session” must not be brokenUser’s “session” must not be broken
If user must get to same resource over and over, the SLB If user must get to same resource over and over, the SLB device must ensure that happens (ie, session persistence)device must ensure that happens (ie, session persistence)
In order to do work, SLB must:In order to do work, SLB must: Know servers – IP/port, availabilityKnow servers – IP/port, availability Understand details of some protocols (e.g., FTP, SIP, etc)Understand details of some protocols (e.g., FTP, SIP, etc)
Network Address Translation, NAT:Network Address Translation, NAT: Packets are re-written as they pass through SLB device.Packets are re-written as they pass through SLB device.
Why to Load-balance?Why to Load-balance?
Scale applications / servicesScale applications / services Ease of administration / maintenanceEase of administration / maintenance
Easily and transparently remove physical servers Easily and transparently remove physical servers from rotation in order to perform any type of from rotation in order to perform any type of maintenance on that server.maintenance on that server.
Resource sharingResource sharing Can run multiple instances of an application / Can run multiple instances of an application /
service on a server; could be running on a service on a server; could be running on a different port for each instance; can load-balance different port for each instance; can load-balance to different port based on data analyzed.to different port based on data analyzed.
Load-Balancing AlgorithmsLoad-Balancing Algorithms
Most predominant:Most predominant: least connectionsleast connections: server with fewest number of flows : server with fewest number of flows
gets the new flow request.gets the new flow request. weighted least connectionsweighted least connections: associate a weight / : associate a weight /
strength for each server and distribute load across server strength for each server and distribute load across server farm based on the weights of all servers in the farm.farm based on the weights of all servers in the farm.
round robinround robin: round robin thru the servers in server farm.: round robin thru the servers in server farm. weighted round robinweighted round robin: give each server ‘weight’ : give each server ‘weight’
number of flows in a row; weight is set just like it is in number of flows in a row; weight is set just like it is in weighted least flows.weighted least flows.
There are other algorithms that look at or try to There are other algorithms that look at or try to predict server load in determining the load of the predict server load in determining the load of the real server.real server.
How SLB Devices Make How SLB Devices Make DecisionsDecisions
The SLB device can make its load-balancing decisions The SLB device can make its load-balancing decisions based on several factors.based on several factors. Some of these factors can be obtained from the packet Some of these factors can be obtained from the packet
headers (i.e., IP address, port numbers, etc.).headers (i.e., IP address, port numbers, etc.). Other factors are obtained by looking at the data beyond the Other factors are obtained by looking at the data beyond the
network headers. Examples:network headers. Examples: HTTP CookiesHTTP Cookies HTTP URLsHTTP URLs SSL Client certificateSSL Client certificate
The decisions can be based strictly on flow counts or they The decisions can be based strictly on flow counts or they can be based on knowledge of application.can be based on knowledge of application.
For some protocols, like FTP, you have to have knowledge For some protocols, like FTP, you have to have knowledge of protocol to correctly load-balance (i.e., control and data of protocol to correctly load-balance (i.e., control and data connection must go to same physical server).connection must go to same physical server).
When a New Flow ArrivesWhen a New Flow Arrives
Determine if virtual server exists.Determine if virtual server exists. If so, make sure virtual server has available resources.If so, make sure virtual server has available resources. If so, then determine level of service needed by that If so, then determine level of service needed by that
client to that virtual server.client to that virtual server. If virtual machine is configured with particular type of If virtual machine is configured with particular type of
protocol support of session persistence, then do that work.protocol support of session persistence, then do that work. Pick a real server for that client.Pick a real server for that client.
The determination of real server is based on flow counts The determination of real server is based on flow counts and information about the flow.and information about the flow.
In order to do this, the SLB may need to proxy the flow to In order to do this, the SLB may need to proxy the flow to get all necessary information for determining the real get all necessary information for determining the real server – this will be based on the services configured for server – this will be based on the services configured for that virtual server.that virtual server.
If not, the packet is bridged to the correct If not, the packet is bridged to the correct interface based on Layer 2.interface based on Layer 2.
SLB: ArchitecturesSLB: Architectures
TraditionalTraditional SLB device sits between the Clients and SLB device sits between the Clients and
the Servers being load-balanced.the Servers being load-balanced. DistributedDistributed
SLB device sits off to the side, and only SLB device sits off to the side, and only receives the packets it needs to based receives the packets it needs to based on flow setup and tear down.on flow setup and tear down.
SLB: Traditional View with SLB: Traditional View with NATNAT
SLBClient
Server1
Server2
Server3
SLB: Traditional View without SLB: Traditional View without NATNAT
SLBClient
Server1
Server2
Server3
Load-Balance: Layer 3 / 4Load-Balance: Layer 3 / 4
Looking at the destination IP address Looking at the destination IP address and port to make a load-balancing and port to make a load-balancing decision.decision.
In order to do that, you can In order to do that, you can determine a real server based on the determine a real server based on the first packet that arrives.first packet that arrives.
Layer 3 / 4: Sample FlowLayer 3 / 4: Sample Flow
SLBClient
Server1
Server2
Server3
1: SYN
2: SLB makes decision on Server
3: SYN
4: SYN/ACK
5: SYN/ACK
Rest of flow continues through HTTP GET and Server response.
Load-Balance: Layer 5+Load-Balance: Layer 5+
The SLB device must terminate the The SLB device must terminate the TCP flow for an amount of time TCP flow for an amount of time BEFORE the SLB decision can be BEFORE the SLB decision can be made. made. For example, the cookie value must be For example, the cookie value must be
sent by the client, which is after the TCP sent by the client, which is after the TCP handshake before determining the real handshake before determining the real server.server.
Layer 5+: Sample FlowLayer 5+: Sample Flow
SLBClient
Server1
Server2
Server3
1: SYN
2: SLB device determines it must proxyflow before decision can be made.
6: SYN
7: SYN/ACK
3: SYN/ACK
Rest of flow continues with Server response.Note: the flow can be unproxied at this point for efficiency.
4: ACK
5: GET w/ Cookie
8: ACK
9: GET w/ Cookie
SLB: Distributed SLB: Distributed ArchitectureArchitecture
Client
FE
FE
FE
Server
Server
Server
SLB
FE: Forwarding Engines, which are responsible for forwarding packets. They ask the SLB device where to send the flow.
FE
SLB
Client
Distributed Architecture: Sample FlowServer1
Server2
Server3
Server4
1: TCP SYN
2: FE asks where to send flow.
3: Service Mgr tells it to use Server2.
4: flow goes to Server2.
Subsequent packets flow directly from Client to Server2 thru the FE.The FE must notify the SLB device when the flow ends.
Server FeedbackServer Feedback
Determining Health of Real Determining Health of Real ServersServers
In order to determine health of real In order to determine health of real servers, SLB can:servers, SLB can: Actively monitor flows to that real Actively monitor flows to that real
server.server. Initiate probes to the real server.Initiate probes to the real server. Get feedback from real server or third Get feedback from real server or third
party box.party box.
Server FeedbackServer Feedback
Need information from real server Need information from real server while it is a part of a server farm.while it is a part of a server farm.
Why?Why? Dynamic load-balancing based on ability Dynamic load-balancing based on ability
of real server.of real server. Dynamic provisioning of applications.Dynamic provisioning of applications.
Server Feedback: Use of Server Feedback: Use of InformationInformation
Availability of real server is reported Availability of real server is reported as a ‘weight’ that is use by SLB as a ‘weight’ that is use by SLB algorithms (e.g., weighted round algorithms (e.g., weighted round robin, weighted least connections).robin, weighted least connections).
As weight value changes over time, As weight value changes over time, the load distribution changes with it.the load distribution changes with it.
How to Get WeightsHow to Get Weights
Statically configured on SLB device – Statically configured on SLB device – never change.never change.
Start with statically configured value Start with statically configured value on SLB device for initial start-up, then on SLB device for initial start-up, then get weight from:get weight from: Real serverReal server Third party box / Collection PointThird party box / Collection Point
It is assumed that if a third party box is being It is assumed that if a third party box is being used, it would be used for all the real servers used, it would be used for all the real servers in a server farm.in a server farm.
Direct Host FeedbackDirect Host Feedback
Description: Have “agents” running Description: Have “agents” running on host to gather data points. That on host to gather data points. That data is then sent to SLB device just data is then sent to SLB device just for that physical server.for that physical server. Note: agent could report for different Note: agent could report for different
applications on that real server.applications on that real server. Agent could be based on available Agent could be based on available
memory, general resources available, memory, general resources available, proprietary information, etc.proprietary information, etc.
Direct Host FeedbackDirect Host Feedback
Pros:Pros: Have some way to dynamically change Have some way to dynamically change
physical server’s capability for SLB flows.physical server’s capability for SLB flows. Cons:Cons:
SLB device must attempt to normalize data for SLB device must attempt to normalize data for all real servers in a server farm. If have all real servers in a server farm. If have heterogeneous servers, it is difficult to do.heterogeneous servers, it is difficult to do.
Difficult for real server to identify itself in SLB Difficult for real server to identify itself in SLB terms for case of L3 vs. L4 vs. L5, etc SLB terms for case of L3 vs. L4 vs. L5, etc SLB scenarios.scenarios.
Third Party Feedback: Third Party Feedback: NetworkNetwork
SLBClient
Server1
Server2
Server3
Collection Point
Host to Third Party Host to Third Party FeedbackFeedback
Description: Real servers report data to a Description: Real servers report data to a ‘collection point’. The ‘collection point’ system ‘collection point’. The ‘collection point’ system can normalize the data as needed, then it can can normalize the data as needed, then it can report for all physical servers to the SLB device.report for all physical servers to the SLB device.
Pros:Pros: Have a device that can analyze and normalize the data Have a device that can analyze and normalize the data
from multiple servers. The SLB device can then just do from multiple servers. The SLB device can then just do SLB functionality.SLB functionality.
Cons:Cons: Requires more communication to determine dynamic Requires more communication to determine dynamic
weight – could delay the overall dynamic affect if it takes weight – could delay the overall dynamic affect if it takes too long.too long.
RSerPoolRSerPool
ASAP
PU
ASAP
ENRPServers
RSerPool: ArchitectureRSerPool: Architecture
PE
PE
PE
RSerPool: OverviewRSerPool: Overview
RSerPool protocols sit between the user application RSerPool protocols sit between the user application and the IP transport protocol (session layer).and the IP transport protocol (session layer).
The application communication is now defined over The application communication is now defined over a pair of logical session layer endpoints that are a pair of logical session layer endpoints that are dynamically mapped to transport layer addresses.dynamically mapped to transport layer addresses.
When a failure occurs at the network or transport When a failure occurs at the network or transport layer, the session can survive because the logical layer, the session can survive because the logical session endpoints can be mapped to alternative session endpoints can be mapped to alternative transport addresses.transport addresses.
The endpoint to transport mapping is managed by The endpoint to transport mapping is managed by distributed servers providing resiliency.distributed servers providing resiliency.
RSerPool / SLB: Unified RSerPool / SLB: Unified ApproachApproach
(A Work in Progress)(A Work in Progress)
Unified View: OverviewUnified View: Overview
Preserve the RSerPool architecture:Preserve the RSerPool architecture: Any extensions or modifications are backwards compatible Any extensions or modifications are backwards compatible
with current RSerPool.with current RSerPool. SLB extensions at ENRP Server and PE are optional based on SLB extensions at ENRP Server and PE are optional based on
pool policy chosen / implemented.pool policy chosen / implemented. Utilize SLB distributed architecture:Utilize SLB distributed architecture:
Introduce FE when using SLB pool policies.Introduce FE when using SLB pool policies. Add SLB technology to the ENRP Server:Add SLB technology to the ENRP Server:
SLB-specific versions of pool policies.SLB-specific versions of pool policies. SLB-<pool_policy>: example SLB-WRR takes into account SLB-<pool_policy>: example SLB-WRR takes into account
additional host feedback such as number of flows on each PE.additional host feedback such as number of flows on each PE. Add server feedback:Add server feedback:
Enable delivery of host feedback from PEs to home ENRP Enable delivery of host feedback from PEs to home ENRP Server.Server.
Enable delivery of host feedback to FE from ENRP Server.Enable delivery of host feedback to FE from ENRP Server.
Unified: Component Unified: Component DescriptionDescription
ASAP:ASAP: Between PE and ENRP Server is extended to include additional Between PE and ENRP Server is extended to include additional
host feedback such as current number of flows on PE.host feedback such as current number of flows on PE. Encapsulation of host feedback protocol in pool element Encapsulation of host feedback protocol in pool element
parameter.parameter. Information will be replicated among peer ENRP Servers.Information will be replicated among peer ENRP Servers.
Subscription service and/or polling between ENRP Server and Subscription service and/or polling between ENRP Server and PU allows delivery of host feedback (membership, weights, PU allows delivery of host feedback (membership, weights, flows, etc).flows, etc).
Subscription is between PU and current ENRP Server (not Subscription is between PU and current ENRP Server (not replicated).replicated).
PU must be re-register subscription upon selection of new ENRP PU must be re-register subscription upon selection of new ENRP Server.Server.
Subscription and polling service previously discussed in design Subscription and polling service previously discussed in design team as an addition to core ASAP functionality.team as an addition to core ASAP functionality.
Make decision on flow destination based on SLB-specific pool Make decision on flow destination based on SLB-specific pool policy (i.e., load-balancing algorithm).policy (i.e., load-balancing algorithm).
Unified: Component Unified: Component DescriptionDescription
FE:FE: RSerPool enabled application (PU):RSerPool enabled application (PU):
Uses RSerPool API for sending flows to PE.Uses RSerPool API for sending flows to PE. ASAP control plane for PE selection.ASAP control plane for PE selection. Bearer plane uses flow-specific protocol (e.g., HTTP, SIP, Bearer plane uses flow-specific protocol (e.g., HTTP, SIP,
etc) and corresponding transport (e.g., TCP, SCTP).etc) and corresponding transport (e.g., TCP, SCTP). Must know which pools support which applications Must know which pools support which applications
(SLB-types). (SLB-types). Add parameter to SLB-enabled PEs?Add parameter to SLB-enabled PEs?
Choose pool handle based on incoming client requests Choose pool handle based on incoming client requests and supported SLB-types (SLB-L4, SLB-HTTP, SLB-SIP, and supported SLB-types (SLB-L4, SLB-HTTP, SLB-SIP, etc).etc).
If no other SLB-type matches, the SLB-L4 will be used.If no other SLB-type matches, the SLB-L4 will be used. NAT, reverse NAT.NAT, reverse NAT. Proxy service.Proxy service.
Unified: Component Unified: Component DescriptionDescription
FE (continued):FE (continued): Configuration:Configuration:
Server Pools:Server Pools: Static configuration of pool handles – pool names are Static configuration of pool handles – pool names are
resolved upon initialization.resolved upon initialization. Static configuration of pool handles and PE detail, Static configuration of pool handles and PE detail,
including initial/default weights.including initial/default weights. Automagic configuration?Automagic configuration?
Protocol Table:Protocol Table: Maps supported SLB-types to pool handles by looking Maps supported SLB-types to pool handles by looking
for best match in incoming packet, e.g.,for best match in incoming packet, e.g., SLB-L4 (must implement).SLB-L4 (must implement). SLB-HTTP.SLB-HTTP. SLB-SIP.SLB-SIP.
Unified: Component Unified: Component DescriptionDescription
PE:PE: SLB-enabled PEs must support dynamic SLB-enabled PEs must support dynamic
host feedback.host feedback.
Unified: Layer 3/4 ExampleUnified: Layer 3/4 Example
PU / FEClient
ENRP Server
PE1
PE2
PE3
1: TCP SYN3: TCP SYN is sent to PE2
4: SYN/ACK5: SYN/ACK
ASAP with host feedback
2: Correlate request to SLB-type; thenchoose pool handle. Then do a send tothat pool handle.
ASAP Pool handle resolution &subscription/polling.
ENRP Server
Server Feedback: How to Server Feedback: How to Implement with RSerPoolImplement with RSerPool
Unified: PE CommunicationUnified: PE Communication
PEs will send their weights to ENRP server via PEs will send their weights to ENRP server via ASAP protocol.ASAP protocol. Server agent on host provides weight to PE application.Server agent on host provides weight to PE application.
There are some protocols that exist for reporting There are some protocols that exist for reporting this information. The current list:this information. The current list: Server/Application State Protocol, SASP:Server/Application State Protocol, SASP:
Joint IBM / Cisco Protocol.Joint IBM / Cisco Protocol. IETF draft is currently available.IETF draft is currently available.
Dynamic Feedback Protocol, DFP:Dynamic Feedback Protocol, DFP: Cisco developed Protocol.Cisco developed Protocol. IETF draft is in progress.IETF draft is in progress.
Design Team Work ItemsDesign Team Work Items
How to Implement: To Do How to Implement: To Do ListList
Details, Details, Details .....Details, Details, Details ..... Reconcile design with pool policy draft:Reconcile design with pool policy draft:
Determine what information needs to be passed.Determine what information needs to be passed. Determine what algorithms need to be added Determine what algorithms need to be added
where.where. Define SLB-<pool policies>.Define SLB-<pool policies>.
Determine best method for implementation Determine best method for implementation of host feedback.of host feedback.
Complete Layer 5 example with session Complete Layer 5 example with session persistence mechanism at FE.persistence mechanism at FE.
How to Implement: To Do How to Implement: To Do ListList
Polling / Subscriptions.Polling / Subscriptions. Complete DFP IETF draft, so it can be Complete DFP IETF draft, so it can be
considered.considered. Everything else.Everything else.