Traffic Engineering in Cable-Data System
: Maximizing CMTS port utilization
Jun-seong Cho, Jongheon Park, Munhee Hong
NMS Development Part, Research and Development Center
Hanaro Telecom, Inc. Seoul Korea
Email: [email protected]: +82-26266-4244
(2)NOMS 2004
Introduction• Broadband Internet Service using HFC network is widely available
• Regarding the traffic in HFC network– The advent of multimedia age has continuously increased the traffic.– Worse than that, P2P traffic has dramatically increased and the increased traffic
lasts all time while its users are on online.
• However, network interface has bandwidth limit – Network Interface(CMTS port) is shared by several users in HFC network– Possibility of some monopolizing a network resource– When monopolized, users per port decreases -> port utilization suffers– Need to control network resource monopolization
• Load balancing of Excessive User(EU) traffic can solve the problem– Mechanism finding excessive users has been introduced in my previous research
“Efficient Upstream Management in Cable-Data System“[1]– Compared to my previous research[1][2], this time it deals with not only upstream,
but also downstream, and traffic in general. – It introduces EU traffic load balancing to achieve the better port utilization, and
shows the field test result of the mechanism.
HFC > 2000 HOMESHeadend N COAXFiberC MTS
Cable Serving AreaCable Serving Area
(3)NOMS 2004
Intro on HFC Network’s Cable Data System
Core part of Cable Data System configuration are CMTS, Port, CM, Cell
※ CMTS: Cable Modem Termination System, CM: Cable Modem
C M
C M
C M
....
C MTS Devic e(ex: C isco uBR 7246)
Ports
HFCNetwork
Internet
Internet
CM
CM
CM
CELL
CMTS ONU
Node Element(tap,amplifier etc.)
Optical Fiber
Coaxial Cable
※ a cell can be provisioned to be handled by several port
(4)NOMS 2004
CMTS Port has limited bandwidth…
• Sharing the same port (bandwidth) leads to Bandwidth Contention
• Excessive user– Endangers CMTS performance (Lowers Port Utilization, leads to busy
port)– Make the line unavailable to the others who share the port
※ We define EU (Excessive User) as the user who generate more than 1 percent of the total traffic in the port where it belongs.
X
1) Let Upstream Port’s max bandwidth: 2) User A consumes the following
3) User B consumes the following
4) User C fails to use
All other users can not use!!
Upstream Port
Users sharing the same port
(5)NOMS 2004
Hot potato: EU's Monopolization
3/0 236 18,847 22 9.32% 78.30%4/0 210 15,677 17 8.10% 79.80%5/0 220 17,035 18 8.18% 76.90%6/0 192 19,229 19 9.90% 90.20%
Average 214.5 17697 19 9% 81%
EU’s trafficoccupying
ratio
downstream -port location
the number ofsubscribers
Sum of trafficgenerated by
EU(Kbps)
Number ofEUs
EU ratioamongUsers
• What’s the reality? – A CMTS has been chosen and the traffic has been monitored and the below
shows one of the outcome of the peak time– EUs who are less than 10 %(9 %) of total subscribers are taking up 81% of
total traffic. – May be fair for free competition environment to the EUs– However, if the monopolization especially in its peak time can lead to
harming resource utilization, bandwidth exhaustion, and make the other users suffer….
– What about the actual traffic trends => next slide shows a snapshot of a day
A CMTS from the HFC network in Hanaro Telecom Inc., Traffic has been monitored through ’03.3.1 14:50~’03. 3. 21 14:50’
(6)NOMS 2004
Hot potato: EU's Monopolization(2)
• 4/0 seems ok except 23:00~24:00
• 6/0 reaches the busy port level almost half the day
• The user
complaints in 6/0 have been piled up due to its slow network (Some are reported that they couldn’t use Internet)
• It would be nice if 4/0 can spare some of its free bandwidth to 6/0 when 6/0 becomes busy
* Traffic computed by five minute interval sampling
(7)NOMS 2004
What can we do about it?• Options
① Purchases enough network resources ② Lowers per-cell subscribers③ Engineers traffic, and minimize the
inconvenience until new resources are added.
• Option ➀ and ➁, every time when it reaches the bandwidth limit, – is not economical – makes service provider’s ROI suffer– Thus, is simply unrealistic
• ➂ Traffic Engineering sounds be nicer– if it solves afore mentioned problems.– The paper introduces a module called “ Load Control
Manager (LCM)” – LCM will monitor the traffic, and is responsible for the
traffic engineering
$$$
(8)NOMS 2004
LCM Basic Idea – Port Load Balancing
1. Detec ts busy port thru resourc e performanc e mgmt func tionality
C M
EU C M
C M
HFC Network
...
C MTS device(ex: C isco uBR 7246)
2. Monitor home- user traffic on the busy port, and filters out the EU(extreme user)s
3. C hec k wether there is a port to move the EU
4. Reloc ate the EU to another port that c an handle the traffic
Yes
Reloc ate!!
5. O ther traffic load c ontroling Managements
No
(9)NOMS 2004
LCM 1: Detection of Busy Port※ Same as the mechanism in my previous research [1]
• Everything starts from discovering busy port– If the port is not busy, there is no point of finding EUs. – Performance management module in NMS can easily find the busy port
because it has got the functionality monitoring every CMTS port performance– NMS notifies the detection of the busy port to LCM
• Defining Busy port– Well defining “busy port” condition is very important (ex: Port Utilization, Load)– The status that can endanger the service in no time.– Usually the traffic reaches near to its maximum bandwidth.
• NMS gives the following info to LCM– Busy Port Information : CMTS ID(CMTS IP), Port Index, Port Usge– CMTS Connection Info such as SNMP read community or telnet login id/password
Load C ontrol
Manager(LC M)
NMS
24 hour, global C MTS
Monitoring
NMS detec ts busy port, & notifies it to LC M
(C MTS ID, P ort, Busy rate, C onnection related Info .)
(10)NOMS 2004
LCM 2: Detection of EUs※ Same as the mechanism in my previous research [1]
• As soon as LCM are notified of the busy port from NMS…– LCM connect to the CMTS where the busy port belongs– Scan all the user upstream status on the busy port– After a monitoring interval, calculates all the users upstream speed– Finally draws EU’s CM Mac address, avg traffic that has been detected.
• How to query the traffic info from CMTS– Depends on the interface device offers– SNMP, as a standardized way of managing nw device, may be one candidate– However, using proprietary command interpreter works fast and simple if there
are only one vendor of devices.
• LCM uses the following info to detect EU– Busy Port Information : CMTS ID(CMTS IP), Port Index, Port Usage– CMTS Connection Info such as SNMP read community or telnet login id/password– EU detection rules and policies : EU upstream.downstream thresholds, Detection
period, Port busy status before and after the monitoring, etc.
LC M Monitors the Busy P ort. Detec ts EUs(C M mac , up/ down traffi c) based on EU detec tion po lic ies
EU detec tion rules & polic ies
(11)NOMS 2004
LCM 3: Port relocation• Up to now
– Busy port(port utilization, bandwidth utilization) is detected– EUs(id, traffic info, service category etc) were detected
• Who are going to give it a break?– Find the port that can handle the same cell. Find the ports which are
provisioned to handle the same cell. – Find the port amongst them, that can hold some of EUs. Ex) n EUs, helper port X For (i = 1; i < n; i++) { if (I-th EU traffic+ X’s port traffic) < busy port level traffic) transfer the I-th EU to the port X }– Relocates EUs from the previous port to the port X, Log and save the
record
• What if there’s no port to hole EUs? => Other traffic Load Controlling Methods– Still need to avoid the panic situation where a lot of ordinary users can
not use Internet – Service provider may opt to limit the EUs bandwidth rate only for the
busy time– Or, it might force many EUs to gather into a single port, so ordinary users
get no harm from them– Still in research on the other load controlling method
(12)NOMS 2004
Regarding Implementation• Setting the right thresholds makes all!
– Thresholds always becomes the bottom line of the mechanism. – Threshold can differ depending on the service provider’s
environment: Various combinations can be produced– The values have to be tested, analyzed, and carefully chosen.
(Due to confidentiality, I can not reveal the details of them, but will present its guidelines)
• Busy port related threshold– Everything starts from busy port– Port status, Port utilization, Port alarm record, Channel
Utilization
• EU threshold– Do not make everybody EU. If EUs are majority, then they are
not EU any more.– EU threshold: EU upstream / downstream thresholds,
sampling period
• Things to be solved further…– While port relocation, the EU will lose the connection
temporarily– EU traffic controlling mechanisms such as EU relocation, rate
limiting may violate the SLA.– EUs after relocated may experience slower speed than before
especially in the peak time
(13)NOMS 2004
Result – After applying LCM
6/0, After applying LCM
4/0, After applying LCM
* Traffic computed by five minute interval sampling
(14)NOMS 2004
Thanks for listening
Closing Remarks & Future Work
• However, there are things to solve…– While port relocation, the EU will lose the connection temporarily– EU traffic control mechanisms such as EU relocation, rate limiting may violate the
SLA.– EUs may experience slower speed than before esp. in the peak time
• Work to be done…– LCM and SLA relationship– Other EU traffic control mechanisms when there is no extra port to relocate– Making LCM’s EU detection rules & policies flexible (Rules are to change!)– LCM Scalability & Load Balancing– Relocation record management and linking it with QOS– Marketwise, service related traffic control mechanism
• EU’s monopolization of CMTS port can endanger the Internet service.
• Applying a Traffic Engineering scheme(LCM)…- Better Port Usage Rate: More subscribers to a CMTS por, more
revenue by resource saving- More satisfaction of home-users: Less complaint, better availability to
customers- Saving resources: Network resource + Operational human resources
Top Related