CNGI/CERNET2 update MA Yan, CERNET/BUPT APAN19, Bangkok Jan.26, 2005.
Support for Video streaming over CERNET-080713 · 2017-07-20 · Support for Video Streaming over...
Transcript of Support for Video streaming over CERNET-080713 · 2017-07-20 · Support for Video Streaming over...
1
Support for Video Streaming over CERNET
Congxiao BaoTsinghua University
2008-7-12
Outline
• Background• Problem & challenges• Design goals• Solutions• Cases study• Conclusion
2
Background-Network deployment &Video streaming
activities
Network deployment
http://lake.net.edu.cn:8033/map-history.html
3
CERNET/CERNET2•IPv4 national backbone•IPv4 unicast and multicast•38 GigaPops in 36 cities •2,200 universities connected•Self-funded operation•High utilization (70%+)
•Pure IPv6 National Backbone•IPv6unicast and multicast•25 GigaPops in 20 cities•120 universities connected•Free usage•Low utilization (10%)
Video Streaming Trials over our networks
• Distance-learning in – Language– Technology– Music– Art performance– Medicine
• Conference attending• Project seminar• Regular NOC working meeting• etc
4
SIGCOMM2007
Tsinghua-KISTI uncompressed HD transmission
APAN telemedicine
5
Video Streaming System• LD video
– compressed 1M H.323/Accessgrid/Isabel• SD video
– Compressed 30 Mbps DVTS– Uncompressed 220 Mbps MCGILL
• HD Video– Compressed 22 Mbps VLC– Uncompressed 800 Mbps –1.2 Gbps KISTI-code
• 3D video– Compressed 60Mbps 2*DVTS
• Huge data transmission– EUChinagrid-1– MaZeP2P 500Mbps
Key technologies
• Network support– Unicast/Multicast– IPv4/IPv6
• protocol feature– HP UDP real-time video applications– HP TCP applications
• Interactive way– Point to point transmission– Multi-points to multi-points transmission
6
Remarks• HP video applications are required over
distributed backbones from users• but……from our experiences
– Multicast based applications is very important among academic communities, but enabling multicast is a hard work
– End to end performance guarantee for HP UDP or TCP applications is impossible without advanced measurement and bandwidth resources adjustment
• Analysis of the fundamental problems and try to find solutions
Problems & challenges-multicast network and distributed UDP/TCP
applications
7
From our previous presentations
Multicast network monitoring
• Enabling multicast on the backbone is a tough work
• The router supported commands are not enough for the multicast debugging
• End to end (user-controlled) monitoring tools are mandatory
8
HP-UDP applications
• Take DVTS as an example – Hardware is based on consumer products– Software is free and open source– Useful for mission-critical applications– Special demand for network support
• 30Mbps/single video stream
DVTS is a good start point!!!
Setup a DVTS demonot easy
1. Simplest case----point to point with intradomain routing
User B
UserA
DVTS unit
DVTS unitsending
receivingsending
receiving
9
Normal procedures for a DVTS setup
Step 1: Solicit an application Step2: Exchange contact info Step3: Mailing list setupStep4: Email bomb (equipment config/test schedule/…)Step5: 1st test - A to B is not ok, B to A is not ok
2nd test - maybe network /application system problemping/traceroute/ network adjusting
3nd test - network is ok. But still can not see or hear …software reinstallation /even change anther pc/camera checking/speak checking….
……10nd test (rehearsal) - Everything is ready, engineers are
exhausted!!! Step6: Formal event - let’s keep fingers cross!!!
Very difficult ……
2. More complex cases• Multi-points with intradomain routing• Point to point with interdomain routing• Multi-points with interdomain routing
Especially with asymmetrical routing….. for international collaboration
10
What is the problem and where are the bottlenecks
AS2AS2 AS3AS3
AS1AS1
access access
Bad cablingWrong speed
Wrong duplex mode
Bad cablingWrong speed
Wrong duplex mode
firewall firewallBandwidth bottleneck
Host parameters
Host parameters
Problem statement• Technique issues
– DVTS application behavior and network test results may have different meaning
• DVTS application: 30Mbps, UDP port 8000, etc• Network test: 1Kbps, ICMP, show AS-path
– The packets are traveling cross difference AS, the path is likely asymmetric, and the end2end path may not be the same as what the NOC sees.
• The end users do not know the network traffic and the NOCs do not know the end2end
– The firewalls and the QoS policies may deployed somewhere in the path
• Ratelimite match acl 140 in g0/0
11
Problem statement(cont.)
– The network behavior may change between the rehearsals
• Murphy’s law– It is not a easy job to setup DVTS system
• DV camera, 1394 cable, computer, clicking the menu
– The end users and the NOCs are usually speaking different languages
• ^_^
Problem statement (cont.)
• Non-technique issues--- heavy burden of negotiation– Language
• Natural language– Time differences (sleepless testing)– POC (right person to find right person)– Etc
• Multi-sites work together– Need enough patience for more than 5
sites to work together at the same time if each site could not fixed everything ASAP
12
Why HP TCP applications
• Reliable mass data transfer
33
TCP performance
use the Principle of Network Efficiency:– only inject more data into a loaded network
when you believe that the receiver has removed the same amount of data from the network
– TCP uses ACKs as the sender’s timer
ACK packets
Data packets
RS
13
36
TCP session behaviour
TCP Rate Control
0
1 0
2 0
3 0
4 0
5 0
6 0
7 0
0 5 1 0 15 20 25 30
TC
P W
indo
w S
ize
Network congestion level
Slow Start Behaviour Congestion Avoidance Behaviour
HP TCP application test
max__ _
_ _throughput
receive window sizeround trip time
=
max_,.
. / secthroughput bytes= =65 535
0 56117 027
max_,.
. / secthroughput bytes= =24 567
0 5643 886
Formula
SUN §WINDOWS
Theory
TCP transfer speed
14
Problem statement (cont.)
• Packet loss– Packet loss is the key factor for auto-
adjustment of TCP rate-control, it should be very very small
• Window size – If the packet loss is zero, but RTT is large for
a specific physical link, increase the window size of the operation system of the end system can improve the transmission throughput
Summary• More and more demonstration and trials on high-
performance applications– Application-specific stream or data– Consume large network bandwidth– Heavily rely on network support
• User controlled tools, work with NOC closely– Measurement and monitoring
• Network performance measurement tool– Unicast/multicast
• Application-specific measurement tools• Distributed measurement servers
– End to end performance guarantee• Inter-AS routing adjustment• Agreement on enabling application-related ACLs
15
Design Goals-Requirements and answers
Goals
• Use measurement tools to– Built a common language between the application
users/engineers and the network administrators• Separate the network problems from the end system
problems
– Help users to do self-service• Test without the help of the counterpart• Simulate the real DVTS behavior without using the real
DVTS system
– Find the network segments which cause the problem• Find the bottleneck of the network
16
Requirements and answers• Open and share the network and application test
statistics both for network administrators ,application system engineers and end-users– Similar 5+ elements for both simulated random data and video
• To do the test automatically with the measurement server at any time without partners– Measurement servers setup and running
• To do the trouble-shooting hop by hop for inter-AS collaboration– Distributed measurement servers in each AS management
How about setup a measurement servers at every AS and user’s subnet?
Solutions-Scalable Application-specific measurement
framework
17
Application-support Architecture
Network Infrastructure(Unicast/Multicast)
Open Network monitoring and management
Middleware(Directory services/dial plan/Security)
Application systemEngineering environment
Application protocols
Otherprotocols
Network protocols
ApplicationSpecialist
Software developer
Networkengineer
Humanprotocols
Application self-testing Tools
SASM building blocks• General client-server architecture
– Unidirectional and bidirectional• User-controlled measurement tools integrated
– Basic network performance monitoring– Application-specific measurement – Real application trial
• Distributed measurement servers– Sending or reflecting mode– Distributed server management
• User access interface– Easy to use & visualization
• web service– Prefix authentication & DDOS
18
User-controlled measurement tools• Basic network performance monitoring
– unicast network connectivity • TCP-ICMP-ping
– multicast network connectivity• Ssmping/asmping
• Application-specific measurement tools– Simulated DVTS streaming
• UDP-throghput-dvping– Simulated UHD streaming
• UDP-throghput-uhdvping– HP TCP throughput measurement
• iperf• Real-application reflecting
– DVTS– UHD– FTP
Focus…….
Operation modes of basic tools
19
The technical specification
tein2-hk2-dvts33 % dvping -hUsage:dvping [-h] [-p snd_port] [-rp rcv_port] [-t period] [-tt tt] [-bw bandwidth] host
tein2-hk2-dvts33 % dvmcast -hdvmcast [-h] [-t time_limit]
[-l log_file] [-lp log_period] [-ot open_th] [-ft fluc_th][-noloop] [-dvf dv_file] [-4|-6] [-rp rcv_port] [-asm grp] [-ssm src grp] [-sf src_flt] [-sff sf_file][-ech ech_port] [-d [dst_addr] [dst_port]] [-df dst_file]
Introduction & usage
20
Sending mode
Reflecting mode
21
Distributed measurement servers
• Server discovery alone the packet traveling path in each AS– AS path and DNS server
• New server registration – By manual or automatically
Sender
Receiver
display
Browser
DMServ
httpd
Sender
Receiver
display
Browser
DMServ
httpd
DMServ
httpd
DMServ
httpd
DMServ
httpd
User AUser B
AS1 AS2 AS3
22
Find the measurement servers
SASM Framework
23
SASM Framework
Web interface
http://202.112.35.200:8056/ddvts.htmlUser: tein2, passwd: tein2
24
TEIN2 DM servers distribution
The TCP measurement Tool
25
The testing steps• Default test
– The default TCP transfer speed for the e2e peer• Window size test
– Check the client window• Parallel session test
– Check the server window– Physical speed limit
• Comparison with ping and UDP (dvping) tests– Check packet loss (ping and dvping)– Check RTT (ping and dvping)– Check throughput (dvping)
A Document
http://noc.tein2.net:8036/support-for-hp/hp-mc-20070507.html
Login: tein2 passwd: tein2
26
Summary of the TEIN2 NOC measurement servers
server clinet
ICMPICMP
U/UDPU/UDPM/UDP
U/UDP
U/UDP
U/TCP
U/TCP
U/UDP
ping
ssmping
dvping
iperf –c
smjoin
iperf –s
smclock
dvmcast
ssmpingd
Socket stack
U – UnicastM - Multicast
The specification is important
• Application Area– Medicine
• Application system– DVTS
• Network specification– UDP/p=8000/bw=30Mbps/RTT<150ms/ Erlang– IP/prefix/ASN
27
Cases study-dvping/dvmcast tools usage
Tsinghua-kISTI UHD transmission trialCERNET2 DVTS deployment
TEIN2 multicast support for canalAVIST
From Arlington to TEIN2 Beijing server
using WALN Not good
using RJ45 Good
28
Tsinghua-kISTI UHD transmission trial
• Basic config– Uhd packet format
• FRC4751
– Changeable Packet size• From 1472Bytes
– Same port • 5004
– Linux configurations• memory:2G• CPU:3.2G(2.4G)*4
Test topology (IPv4/IPv6)
s
s
BJ(6IX-CERNET2)
HK
KOREANet2
s
1G
1G
1G1G
1G
1G1G
10G
210.25.189.1262001:252:0:2::1001
210.25.189.1252001:252:0:2::1002
MSSender/receiver
134.75.21.902001:320:10:4000:/64
MSSender/receiver
Video senderVideo receiver
134.75.21.802001:320:10:4000::1001
134.75.21.842001:320:10:4000::1002
29
Test results-IPv4UHD-specific measurement (1)
• Sending mode( from BJ-KISTI)[cernet@localhost bin]$ ./hdvmcast -4 -l<source_log ip="210.25.189.126" port="32771" start="2007/10/30 16:20:44" stop="2007/10/30 16:20:54" packets="764166" lost="0%" bytes="1124852352" avg_bw="900Mbps"><bw_change time="2007/10/30 16:20:45" bandwidth="897Mbps"/>
<bw_change time="2007/10/30 16:20:46" bandwidth="905Mbps"/><bw_change time="2007/10/30 16:20:47" bandwidth="900Mbps"/><bw_change time="2007/10/30 16:20:48" bandwidth="899Mbps"/><bw_change time="2007/10/30 16:20:49" bandwidth="901Mbps"/><bw_change time="2007/10/30 16:20:50" bandwidth="896Mbps"/>
<bw_change time="2007/10/30 16:20:51" bandwidth="905Mbps"/><bw_change time="2007/10/30 16:20:52" bandwidth="899Mbps"/>
<bw_change time="2007/10/30 16:20:53" bandwidth="898Mbps"/><bw_change time="2007/10/30 16:20:54" bandwidth="905Mbps"/>
</source_log>• Sending mode( from KISTI-BJ)
bao@debian:~/hdvts/bin$ ./hdvmcast -l<source_log ip="134.75.21.90" port="32773" start="2007/10/30 15:33:39" stop="2007/10/30 15:33:49"
packets="764008" lost="1.08%" bytes="1112475776" avg_bw="889Mbps"><bw_change time="2007/10/30 15:33:40" bandwidth="890Mbps"/><bw_change time="2007/10/30 15:33:41" bandwidth="893Mbps"/><bw_change time="2007/10/30 15:33:42" bandwidth="889Mbps"/><bw_change time="2007/10/30 15:33:43" bandwidth="890Mbps"/><bw_change time="2007/10/30 15:33:44" bandwidth="892Mbps"/><bw_change time="2007/10/30 15:33:45" bandwidth="890Mbps"/><bw_change time="2007/10/30 15:33:46" bandwidth="894Mbps"/><bw_change time="2007/10/30 15:33:47" bandwidth="894Mbps"/><bw_change time="2007/10/30 15:33:48" bandwidth="891Mbps"/><bw_change time="2007/10/30 15:33:49" bandwidth="894Mbps"/>
</source_log>
Test results-IPv4UHD-specific measurement (2)
• Reflecting mode
bao@debian:~/hdvts/bin$ ./hdvping 134.75.21.90 -bw 900M -t 300ping 134.75.21.90 with uncompressed HDV data:send port:5004recv port:5004packet size:1472period :300 ssnd_pkt rcv_pkt snd_rate rcv_rate loss rtt22906891 21542645 899Mbps 846Mbps 5.96% 4856ms
Due to CPU usage problem………………
30
Test results-IPv4UHD-specific measurement (3)
Test results-real video transmission (IPv4)
BJ
HK
KOREANer2
10G1G
receiver
sender 1G1G sender
receiver
Tsinghua University
6IX/CERNET2
900Mbps
900Mbps
KISTI
31
Test results-IPv6(1)• Sending mode (from BJ-KISTI)<source_log ip="2001:252:0:2::1001" port="32770"
start="2007/10/29 18:37:23" stop="2007/10/29 18:37:33" packets="812201" lost="47%" bytes="624801408" avg_bw="499Mbps">
• Sending mode (from KISTI-BJ) <source_log ip="2001:320:10:4000:230:48ff:fe90:564f"
port="32771" start="2007/10/28 17:42:45" stop="2007/10/28 17:42:55" packets="808379" lost="47.2%" bytes="619944468" avg_bw="496Mbps">
Test results-IPv6(2)video-reflecting-measurement
Dear Zhang Xuan,
KREONET NOC found a number of bottleneck points in domestic IPv6 network.I guess it is coupled with using dual-stack ipv4 & ipv6 in KR side.
Currently, ipv6 performance is confined with 450Mbps so i'm looking for the other ways tocontinue our tests for ipv6.
Best Regards,
Jinyong Jo from KISTI.
210.25.189.125(BJ receiver)210.25.189.123(BJ sender)134.75.21.90(korea echo)
32
CERNET2 DVTS deployment
• Connection topology• Node configuration• Data flow• Reflector development• Traffic monitoring• Website support• DVTS-specific measurement tools
• 25 DVTS nodes (diameter is about 1000+ kms)
• Bidirectional transmission
• 750MbpsDVTS
DVTS
DVTS
DVTS
DVTS
DVTS DVTS
DVTS DVTS
THU
XA
WH
HF
SJTU
GZ
CD
SY
TJ
CHQ
XM
NJ
LZ
DL
HABCHC
JN
HZ
CHS
ZZ
PKUBUPT
BUAA
FDU
TJU
DVTS DVTS DVTS
DVTS
DVTS
DVTS
DVTSDVTS
DVTS
DVTS
DVTSDVTS
DVTS
DVTSDVTSDVTS
BJ
SH
CERNET2 DVTS Connection Topology
33
CERNET2 DVTS Node configuration
CERNET2CERNET2
DVTSsender
DVTS Testing server
R
SW
DVTSreceiver
Node configuration—Central Node (1)
R R RReflector
array
sender
switch GE
Multicast/unicast
T640
12 receivers 13 receiversbackup
CE
RN
ET
2
THU sending
Reflector array
Each reflector receives
multicast/unicast stream from sender and converts to
12(13)unicast duplicated copies
to 25 different IPv6 hosts
distributed in 25 Pops
30Mx25=750Mbps
34
Equipment Configuration— Central Node (2)
bj wh gz nj sh
xa cd sy tjn lzh
chq chs zhz hef jnn
xmn hzh dln chc hrb
pku bupt buaa fdu tju
PAD displayarray
PCcluster
switchGE
30Mx25=750Mbps
PC clusterEach PC receives a
DVTS stream from one of the
25 Pops, respectively T640
CERNET2
THU Receiving
SJTU XJTU …… JLU
CERNET2
switch switch
………
…
北大
吉大 网管
DV
CERNET2 national Center
THU
R R RGE GE
CERNET2 25 DVTS nodes
DVTSsending PKU …… JLU
CERNET2
switch switch
………
…
PKU
JLU NMS
DV
CERNET2 national center
THU
R R RGE GE
DVTSsending
DVTSreceiving
DVTSreceiving
SJTU
Data Flow
35
Video Wall
Reflector Development
• It runs on Linux OS and developed using g++• It supports IPv4 and IPv6• It supports Unicast and multicast (SSM and
ASM)• It can reflects 20+ streams with gigabit
network card• It will be released as Open source soon
source
reflector
reflector
reflector
SSSM multicast
unicast
36
Network Traffic Monitoring
CERNET2 DM servers distribution
37
Website Support
TEIN2 multicast support for CanalAVIST
• Support for distributed DVTS application• User-controlled monitoring tools &
distributed servers • All protocols work together• documentation
38
Conclusion
39
Conclusion
• High performance UDP (real-time video) enabling is not only a pure application test
• Involves network performance monitoring and application-specific measurement
• Distributed servers are important for user-controlled tools
• With the measurement results, users can enable a HP application sucessfully
Thank You!