ITTC © James P.G. Sterbenz Communication Networksjpgs/courses/nets/lecture-mm... · 2017-12-11 ·...
Transcript of ITTC © James P.G. Sterbenz Communication Networksjpgs/courses/nets/lecture-mm... · 2017-12-11 ·...
© James P.G. SterbenzITTC
04 December 2017 © 2004–2017 James P.G. Sterbenzrev. 17.0
Communication NetworksThe University of Kansas EECS 780
Multimedia and Session Control
James P.G. Sterbenz
Department of Electrical Engineering & Computer Science
Information Technology & Telecommunications Research Center
The University of Kansas
http://www.ittc.ku.edu/~jpgs/courses/nets
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-2
Multimedia and Session ControlMS.1 Multimedia Applications
MS.1 Multimedia applications
MS.2 Multimedia streaming and transport
MS.3 Session Control
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-3
Multimedia ApplicationsOverview
• Multimedia applications
– involve audio and/or video
– perhaps in addition to conventional data
• Modes of operation
– streaming
– interactive
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-4
Multimedia ApplicationsStreaming
• Streaming multimedia
– media is streamed from server to client
• not download and then play
– user may have back-channel to server to control playback
• Examples
– Youtube, Dailymotion, Netflix, Hulu Lecture AL
– IPTV: television broadcast over IP
– Internet VOD (video on demand)
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-5
Multimedia ApplicationsInteractive
• Interactive multimedia
– users communicate with using audio and/or video
– peer-to-peer interaction
• E2E or shared multicast with reflector Lecture AL
– may combine with collaborative data sharing
• document sharing or shared whiteboard
• distance learning with information access
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-6
Multimedia ApplicationsInteractive Examples
• Examples
– VoIP (voice over IP) telephony and video conferencing
• e.g. skype
– shared collaboration environments
– virtual reality interactions, e.g Second Life
– multiuser games
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-7
Multimedia and Session ControlMS.2 Multimedia Streaming and Transport
MS.1 Multimedia applications
MS.2 Multimedia streaming and transportMS.2.1 Multimedia streaming and RTSP
MS.2.2 Multimedia transport: RTP and RTCP
MS.3 Session Control
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-8
Multimedia Streaming and TransportMS.2.1 Multimedia Streaming and Transport
MS.1 Multimedia applications
MS.2 Multimedia streaming and transportMS.2.1 Multimedia streaming and RTSP
MS.2.2 Multimedia transport: RTP and RTCP
MS.3 Session Control
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-9
Multimedia Streaming Stream Transfer Mode
• Various mechanisms to start stream– explicit client request
– server push
– may or may not establish connection state
• Data flow – synchronisation and control
• embedded or
• out-of-band
REQUEST
RELEASE
CONNECT
SETUP
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-10
Multimedia StreamingInternet-Based Techniques
• Server
– source of media stream
• Client
– media player
• Transport
– end-to-end data transfer and control protocols
is TCP appropriate?
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-11
Multimedia StreamingDelay and Jitter
• Assume CBR source
– note that some codecs are VBR
constant bit
rate video
transmission
time
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-12
Multimedia StreamingDelay and Jitter
• Assume CBR source
• Network imposes jitter
why?
constant bit
rate video
transmission
time
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-13
Multimedia StreamingDelay and Jitter
• Assume CBR source
• Network imposes jitter
– variable network delay
Consequences?
constant bit
rate video
transmission
time
variable
network
delay
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-14
Multimedia StreamingDelay and Jitter
• Assume CBR source
• Network imposes jitter
• Received flow no longer CBR
– implication?
constant bit
rate video
transmission
time
variable
network
delay
client video
reception
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-15
Multimedia StreamingDelay and Jitter
constant bit
rate video
transmission
time
variable
network
delay
client video
reception constant bit
rate video
playout at client
client playout delay
buff
ere
d
vid
eo
• Assume CBR source
• Network imposes jitter
• Received flow no longer CBR
– receiver must buffer to compensate and playback CBR
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-16
Multimedia Streaming Client Playback Buffer
• Playback buffer absorb jitter– adds delay proportional to jitter
– may also be used to reorder if misordering permitted by TP
maximum playout point
1
application
2 3 4 6
5 6 7
playout buffer
receiving end system
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-17
Multimedia StreamingServer Operation
• Server
– match stream rate to path bandwidth
what is this?
– match stream rate to client capabilities
what is this?
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-18
Multimedia StreamingServer Operation
• Server
– match stream rate to path bandwidth
• congestion avoidance
– match stream rate to client capabilities
• flow control
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-19
Multimedia StreamingClient Operation
• Client: media player
– adaptive playout to compensate for jitter
– decompression
– error concealment
– GUI (graphical user interface)
• controls for interactivity
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-20
Multimedia StreamingInternet Transport
Is TCP appropriate for media streaming?
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-21
Multimedia StreamingInternet Transport
• Transport: RTP over UDP
– avoid TCP control loop delays
– avoid TCP congestion control throttling
– UDP: datagram-based transport with loss tolerance
– RTP: synchronisation added
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-22
Multimedia StreamingTypical Web-Based Architecture
• Web browser– GETs metafile
– launches or embeds player, passing metafile
• HTML5 replacing Flash for embedded streaming
• Media player contacts server
• Server streams audio/video to player
[Kurose]
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-23
Multimedia Streaming ControlRTSP Overview
• RTSP: real time streaming protocol [RFC 2326]
– client-server application layer protocol
• User to control of streaming
– rewind
– fast forward
– pause
– resume
– repositioning …
• Out-of-band control
– port 554
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-24
Multimedia Streaming ControlRTSP Operation
• Metafile communicatedto web browser
• Web browserlaunches player
• Player sets up
– RTSP control connection
– data flow withstreaming server
[Kurose]
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-25
Multimedia Streaming ControlRTSP Metafile Example
<title>Twister</title>
<session>
<group language=en lipsync>
<switch>
<track type=audio
e="PCMU/8000/1"
src="rtsp://audio.example.com/twister/audio.en/lofi">
<track type=audio
e="DVI4/16000/2" pt="90 DVI4/8000/1“
src="rtsp://audio.example.com/twister/audio.en/hifi">
</switch>
<track type="video/jpeg"
src="rtsp://video.example.com/twister/video">
</group>
</session>
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-26
Multimedia Streaming ControlRTSP Example
C: SETUP rtsp://audio.example.com/twister/audio RTSP/1.0
Transport: rtp/udp; compression; port=3056; mode=PLAY
S: RTSP/1.0 200 1 OK
Session 4231
C: PLAY rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0
Session: 4231
Range: npt=0-
C: PAUSE rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0
Session: 4231
Range: npt=37
C: TEARDOWN rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0
Session: 4231
S: 200 3 OK
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-27
Multimedia Streaming and TransportMS.2.1 Multimedia Transport: RTP and RTCP
MS.1 Multimedia applications
MS.2 Multimedia streaming and transportMS.2.1 Multimedia streaming and RTSP
MS.2.2 Multimedia transport: RTP and RTCP
MS.3 Session Control
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-28
Real-Time ProtocolOverview and Transfer Mode
• RTP: real-time protocol [RFC 3550 / STD 0064]
• Streaming of data with real-time properties
– uses UDP for basic transport
• no connection establishment
• basic end-to-end multiplexing
• no reliability
• no flow or congestion control
– adds real-time support fields
• sequence number
• timestamp
• source identifiers
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-29
Real-Time ProtocolSegment Format
• Encapsulated in UDP• RTP header
– version = 02– P: padding bytes at end– X: extension header– CC: CSRC count [4b]– PT: payload type [7b]– sequence # [16b]– timestamp [32b]
• resolution app dependent
– source identifiers• SSRC• CSRC (mixed in)
destination port #source port#
length checksum
CSRC list: contributing source ids
. . .
optional padding
application payload
SSRC: synchronisation source id
timestamp
sequence #02 P X CC PT
#B pad
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-30
Real-Time ProtocolRTP Payloads
• PT: payload type– [www.iana.org/assignments/rtp-parameters]
• Payload formats and encodings
– specified in various RFCs
– audio: RFC 3551, etc.
– video: RFC 2250, etc.
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-31
Real-Time ProtocolControl Protocol Overview
• RTCP: real-time control protocol [RFC 3550 / STD 0064]
• Protocol to control RTP streams
– monitoring quality of service
– convey participant status information
• periodically transmits RTCP control packets to others
• RTCP packet
– contains sender and/or receiver reports
– report statistics useful to application
• # packets sent, # packets lost, interarrival jitter, …
• Feedback can be used to control performance
– sender may modify transmission based on feedback
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-32
Real-Time ProtocolRTCP Functions
• Stream synchronisation
• Bandwidth scaling
todo: expand from Kurose foils
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-33
Multimedia Streaming and TransportMS.3 Session Control
MS.1 Multimedia applications
MS.2 Multimedia streaming and transport
MS.3 Session ControlMS.3.1 Session layer functions and services
MS.3.2 Session control protocols: SIP
MS.3.3 Session control protocols: H.323
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-34
SessionDefinition
• Session is association among usersor application entities
– e.g. teleconference, distance learning session, game
• Participants
– end-system users or application programs
– network-embedded resources
• e.g., transcoders, audio mixers
• Topology
– set of end-to-end transport flows
– multipoint (or point-to-point if only 2 participants)
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-35
Session ControlHybrid Layer/Plane Cube
physical
MAC
link
network
transport
session
application
L1
L7
L5
L4
L3
L2
L1.5
data plane control plane
management plane
socialL8
virtual linkL2.5
Session layer (layer 5):
control plane
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-36
Session LayerSession Control Protocol
• Session protocol
– is responsible for coordination/control of application sessions
network
application
session
transport
network
link
end system
network
link
intermediate
system
network
link
intermediate
systemnetwork
link
intermediate
system
application
session
transport
network
link
end system
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-37
Multimedia Streaming and TransportMS.3.1 Session Layer Functions and Services
MS.1 Multimedia applications
MS.2 Multimedia streaming and transport
MS.3 Session ControlMS.3.1 Session layer functions and services
MS.3.2 Session control protocols: SIP
MS.3.3 Session control protocols: H.323
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-38
Session LayerService and Interfaces
• Session layer (L5) service to application layer (L7)
– establishes and maintains sessions among users/applications
– naming and addressing : to identify and locate participants
– signalling : messages to control application sessions
– may perform routing functions among session resources
• e.g. to transcoders or reflectors
– higher layer analogue of network layer services
• Session layer uses transport layer (L4) services
– a session consists of a coördinated set of end-to-end flows*
* this is not the OSI definition,
but the layer happens to be in the right place (L5)
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-39
Session Control ProtocolSession Establishment
• Establish session: signalling
– assist in location and invitation of participants
– establish one or more transport-layer associations
• flows or connections
– discovery of needed resources
• e.g. transcoders
– routing among participants and resources
– establish session state
• distributed among participants (good scalability)
• centralised in a session controller (efficient coördination)
• combination
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-40
Session Control ProtocolSession Maintenance
• Maintain session
– adjust to dynamic session group membership
• add and removal of participants
• merge and split of sessions
– initiate and terminate transport associations as needed
– add and remove resources as needed
– update session state
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-41
Session Control ProtocolSession Termination
• Terminate session
– signal termination to all participants
– teardown all transport layer flows and connections
– release resources
– remove session state
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-42
Session Control ProtocolSignalling Flow
1. Session signalling
2. Flow setup
3. Data transfer
4. Termination
CONNECT
SESS-REQUEST
SESS-ESTABLISH
SETUP
user negotiation
user initiation
end system end system user user
session establishment
connection establishment
data transfer
network
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-43
Multimedia Streaming and TransportMS.3.2 Session Control Protocols: SIP
MS.1 Multimedia applications
MS.2 Multimedia streaming and transport
MS.3 Session ControlMS.3.1 Session layer functions and services
MS.3.2 Session control protocols: SIP
MS.3.3 Session control protocols: H.323
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-44
Session Control ProtocolsExample: SIP
• SIP (session initiation protocol)
– IETF session protocol
– Internet style signalling based on HTTP messages
– single component
• H.323
– ITU session protocol
– telephony style signalling
– integrated protocol suite for multimedia conferencing:
• signaling, registration, admission control, transport, codecs
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-45
Session Initiation ProtocolOverview
• SIP (session initiation protocol)
– IETF session protocol [RFC 3261]
• Internet style signalling
– based on HTTP-like messages
– SDP typically used to describe media characteristics(session description protocol) [RFC 2327]
• Single component
– works with RTP but does not mandate it
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-46
Session Initiation ProtocolAssumptions and Design Goals
• Internet-based telephone video conference calls
• Addressing
– people identified by names or e-mail addresses
• Support for:
– roaming
– heterogeneous IP-based devices
• Simplicity
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-47
Session Initiation ProtocolCall Services
• Call setup
– mapping of user identifier to IP address
– caller invitation of callee
– negotiation of media type and encoding
– unique call identifier
– mechanisms to end call
• Call management
– add new media streams during call
– change encoding during call
– invite others to join call
– transfer and hold calls
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-48
Session Initiation ProtocolSIP Message
INVITE sip:[email protected] SIP/2.0
Via: SIP/2.0/UDP 167.180.112.24
From: sip:[email protected]
To: sip:[email protected]
Call-ID: [email protected]
Content-Type: application/sdp
Content-Length: 885
c=IN IP4 167.180.112.24
m=audio 38060 RTP/AVP 0
• Bob’s IP address unknown
– SIP servers will resolve
• Alice specifies in Via:SIP over UDP
• SDP used for sessiondescription
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-49
Session Initiation ProtocolCall Example
• Alice’s SIP INVITE message:
– port number & IP address
– PCM preferred audio encoding
• Bob’s reply message 200 OK
– his port number & IP address
– GSM preferred audio encoding
• SIP messages
– HTTP message syntax
– sent over TCP or UDP
• here sent over RTP/UDP
– default SIP port is 5060 time time
Bob's
terminal rings
Alice
167.180.112.24
Bob
193.64.210.89
port 5060
port 38060
m Law audio
GSMport 48753
INVITE [email protected]=IN IP4 167.180.112.24m=audio 38060 RTP/AVP 0port 5060
200 OK
c=IN IP4 193.64.210.89
m=audio 48753 RTP/AVP 3
ACKport 5060
[Kurose]
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-50
Session Initiation ProtocolCall Negotiation
• Media can be sent over RTP or some other protocol
• Codec negotiation:
– suppose Bob doesn’t have PCM μlaw encoder
– Bob will instead reply with 606 Not Acceptable Reply
• and list encoders he can use
– Alice can then send a new INVITE message
• advertising an appropriate encoder
• Call be rejected with replies:
– busy, gone, payment required, forbidden
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-51
Session Initiation ProtocolName Translation and User Location
• Caller has callee’s “e-mail” address, not IP address
• Need to get IP address of callee’s current host:
– user moves around
– DHCP protocol
– user has different IP devices (PC, PDA, car device)
• Result can be based on:
– time of day (work, home)
– caller (don’t want boss to call you at home)
– status of callee
• e.g. sent to voicemail when callee is already talking
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-52
Session Initiation ProtocolSIP Proxy
• SIP Proxy used to resolve to IP address
• Alice sends invite message to her proxy server– contains address sip:[email protected]
• Proxy responsible for routing SIP messages to callee
– possibly through multiple proxies
• Callee sends response back
– through the same set of proxies.
• Proxy returns SIP response message to Alice
– contains Bob’s IP address
Note: proxy is analogous to local DNS server
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-53
Session Initiation ProtocolSIP Registrar
• SIP Registrar keeps track of users
– queried by SIP proxy
• When SIP client started
– REGISTER message sent to registrar server
• Register message exampleREGISTER sip:domain.com SIP/2.0
Via: SIP/2.0/UDP 193.64.210.89
From: sip:[email protected]
To: sip:[email protected]
Expires: 3600
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-54
Session Initiation ProtocolExample of Location and Name Resolution
• Example: [email protected] calls [email protected]. Jim sends INVITE mssage to umass SIP proxy
2. Proxy forwards request to upenn registrar server
3. upenn server returns redirect response to [email protected]
4. umass proxy sends INVITE
to eurecom registrar
5. eurecom registrar forwardsINVITE to 197.87.54.21,
which runs Keith’s SIP client
6–8. SIP response sent back
9. media sent between clients
Note: ACK message not shownSIP client
217.123.56.89
SIP client
197.87.54.21
SIP proxy
umass.edu
SIP registrar
upenn.edu
SIP
registrar
eurecom.fr
1
2
34
5
6
7
8
9
[Kurose]
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-55
Multimedia Streaming and TransportMS.3.3 Session Control Protocols: H.323
MS.1 Multimedia applications
MS.2 Multimedia streaming and transport
MS.3 Session ControlMS.3.1 Session layer functions and services
MS.3.2 Session control protocols: SIP
MS.3.3 Session control protocols: H.323
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-56
Session Control ProtocolsExample: H.323
• SIP (session initiation protocol)
– IETF session protocol
– Internet style signalling based on HTTP messages
– single component
• H.323
– ITU standard session protocol
– telephony style signalling
– integrated protocol suite for multimedia conferencing:
• signaling, registration, admission control, transport, codecs
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-57
Multimedia and Session ControlAdditional References
© James P.G. SterbenzITTC
04 December 2017 KU EECS 780 – Comm Nets – Multimedia & Session Control NET-MS-58
Multimedia and Session ControlAcknowledgements
Some material in these foils comes from the textbook supplementary materials:
• Kurose & Ross,Computer Networking:
A Top-Down Approach Featuring the Internet
• Sterbenz & Touch,High-Speed Networking:
A Systematic Approach to
High-Bandwidth Low-Latency Communication
http://hsn-book.sterbenz.org