Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE...
-
Upload
ruby-harmon -
Category
Documents
-
view
219 -
download
0
Transcript of Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE...
![Page 1: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/1.jpg)
Real-Time Networking:Real-Time Networking:Data, Voice, and VideoData, Voice, and Video
Dan Schonfeld
Multimedia Communications Laboratory
ECE Department
University of Illinois
Chicago, Illinois
![Page 2: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/2.jpg)
Multimedia Communications Multimedia Communications LaboratoryLaboratory
Rashid AnsariRashid AnsariAshfaq KhokharAshfaq KhokharDan SchonfeldDan Schonfeld
Oliver YuOliver Yu
A Vision To The Future!A Vision To The Future!Real-Time High-Quality Global Video Communications over High-Real-Time High-Quality Global Video Communications over High-
Speed NetworksSpeed Networks
![Page 3: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/3.jpg)
Research AreasResearch Areas
Video NetworkingVideo TrackingVideo Retrieval
![Page 4: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/4.jpg)
Video NetworkingVideo Networking
Collaborators:Collaborators:Rashid Ansari (UIC)Rashid Ansari (UIC)Bulent Cavusoglu (UIC)Bulent Cavusoglu (UIC)Tom DeFanti (UIC)Tom DeFanti (UIC)Jason Leigh (UIC)Jason Leigh (UIC)Emir Mulabegovic (UIC)Emir Mulabegovic (UIC)Oliver Yu (UIC)Oliver Yu (UIC)
![Page 5: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/5.jpg)
Homeland Security Homeland Security Application TestbedApplication Testbed
Chicago Video Surveillance Monitoring of Biochemical Sensor Arrays Helmet Mounted Cameras and Portable Sensors
for Search/Rescue Operations Dedicated Emergency Telephony Network
– Public network lack of reliability during emergency
![Page 6: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/6.jpg)
Real-Time Real-Time Surveillance/MonitoringSurveillance/Monitoring
Central Monitoring– Ok as long as links are
dedicated
Central Monitoring
Search and rescue
Remote sharing with agencies in the field
Mobile and Portable Emergency Response Center
![Page 7: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/7.jpg)
Next Generation Internet Next Generation Internet ArchitectureArchitecture
CR-LDPBGPIGP
ICMPRSVP
Signaling Protocol
Out-of-Band Associated
Signaling Transport
Traffic Control
Fiber/DWDM
SONETATM/PPP/HDLC
IP Routing
Path SelectionMPLS Forwarding
IP QoS Control
HTTP/SMTP/FTP/TELNET
Real-time Multimedia
Middleware
RTPUDP
TCP
Best Effort Guaranteed & Controlled-Load
Non-Real-time Real-time
IP / MPLS
Topology Distribution
Transport
User Application
Current
Internet NGI
Extension
Net
wor
k &
Lay
er
Man
agem
ent
![Page 8: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/8.jpg)
VIPER: FEC/UDP Over VIPER: FEC/UDP Over STARTAPSTARTAP
(Chicago-Amsterdam)
Latency of transmitting 100 packets underUDP, TCP, FEC/UDP with 3:1 redundancy.
0
50
100
150
200
250
300
350
400
0 500 1000 1500 2000 2500
Packet size in bytes
1-way latency in ms
UDP
TCP
FEC over UDP
![Page 9: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/9.jpg)
APPLICATION
RTP
UDP
IP
DATA LINK
PHYSICAL
TRANSPORT
Picture Type
f_code
MBZ T TR AN N S B E P FBV BFC FFV FFC
MPEG-1 Header extension
X Ef(0,0)
f(0,1) f(1,0) f(1,1) DC PS T P C Q V A R H G D
MPEG-2 Header extension
Real-Time Real-Time Transport Transport Protocol Protocol
(RTP)(RTP)
![Page 10: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/10.jpg)
AdaptiveAdaptiveFECFEC
RTP MEDIAPACKETS
ADAPTIVEFEC
ENCODER
RTP FECPACKETS
MPEG-2 RTP NETWORK RECEIVER
1 2 3 4 5 6 7 8 9 1030
32
34
36
38
40
42
44
46
48
50
Packet Loss Ratio Percentage
PSNR in dB
AFECstaticIPBstaticFECoptimalFEC
![Page 11: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/11.jpg)
(c) AFEC
(b) Static IPB(a) Static FEC
(d) Original
FEC SimulationsFEC Simulations
![Page 12: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/12.jpg)
DiffServDiffServ
HOST
EDGE ROUTER CORE ROUTER CORE ROUTER EDGE ROUTER
HOST
TOKEN BUCKET
WEIGHTED FAIRQUEUING
(WFQ)
CHECK IF THERE IS ANYTOKENS AVAILABLE FOR
THE ASSIGNED DSCP
YES
ASSIGN NEXT AVAILABLEDSCP FROM THE TABLE
TOKENS
NO DSCP TABLEAF31AF32AF33AF21AF22AF23AF11AF12AF13
BEST EFFORT
QUEUESAF3
AF2
BESTEFFORT
RANDOM EARLY DETECTION(RED)
TRANSMITTED PACKETS
DROPPED PACKETS
MARKERWEIGHTS
FOR PACKETS
HOSTHOST
HOST
OTHER SOURCES THATCONTRIBUTES TRAFFIC
VIDEOSOURCES
VIDEO SOURCE WITH THEPROPOSED MARKING
ALGORITHM
RANDOM EARLY DETECTION(RED)
RANDOM EARLY DETECTION(RED)
5.7 5.75 5.8 5.85 5.9 5.95 6 6.05 6.1 6.1518
19
20
21
22
23
24
25
26
27
28
Mbps
PSNR in dB
MotionIPBGreedyOptimum
![Page 13: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/13.jpg)
(c) Motion
(b) IPB(a) Greedy
(d) Original
DiffServ SimulationsDiffServ Simulations
![Page 14: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/14.jpg)
Rate-ControllerRate-Controller
NETWORK
DROPPEDPACKETS
HOST HOST
HOST
HOST
HOST
OTHER FLOWS
HOST
HOST
HOST
OTHER FLOWS
RATE CONTROLLED FLOW
( )ip μ
LINEAR STATEFEEDBACKREGULATOR
$
$[ ]
( )
( ),
min( )i
tot
D
E Dp µ
p µ
$( )p µ
Nack packetsor RTCP report
( )iμ
µ
Networkpk
KalmanPredictor
vk
zk
wk
z-1 inv()
1kμ +
( ) 1kμ
−
kα inv()
1kα−
1
−
+
1( )k kp μ+)
11 kα−−
uk
uk
11(1 ) ( )k k kpα μ−+− )
−
+kp
z-1( )k kp μ)
1( )kp + µ)1 1
1
( )
( )
dk k
k
choosep μ
μ+ +
+ ∈µ
)
1kμ +
( )k
Generate
u µ
5
1
2
7
2
5
2
2 4
4
4
7 5
4
6
![Page 15: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/15.jpg)
(b) Choke-ORCA(a) Choke-Only
Rate-Control SimulationsRate-Control Simulations
![Page 16: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/16.jpg)
Lightweight Streaming Lightweight Streaming Protocol (LSP)Protocol (LSP)
Videofile
Framer Discriminator Packetizer Packetbuffer
Sender
ReceiverControlShared
parametersand statistics
RTX
To client
Control messages from theclient
Nominal frame rate (NFR)
Actual frame rate (AFR)
Server architecture
SEQ1 SEQ2(lost) SEQ3 SEQ4(lost) SEQ5 SEQ2 (rtx) SEQ4(rtx) NACK2 (ignored)
NACK2 (NACK2 + NACK4)
Receiver Sender
PacketsTrans-mitted
Packets received
UDP LSP LSP-PMN
DSL 94% 99% 100%
Wi-Fi 95% 96% 98%
![Page 17: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/17.jpg)
LSPLSP
PSNR = 18 dBms PSNR = 33 dBms
![Page 18: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/18.jpg)
QoS Control QoS Control Over Wireless & Over Wireless &
Core NetworkCore Network
CDMA-Based CDMA-Based Admission & Admission & SchedulingScheduling
(Oliver Yu)(Oliver Yu)
Internet
Backbone
Gateway
Backbone
Wireless QoS Control
Core QoS Control
Internet QoS Control
Router
MSC
AP-CAC
Backbone
BS
FDWFQ-MAC
AMDG-CAC
![Page 19: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/19.jpg)
Wireless MAC ProtocolsWireless MAC Protocols(Khokhar)(Khokhar)
Listen Idle
Zzzz
Zzzz
Zzzz
Zzzz
Zzzz
Zzzz
Zzzz
Zzzz
Zzz
z
Zzzz
Duty Cycle
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
Event Arrival Rate
Normalized Power Consumption
TDMA-W, 50 nodesTDMA-W, 100 nodesTDMA-W, 200 nodes10% S-MAC, 50 nodes10% S-MAC, 100 nodes10% S-MAC,200 nodes
9.30%
8.16%
5.13%
0.56%
Channel
Call AdmissionController
Slot Level Scheduler
X
X
![Page 20: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/20.jpg)
Center for Global Multimedia Center for Global Multimedia Mobile CommunicationsMobile Communications
![Page 21: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/21.jpg)
Internet over CableInternet over Cable
![Page 22: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/22.jpg)
Digital Subscriber LinesDigital Subscriber Lines
A typical ADSL equipment configuration.
![Page 23: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/23.jpg)
Low-Earth Orbit SatellitesLow-Earth Orbit SatellitesIridiumIridium
(a) The Iridium satellites form six necklaces around the earth.
(b) 1628 moving cells cover the earth.
![Page 24: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/24.jpg)
GlobalstarGlobalstar
(a) Relaying in space.(b) Relaying on the ground.
![Page 25: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/25.jpg)
The 802.16 Physical LayerThe 802.16 Physical Layer
The 802.16 transmission environment.
![Page 26: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/26.jpg)
Computer Network Computer Network InfrastructureInfrastructure
UICNU
NCSA
UCSD
Optical Networking Transport Encoding
& Protocols Wired and Wireless
Network Integration Circuit and Packet Switched
Network Deployment
![Page 27: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/27.jpg)
Wireless CommunicationsWireless Communications
Router Wireless Access Pointoptical Switch Wireless Terminal
Optical Core
PacketData Network
Wireless Access Network:• IEEE 802.11 (Year 1)• IEEE 802.16 (Years 2 & 3)• IEEE 802.20 (Optional)
Wireless AccessNetwork
Opportunistic Resource Allocation & Admission Control
Channel Estimation Power-Efficient
Wireless Protocols High-Capacity
Wireless Networks
![Page 28: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/28.jpg)
Applications & PrototypesApplications & Prototypes
Video Communications Tele-Education Natural Event Monitoring Geosciences Monitoring Environmental Assessment Emergency Management Elderly Care Medical Diagnosis Remote Robotic Surgery
![Page 29: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/29.jpg)
Visualization & DevicesVisualization & Devices
High-Resolution Scalable Displays
High-Resolution Capture
Interactive Tools
![Page 30: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/30.jpg)
Intelligence SharingIntelligence Sharing
Real Time Monitoring and Real Time Multimedia Retrieval and Sharing across the continent
![Page 31: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/31.jpg)
Video TrackingVideo Tracking
Collaborators:Collaborators:Nidhal Bouaynaya (UIC)Nidhal Bouaynaya (UIC)Karthik Hariharakrishnan (Motorola Research)Karthik Hariharakrishnan (Motorola Research)Dan Lelescu (NTT DoCoMo Research)Dan Lelescu (NTT DoCoMo Research)Josh Meir (NeoMagic)Josh Meir (NeoMagic)Magdi Mohamed (Motorola Research)Magdi Mohamed (Motorola Research)Wei Qu (UIC)Wei Qu (UIC)Philippe Raffy (R2 Technology)Philippe Raffy (R2 Technology)Fathy Yassa (NeoMagic)Fathy Yassa (NeoMagic)
![Page 32: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/32.jpg)
MotivationMotivation
Target Tracking Surveillance Retrieval Video Coding Video Communications Videoconferencing Virtual Reality Human-Computer Interaction Computer Animation
![Page 33: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/33.jpg)
VORTEXVORTEX
Reference frame
Object cluster
VORTEX: Video Retrieval and Tracking from Compressed Multimedia Databases
![Page 34: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/34.jpg)
Template Template Matching [sec] VORTEX [sec]
Object #1 45.22 0.0084
Object #2 39.36 0.0092
VORTEXVORTEX
![Page 35: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/35.jpg)
Adaptive Block Matching Adaptive Block Matching (ABM)(ABM)
Method Time [sec]
ABM 10
Partition Projection 165
Partition Lattice Operators 193
![Page 36: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/36.jpg)
MBPFCondensation filter
Motion-Based Motion-Based Particle FiltersParticle Filters
![Page 37: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/37.jpg)
Multi-Object Particle FiltersMulti-Object Particle Filters
. . .. . .
. . .
......
...
......
...
21x
1
mx 2mx
22x
1
2x1
1x 1
tx
2tx
m
tx
1
1z
2
1z
1mz 2
mz
2
2z
1
2z 1
tz
2
tz
mtz
HMM
MHMM
x1 x2 xt. . . .
z1 ztz2
p(zt|xt)
p(xt|xt-1)
![Page 38: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/38.jpg)
Experimental ResultsExperimental Results
![Page 39: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/39.jpg)
The Dynamic Graphic Model for Multiple Interactive Objects In Two Frames
IDMOTIDMOT
![Page 40: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/40.jpg)
Magnetic-Inertia ModelMagnetic-Inertia Model
Reward
Punish
![Page 41: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/41.jpg)
Video Tracking and Video Tracking and FoveationFoveation
(Ansari & Khokhar)(Ansari & Khokhar)
![Page 42: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/42.jpg)
Future Research:Future Research:Video TrackingVideo Tracking
Randomly Perturbed Active Surfaces Video Stabilization Auto-Focus Recovery Pose Estimation and Feature Tracking Video Animation Stereography from a Single Camera Multiple Camera Mosaics Multiple Camera Tracking Low-Power Particle Filters
![Page 43: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/43.jpg)
Video RetrievalVideo Retrieval
Collaborators:Collaborators:Faisal Bashir (UIC)Faisal Bashir (UIC)Ashfaq Khokhar (UIC)Ashfaq Khokhar (UIC)Dan Lelescu (NTT DoCoMo Research)Dan Lelescu (NTT DoCoMo Research)Fatih Porikli (Mitsubishi Research)Fatih Porikli (Mitsubishi Research)
![Page 44: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/44.jpg)
MotivationMotivation
• Video Surveillance• Sign Language Recognition• Sports Video Analysis• Animal Mobility Experiments• Moving Object Databases• Video and Sensor Databases
![Page 45: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/45.jpg)
Spectral ClusteringSpectral Clustering
![Page 46: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/46.jpg)
Trajectory RetrievalTrajectory Retrieval
0
500
1000
1500
2000
2500
3000
1 2 3 4 5 6
PCA-Global
PCA-Seg Euc
PCA-Seg Str
Lei Chen
![Page 47: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/47.jpg)
Gaussian Mixture ModelsGaussian Mixture Models
Figure: 1-Sigma contours of GMM’s learnt from three classes.(a) ‘Norway’. (b) ‘Alive’. (c) ‘Crazy’.
1
cN
i i ii
P( y ) ( y; , )π μ=
Θ = ∑∑ ¥
![Page 48: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/48.jpg)
HMM for Class NHMM for Class 1
ClassificationClassification
…
…
Gaussian Mixtures
Training Set Database
Classification:
[ ]( )1
1m i
i , ,Larg max p Y , ,Y λ
∈ LL
![Page 49: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/49.jpg)
AccuracyAccuracy
Datasets
ASL
HJSL#Classes : # Trajectories
2:138 4:276 8:552 16:1104 29:2001 38:2622
HMM 0.9638 0.9167 0.8587 0.7790 0.6882 0.6609 0.9074
GMM 0.9855 0.8949 0.8514 0.7455 0.6672 0.6400 0.8981
Moghaddam 0.9420 0.9312 0.8297 0.7283 0.5592 0.6175 0.4537
Accuracy values for various class sizes from ASL data set and the HJSL dataset (last column). Column headings are shown as (number of classes:number of trajectories) for the ASL dataset at different sizes.
![Page 50: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/50.jpg)
Shape RepresentationShape Representation
![Page 51: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/51.jpg)
Curvature Scale SpaceCurvature Scale Space
CSS Images of a Trajectory and its 36-degree rotated versionFigure: An example high jump trajectory and its translated, rotated and non-uniformly scaled version, along with their CSS images.
![Page 52: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/52.jpg)
PerformancePerformance
Indexing Time (sec.)
(408 Traj.)
Retrieval Time (sec.)
(15 Traj.)
PCA Centroid 178.9270 8.2920
Hybrid PCA 175.2020 27.5400
CSS 1508.3 28.0500
![Page 53: Real-Time Networking: Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649e4a5503460f94b3ecd3/html5/thumbnails/53.jpg)
Future Research:Future Research:Video RetrievalVideo Retrieval
Trajectory OcclusionCamera MotionMultiple CamerasMultiple TrajectoriesVideo MiningJoint Retrieval, Recognition, & MiningMulti-Modality Feature Integration