Study-Element Based Adaptation of Lecture Videos to Mobile Devices
description
Transcript of Study-Element Based Adaptation of Lecture Videos to Mobile Devices
Study-Element Based Adaptation of Lecture Videos to Mobile Devices
Ganesh Narayana Murthy (M.Tech IIT Bombay)Sridhar Iyer (Associate Professor, IIT Bombay)
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 1
Problem Definition• Adapt CDEEP videos to be viewable on mobile devices:
– Viewable at low network bandwidths (like GPRS)– Viewable at low cost
• Video bit-rate – Size of video stream over time– Total size = bit-rate * total time– CDEEP video bit-rate: 1150kbps– GPRS bit-rate: 40kbps
• The problem:– Video playing incurs delays if available network bandwidth is less than
video-bit rate
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 2
Video Transcoding
• Converting from one video format to another– Changing video bit rate– Changing other parameters like frame rate, screen
resolution
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 3
Format Name Typical Bit Rate ApplicationMPEG-1 1.5Mbps or less CD-ROM
MPEG-2 5-8Mbps DVD, HDTV
H.263 Typically low bit rates Low bit-rate video conferencing
MPEG-4 / H.264 40Kbps to 10Mbps and above
Internet Streaming, Video Telephony
Flash Video (FLV) Typically low bit rates Embedded video in websites
Video Quality at low-bit rates
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 4
(a) MPEG-1 (b) MPEG-2
Images from transcoded videos (Target bit rate : 40kbps, No audio)
Video Quality at low bit-rates (contd.)
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 5
(c) H.264 (mp4) (d) H.263 (3gpp)
Images from transcoded videos (Target bit rate : 40kbps, No audio)
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 6
(e) Flash Video (flv)
Images from transcoded videos (Target bit rate : 40kbps, No audio)
Comparison of Video CodecsFormat Name
Original VideoSize
Converted VideoSize
Video Quality at low-bit rates
Remarks
MPEG-1 432MB 26MB Poor Cannot be used at low bit-rates
MPEG-2 432MB 29.12MB Poor Cannot be used at low bit rates
H.263 432MB 38.3MB Poor Cannot be used at low bit rates
H.264 432MB 16.9MB Good Processing power / Decoding complexity is high.[1]
Flash 432MB 20.5MB Good Can be used, but cost is still high.
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 7
(Note: Video bit rate = 1150kbps, No audio, Target bit rate = 40kbps, No audio)
Video Sizes are still high forviewing over GPRS
Study-Element Based Adaptation
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 8
Motivation• CDEEP video usually consists of
– Presentation slides– Instructor explaining on white paper– Video of instructor talking
• Presentation slide is usually not changing– Video of slide is not required. One image is sufficient
• Idea– Extract one image every ‘n’ seconds and send to client.– This would reduce amount of data sent for showing one
slide.
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 9
Method-1
• Send one image every ‘n’ seconds– Server sends one image every ‘n’ seconds to client– Audio is simultaneously streamed
• Network bandwidth and Size – Network Overhead (NO) = Image Size / n– Size Overhead (SO) = Total size of images
• What is the user experience?
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 10
User Experience Basis• Presentation Study Element
– Portion of video showing one slide
• White Paper Study Element– Portion of video showing instructor writing on
white paper
• Instructor Study Element– Portion of video showing instructor talking
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 11
…………..
0 5 10 15
Presentation Slide
Delay in start of slide
3
………..
25 30 35
White Paper
Video Time(secs)
User Experience• Presentation Element
– Delay Experienced (D2) = • Delay in start of slide as compared to audio
• White Paper Element– Delay Experienced (D1) =
• Delay between any two consecutive images = Sending Rate
• Instructor Element– Only audio important. No image need be sent.
• User Experience is assumed to be one
• User Experience (Ui) = 1 sec / Di
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 12
Method-2• Trade-off for user experience
• Cost incurred in terms of number of images sent
• Same sending interval for all elements, cannot balance user experience and cost.
• Choose different sending interval for each study element
• Probably:– Higher user experience for white paper
element– Lower user experience for presentation
element
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 13
User Experience
Cost
Sending Interval
Trade-Off Relation
System Overview
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 14
Building the index• Corpus of 10 videos
– Representative of various departments
• Consider different sending intervals ‘r’– For each ‘r’ find NO,SO and U for every study element in a
video. – Repeat for all videos and take average.
• This relation can be used backwards:– For calculating sending interval, given network bandwidth
and user experience.
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 15
Graphs of User Experience
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 16
Graphs of overheads
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 17
Results
Original Video Size(MB)
Images Size (MB)
Reduction (%)
U1 U2 SupportedNetwork Bandwidth
432 2.85 97% 0.2 0.38 20kbps and above
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 18
Achieved Size Reduction
Fig: Video stream size reduction (note: Original video bit-rate = 1150kbps, No audio)
Results (contd.)
Sending Interval U1 U2 NO1(kbps)
S01(MB)
NO2(kbps)
SO2(MB)
Total Size (SO)
White Paper Element
Presentation Element
5 5 0.2 0.337 15.12 1.25 23.43 1.6 5.46
5 15 0.2 0.115 15.12 1.25 7.81 0.53 1.78
15 15 0.067 0.115 5.254 0.781 7.63 1.03 1.81
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 19
Balance User Experience and Cost
Required Network Bandwidth =max(NO1,NO2) is reduced
Reduction in size user experience for white paper element remaining same
Conclusion
• Large size reduction can be achieved by using the concept of slideshows
• Identifiying study-elements within the video helps define user-experience of the slideshow.
• CDEEP Lecture videos can be adapted to low network bandwidths and in a cost-controlled manner.
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 20
Future Work
Automated tagging– Identifying study element boundaries – Shot detection techniques
User Experience Correlation– Identifying relation between obtained user
experience and actual user values
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 21
References1. H.264 white paper.
http://ati.amd.com/products/pdf/h264_whitepaper.pdf.2. Real-time Content-Based Adaptive Streaming of Sports Videos.
Shih-Fu Chang, Di Zhong, and Raj Kumar. In CBAIVL '01: Proceedings of the IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL'01), page 139, Washington, DC, USA, 2001. IEEE Computer Society.
3. Content-aware video adaptation under low-bitrate constraint. Ming-Ho Hsiao, Yi-Wen Chen, Hua-Tsung Chen, Kuan-Hung Chou, and Suh-Yin Lee. EURASIP J. Adv. Signal Process,2007(2):27-27, 2007.
4. A Characteristics-Based Bandwidth Reduction Technique for Pre-recorded Videos. Wallapak Tavanapong and Srikanth Krishnamohan. In IEEE International Conference on Multimedia and Expo (III), pages 1751-1754, 2000.
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 22
Questions?
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 23
Content-Aware Adaptation
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 24
Method Name Adaption Mechanism Video Quality Remarks
Hsiao et.al.[2] Identify visual attention regions in a frame. Encode them at high quality.
Poor Quality of important objects still depends on network bandwidth
Chang.et.al [3] Identify events in sports videos at high quality. Other regions as slideshows.
Good Slideshow of images reduces network bandwidth and size
Tavanapong. et. Al. [4]
Identify non-changing portions of lecture video and extract one image from them
Good Exploits redundancy in lecture videos