AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT...
-
Upload
linda-barton -
Category
Documents
-
view
213 -
download
0
Transcript of AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT...
![Page 1: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/1.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
Assessing Audio Visual Quality
P905 - AQUAVIT
Assessment of Quality for audio-visual signals over Internet and UMTS
Geoff Morrison, BT Labs
![Page 2: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/2.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
Contents
• Why we are doing this
• What we want to achieve
• How we are going about it
• Who is doing it– BT, CSELT, Berkom
• When are we doing it– February 1999 to December 2000
• The Results
![Page 3: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/3.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
• Coding algorithms, such as MPEG-4 and H.263, can compress audio and video signals to bit rates<1 Mbit/s as available over mobile and IP networks.
• The impact of impairments, from compression and transmission, on users' opinion must be understood.
• Audiovisual communication must be evaluated under representative conditions and real tasks, such as conversational services and AV database retrieval.
• Network performance must be monitored to guarantee a minimum QoS. Need single-ended models to check audio and video transmission quality in real time.
Rationale
![Page 4: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/4.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
Is the resulting application usable for purpose?
How best to assign a given total bit rate between video and audio?
Compression Impairments• Digital TV in studios is 216 Mbit/s
• Lossless compression typically only halves the bit rate
• To reach very low rates requires lossy audio and video compression, introducing significant distortions
![Page 5: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/5.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
Transmission Impairments • AV bits <> data
• Audiovisual content has a “timeliness” aspect– Critical in conversational applications– Important in database retrieval applications
• Error detection and retransmission protocols not ideal for audiovisual content
– variable delay– variable throughput
• Compressed AV is very sensitive to transmission errors, packet loss
– more compression more sensitivity
![Page 6: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/6.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
Error Handling• Error resilience tools for encoder
– structure compressed data so that damage is localised– optimise encoder options for the network performance
• Forward Error Correction– improve apparent network performance seen by decoder
• Error concealment at decoder– reconstruct approximations for damaged parts of pictures
and sound
• Combinations of above
How well do these work in practice?
![Page 7: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/7.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
Project Objectives• Understand the potential of audio, video and AV
communication at bit rates < 1Mbit/s for UMTS and IP.
• Develop tools and techniques for simulation of complete audiovisual transmission systems.
• Develop quality evaluation methods for complete audiovisual transmission systems.
• Investigate the relationship between system parameters (network and terminal) and subjective quality.
• Provide guidelines for the system parameters to be used for new applications.
• Encourage the use of audiovisual communication on these dominant networks of the future
![Page 8: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/8.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
Work Items
• Define the tests to be performed
• Develop/implement a UMTS test bed
• Develop/implement an IP test-bed
• Conduct subjective tests
• Make objective measurements and relate them to subjective results
• Disseminate the findings
![Page 9: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/9.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
Definition of tests
• Literature survey for available test methods
• Select appropriate methods
![Page 10: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/10.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
UMTS Test Bed• Video codec (MPEG-4 and/or H.263)
• Audio codec (MPEG AAC, scalable and non-scalable)
• Error resilience tools (MPEG-4)
• MPEG-4 multiplex
• Transmission errors - simulated and perhaps actual
• Error recovery
• Demux
• Optimisation of parameters
• Tools to make objective measurements
• Production of material for subjective and objective tests
![Page 11: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/11.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
UMTS Test Bed
• Baseline non real-time test bed– MPEG-4 baseline video codec, AAC and G.723.1 audio, AL
layer and MPEG DMIF
• Enhanced non real-time test bed – Error resilience tools, layered audio
![Page 12: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/12.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
UMTS Test Bed
Server Application Audio decoder Video decoder
Network interface Network interface
Network
Server Client
![Page 13: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/13.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
IP Test Bed• Video codecs (H.263 and/or MPEG-4)
• Audio codecs (G.723.1, MPEG AAC scalable and n/s)
• Error resilience tools (ITU-T, MPEG, IETF)
• Packetisation
• Transmission errors - simulated, measured profiles and actual
• Error recovery
• Optimisation of parameters
• Tools to make objective data measurements
• Production of material for subjective and objective tests
![Page 14: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/14.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
IP Test Bed
• Baseline non real-time test bed– G.723.1, H.263, simplified RTP, simple network impairment
simulator
• Enhanced non real-time test bed– more codecs including MPEG, error resilience, complete
RTP, error concealment, enhanced network simulator
• Enhanced real-time test bed– real time image capture, encoding and decoding
![Page 15: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/15.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
IP Test Bed
Encoder Application
Audio Encoder
Server Application
Video Encoder
Network Interface
Network
Decoder Application
Audio Decoder Video Decoder
Network Interface
Encoder Decoder
![Page 16: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/16.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
Subjective tests
• “Conventional” subjective tests covering a suitably wide range of variables
– the codecs– bit rate, video frame rate, audio bandwidth– Error conditions (packet loss, delay variability, …)– Protocols (CBR, VBR, ABR, RSVP, ..)– Application (one-way, conversational)
• Users may accept lower quality in return for mobility. Testing will seek an estimation of the threshold of acceptability of audio and video quality in mobile applications
![Page 17: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/17.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
Objective measurements
• Objective assessment of audio quality– TOSQA, PAMS, ...
• Objective assessment of video quality– model of human vision– other algorithms to objectively estimate user’s opinion
• Compare subjective and objective assessments
• Investigate relationship between parameter values and subjective audio, video and AV quality assessments
![Page 18: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/18.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
Double Ended Model
• Measuring instrument needs both original input to encoder and delivered version at decoder output
SystemunderTest
Meter
![Page 19: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/19.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
Single Ended Model
• Measuring instrument needs only decoder output– more suitable for live performance monitoring
SystemunderTest
Meter
![Page 20: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/20.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
Project Resources
20 man-monthsProjectManagementDissemination ofResultsDefinition ofTestsUMTS Testbed
IP Testbed
Subjective Tests
Objective Tests
![Page 21: AIMS’99 Workshop Heidelberg, 11-12 May 1999 Assessing Audio Visual Quality P905 - AQUAVIT Assessment of Quality for audio-visual signals over Internet.](https://reader036.fdocuments.in/reader036/viewer/2022062805/5697bfdd1a28abf838cb1a52/html5/thumbnails/21.jpg)
AIMS’99 Workshop
Heidelberg, 11-12 May 1999
Results• Reports due in Q4 of 2000.
• The findings should be of benefit to at least 3 groups:– Network Providers. Results such as minimum desirable bit
rates and error characteristics should influence the planning and operation of IP and UMTS networks intended to carry audiovisual services.
– Application Providers. The results will provide reliable guidance about which applications and operating parameters will satisfactorily meet customers' expectations and those which will not.
– Researchers in the field of "human factors". It is expected that the project will advance the state of the art in perceptual modelling and in the understanding of cross-modal effects.