Perceptual Evaluation of Speech Quality (PESQ)
description
Transcript of Perceptual Evaluation of Speech Quality (PESQ)
11 TAC2000/2000.7
802.16 IP Telephony Lab
Perceptual Evaluation of Speech Quality (PESQ)
Speaker: Wen-Jen LinSpeaker: Wen-Jen LinDate: Dec. 3 2009Date: Dec. 3 2009
22 TAC2000/2000.7
802.16 IP Telephony Lab
Outline IntroductionIntroduction ITU-T P.862ITU-T P.862 P.862.3P.862.3 P.862.2P.862.2 P.862.1P.862.1 P.862 - Overview of the basic philosophy used in P.862 - Overview of the basic philosophy used in
PESQPESQ ConclusionConclusion ExperienceExperience Demo PESQ programDemo PESQ program ReferenceReference
33 TAC2000/2000.7
802.16 IP Telephony Lab
IntroductionClassic Quality MeasurementClassic Quality Measurement
Noise RatioNoise RatioFrequency Response Functions etc.Frequency Response Functions etc.Speech coding ( ETSI GSM EFR/AMR, ITU-T Speech coding ( ETSI GSM EFR/AMR, ITU-T
G.728/729/723.1 etc)G.728/729/723.1 etc)New types of distortionsNew types of distortions
Voice over IP (packet loss and variable delay)Voice over IP (packet loss and variable delay)Voice over ATM (cell loss)Voice over ATM (cell loss)Voice over mobile (GSM, UMTS, frame repeat, Voice over mobile (GSM, UMTS, frame repeat,
front end clipping, comfort noise generation)front end clipping, comfort noise generation)
44 TAC2000/2000.7
802.16 IP Telephony Lab
Introduction (cont.)PSQM (Perceptual Speech Qualify MeasurPSQM (Perceptual Speech Qualify Measur
e)e)P.861 - The first international standard for the P.861 - The first international standard for the
perceptual quality measurement of telephone-baperceptual quality measurement of telephone-band (300-3400 Hz) speech signals.nd (300-3400 Hz) speech signals.
The scope of recommendation P.861 was limiteThe scope of recommendation P.861 was limited to the assessment of telephone-band speech cd to the assessment of telephone-band speech codecs only.odecs only.
55 TAC2000/2000.7
802.16 IP Telephony Lab
ITU-T P.862 P.862P.862
Perceptual evaluation of speech quality (PESQ): An objective methoPerceptual evaluation of speech quality (PESQ): An objective method for end-to-end speech quality assessment of d for end-to-end speech quality assessment of narrow-bandnarrow-band telephon telephone networks and speech codecse networks and speech codecs
Amendment 2Amendment 2 P.862.1P.862.1
Mapping functionMapping function for transforming P.862 raw result scores to MOS- for transforming P.862 raw result scores to MOS-LQOLQO
P.862.2P.862.2 Wideband Wideband extension to Recommendation P.862 for the assessment oextension to Recommendation P.862 for the assessment o
f wideband telephone networks and speech codecsf wideband telephone networks and speech codecs P.862.3P.862.3
Application guideApplication guide for objective quality measurement based on Reco for objective quality measurement based on Recommendations P.862, P.862.1 and P.862.2mmendations P.862, P.862.1 and P.862.2
66 TAC2000/2000.7
802.16 IP Telephony Lab
ApplicationP.862.3 – Example of measurement set-up P.862.3 – Example of measurement set-up
and terminologyand terminology
77 TAC2000/2000.7
802.16 IP Telephony Lab
P.862.1MOS-LQO (P.800.1)MOS-LQO (P.800.1)ITU-T Rec. P.862 provides raw scores in ITU-T Rec. P.862 provides raw scores in
the range –0.5 to 4.5.the range –0.5 to 4.5.The mapping function:The mapping function:
88 TAC2000/2000.7
802.16 IP Telephony Lab
Figure of Mapping Function
99 TAC2000/2000.7
802.16 IP Telephony Lab
P.862 - Overview of the basic philosophy used in PESQ
1010 TAC2000/2000.7
802.16 IP Telephony Lab
P.862 - Overview of the basic philosophy used in PESQ (cont.)
1111 TAC2000/2000.7
802.16 IP Telephony Lab
Addition of background noise
1212 TAC2000/2000.7
802.16 IP Telephony Lab
Factors for which PESQ had demonstrated acceptable accuracy
Speech input levels to a codecSpeech input levels to a codecErrors in the transmission channel Errors in the transmission channel
between an encoder and a decoderbetween an encoder and a decoderBit rates if a codec has more than oBit rates if a codec has more than o
ne bit-rate modene bit-rate modeTranscodingsTranscodingsEnvironmental noise in the sending Environmental noise in the sending
sideside
1313 TAC2000/2000.7
802.16 IP Telephony Lab
PESQ is not intended to be used to assess
Effect of listening levelEffect of listening levelConversational delayConversational delayTalker echo, where a subjects hears his own Talker echo, where a subjects hears his own
voice delayedvoice delayedTalker sidetone, where a subjects may hear iTalker sidetone, where a subjects may hear i
ts own voice distortedts own voice distortedNon-intrusive measurements, where only outNon-intrusive measurements, where only out
put signals are available from the system muput signals are available from the system musicsic
1414 TAC2000/2000.7
802.16 IP Telephony Lab
which PESQ has not currently been validated
1515 TAC2000/2000.7
802.16 IP Telephony Lab
ConclusionPESQ has been evaluated on a very wide raPESQ has been evaluated on a very wide ra
nge of speech codecs and telephone networnge of speech codecs and telephone network tests.k tests.
It has been found to produce accurate predicIt has been found to produce accurate predictions of quality in the presence of diverse entions of quality in the presence of diverse end-to-end network behaviors.d-to-end network behaviors.
PESQ represents a significant step forward iPESQ represents a significant step forward in the accuracy and range of applicability of n the accuracy and range of applicability of objective speech quality assessment methodobjective speech quality assessment methods.s.
1616 TAC2000/2000.7
802.16 IP Telephony Lab
Head and Torso Simulator(HATS)
1717 TAC2000/2000.7
802.16 IP Telephony Lab
Spirent Abacus
1818 TAC2000/2000.7
802.16 IP Telephony Lab
Abacus Control GUI
1919 TAC2000/2000.7
802.16 IP Telephony Lab
Testbed
DSLAIIHandset Interface
ControllerMalden
2020 TAC2000/2000.7
802.16 IP Telephony Lab
RTP Flow
2121 TAC2000/2000.7
802.16 IP Telephony Lab
Demo PESQ program
2222 TAC2000/2000.7
802.16 IP Telephony Lab
Reference HATS - HATS - http://www.bksv.com/doc/bp2240http://www.bksv.com/doc/bp2240.pdf.pdf ITU-T Rec. P.862, “Perceptual Evaluation of Speech QualiITU-T Rec. P.862, “Perceptual Evaluation of Speech Quali
ty (PESQ), an Objective Method for End-to-end Speech Qty (PESQ), an Objective Method for End-to-end Speech Quality Assessment of Narrowband Telephone Networks anuality Assessment of Narrowband Telephone Networks and Speech Codecs”, International Telecommunication Uniod Speech Codecs”, International Telecommunication Union, Geneva, Switzerland (2001 Feb.)n, Geneva, Switzerland (2001 Feb.)
Malden - http://www.malden.co.uk/dsla.htmMalden - http://www.malden.co.uk/dsla.htm Perceptual Evaluation of Speech Quality (PESQ), the new Perceptual Evaluation of Speech Quality (PESQ), the new
ITU standard for end-to-end speech quality assessment. PaITU standard for end-to-end speech quality assessment. Part II – Psychoacoustic model - rt II – Psychoacoustic model - http://www.mp3-tech.org/programmer/docs/2001-P03b.pdfhttp://www.mp3-tech.org/programmer/docs/2001-P03b.pdf
Spirent - http://www.spirent.com/Spirent - http://www.spirent.com/