QUANTIZATION FOR CLASSIFICATION ACCURACY IN HIGH-RATE QUANTIZERS

QUANTIZATION FOR CLASSIFICATION ACCURACY IN HIGH-RATE QUANTIZERSBehzad M. DogaheManohar N. Murthi

Department of Electrical and Computer Engineering

IEEE DSP Workshop, January 2011

1

OutlineMotivationBackgroundProblem Statement and SolutionSimulationsConcluding Remarks

2

Motivation

Quantization of signals is required for many applicationsThe original signal is quantized at the encoder and at the decoder side a replica that should resemble the original signal in some sense is recoveredPresent quantizers make an effort to reduce the distortion of the signal in the sense of reproduction fidelityConsider scenarios in which signals are generated from multiple classes. The encoder focuses on the task of quantization without any regards to the class of the signalThe quantized signal reaches the decoder where not only the recovery of the signal should take place but also a decision is to be made on the class of the signal based on the quantized version of the signal only

3

Motivation

Goal: Design of a quantizer that is optimized for the task of classification at the decoderApplication Scenarios:Want to have good sound fidelity (good voice/audio quality) but also want to be able to perform speaker recognitionSensor network where the sensors have low complexity, simple quantizers, but the decoder/sensor sink node does more sophisticated processing (so the raw signal value is needed, but we also want to be able to classify the sensed signal)

4

Background

Quantizer

In high-rate theory point density function represents the density of codebook points in any region for a quantizer. The design of a quantizer is equivalent to design of the optimal point density function.

: Probability Density Function

5

Background

Design of Quantizer involves minimizing:

where is Distortion Measure

Examples of Distortion Measure: MSE Log Spectral Distortion

High-Rate Theory:

Optimization Problem

6

Background

Following the steps in [Gardner and Rao] point density function will be derived as

(n is the dimension of x)

W.R. Gardner and B.D. Rao, Theoretical analysis of the high-rate vector quantization of lpc parameters, Speech and Audio Processing, IEEE Transactions on, vol. 3, no. 5, pp. 367 381, sep 1995.

7

Problem Statement

We are looking for a point density function that is representative of a quantizer that performs well in the classification taskWe have to select a distortion measure that is well defined for classification purposesWe chose the symmetric Kullback-Leibler divergence measure between probability of class given the signal before and after quantization

8

Problem Statement & SolutionWe assume a generativemodel for classifier. Hence and are known a priori.

Trade-off Distortion Measure:

9

Simulations

Signal is from two classes with known conditional PDFsDashed lines represent the decision boundaries Point density function dedicates codebook points to the boundaries

10

Simulations

only dedicates codebook points where the signal is concentratedBy introducing tradeoff between MSE and classification, codebook points move to the classification boundaries

11

Simulations

The higher the bit rate of quantizer the better classification accuracy

As we move from MSE to KL, the classification accuracy improves

12

Simulations

Pure KL performs poorly as far as the distortion of the signal

However, introducing the slightest tradeoff with MSE improves distortion significantly

13

Concluding Remarks

A solution for quantization of signals for the purpose of obtaining a more accurate classification at the decoder was proposed High-rate theory for quantizer design was employed An optimal point density function was derived The performance of this method on synthetically generated data was examined and observed to be superior in the task of classification of signals at the decoder The tradeoff between the reproduction fidelity and classification accuracy was studied as well In our future work, we will study the practical vector quantizer design based on the high-rate theory

14