國立雲林科技大學National Yunlin University of Science and Technology

Application of LVQ to novelty detection using outlier training data

Hyoung-joo Lee, Sungzoon Cho, Pattern Recognition Letters, 2006. (article in press)

.

Presenter : Wei-Shen TaiAdvisor : Professor Chung-Chian Hsu

2006/7/5

N.Y.U.S.T.

I. M.

Outline Introduction Learning vector quantization for novelty detection

Codebook update for an LVQ for novelty detection Determining local thresholds Parameters for the proposed approach

Experimental results Conclusion and discussion Comments

N.Y.U.S.T.

I. M.

Motivation Novelty detection

A model learns the characteristics of normal patterns in training data and detects outliers or novel patterns.

Original LVQ problem Cannot deal with a highly imbalanced dataset, Codebook update is modified

codebooks should be located close to normal patterns and far away from novel patterns.

N.Y.U.S.T.

I. M.

Objective Local thresholds to determine

Effectively exclude novel patterns outside boundaries of the normal class.

LVQ for novelty detection (ND) generate more accurate and tighter boundaries

than other approaches that use only the normal class of patterns.

N.Y.U.S.T.

I. M.

Training algorithm and classification

N.Y.U.S.T.

I. M.

Results on an artificial dataset

Effects of the modified LVQ update.(a) True boundaries, (b) SOM, (c) LVQ-ND and

(d) LVQ.

No training at all since all codebooks were assigned to the normal class while training was prematurely stopped due to the class imbalance .(a) SOM-G, (b) SOM-L, (c) LVQ-ND and (d) LVQ.

N.Y.U.S.T.

I. M.

Results on real-world datasets When applied to the Ratsch’s benchmark datasets and the p

ump vibration dataset, It performed better than other widely-used novelty detectors.

N.Y.U.S.T.

I. M.

Codebook update rule Initial codebooks

Generated by training a SOM. Note that only the normal patterns are used in this process.

A modified error function (yi = +1, -1)

Codebooks can be written as if xi does not belong to Voronoi region Sk that wk represents, w

k remains unchanged. If xi does belong to Sk, wk moves toward xi if xi is normal, or moves away from xi otherwise.

k Tx Ox

kikii

iiiki ki

wxwxN

xmxyN

xe 222 1)(1)(

ki

ki

ki kTi

Sx i

Sx i

Tx x ii

k kyy

xxw ,0,*

N.Y.U.S.T.

I. M.

Determining local threshold Voronoi region Sk

A hypersphere with a center at wk and a minimal radius can be obtained so that it surrounds as many normal patterns and as few novel patterns as possible.

Find the radius an ‘‘optimization’’ problem a large radius can surround many normal patterns, but may increas

e false acceptance. a small radius can exclude many novel patterns, but may increase

false rejection.

22 )(

22

)(

21

22 ))(())(()(min

ki

ki

ki

kik

rxeOx

ik

rxeTx

kikkkrxerCrxeCrrE

N.Y.U.S.T.

I. M.

Parameter setting The number of codebooks, K,

Minimize the misclassification error C1 , C2

While larger normal regions are defined with a larger C1, tighter boundaries are obtained with a larger C2.

Suppose k, Ok = ; and x1; . . . ; x|Tk| Tk. If FRk denotes the FRR (false rejection rate) in Voronoi region Sk, the following holds (|Tk|-uk means normal pattern outside the hypersphere)

kkkk FRTC

FRT1

11

1

kkkk TuTFR /)(

N.Y.U.S.T.

I. M.

Average AUROCs (%) with respect to |O|/|T|

(a) Banana, (b) Breast-cancer, (c) Diabetes, (d) German, (e) Heart and (f) Titanic.

N.Y.U.S.T.

I. M.

Conclusions Utilizing information on the novel class

LVQ-ND and SVDD, outperformed their counterparts at least slightly, it can improve novelty detection performance.

Well determined thresholds A codebook-based method with well determined

thresholds can be good enough for novelty detection tasks. (in SOM-L)

The number of novel patterns gradually increases. As |O|/|T| increases, however, the LVQ-ND excels

other models.

N.Y.U.S.T.

I. M.

Comments Is it feasible for classification of two more clas

ses? Focus on outlier processing, but those functions se

ems cannot be utilized in the experiments. If it do so, the effectiveness of LVQ-ND merely wa

s applied in binary classification so far. Further experiments for multiple classes

It is essential for demonstrating those effectiveness of the proposed method or functions.