Faculty of Information Engineering, Shenzhen University Liao Huilian SZU TI-DSPs LAB Aug 27, 2007...

Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University

Liao HuilianLiao HuilianSZU TI-DSPs LABSZU TI-DSPs LAB

Aug 27, 2007Aug 27, 2007

Optimizer based on particle swarm Optimizer based on particle swarm optimization and LBG (PSO-LBG)optimization and LBG (PSO-LBG)

—— application in vector quantizationapplication in vector quantization

School of Software Engineering, Shenzhen University


OutlineOutlineVector quantization (VQ)Vector quantization (VQ) LBGLBGParticle swarm optimization (PSO)Particle swarm optimization (PSO)Optimizer based on PSO and LBG (PSO-Optimizer based on PSO and LBG (PSO-LBG)LBG) PSO-LBGPSO-LBG 2-dimensional simulation2-dimensional simulation Performance comparisonPerformance comparisonConclusionConclusionAcknowledgementAcknowledgement



Vector quantization (VQ)Vector quantization (VQ)



LBGLBG

LBG, a well-known method of VQ, was proposed LBG, a well-known method of VQ, was proposed by Linde, Buzo and Gray in 1980 by Linde, Buzo and Gray in 1980 Apply two optimality criteria iteratively:Apply two optimality criteria iteratively: Nearest neighbour criterion during assigning training vNearest neighbour criterion during assigning training v

ectorsectors Centroid criterion during updating codewords (code vCentroid criterion during updating codewords (code v

ectors)ectors)

Drawbacks: Drawbacks: Local optimization Local optimization Sensitive to the selection of initial codebookSensitive to the selection of initial codebook



LBGLBG



Particle swarm optimizationParticle swarm optimization

PSO was proposed by Eberhart and Kennedy in PSO was proposed by Eberhart and Kennedy in 19951995Advantages:Advantages: Simplicity of implementationSimplicity of implementation Few parameters Few parameters High convergence rate High convergence rate

Population based optimizationPopulation based optimization Remember the best location of itself (Pbest)Remember the best location of itself (Pbest) Remember the best experience in the swarm (Gbest)Remember the best experience in the swarm (Gbest)



Particle swarm optimizationParticle swarm optimization



PSO-LBGPSO-LBG

Based on conventional PSO and LBG Based on conventional PSO and LBG algorithmsalgorithmsPSO-LBGPSO-LBG Structure of particleStructure of particle Particle-pair model Particle-pair model Updating processUpdating process

Apply in Vector Quantization (VQ)Apply in Vector Quantization (VQ)



Structure of particleStructure of particle



Updating model Updating model



Updating processUpdating process

PSO-LBG performs three steps at each itePSO-LBG performs three steps at each iteration:ration: Step1: Basic PSO operationsStep1: Basic PSO operations Step2: Classical vector quantizer, i.e. LBG algStep2: Classical vector quantizer, i.e. LBG alg

orithmorithm Step3: Deal with codewords “flying” over the bStep3: Deal with codewords “flying” over the b

oundary of training vector spaceoundary of training vector space



Step1Step1 －－ Basic PSO operationsBasic PSO operations

Difference between PSO-LBG and PSODifference between PSO-LBG and PSO Velocity updating: (additive inertia weight )Velocity updating: (additive inertia weight )

The parametersThe parameters , and are much smaller than general PSO-based , and are much smaller than general PSO-based algorithmsalgorithms

Apply a particle-pair instead of a large number of particlesApply a particle-pair instead of a large number of particles

1c 2c



Why small parameters?Why small parameters?One point larger parametersOne point larger parametersThe solution of PSO-LBG represents The solution of PSO-LBG represents NN points in the training vector spacepoints in the training vector space



Why just two particles?Why just two particles?

Three particles consisting of two codewordThree particles consisting of two codewords: s: PP11={={yy11, , yy22}; }; PP22={={yy22, , yy11} and } and PP33={={yy33, , yy44}. }. PP33 has a poorer performance has a poorer performance During the following iterations, particle During the following iterations, particle PP11 aandnd PP22 are comparative are comparative

The fly direction of particle The fly direction of particle PP33 is uncertain is uncertain



Stable convergence Unstable convergenceStable convergence Unstable convergence

Why just two particles?Why just two particles?



Updating steps 2 & 3Updating steps 2 & 3

Apply LBG with only 3 iterations to avoid cApply LBG with only 3 iterations to avoid converging earlyonverging early

Deal with the codewords “flying” over the bDeal with the codewords “flying” over the boundary of search space: Replace this kinoundary of search space: Replace this kind of codeword with the training vector that d of codeword with the training vector that has higher distortionhas higher distortion



Demonstration in Demonstration in 2-dimensional space2-dimensional space

Three objectives Three objectives PSO-LBG intends to achieve: intends to achieve: Disperse codewords Disperse codewords Move towards global optimum codebook Move towards global optimum codebook Codewords are settled reasonably both in high densitCodewords are settled reasonably both in high densit

y regions and low density areas of training vectors spy regions and low density areas of training vectors space ace

0 50 100 150 200 2500

50

100

150

200

250

0 50 100 150 200 2500

50

100

150

200

250

0 50 100 150 200 2500

50

100

150

200

250



Demonstration in Demonstration in 2-dimensional space2-dimensional space

Initial codebook =561.15

LBG =61.26

PSO-LBG =46.85

0 50 100 150 200 2500

50

100

150

200

250


0 50 100 150 200 2500

50

100

150

200

250

0 50 100 150 200 2500

50

100

150

200

250


Performance comparisonPerformance comparison

Performance is evaluated by Performance is evaluated by and PSN and PSNRR : : Mean square error between the training vMean square error between the training v

ectors and corresponding nearest codeworectors and corresponding nearest codewordsds

PSNR: Peak signal to noise ratioPSNR: Peak signal to noise ratio2255PSNR 10log (dB)

/D L



Performance comparisonPerformance comparison

Comparison is conducted among:Comparison is conducted among: LBGLBG Fuzzy k-means (FKM)Fuzzy k-means (FKM) Fuzzy reinforced learning vector quantization Fuzzy reinforced learning vector quantization

(FRLVQ)(FRLVQ) FRLVQ-FVQ: Apply FRLVQ as the pre-FRLVQ-FVQ: Apply FRLVQ as the pre-

process of FVQ process of FVQ PSO-LBGPSO-LBG



Experimental imagesExperimental images

LenaPepper

Cameraman

Kgirl



PSNR comparison on LenaPSNR comparison on Lena


1 2 3 4 5 6 7 8 9 10

28.8

29

29.2

29.4

29.6

29.8

30

30.2

30.4

30.6

Number of runs

PS

NR

(dB

)PSO-LBGFRLVQ-FVQFRLVQFKMLBG


Convergence comparison on LenaConvergence comparison on Lena


5 10 15 20 25 3026

27

28

29

30

31

32

Number of iterations

PS

NR

(dB



Computation time on LenaComputation time on Lena

0102030405060708090

100

FKM FRLVQ FRLVQ-FVQ

PSO-LBG

ti me (mi n)



Codebook characteristic on LenaCodebook characteristic on Lena

D

0 100 2000

50

100

150

200

250

0 2 4

x 104

0

50

100

150

200

250

0 2000 40000

50

100

150

200

250Codebook Size AV. D D



PSNR comparison on pepperPSNR comparison on pepper


1 2 3 4 5 6 7 8 9 1029.5

30

30.5

31

31.5

Number of runs

PS

NR

(dB



PSNR comparison on cameramanPSNR comparison on cameraman


1 2 3 4 5 6 7 8 9 10

27

27.5

28

28.5

29

29.5

30

30.5

31

Number of runs

PS

NR

(dB



PSNR comparison on KgirlPSNR comparison on Kgirl


1 2 3 4 5 6 7 8 9 1031

31.5

32

32.5

33

33.5

34

Number of runs

PS

NR

(dB



ConclusionConclusion

Experimental results demonstrate that PSExperimental results demonstrate that PSO-LBG Outperforms existing algorithms in tO-LBG Outperforms existing algorithms in the field of vector quantizationhe field of vector quantization

Future workFuture work Application in gene clusteringApplication in gene clustering



AcknowledgementAcknowledgement

My supervisor:My supervisor: Prof. JiProf. JiAll of youAll of you



Thank you!Thank you!


Faculty of Information Engineering, Shenzhen University Liao Huilian SZU TI-DSPs LAB Aug 27, 2007...

Documents

Transcript of Faculty of Information Engineering, Shenzhen University Liao Huilian SZU TI-DSPs LAB Aug 27, 2007...