Enhancing Spectral Classification Using Adaboost Khairul Muzzammil Saipullah and Nurul Atiqah Ismail,

Faculty of Electronic and Computer Engineering, UniversitiTeknikal Malaysia Melaka (UTeM),

Melaka, Malaysia muzzammil@utem.edu.my and m021110036@student.utem.edu.my

Abstract—Spectral classification for hyperspectral image is a challenging job because of the number of spectral in a hyperspectral image and high dimensional spectral. In this paper, we proposed a method to enhance the spectral classification using the Adaboost for hyperspectral image analysis. By applying the Adaboost algorithm to the classifier, the classification can be executed iteratively by giving weight to the spectral data, thus will reduce the classification error rate. The Adaboost is implemented to spectral angle mapper (SAM), Euclidean distance (ED), and city block distance (CD). From the experimental results, the Adaboost increases the average classification accuracy of 2000 spectral up to 99.63% using the CD. Overall, Adaboost increases the average classification accuracy of ED, CD, and SAM by 2.54%, 1.95%, and 1.67%.

Keywords-Adaboost; SAM; hyperspectral; spectral similarity; spectral classification

I. INTRODUCTION With the development of the hyperspectral sensor,

researchers in the remote sensing field are now able to extract ample spectral information to identify and discriminate between spectral similarities of materials. The hyperspectral image (HSI) of remotely sensed data covers large number of bands which results in more accurate and detailed information of each material than multispectral image does. A typical HSI covers up to several hundreds of bands.

In order to capture and characterize the spectral properties provided in a single pixel vector by hundreds of spectral channels, a new stochastic measure, called the spectral information measure (SIM), was introduced [1][2]. It models the spectrum of a pixel vector as a probability distribution so that the spectral properties of the pixel vector can be further characterized by its statistical moments of any order, such as the mean, variance, skewness, and kurtosis. More importantly, by virtue of the SIM, many concepts in information theory are readily applied to spectral characterization. For example, the self-information derived from the SIM can be used to describe the information provided by a particular spectral channel within a pixel vector. Using such self-information, a discrimination measure, called the spectral information divergence (SID) can be derived and used to measure the spectral similarity between two pixel vectors [1].

In the remote sensing community, the spectral angle mapper (SAM) has been widely used as a spectral similarity measure for material identification [3][4]. It calculates the angle between two spectra and uses it as a measure of discrimination. Another popular spectral similarity measure,

Euclidean distance (ED) has also been used to calculate the distance between two spectra as a spectral similarity measure [8]. It is shown in [1][2] that when the angle is small, the SAM and ED yield very close results and they are essentially equivalent in terms of spectral discrimination. In contrast with the SID, the SAM and the ED are deterministic measures and consider a spectrum of a pixel vector as a vector rather than a probability distribution as modeled by the SIM. In the SAM classification algorithm [5], each query spectral is compared with all the spectra in the spectral database using the SAM. Then the class of the query spectral is assigned as the class of spectral in the database that achieve the SAM value smaller than a certain threshold.

However, the performances those spectral similarity measures are depending on the dimension of the feature vector. Typically, the larger the feature vector, the better the performance is. For feature extraction methods that have small dimension of feature vectors, they usually have high probability to perform purely with any spectral similarity measures. To solve this problem, the classifiers that utilize those spectral similarity measures need to be enhanced so that they can increase their performances even with small dimensional feature vector.

In this paper, we enhance the performance of SAM, ED, and city block distance (CD) classification algorithms by boosting the classifier using Adaboost algorithm [6]. By applying the Adaboost algorithm to the classifiers, the classification can be executed iteratively by giving weight to the spectral data, thus will reduce the classification error rate. The classification performance of classification algorithms that are enhanced with Adaboost is depending on the classification iteration and no longer depending on the size of the feature vector.

The remainder of this paper is organized as follows: in Section II, related work is presented; in Section III, the proposed method; in Section IV the experimental evaluations are described and finally conclusions and future works are given in Section V.

II. RELATED WORKS

A. AdaBoost Boosting is as a learning classifier that selects a small

number of week classifier from the large weak classifier hypothesis to construct a strong classifier. The most popular boosting algorithm is an AdaBoost come from the word Adaptive Boosting. AdaBoost is a meta-algorithm; it is used

2012 IEEE Asia-Pacific Conference on Applied Electromagnetics (APACE 2012), December 11 - 13, 2012, Melaka, Malaysia

9978-1-4673-3115-9/12/$31.00 ©2012 IEEE 17

as conjunction with combination with other learning algorithm to improve the accuracy of the learning algorithm [7]. Furthermore AdaBoost is a simple to use and implement and will give better results and sensitive to the noise data [9].

AdaBoost can be derived the method to find �t at each stage of the algorithm. Consider the algorithm running at some stage, as shown in Table I [9]. Generally, basic theoretical of AdaBoost concerns the ability to reduce the training error. Unfortunately the hypothesis will produce an error rate comes from the binary problems. Overcome of this problem, the training error with the final hypothesis H using this equation [8]:

( ) 2 22 1 1 4 exp 2t t t ttt t

ε ε γ γ� �� �− = − ≤ −� �� ��∏ ∏

(1)

Thus, if each weak hypothesis is slightly better than random so that �t � � for some � > 0, then the training error drops exponentially fast. From the Eq. (1), the boosting algorithm proves that the AdaBoost can efficiently convert a weak learning algorithm where it is can always generate a hypothesis with a weak edge for any distribution into a strong learning algorithm.

TABLE I. THE ADABOOST ALGORITHM.

Given: ( ) ( )1 1, ,... ,m mx y x y , ix χ∈ , [ ]1, 1iy ∈ − + and the size

of the final strong classifier T. Initialize:

( )1 1 / .D i m=

For t = 1…T

1) Find ( )1

arg min ( )j

m

t t t t i ih H ih D i h x yε

∈ =

= = ≠� � �� �

2) When 12tε ≥ , 11 ln

2t

tt

εαε

� �+= � � �

3) Update distribution ( ) ( ) ( )

1

t i t iY h xt

tt

D i eD i

Z

α−

+ = , where tZ

is a normalization factor Final classifier: ( ) ( )( )1

Tt tt

H x sign h xα=

= �

B. Spectral Angle Mapper The SAM [5] measures spectral similarity by finding the

angle between the spectral signatures s and s’ of two pixel vectors r and r’:

1 ,( , ) cos

s sSAM s s

s s− � �′

′ = � �� �′ � (2)

1, L

l lls s s s

=′ ′= � and ( )

122

1

Lll

s s=

′ ′= � (3)

This measure of similarity is insensitive to gain factors because the angle between two vectors is invariant to the lengths of the vectors. As a result, spectra can be directly compared to remotely sense apparent reflectance spectra, which inherently have an unknown gain factor related to

topographic illumination effects.

For each reference spectrum, the spectral angle for SAM is determined with respect to every pixel spectrum and this value is assigned to that pixel in the output SAM image. A unique spectral range may be chosen for each reference spectrum. The derived spectral angle maps form a new data cube with the number of bands equal to the number of reference spectra used in the mapping [6].

C. Euclidean Distance The Euclidean distance classifier has the similar concept of

SAM classifier. The difference is only the distance it uses to measure the similarity between two spectra. Instead of using the angle, it uses the Euclidean distance which is the magnitude between two vectors. The similarity between the spectral signatures s and s’ of two pixel vectors r and r’ is measured using the following formula:

( )2( , )ED s s s s′ ′= − (4)

The smaller the ED is the high similarity between two spectra. The ED is measured between the target spectral and the reference spectra. The target spectral will be classified as the same group of the reference spectral that have the smallest ED with the target spectral.

D. City Block Distance City block distance or the Manhattan distance is the

alternative way to calculated distance between two vectors or points other that the Euclidean distance [10]. It is actually the absolute difference between to vectors. For spectra, the similarity between the spectral signatures s and s’ of two pixel vectors r and r’ is measured using the following formula:

( , )CD s s s s′ ′= − (5)

Same as the ED, the smallest distance criteria is used to classify the target spectral.

III. THE PROPOSED METHODS To increase the spectral classification accuracy, the SAM,

ED, and CD classifiers are embedded with the Adaboost. The detail of the combination between the Adaboost and the classifiers will be explained in the following subchapters.

A. Adaboost Spectral Angle Mapper The Adaboost Spectral Angle Mapper (ASAM) applies the

Adaboost algorithm with SAM classifier as its weak learner. For each query spectrum, the SAM between the query spectrum and the spectra in the database are determined. Instead of setting a unique range of threshold for the SAM classifier, the smallest spectral angle for SAM is used in determining the similarity between the query and the database spectra. Then the weight is given to the spectra and the distribution over the training set is updated repeatedly. The detailed algorithm of ASAM is as follows:

1. Initialize distribution over the all of the training spectra

11( )D im

= , where m is the number of spectra.

2012 IEEE Asia-Pacific Conference on Applied Electromagnetics (APACE 2012), December 11 - 13, 2012, Melaka, Malaysia

18

2. For t=1, 2 ,…,T : A. Train the SAM classifier using training spectra with

distribution tD .

B. Choose a weight (or confidence value) Rtα ∈ . C. Update the distribution over the training spectra:

( )

1( )( )

t t t ty h xt

tt

D i eD iZ

α−

+ =

3. Final vote H(x) is a weighted sum:

1( ) ( ( )) ( )

T

t tt

H x sign f x sign h xα=

� �= = � � �� (6)

where T is the number of iterations.

B. Adaboost Euclidean Distance The Adaboost Eucliden Distance (AED) is the nearest

distance classifier using the ED to measure the distance between spectra. The steps to conducting AED are the same as ASAM except the classifier it uses for training is the ED classifier. For each query spectrum, the ED between the query spectrum and the spectra in the database are computed. The smallest ED is used in determining the similarity between the query and the database spectra. Then the weight is given to the spectra and the distribution over the training set is updated repeatedly.

C. Adaboost City Block Distance Same as the AED, the Adaboost city block distance (ACD)

uses the same algorithm in the classification except for the similarity measures where ACD uses the CD in measuring the similarity. For each query spectral, the CD between it and the spectra in the training or the database will be calculated and the shortest CD is used to determine the query spectral.

IV. EXPERIMENTAL RESULTS

Figure 1. The RGB band images of Seoul (left) and seosan (right)

We carefully extract six classes of spectral from two HSIs. The list and the number of spectra are listed in Table II. Two HSIs used in this experiment are the Seoul HSI (Upper left L/L: 37.661257N/126.785989E, Lower right L/L: 37.510587N /126.922847E) and Seosan HSI (Upper left L/L: 36.742125N/

126.521865E, Lower right L/L: 37.510587N/126.922847E). The resolution of each HSI is 559x403 and 439x367 pixels with a scale of 30m per pixel. RGB band images of these HSIs are shown in Fig. 1. These HSIs are captured using Hyperion sensors with a total of 229 bands. After removing the noise, the remaining bands are 145 bands with the wavelength in range from nm to 406.46nm to 2496.39nm. All of the experiment is conducted using Matlab.

The ASAM, AED, and ACD are used to classify the spectra. For the classifier, the classification is iterated 15 times (T values of the Adaboost algorithm is equal to 15). The result of the first iteration is the same result of the original SAM, ED, and CD without the Adaboost enhancement. A cross-validation scheme is applied in order to reduce the dependency on the training set. Each class of the spectral is equally divided into four groups and for each pair of groups, one group is used as testing set and the others are used as training set. Because we divided the spectral into four groups, by replacing the testing and training set with each of the group, there are twelve classifications will be conducted. The average of these twelve classification accuracies is recorded as the overall classification accuracy.

TABLE II. THE CLASS AND NUMBER OF SPECTRA USED IN THE CLASSIFICATION EXPERIMENT.

Class Number of spectra Water 1000

Buildup 1000 Barren 1000 Grass 1000

Coniferous 2000 Agricultural 2000

The spectral classification accuracies with respect to each class for SAM and Adaboost SAM classification algorithms are shown in Fig 2 to Fig 7. For each of the graphs, the performance of classification without Adaboost is the same as performance with iteration of one. As we can see, ASAM, AED, and ACD increase the classification accuracy for every spectral class. For the water class, there is no room for increase as the first iteration already achieves perfect 100% accuracy. For the buildup class, ASAM achieves the highest accuracy of 100% after increases it accuracy from 97.2%. ACD achieves the higher increment from 94.8% to 99.6% whereas AED rises from 95.2% to 99.2%.

Among all of the spectra classes, barren is the most hard to be classified. However, for the barren class, Adaboost increases the average accuracy of all of classifiers from 92% to 99.6%. This class achieves the highest accuracy increment with an increase of 8% of accuracy. For the grass class, the accuracy increases from 98.4% to 100% and for the coniferous class, we could see the consistent and high performance of the ASAM with the starting accuracy of 99.6%. The Adaboost iteration manages to increase its accuracy to 100%. On the other hand, ACD and AED increase their accuracy up to 99.8%, respectively. And lastly, for the agricultural class, Adaboost increases all of the classifiers to a perfect 100% of accuracy.

2012 IEEE Asia-Pacific Conference on Applied Electromagnetics (APACE 2012), December 11 - 13, 2012, Melaka, Malaysia

19

Figure 2. The classification accuracy of water class

Figure 3. The classification accuracy of build up class

Figure 4. The classification accuracy of barren class

The spectral average classification accuracies of all of the spectra for ASAM, AED, and ACD are shown in Fig 8. By embedding the Adaboost to the classifiers, the average spectral classification accuracy increases from 97.6% to 99.6%. From the average accuracy, we can see that the highest accuracy is achieved by ACD at iteration of 13 with 99.63% of accuracy. The summary of the average classification accuracies of all of the spectra is shown in Table III. The original accuracy is the accuracy of the first iteration whereas the highest accuracy is the highest accuracy among the accuracies in 15 iterations.

Among all of the classifiers, ACD is able to achieve the higher accuracy of 99.63% and also the highest increment of 2.54%.

Figure 5. The classification accuracy of grass class

Even though the task to classify 8000 spectra is not easy, the classifiers used in this experiment manage to achieve very high accuracies. Among all of the classifiers, ASAM shows the best performance with high original accuracy. From the class by class accuracies, we can conclude that different classifiers will lead to different result due to various accuracies of each classifier in classifying each of the spectra. However, Adaboost is able to shift all of the accuracies for each classifier to a same level where all of their accuracies have minimum differences compared the original accuracies. As a surplus, the same level that Adaboost bring them is the highest accuracies for each classifier which are incredibly near to the perfect 100% accuracy.

TABLE III. THE SUMMARY OF ALL OF THE SPECTRA USING ASAM, AED, AND ACD

Classifier Original Accuracy

Highest Accuracy

Increment

ASAM 97.97% 99.60% 1.67% ACD 97.17% 99.63% 2.54% AED 97.60% 99.50% 1.95%

Figure 6. The classification accuracy of coniferous class

2012 IEEE Asia-Pacific Conference on Applied Electromagnetics (APACE 2012), December 11 - 13, 2012, Melaka, Malaysia

20

Figure 7. The classification accuracy of agricultural class

Figure 8. The classification accuracy of all classes

V. CONCLUSION AND FUTURE WORKS In this paper, we proposed an enhancement of SAM, ED,

and CD classification algorithm using the Adaboost for hyperspectral image analysis. The contribution of this work is to apply the Adaboost to those classifiers and increase their classification accuracy while remove the dependency on feature vector size. The experimental result shows that the proposed method achieves the highest increase in classification accuracy in case of the barren class. Existing classifiers may not entirely discriminate the material between barren class and other classes. However, with Adaboost, the

classification of the barren class can be increased to an almost perfect performance. For the future work, we would like to construct a method to find the optimum iteration number in order to obtain the highest classification accuracy using the Adaboost SAM.

ACKNOWLEDGMENT This paper is supported by UniversitiTeknikal Malaysia

Melaka under PJP/2011/FKEKK(44B)/S00979.

REFERENCES [1] C.-I. Chang, “An information theoretic-based approach to spectral

variability, similarity and discriminability for hyperspectral image analysis,” IEEE Trans. Inf. Theory, vol. 46, no. 5, pp. 1927-1932, 2000.

[2] C.-I. Chang, Hyperspectral Imaging: Techniques for Spectral Detection and Classification, Chapter 2, Kluwer Academic/Plenum Publishers, New York, 2003.

[3] Y. Du, C.-I., H. Ren, C.-C. Chang, J. O. Jensen, and F. M. D'Amico, "New hyperspectral discrimination measure for spectral characterization," Opt. Eng. 43, pp. 1777-1786, 2004.

[4] F.A. Kruse, A.B. Lefkoff, J.W. Boardman, K.B. Heidebrecht, A.T. Shapiro, P.J. Barloon, A.F.H. Goetz, “The spectral image processing system (SIPS)--interactive visualization and analysis of imaging spectrometer data,” Remote Sensing of Environment, vol. 44, issue 2-3, pp. 145-163, 2003.

[5] D. Manolakis and G. Shaw, “Detection algorithms for hyperspectral imaging applications,” IEEE Signal Process. Mag., vol. 19, no. 1, pp. 29–43, Jan. 2002.

[6] K. Beyer, J. Goldstein, R. Ramakrishnan and Uri Shaft, “When Is “Nearest Neighbor” Meaningful?,” Proc. ICDT 99, LNCS, vol. 1540, pp. 217-235, 1999.

[7] Khairul-Muzzammil Saipullah, Byung-Choon Yun, Deok-Hwan Kim, “Unsupervised Multispectral Image Segmentaion Based on 1D Combined Neighborhood Differences”, 34th Korea Information Processing Society Spring Conference, November, 2010.

[8] Yoav Freund, Robert E. Schapire, "A Short Introduction to Boosting",Journal of Japanese Society for Artificial Intelligence, Vol.14(5):771-780, Sept. 1999.

[9] G. Rätsch, T. Onoda and K.-R. Müller, “Soft Margins for AdaBoost”, Machine Learning, Vol. 42(3), pp.287-320, Mac 2001.

[10] Khairul Muzzammil, Shao-hu Peng, and Deok-Hwan Kim, “Texture Descriptor for Texture-Based Image Retrieval and Its Application in Computer-Aided Diagnosis System”, Journal of the Institute of the Electronics Engineers of Korea, Vol. 47-CI, No.4, July, 2010.

2012 IEEE Asia-Pacific Conference on Applied Electromagnetics (APACE 2012), December 11 - 13, 2012, Melaka, Malaysia

21