Time Series Forecasting Using Wavelet-Least Squares ...file.scirp.org/pdf/OJS_   Time Series...

Click here to load reader

download Time Series Forecasting Using Wavelet-Least Squares ...file.scirp.org/pdf/OJS_   Time Series Forecasting

of 12

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Time Series Forecasting Using Wavelet-Least Squares ...file.scirp.org/pdf/OJS_   Time Series...

  • Open Journal of Statistics, 2013, 3, 183-194 http://dx.doi.org/10.4236/ojs.2013.33021 Published Online June 2013 (http://www.scirp.org/journal/ojs)

    Time Series Forecasting Using Wavelet-Least Squares Support Vector Machines and Wavelet Regression Models

    for Monthly Stream Flow Data

    Siraj Muhammed Pandhiani, Ani Bin Shabri Faculty of Science, Department of Mathematics, Universiti Teknologi Malaysia, Skudai, Malaysia

    Email: pandhiani@hotmail.com, ani@utm.my

    Received March 24, 2013; revised April 25, 2013; accepted May 2, 2013

    Copyright 2013 Siraj Muhammed Pandhiani, Ani Bin Shabri. This is an open access article distributed under the Creative Com-mons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

    ABSTRACT This study explores the least square support vector and wavelet technique (WLSSVM) in the monthly stream flow fore- casting. This is a new hybrid technique. The 30 days periodic predicting statistics used in this study are derived from the subjection of this model to the river flow data of the Jhelum and Chenab rivers. The root mean square error (RMSE), mean absolute error (RME) and correlation (R) statistics are used for evaluating the accuracy of the WLSSVM and WR models. The accuracy of the WLSSVM model is compared with LSSVM, WR and LR models. The two rivers surveyed are in the Republic of Pakistan and cover an area encompassing 39,200 km2 for the Jhelum River and 67,515 km2 for the Chenab River. Using discrete wavelets, the observed data has been decomposed into sub-series. These have then appropriately been used as inputs in the least square support vector machines for forecasting the hydrological variables. The resultant observation from this comparison indicates the WLSSVM is more accurate than the LSSVM, WR and LR models in river flow forecasting. Keywords: River Flow; Time Series; Least Square Support Machines; Wavelet

    1. Introduction water modeling and water resource processed such as flood stage forecasting [6], rainfall runoff modeling [7-9] and stream flow forecasting [10-12]. Researcher solved the standard SVM models using the complicated compu- tational programming techniques, which are very expen- sive in time to handle the required optimization pro- gramming.

    Pakistan is the home to the worlds largest contiguous irrigation system. This is an economy that is largely de- pendent on the agricultural sector; therefore, it has a vast irrigation network. The dependence on this form of farm- ing is developed by a persistent rainfall pattern that keeps the rivers flowing, which in turn leads to the availability of irrigation waters flowing downstream. The Jhelum and Chenab rivers provide the biggest percentage of irrigation waters in various provinces in Pakistan.

    The Suykens and Vandewalls [13] proposed the least square support vector machines (LSSVM) model to sim- plify the SVM. In the various areas, such as regression problems and pattern recognition [14,15], the LSSVM model has been used successfully. LSSVM and SVM have almost similar advantages but the LSSVM has an additional advantage, e.g. it needs to solve only a linear system of equation, which is much easier to solve and predict results [1-4]. The LSSVM and SVM mathematical models have been used in predicting ad analyzing the future flow of rivers.

    These days, river flow forecasting has a major role to play in water resources system planning. The sources of water with many activities such as planning and operating system component estimate for future demand. The composition of water is necessary for both short-term and long-term forecasts of the event flow to optimize system or an application for the growth or decline in the future. There are many mathematical models to predict future flow of rivers such as discussed by [1-4].

    Recently, wavelet theory has been introduced in the field of hydrology, [16-18]. Wavelet analysis has recently been identified as a useful tool for describing both rainfall and runoff time series [16,19]. In this regard there has

    The Support Vector Machine (SVM) [5] forecasting method has been used in various studies of hydrology

    Copyright 2013 SciRes. OJS


    been a sustained explosion of interest in wavelet in many diverse fields of study such as science and engineering. During the last couple of decades, wavelet transform (WT) analysis has become an ideal tool studying of a measured non-stationary times series, through the hydrological pro- cess.

    An initial interest in the study of wavelets was devel- oped by [20-22]. Daubechies [22] employed the wavelets technique for signal transmission applications in the electronics engineering. Foufoula Georgiou and Kumar [23] used geophysical applications. Subsequently, [17] attempted to apply wavelet transformation to daily river discharge records to quantify stream flow variability. The wavelet analysis, which is analogous to Fourier analysis is used to decomposes a signal by linear filtering into components of various frequencies and then to recon- struct it into various frequency resolutions. Rao and Bopardikar [24] described the decomposition of a signal using a Haar wavelet technique, which is a very simple wavelet.

    Wavelet spectrum, based on the continuous wavelet transform, has been proven to be a natural extension of the much more familiar conventional Fourier spectrum analy- sis which is usually associated with hydro metrological time series analysis [25]. Instead of the results being presented in a plot of energy vis a vis frequency for energy spectrum in Fourier Transform (FT) as well as FFT (Fast Fourier Transform), the wavelet spectrum is three di- mensional and is plotted in the time frequency domain in which the energy is portrayed as contours. The wavelets are mathematical functions that break up a signal into different frequency components so that they are studied at different resolutions or scales. They are considered better than the Fourier analysis for their distinct signals that pose discontinuities and sharp spikes.

    The main purpose of the this study is to investigate the performance of the WLSSVM model for streamflow forecasting and to compare it with the performance of the least square support vector machines (LSSVM), linear regression (LR) and wavelet regression models (WR).

    2. Methods and Materials 2.1. The Least Square Vector Machines Model LSSVM is a new version of SVM modified by [13]. LSSVM involves the solution of a quadratic optimization problem with a least squares loss function and equality constraints instead of inequality constraints. In this sec- tion, we briefly introduce the basic theory LSSVM in time series forecasting. Consider a training sample set

    i i , x y with input i nx R iy R and output . In fea- ture space SVM models take the form

    Tw x b

    y x (1)

    xwhere the nonlinear mapping

    maps the input data into a higher dimensional feature space. LSSVM intro- duces a least square version to SVM regression by for- mulating the regression problem as

    T 2


    1min ,2 2



    R w e w w e


    subject to the equality constraints

    T , 1,2, ,i iy x w x b e i n (3) To solve this optimization problem, Lagrange function

    is constructed as

    T 2 T1 1

    , , ,

    12 2

    n n

    i i i i ii i

    L w b e

    w w e w x b e y


    where, i is Lagrange multipliers. The solution of (6) can be obtained by partially differentiating with respect to and , , iw b e i



    i ii

    L w xw


    10 0





    0 i ii

    L ee (7)

    T0 0,

    1, 2, ,

    i i ii

    L w x b e y

    i n



    then the weight w can be written as combination of the Lagrange multipliers with the corresponding data train- ing x .

    1 1

    n n

    i i i ii i

    w x e x





    i i iin

    i ii

    y x x b

    K b


    If we put the result of Equation (9) into Equation (3), then the following result is obtained:


    x x

    T,i i i


    where, a positive definite kernel is defined as follows:

    x x x xK (11)

    The vector and b can be found by solving a set of linear equations:


    T 1

    0 0

    i j

    byx x I


    1 (12)

    Copyright 2013 SciRes. OJS


    where, 1 1n n; ; , 1; ;1 ,y y y ; ;

    ,i i

    1 . This finally leads to the following LSSVM model for function estimation:



    iy K b x x

    x (13)

    where i , are the solution to the linear system. Ker- nel function, i represents the high dimensional feature space that is nonlinearly mapped from input space x. The typical examples of the kernel function are as follows:

    b ,K x x


    n Kernel :



    , 0

    K r



    , r




    Linear :

    ,Multi - Layer Perceptro

    , tanh

    Polynomial :

    , ,

    Radial basis function


    i i

    i i


    i i

    i i

    K x x x x

    K r


    x x x x

    x x x x

    x , x x


    Here and are the kernel parameters. The ar- chitecture of LSSVM is shown in Figure 1.


    2.2. Wavelet An