Interest Rate Forecasting by Term Structure: SVM vs. ANN 1 Intelligent Systems in Accounting,...

1

Interest Rate Forecasting by Term Structure: SVM vs. ANN

Intelligent Systems in Accounting, Finance and Management, AI’08, Cambridge

Interest Rate Forecasting: Support Vector Machines vs. Neural

Networks Rong Qu

School of Computer Science, The University of [email protected]

based on Andreas Jacovides, MSc Dissertation on Computational Finance,

Business School & School of Computer Science, University of Nottingham

2



• Problem of interest rate forecasting

• SVM and ANN in computational finance

• Comparing SVM and ANN on interest rate prediction

• Conclusions and future work

Content

3



• Interest rate: the percentage of principal amount borrowed, which is paid as a fee to the lender

• Interest Rate prediction

– One of the most challenging tasks

– Provide important information• Investment decisions• Reduce the associated interest rate risk, i.e. in capital

budgeting

Forecasting the Interest Rate

4



• Classical techniques

– Economic indicators and explanatory variables

– Expectations Hypothesis (EH)• Slope of the yield curve presents future expectations

of interest rates levels• Forward and spot rates• Spreads between long-term and short-term rates [FB87,

CS91]

• Little predictive power compared to random walk– Failure to identify exact relations between the spread

and future interest rate changes, due to use of linear models [M08]


5




– Expectations Hypothesis (EH)• The term structure reflects market expectations for

future interest rates• A rising term structure or a rise in the spread

between long-term and short term rates– Increase in the short-term rate– Vice versa

S(n,m) = Rtn - Rt

m


6




– Equilibrium term structure models• Explain the shape and informative content of the

term structure• Statistical properties of interest rate movement for

probabilistic description of future changes [V77, CIR95]

• Restrictive assumption on variables and linearity [B97]


7



• Recent research– No clear conclusion from literature– Results vary and mixed depend on models– Models not able to beat random walk

– Non-linear modelling of financial problems• Favourable compared to parametric methods• Although EH is rejected

– Informative term structure slope has been confirmed– Evidence in support of the predictive power of spreads


8



• Neural networks– Limited work on interest rate prediction

• Support vector machines– No published work so far on interest rate prediction

• Comparison between these techniques on interest rate prediction in this work


9



• One of machine learning methods– Fully connected network of small computation units,

neurons– Neuron: output calculated by weight sum of inputs to an

activation function

• Supervised learning (training)– Adjust weights– Cost function, e.g. SSE, MSE, DS

Artificial Neural Networks (ANN)

10



• One of the most popular prediction/classification tools– Data driven non-parametric statistical method– Generalise to noisy data– Approximate any continuous functions to desired level of

accuracy

• Learn and identify complex patterns and non-linear relationships for new data (multi-layer perceptron)

• Limited work on interest rate prediction[WD95,T98,AS02]

Artificial Neural Networks (ANN)

11



• Classification and regression– Hard margin SVM

• Find decision boundaries that not only correctly separate classes in the train set, but also lies as far from the training examples as possible– Problem transferred to optimisation problem

– Soft margin SVM

– By how much the margin is violated C

Support Vector Machines (SVM)

12



• Most real world problems are non linear– Mapping function (Kernel function)– Kernel trick: map the input space to a feature space,

solved by linear models– How to choose the best kernel for the problem in hand– Finding the optimal hyperplane, determined by support

vectors– Optimisation: Qudratic Programs problems


13



• SVM in financial applications– Mainly in financial time series forecasting, i.e. daily

closing prices of firms, treasury bills, daily index, future contracts forecasting, exchange rates, etc

– Also on option pricing, credit rating, bankruptcy prediction

– Only one study of SVM on interest rate prediction so far

• In general, it’s expected that SVNs perform at least as good as ANNs


14



• UK spot rates from government liability yield curve with maturities of different lengths– Bank of England web site, Mar 1997 – May 2008– Six months, one/three/five/ten/twenty years

• Initial statistics– Short maturity rates higher than longer maturity rates

and more volatile– Correlation coefficients between different maturity

spot rates• Highly correlated• Weaker as the difference increases

Problem Data

0.0200

0.0250

0.0300

0.0350

0.0400

0.0450

0.0500

0.0550

0.0600

0.0650

0.0700

0.0750

0.0800

0.0850

03/0

3/19

97

03/0

9/19

97

03/0

3/19

98

03/0

9/19

98

03/0

3/19

99

03/0

9/19

99

03/0

3/20

00

03/0

9/20

00

03/0

3/20

01

03/0

9/20

01

03/0

3/20

02

03/0

9/20

02

03/0

3/20

03

03/0

9/20

03

03/0

3/20

04

03/0

9/20

04

03/0

3/20

05

03/0

9/20

05

03/0

3/20

06

03/0

9/20

06

03/0

3/20

07

03/0

9/20

07

03/0

3/20

08

6- month spot rate 1- year spot rate 3- year spot rate 5- year spot rate

10- year spot rate 20- year spot rate

15



• Inputs & outputs– Mar 1997 – Nov 2007 & Sep 1997 – May 2008

• 2700 daily observations excl. holidays– 70% for training– 15% for validation– 15% for testing (unknown)

• Based on EH and previous work onpredictive power of spreads

Training Data

0.0200

0.0250

0.0300

0.0350

0.0400

0.0450

0.0500

0.0550

0.0600

0.0650

0.0700

0.0750

0.0800

0.0850

03/0

3/19

97

03/0

9/19

97

03/0

3/19

98

03/0

9/19

98

03/0

3/19

99

03/0

9/19

99

03/0

3/20

00

03/0

9/20

00

03/0

3/20

01

03/0

9/20

01

03/0

3/20

02

03/0

9/20

02

03/0

3/20

03

03/0

9/20

03

03/0

3/20

04

03/0

9/20

04

03/0

3/20

05

03/0

9/20

05

03/0

3/20

06

03/0

9/20

06

03/0

3/20

07

03/0

9/20

07

03/0

3/20

08

6- month spot rate 1- year spot rate 3- year spot rate 5- year spot rate

10- year spot rate 20- year spot rate

16



• Research finding– Level of interest rates, shape of term strcture in inputs

add additional power of predictions

• Function of interest rates spreads and interest rate levels, both tested by ANN and SVM

Function 1

f(spreads between subsequent interest rates, interest rates levels)

Function 2

f(interest rates level, spreads from all other maturities)

Problem Formulation

17



• Initial statistics (correlations)

– Level of interest rates vs. change six month forward• Negative• Stronger for longer maturities

– Different spreads vs. interest rate changes• Positive although not strong

– Can nonparametric approaches extract the relations and predict interest rates?

Problem Formulation

18



ANN Implementations

– One hidden layer– Hidden nodes < 20, by experiments– Data normalised to faciliate training– Training: Levenberg-Marquardt (LM)

algorithm[D08] (accurate and efficient)– Stopping: 1000 epochs or gradient <1e-10

19



ANN Implementations

– One hidden layer– Hidden nodes < 20, by experiments– Data normalised to faciliate training– Training: Levenberg-Marquardt (LM)

algorithm[D08] (accurate and efficient)– Stopping: 1000 epochs or gradient <1e-10

11-3-1

1y

6-14-1

3y

6-10-1

5y

11-6-111-2-1topology

10y6m

Different specs required to obtain the best predictions.Different functions different performance (11 vs. 6 inputs).

20



SVM Implementations

– Kernel function: Gaussian Radial Basis Function[CT01]

• Good on time series forecasting

– ε-insensitivity loss: [0.0005, 0.5]– Upper bound C: different values in [1, 200]– γ: [0.001, 0.05]– Parameter settings by experiments

21



SVM Implementations

– Kernel function: Gaussian Radial Basis Function[CT01]

• Good on time series forecasting

– ε-insensitivity loss: [0.0005, 0.5]– Upper bound C: different values in [1, 200]– γ: [0.001, 0.05]– Parameter settings by experiments

110.50.01

501y

110.50.05

203y

110.50.155y

110.050.01

506m

0.5

γ

0.0005

ε

6100

10y

InputsC

Different specs required to obtain the best predictions.Function 1 performed better (11 inputs).

22



Comparisons

– Simple random walk model• Prediction of next period is simply the current actual

value

– Both SVM and ANN outperformed the simple random walk model

– ANN observations• With maturity increases: average errors decreased

but the range of individual errors increases.

6-months spot rate 1-year spot rate

23



Comparisons

– SVM in general performed better than ANN• ANN better on 6 months predictions• SVM better on all others• SVM has fewer parameters and is less sensitive to

parameter settings

– Prediction of direction monthly• SVM and ANN predict correct directions for 19 and 18

months out of 20 months, respectively

6-months spot rate 5-year spot rate 10-year spot rate3-year spot rate

24



Discussions• Both SVM and ANN beat no-change model

– Several studies were unable to beat no-change model, especially for short-term interest rate changes

– Short-term interest rate changes are usually small

• Is the interest rate in month x increasing?– SVM: 85.04%; ANN: 78.32%– Main concern in practice

• Predictive content of the term structure– Evidence in support: spreads of interest rate levels– The term structure contains valuable content

25



Discussions• SVM performed better than ANN

– SRM principle incorporated– Optimisation in SVM is convex (global optimal solutions)– Once kernel function chosen, small sensitvity to C

• ANN– Stuck in local optima (minimising cost function)– Errors are very different from each random initialisation– Sensitive to weights, the number of hidden nodes– Lot of parameter settings: activation function, hiddern

layer, training algorithm, weights

26



Future Work & Research Questions• How does data pre-processing affect the

performance?– Does different transferring and segmenting help?– Forecasting ability specific to the specific time period

division?– Different data smoothing or normalisation methods

• Problem formulation– Additional factors not from the term structure, i.e.

inflation, measure of money supply– How to model uncertainty of economy?

27



Future Work & Research Questions• Can ANN be further improved?

– GAs/SA to search for the real optimal ANN topology– Are all inputs necessary, i.e. subset of inputs?– Self Organising Maps

• Can SVM be improved?– What’s the real implication of SRM in SVM?– Is other kernel function better?– More analysis on parameter settings for this newly

emerged technique

28



References• [BBO97] J. Barkoulas, C. Baum, and J. Onochie. A nonparametric

investigation of the 90-day T-bill rate. Working Paper in Economics, Economics Department, Boston College, 1997.

• [CIR85] J. C. Cox, J. E. Ingersoll, and S. A. Ross. A theory of the term structure of interest rates. Econometrica, 53(385-407), 1985.

• [CS91] J. Campbell and R. Shiller. Yield spread and interest rate movements: a bid's eye view. Review of Economic Studies, 58:495-514, 1991

• [CT01] L. J. Cao and E. H. Tay. Financial forecasting using support vector machines. Neural Computing and Applications, 10:184-192, 2001.

• [D08] H. Demuth, M. Beale, and M. Hagan. Neural network toolboxTM6. Matlab, 2008.

• [FB86] E. F. Fama and R. R. Bliss. The information in long-maturity forward rates. American Economic Review, 77:680-692, 1986.

• [M08] M. Modena. The term structure and the expectations hypothesis: a threshold model. MPRA Paper No. 9611, 2008.

• [V77] O. A. Vasicek. An equilibrium characterization of the term structure. Journal of Financial Economics, 5:177-188, 1977.

29



Questions/Discussions?

Thank you!

Interest Rate Forecasting by Term Structure: SVM vs. ANN 1 Intelligent Systems in Accounting,...

Documents

Transcript of Interest Rate Forecasting by Term Structure: SVM vs. ANN 1 Intelligent Systems in Accounting,...