N-tuple S&P Patterns Across Decades, 1950s to 2011

A. G. (Tassos) Malliaris

Mary MalliarisLOYOLA UNIVERSITY CHICAGO

The Korea Institute of Finance and

The Athenian Policy Forum Conference

Seoul, Korea

June 16-18, 2013

Purpose

• To investigate the Up and Down movements of the S&P 500 from 1950 through 2011

• And to use that information to forecast the direction tomorrow of the S&P 500

Data

• Daily closing prices from 1/3/1950 to 7/19/2011

• Over 15,000 observations

• were transformed into Up [U] or Down [D] by comparing today’s value to yesterday’s

• Up and Down movements were recorded from 1 up to 7 days

Up [U] and Down [D] movements per decade, 1950 through 2011.

Two-Day Patterns Across Decades

Three-Day Patterns

Four to Seven-Day patterns

• Strings were formed, for example, UDDU through DUDUDDU

• From 16 to 128 patterns [too many to display on a slide]

• Columns also created counting the number of Up movements within a string

Number of Ups in Two Days

Number of Ups in Three Days

Number of Ups in Four Days

Number of Ups in Five Days

Forecasting

• Training set: data from January 1950 through December 2009 [15,087 rows].

• Validation set: January 2010 through mid-September 2011 [387 rows]

• Training set patterns were used as inputs for models that were then used to predict the Validation set

Forecasting Methodologies

• Pattern Forecasts with 1 to 7 Days

• Decision Tree

• Neural Network

• Random Forecasts

The Forecast Decision by Pattern, an Example

Training set 4-day patterns

Count

DDUD 760

DDUU 1050

Validation Set Pattern Up through Today

Most Likely Move Tomorrow

DDU U

Decision Tree Forecasts

• Decision Tree Methodology: C5.0

• Software Package: SPSS Modeler

• Target: Tomorrow’s Direction

• Inputs: : the up-down patterns from one to seven days, the number of up days in 1 to five days, and the closing value today

Decision Tree Variable Importance• This Modeler technique also ranks the input

variables in terms of importance, with more important variables occurring higher up on the tree.

• The variables ranked highest in importance to the forecast were

the direction today

the closing value today

the 7-day pattern [for example UDUUDDU]

number of Up movements in the last three days.

Neural Network Forecasts

• Neural Network Methodology: Back Propagation; two hidden layers

• Software Package: SPSS Modeler

• Target: Tomorrow’s Direction

• Inputs included: the up-down patterns from one to seven days, the number of up days in 1 to five days, and the closing value today

Neural Network Variable Importance

• SPSS Modeler also ranks the input variables in terms of their importance to the neural network

• The variables ranked highest in importance with this methodology were

the patterns in the 4 day through 7 day strings

the number of Ups in the past three days

the closing value

Comparing Methods by Calculating Gain or Loss

Daily Gain/Loss Summed Over 387 Days

Change in Index

Actual Direction

Predicted Direction

Amount of Gain/Loss

-5 Down Down 5-5 Down Up -55 Up Down -55 Up Up 5

Random Forecast

• A random number was generated for each day in the validation set

• If the random value was less than .5 then Down was predicted, otherwise the forecast was Up.

• The amount of gain/loss was calculated over the Validation set

• This was repeated 100,000 times

RANDOM WALK SIMULATIONSTotal gains/losses for 100,000 simulated random forecasts for 387 days

Minimum -1028.62Maximum 1048.12Mean -0.0995Std Dev 235.4787

Number of correct directional forecasts for each methodology

Method Down UpNumber Correct

2 Day String 72 123 195

3 Day String 41 164 205

4 Day String 41 164 205

Decision Tree 43 168 211

5 Day String 45 167 212

7 Day String 48 164 212

6 Day String 40 178 218

1 Day String 0 219 219

Neural Net 61 161 222

Total amount gained or lost per strategy, and comparison to the random simulations

Type of ForecastTotal Amount Gained/Lost

Amount as Std Deviations in the Random

Distribution2 Day String -21.08 -0.087Decision Tree 168.16 0.7153 Day String 174.34 0.7424 Day String 174.34 0.7421 Day String 193.96 0.8257 Day String 306.92 1.3035 Day String 307.22 1.3056 Day String 335.84 1.426Neural Net 393.4 1.67

Conclusions• The number of up movements is greater than

down movements across the decades.

• Patterns of Up and Down movement from the past can be useful for forecasting future movement

• The highest gain from simple patterns came from using the 6-day strings

• The neural network, a more complex methodology, yielded the greatest gain