Predictions from MARS

May 2012Maria Lupetini

Engineering Asset Management & AnalyticsQualcomm Incorporated

Advantages of MARS Modeling Predicting Demand for an Asset Capturing Trends and Seasonal Effects Finding Interactive Effects Weighting More Recent Data Autoregressive Model for Time Series Using Lag Variables Don’t be Afraid of Missing Values Summary of Findings

Predictions from MARS: Overview

Regression: Linear, Logistic, GLM, MARS ARIMA Time Series Decision Trees Neural Networks Support Vector Machines And more

Need to pick one or more approaches tailored to problem you are tackling

Many Prediction Methods

Sales - Dollars, Number of Chips

Resources - People, Software Assets

Performance of a Semiconductor - Seconds to load a web page

…You name it.

Many Examples: Need to Predict a Continuous Variable

Data contains continuous numbers $123,456.00 Number of employees

Understand influences of categories Geographical regions Operating system: Windows, Android

Seasonal or repeated trends Months of the year Christmas season

Special Effects Consumer Promotions and Advertising Switch turned on

Typical Business and Engineering Data

What do you do if you want to predict a trend or find a pattern in data….and

There are hundreds of possible variables that influence your outcome -◦ Which ones matter?

What if the variables interact with each other and effect the outcome◦ How do you find that those relationships?

What if variables are not linearly related to the outcome◦ How do determine the what the relationship curves will look like?◦ Threshold or plateau relationship

What if the data you are using to predict is a mixture of numbers and categories◦ How do you build a prediction formula?

How do I build a prediction model that is easy to understand?

… USE MARS

The Challenges in Prediction Modeling

MARS short for Multivariate Adaptive Regression Splines

Technique introduced in 1991, Jerome Friedman, Stanford University

Nonparametric, data driven algorithm

Prediction is a regression model with additional side equations (basis functions)

Uses piecewise regression splines to build the prediction

Provides data reduction to select which variables matter

MARS Overview

Software Used in Designing Semiconductor Chips

Is the use of the software growing?

What time of day are the software licenses most demanded?

Does demand change over the weekend?

How many copies do we need next week?

Example – Demand Planning

8/28/2011 4 AM 9/26/2011 4 PM 10/26/2011 4 AM11/24/2011 4 PM12/24/2011 4 AM 1/22/2012 4 PM 2/21/2012 4 AM 3/21/2012 6 PM0

50

100

150

200

250

300

350

Number of Software Licenses Used in an Hour from Aug 2011 to April 2012

How do you forecast this time series of demand data?

Example Data

Time

Actual Licenses

UsedWeek

NumberWeekDa

yDay

NameWeek end Holiday Hour

9/4/2011 9 PM 5837 1 Sun 1 Y 21

9/4/2011 10 PM 7537 1 Sun 1 Y 22

9/4/2011 11 PM 8837 1 Sun 1 Y 23

9/5/2011 12 AM 8137 2 Mon 0 Y 0

9/5/2011 1 AM 7437 2 Mon 0 Y 1

9/5/2011 2 AM 8037 2 Mon 0 Y 2

9/5/2011 3 AM 8137 2 Mon 0 Y 3

• Real Continuous or Integer Variables: License Counts, Week Number• Categorical Text Variables: Holiday flag, Day Name• Binary Numbers: Weekend flag• Choice of Categorical or Real Number: Week Day, Hour

Can we building a prediction model of the form?

Demand = Constant Base+Baseline trend +Hour of day effect +Day of Week effect +Holiday effect

Building a Model

Setting Up Model in MARS

MARS found underlying trend when adjusting for other factors like time of day, day of week, and holidays

Trend line captures:• Growing use of this software product from Sep 20112 to Apr 2012• Deadlines of semiconductor chip projects (Jan. and March)

MARS quantified contribution of hour of the day

Additional licenses

needed as function of hour of the

day

Hour Predictor Captures:• Highest use of licenses during 10 to 1pm US Pacific

time• Effect of Use in European/Indian time zones

MARS quantified contribution of day of the week

Additional licenses

needed as function of day of the

week

Day of Week Predictor Captures:• Highest use of licenses during Wednesday to Friday

Weekday was coded as a continuous variable.Coding it as a categorical can also work here.1= Sunday, 2=Monday, etc

Possible Interactive Effects Between Variables

Look to find an interactive effects between hour of day and day of week.

Did not want to allow interactive effects between week_number and holiday variables with other variables

MARS captures interactive effect between day of week and hour of day

Additional licenses

needed as function of

hour and day

Interactive effect • Work patterns are different on the weekends when

compared to the work week.

MARS quantified effect of holidays

Additional licenses

needed on non-holidays

Holiday Predictor Captures:• The difference in demand in a hour if it is a holiday

MARS enables you to weight the importance of more recent observations over older observations

0.5 1 1.5 2 2.5 3 3.5 44/17/2011 12 AM

6/6/2011 12 AM

7/26/2011 12 AM

9/14/2011 12 AM

11/3/2011 12 AM

12/23/2011 12 AM

2/11/2012 12 AM

4/1/2012 12 AM

5/21/2012 12 AM

Weighting of Observations

Weight Applied to Observations

Day a

nd H

our

Observ

ati

on

MARS will consider a “variable” as a weighting factor.Here, the observations in April 2012 were 3 times more important than observations in Sep 2011.

4/8/2012 12 AM 4/9/2012 4 PM 4/11/2012 9 AM 4/13/2012 3 AM 4/14/2012 7 PM4/16/2012 12 PM4/18/2012 6 AM4/19/2012 10 PM4/21/2012 3 PM0

50

100

150

200

250

300

350

Number of Software Licenses Used and Predicted

Part of the Training Dataset

Prediction on Unseen Data

Blue line Actual Licenses Used Red line is MARS fit on Training Data for 4/18 to 4/15 and Prediction on 4/15 to 4/21 data

How well did MARS predict next week?

How well did MARS fit the training dataset?

8/28/2011 4 AM 9/27/2011 4 AM 10/27/2011 4 AM11/26/2011 4 AM12/26/2011 4 AM 1/25/2012 4 AM 2/24/2012 4 AM 3/25/2012 6 AM0

50

100

150

200

250

300

350

Number of Software Licenses UsedTraining Dataset

Prediction Model Actual

MARS was able to capture:• Overall trend• Hourly and Week Day effect• Somewhat captured US holidays

Statistically speaking, how good is the fit?

Variable Importance -gcv --------------------------------------------------------------- WEEKDAY 100.00000 2713.86182 HOUR 93.20326 2418.96997 WEEK_NUMBER 44.00605 903.06390 HOLIDAY$ 21.76427 574.55463

==============================

N: 15217.52 R-SQUARED: 0.90281 MEAN DEP VAR: 158.15640 ADJ R-SQUARED: 0.90214 UNCENTERED R-SQUARED = R-0 SQUARED: 0.98493

F-STATISTIC = 1344.99320 S.E. OF REGRESSION = 35.12427 P-VALUE = 0.00000 RESIDUAL SUM OF SQUARES = .678790E+07 [MDF,NDF] = [ 38, 5502 ] REGRESSION SUM OF SQUARES = .630548E+08

MARS tells you which variables are most important.

Great R-Squared of 90%. Other diagnostics, not presented here, looked good too.

Actual Used: Range 45 to 344 LicensesAverage 95Standard Dev. 70

Another Modeling Approach: ARIMA-like

Can we build a prediction model of the autoregressive form?

Demand = Constant Base+Baseline trend +Effect of Licenses Used from a week ago +Workweek vs. Weekend effect +Holiday effect

Set up Autoregressive like model, Step 1

Set Up Autoregressive Model, Part 2

Creating lag variable for “Used Lag168.” This predictor is the number of licenses used in the same hour, in the same day, in the prior week.

MARS found underlying trend when adjusting for other factors in the Autoregressive model version.

Adjusting for underlying trend makes series stationary. This is necessary for ARIMA models.

MARS captures contribution of Used Lag 168 hours variable

BF1 = ( USED<168> ne . ); BF2 = ( USED<168> = . ); BF3 = max( 0, USED<168> - 42) * BF1; BF4 = max( 0, 42 - USED<168>) * BF1; BF5 = (HOLIDAY$ in ( "Y" )); BF7 = (MON_TO_FRI in ( 0 )); BF9 = max( 0, WEEK_NUMBER - 50) * BF1; BF10 = max( 0, 50 - WEEK_NUMBER) * BF1; BF11 = max( 0, USED<168> - 137) * BF1; BF13 = max( 0, USED<168> - 265) * BF1; BF15 = (MON_TO_FRI in ( 0 )) * BF2;

Number of Lucenses Needed = 134- 39 * BF1 + 0.58 * BF3 - 2.12 * BF4 - 42* BF5 - 21.6 * BF7 - 0.235 * BF9 - 1.598 * BF10 + 0.338 * BF11 - 0.535 * BF13 - 38 * BF15;

N: 15055.88 R-SQUARED: 0.82525 MEAN DEP VAR: 158.75413 ADJ R-SQUARED: 0.82493

F-STATISTIC = 2533.14901 S.E. OF REGRESSION = 47.37796

Selected MARS Output Showing Model Form and Fit

Basis Functions and Prediction Equation from MARS.

Note the handling of missing values.

Reasonable fit with 82% R-squared

MARS Autoregressive-Like Model in More Understandable Terms

For observations where the 168 lag of the “Used” variable is not missing:

Holiday = 1 if it’s a holiday, else 0 Weekend = 1 if it’s Saturday or Sunday, else 0

A = max( 0, USED<168> - 42) B = max( 0, 42 - USED<168>) C = max( 0, USED<168> - 137) D = max( 0, USED<168> - 265)

E = max( 0, WEEK_NUMBER - 50) F = max( 0, 50 - WEEK_NUMBER)

Forecasted License Need= 95 - 42*Holiday - 22 * Weekend [0.6 * A - 2.1 * B + 0.3 * C - 0.5 * D] +

[- 0.2 * E - 1.6 * F]

Autoregressive Splines

Trend line Splines

Autoregressive Model Fit on Training and Test Data

9/4/2011 4 AM 10/5/2011 4 AM11/5/2011 4 AM12/6/2011 4 AM 1/6/2012 4 AM 2/6/2012 4 AM 3/8/2012 4 AM 4/8/2012 6 AM0

50

100

150

200

250

300

350

400

PredictedUSED

How well did the Autoregressive-like model forecast?

4/8/2012 12 AM 4/9/2012 6 PM 4/11/2012 1 PM 4/13/2012 9 AM 4/15/2012 3 AM 4/16/2012 10 PM 4/18/2012 6 PM 4/20/2012 12 PM0

50

100

150

200

250

300

350

400

Part of Training Dataset

Forecasting Unseen Data

Number of Licenses Used and Predicted

Blue line is Actual UsedRed line is MARS fit on Training data for 4/8 to 4/14 and Prediction on 4/15 to 4/21 data

4/8/2012 12 AM 4/9/2012 7 PM 4/11/2012 3 PM 4/13/2012 12 PM 4/15/2012 7 AM 4/17/2012 3 AM 4/19/2012 12 AM 4/20/2012 7 PM0

50

100

150

200

250

300

350

400

Compare Forecast of Two Models to Actual Licenses Used

Predicted_AutoRegressive Actual Used Predicted Not Auto Reg

Num

ber

of

Lic

enses

Mathematically MARS is versatile; it models most data types Selects best predictors Models nonlinear relationships Easily finds selective interactive effects Simple to create lag variables as predictors Flexible weighting schemes for observations Can handle missing values

Operationally Don’t call me for more software license copies on

Thursday at noon; everyone else is!

Summary