Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke,...
-
Upload
quantopian -
Category
Economy & Finance
-
view
748 -
download
3
Transcript of Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke,...
![Page 1: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/1.jpg)
Automated Selection and
Robustness
For Systematic Trading Strategies
Dr Tom Starke
![Page 2: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/2.jpg)
• Traditional approachParameter optimisation on train set
Validation on test set
This is what we can do now:System Parameter Permutation
Supervised compound metric optimisation
Domain Regression
Step-Down Correlation Minimisation
Strategy Selection Automation
Walk-forward optimisation
![Page 3: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/3.jpg)
Traditional Approach
![Page 4: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/4.jpg)
Train Data
Test Data
![Page 5: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/5.jpg)
Bad result - bad luck
Is there another way?
![Page 6: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/6.jpg)
Too many parameters is bad news
for systematic trading strategies
But what is too many?
![Page 7: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/7.jpg)
• Data mining bias very real
Overfitting is hard to detect
Solution:
Use all available backtest data
![Page 8: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/8.jpg)
System Parameter
PermutationEvaluation all possible outcomes*
Turning Data Mining from Bias to Benefit Through System Parameter Permutation
![Page 9: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/9.jpg)
Mean should
be positive
PnL Distribution of all backtest resultsPitfall: parameter space has unreasonable bounds
![Page 10: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/10.jpg)
Domain Regression:The train performance informs the test
performance.
![Page 11: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/11.jpg)
Train performance informs test performance
Correlation of results from backtest of train and test sets
![Page 12: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/12.jpg)
BAD!
![Page 13: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/13.jpg)
• Picking a single parameter set may not
produce very good results
Selecting several of the best strategies increases
the probability of arriving closer to the
regression line
![Page 14: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/14.jpg)
Step-Down Low-Correlation Approach
![Page 15: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/15.jpg)
Parameter sets should be as uncorrelated as possible
![Page 16: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/16.jpg)
One More:
Use random subsamples instead of fixed test
period
![Page 17: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/17.jpg)
Train
Test
![Page 18: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/18.jpg)
Randomised test sets
![Page 19: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/19.jpg)
Estimating PnL Shortfall
![Page 20: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/20.jpg)
BRAC*- Build,Rebuild and Compare
Out-of-sample performance scales with the
mean of the PnL distribution of N
subsamples.
*Building Reliable Trading Systems - Keith
Fitschen
![Page 21: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/21.jpg)
Choosing subsamples for
BRAC test
![Page 22: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/22.jpg)
Mean PnL shortfall scales with OOS performance
Performance of full data set
Adjusted mean PnL of
subsamples
![Page 23: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/23.jpg)
Bootstrapping
Expected
OOS PnL
![Page 24: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/24.jpg)
Compound Metrics
Manually supervised ML
![Page 25: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/25.jpg)
Optimisation: Monte-Carlo vs Grid Search
GA’s may not give the full story.
![Page 26: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/26.jpg)
Walk-forward optimisation
![Page 27: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/27.jpg)
System Parameter Permutation Profitability
Domain Regression OOS Expectation
Step-Down Selection Optimising Outcome
BRAC PnL Shortfall
Walk-Forward Optimisation Algorithm
Human-Supervised Metric
ConstructionOutput Metric
Bootstrapping PnL Shortfall
![Page 28: Automated Selection and Robustness for Systematic Trading Strategies by Dr. Thomas Starke, Quantitative Trader at Vivienne Court](https://reader034.fdocuments.in/reader034/viewer/2022051318/5a647bf37f8b9a46568b498d/html5/thumbnails/28.jpg)
Questions?