Statistically sound machine learning for algorithmic trading of financial instruments

Table of Contents......Page 6
Two Approaches to Automated Trading......Page 17
Indicators and Targets......Page 20
Converting Predictions to Trade Decisions......Page 21
Testing the Trading System......Page 23
Walkforward Testing......Page 25
Cross Validation......Page 26
Overlap Considerations......Page 28
Model Performance Versus Financial Performance......Page 30
Financial Relevance and Generalizability......Page 31
Performance Statistics in TSSB......Page 32
Desirable Program Features......Page 36
The Script File......Page 38
The Audit Log......Page 42
A Walkforward Fold......Page 46
Out-of-Sample Results for This Fold......Page 50
The Walkforward Summary......Page 52
The Trade File......Page 54
The Script File......Page 57
The Audit Log......Page 61
Out-of-Sample Results for This Fold......Page 63
The Walkforward Summary......Page 65
Common Initial Commands......Page 67
Quick Reference to Initial Commands......Page 68
INTRADAY BY SECOND......Page 72
MARKET DATE FORMAT AUTOMATIC......Page 73
READ MARKET LIST......Page 74
READ MARKET HISTORIES......Page 75
MARKET SCAN......Page 76
RETAIN YEARS......Page 77
RETAIN MOD......Page 79
CLEAN RAW DATA......Page 81
INDEX......Page 82
OUTLIER SCAN......Page 83
DESCRIBE......Page 85
CROSS MARKET AD......Page 86
CROSS MARKET IQ......Page 87
STATIONARITY......Page 88
A Final Example......Page 92
Reading and Writing Databases......Page 97
Detailed Descriptions......Page 98
VARIABLE IS TEXT......Page 99
WRITE DATABASE......Page 100
READ DATABASE......Page 101
APPEND DATABASE......Page 102
A Saving/Restoring Example......Page 105
Creating Variables......Page 107
Index Markets and Derived Variables......Page 108
An Example of IS INDEX and MINUS INDEX......Page 110
Multiple Indices......Page 111
Centering......Page 112
Scaling......Page 113
An Example of Centering, Scaling, and Normalization......Page 114
Cross-Market Normalization......Page 118
Pooled Variables......Page 119
MEDIAN pooling......Page 120
CLUMP60 Pooling......Page 121
Mahalanobis Distance......Page 123
Absorption Ratio......Page 125
LINEAR PER ATR HistLength ATRlength......Page 127
CUBIC PER ATR HistLength ATRlength......Page 128
PRICE MOMENTUM HistLength StdDevLength......Page 129
RESIDUAL MIN ADX HistLength MinLength......Page 130
DELTA ADX HistLength DeltaLength......Page 131
INTRADAY INTENSITY HistLength......Page 132
REACTIVITY HistLength......Page 133
MIN REACTIVITY HistLength Dist......Page 135
MAX REACTIVITY HistLength Dist......Page 136
N DAY HIGH HistLength......Page 137
N DAY LOW HistLength......Page 138
LINEAR DEVIATION HistLength......Page 139
CUBIC DEVIATION HistLength......Page 140
DETRENDED RSI DetrendedLength DetrenderLength Lookback......Page 141
PRICE VARIANCE RATIO HistLength Multiplier......Page 142
CHANGE VARIANCE RATIO HistLength Multiplier......Page 143
DELTA PRICE VARIANCE RATIO HistLength Multiplier......Page 144
BOLLINGER WIDTH HistLength......Page 145
N DAY NARROWER HistLength......Page 146
N DAY WIDER HistLength......Page 147
DELTA INDEX CORRELATION HistLength DeltaLength......Page 148
PURIFIED INDEX Norm HistLen Npred Nfam Nlooks Look1......Page 149
CHANGE SKEWNESS HistLength Multiplier......Page 152
DELTA CHANGE KURTOSIS HistLen Multiplier DeltaLen......Page 153
VOLUME WEIGHTED MA OVER MA HistLength......Page 155
PRICE VOLUME FIT HistLength......Page 156
ON BALANCE VOLUME HistLength......Page 157
POSITIVE VOLUME INDICATOR HistLength......Page 158
PRODUCT PRICE VOLUME HistLength......Page 159
SUM PRICE VOLUME HistLength......Page 160
DELTA SUM PRICE VOLUME HistLen DeltaDist......Page 161
PRICE ENTROPY WordLength......Page 162
PRICE MUTUAL INFORMATION WordLength......Page 163
VOLUME MUTUAL INFORMATION WordLength......Page 164
Indicators Based on Wavelets......Page 165
REAL DIFF MORLET Period......Page 168
IMAG MORLET Period......Page 169
IMAG PRODUCT MORLET Period......Page 170
DAUB MEAN HistLength Level......Page 171
DAUB CURVE HistLength Level......Page 172
Follow-Through-Index (FTI) Indicators......Page 173
Block Size and Channels......Page 174
Essential Parameters for FTI calculation......Page 175
Computing FTI......Page 178
Automated Choice of Filter Period......Page 179
Trends Within Trends......Page 180
FTI LOWPASS BlockSize HalfLength Period......Page 181
FTI MINOR LOWPASS BlockSize HalfLength LowPeriod HighPeriod......Page 182
FTI LARGEST FTI BlockSize HalfLength LowPeriod HighPeriod......Page 183
FTI LARGEST PERIOD BlockSize HalfLength LowPeriod HighPeriod......Page 184
FTI MAJOR PERIOD BlockSize HalfLength LowPeriod HighPeriod......Page 185
FTI MINOR BEST CRAT BlockSize HalfLength LowPeriod HighPeriod......Page 186
FTI BOTH BEST CRAT BlockSize HalfLength LowPeriod HighPeriod......Page 187
NEXT DAY ATR RETURN Distance......Page 188
NEXT MONTH ATR RETURN Distance......Page 189
FUTURE SLOPE Ahead ATRdist......Page 190
RSQ FUTURE SLOPE Ahead ATRdist......Page 191
Screening Variables......Page 192
Chi-Square Tests......Page 194
Options for the Chi-Square Test......Page 195
Output of the Chi-Square Test......Page 196
Running Chi-Square Tests from the Menu......Page 197
Nonredundant Predictor Screening......Page 200
Options for Nonredundant Predictor Screening......Page 203
Running Nonredundant Predictor Screening from the Menu......Page 204
Examples of Nonredundant Predictor Screening......Page 207
Models 1: Fundamentals......Page 213
Overview and Basic Syntax......Page 216
The INPUT list......Page 218
Number of Inputs Chosen by Stepwise Selection......Page 219
The Criterion to be Optimized in Indicator Selection......Page 220
A Lower Limit on the Number or Fraction of Trades......Page 224
Summary of Mandatory Specifications for All Models......Page 225
Mitigating Outliers......Page 227
Testing Multiple Stepwise Indicator Sets......Page 228
Stepwise Indicator Selection With Cross Validation......Page 229
When the Target Does Not Measure Profit......Page 230
Multiple-Market Trades Based on Ranked Predictions......Page 231
Prescreening For Specialist Models......Page 232
Building a Committee with Exclusion Groups......Page 234
Building a Committee with Resampling and Subsampling......Page 236
Avoiding Overlap Bias......Page 237
A Popularity Contest for Indicators......Page 239
Bootstrap Statistical Significance Tests for Performance......Page 241
Monte-Carlo Permutation Tests......Page 243
An Example Using Most Model Specifications......Page 245
Sequential Prediction......Page 246
The MODEL CRITERION Specification for LINREG Models......Page 250
The Identity Model......Page 253
Quadratic Regression......Page 254
The General Regression Neural Network......Page 257
The Number of Neurons in the First Hidden Layer......Page 259
Functional Form of the Output Neuron......Page 260
A Basic MLFN Suitable for Most Applications......Page 261
A Complex-Domain MLFN......Page 262
The Basic Tree Model......Page 264
A Forest of Trees......Page 266
Boosted Trees......Page 268
Operation String Models......Page 271
Use of Constants in Operation Strings......Page 273
Split Linear Models for Regime Regression......Page 277
An Ordinary SPLIT LINEAR Model......Page 279
The NOISE Version of the SPLIT LINEAR Model......Page 280
Committees......Page 283
Model Specifications Used by Committees......Page 284
The AVERAGE Committee......Page 286
The LINREG (Linear Regression) Committee......Page 287
Constrained Linear Regression Committee......Page 288
Models as Committees......Page 289
Creating Component Models for Committees......Page 290
Exclusion Groups......Page 291
Using Different Selection Criteria......Page 292
Varying the Training Set by Subsampling......Page 293
Varying the Training Set by Resampling......Page 294
Oracles......Page 295
Model Specifications Used by Oracles......Page 297
Traditional Operation of the Oracle......Page 298
The PRESCREEN ONLY Option......Page 301
An Example of Prescreen Operation......Page 303
More Complex Oracles......Page 304
Testing Methods......Page 306
Performance for the Entire Dataset......Page 307
Walkforward Testing......Page 308
Cross Validation by Time Period......Page 310
Cross Validation using a Control Variable......Page 311
Cross Validation by Random Blocks......Page 313
Preserving Predictions for Trade Simulation......Page 314
Market States as Trade Triggers......Page 315
An Example of Simple Triggering......Page 316
Triggering Based on State Change......Page 319
Triggering Versus Prescreening......Page 321
Commands Common to All Four Examples......Page 322
Example 1: Model Specialization via PRESCREEN......Page 324
Example 2: Unguided Specialization......Page 329
Example 3: Triggering on High Volatility......Page 334
Example 4: Triggering on Low Volatility......Page 339
Permutation Training......Page 344
The Components of Performance......Page 348
Permutation Training and Selection Bias......Page 353
Multiple-Market Considerations......Page 358
Transforms......Page 360
Expression Transforms......Page 362
Quantities That May Be Referenced......Page 363
Vector Operations in Expression Transforms......Page 366
An Example with the @SIGN_AGE Function......Page 368
An Example with Logical Expressions......Page 371
A More Complex Example......Page 372
Principal Component Transforms......Page 374
Invoking the Principal Components Transform......Page 375
Tables Printed......Page 376
An Example......Page 378
Linear and Quadratic Regression Transforms......Page 381
A Regression Transform Example......Page 382
Inputs and the Target......Page 384
Gates......Page 385
Focusing on Extreme Targets......Page 386
Declaring the Transform and its Options......Page 387
A Nominal Mapping Example......Page 390
The ARMA Transform......Page 394
Defining the Purified and Purifier Series......Page 402
Specifying the Predictor Functions......Page 404
Miscellaneous Specifications......Page 406
Usage Considerations......Page 407
A Simple Example......Page 409
Complex Prediction Systems......Page 412
Stacking Models and Committees......Page 414
Graphics......Page 419
Series Plot......Page 421
Series + Market......Page 423
Histogram......Page 424
Thresholded Histogram......Page 426
Density Map......Page 429
Bivariate and Trivariate Plots......Page 438
Trivariate Plots......Page 445
Equity......Page 448
Prediction Map......Page 452
Indicator-Target Relationship......Page 455
Isolating Predictability of Direction Versus Magnitude......Page 459
Finding Independent Predictors......Page 462
A FIND GROUPS Demonstration......Page 465
Market Regression Classes......Page 471
The Hierarchical Method......Page 473
The Sequential Method......Page 476
The Leung Method......Page 480
Developing a Stand-Alone System......Page 485
Choosing the Target......Page 486
Quality Does Not Equal Quantity for Predictors......Page 489
Predictor and Target Selection for this Study......Page 491
Stationarity......Page 494
The Problem of Outliers......Page 495
Cross-Market Compatibility......Page 497
Data Snooping: Friend or Foe?......Page 499
Checking Stability with Subsampling......Page 501
How Long Does the Model Hold Up?......Page 505
Finding Models for a Committee......Page 507
The Trading System......Page 509
The Final Test......Page 512
Trade Simulation and Portfolios......Page 514
Writing Equity Curves......Page 517
Performance Measures......Page 521
Portfolios (A File-Based Example)......Page 523
A Portfolio Example......Page 527
Integrated Portfolios......Page 534
A FIXED Portfolio Example......Page 537
An OOS Portfolio Example......Page 538