The Evaluation and Optimization of Trading Strategies

Size: px

Start display at page:

Download "The Evaluation and Optimization of Trading Strategies"

Cori Hodge
6 years ago
Views:

2 The Evaluation and Optimization of Trading Strategies Second Edition ROBERT PARDO John Wiley & Sons, Inc.

4 Additional Praise for The Evaluation and Optimization of Trading Strategies The decisive step in system trading is the determination of the reliability and robustness of your system. This greatly expanded and very accessible new edition of Bob s classic presents a thorough yet easy to apply and timetested methodology to accurately make this determination. This makes it possible to form a realistic idea of how your system should perform in the future and increases the likelihood of lasting real-time trading profit. Murray Ruggerio, Vice President of Research and Development for TradersStudio; Contributing Editor Futures Magazine. Bob Pardo has re-written his own excellent book, and made it even better. It contains a basic step-by-step guide to building a trading system, along with an introduction to advanced system building concepts and tools. Simply avoiding the problems and errors Bob warns about can help lead to real time success. A must read for anyone interested in a systematic approach to trading. Michael Tepper, Atlas Capital Management, Inc. A significant contribution that will present, explain, clarify, and illustrate Algo Trading strategies and how to properly test/optimize trading models, thoroughly researched and perceptive. It will give the reader very practical and seasoned insight into the world of Algo Trading. Bob Pardo brings a very lucid approach to a very esoteric subject; a welcome departure from most texts. It is both accessible and rigorous, which is quite rare. Bruce J. Serra, Vice President, Institutional Sales, MF Global Inc.

6 The Evaluation and Optimization of Trading Strategies

7 Founded in 1807, John Wiley & Sons is the oldest independent publishing company in the United States. With offices in North America, Europe, Australia and Asia, Wiley is globally committed to developing and marketing print and electronic products and services for our customers professional and personal knowledge and understanding. The Wiley Trading series features books by traders who have survived the market s ever-changing temperament and have prospered some by reinventing systems, others by getting back to basics. Whether a novice trader, professional or somewhere in between, these books will provide the advice and strategies needed to prosper today and well into the future. For a list of available titles, please visit our Web site at www. WileyFinance.com.

8 The Evaluation and Optimization of Trading Strategies Second Edition ROBERT PARDO John Wiley & Sons, Inc.

9 Copyright C 2008 by Robert Pardo. All rights reserved. Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) , fax (978) , or on the Web at Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) , fax (201) , or online at Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) , outside the United States at (317) or fax (317) Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. For more information about Wiley products, visit our Web site at Library of Congress Cataloging-in-Publication Data: Pardo, Robert, 1951 The evaluation and optimization of trading strategies / Robert Pardo. 2nd ed. p. cm. (Wiley trading series) Rev. ed. of : Design, testing, and optimization of trading systems. c1992. Includes index. ISBN (cloth) 1. Investments Data processing. 2. Futures Data processing. 3. Options (Finance) Data processing. I. Pardo, Robert, 1951 Design, testing, and optimization of trading systems. II. Title. HG P dc Printed in the United States of America

10 Contents Foreword Preface Acknowledgments xv xvii xxix Introduction 1 CHAPTER 1 On Trading Strategies 5 Why This Book Was Written 7 Who Will Benefit from This Book? 8 The Goals of This Book 10 The Lay of the Land 10 CHAPTER 2 The Systematic Trading Edge 17 Discretionary Trading 18 Raising the Bar 19 Verification 22 Quantification 24 Risk and Reward 24 The Performance Profile 26 Objectivity 27 Consistency 29 Extensibility 32 The Benefits of the Historical Simulation 33 Positive Expectancy 34 The Likelihood of Future Profit 34 The Performance Profile 35 Proper Capitalization 35 vii

11 viii CONTENTS A Measure of Real-Time Trading Performance 36 The Benefits of Optimization 37 The Benefits of the Walk-Forward Analysis 38 The Advantages of a Thorough Understanding 39 Confidence 40 Strategy Refinement 41 CHAPTER 3 The Trading Strategy Development Process 43 Two Philosophical Approaches to Strategy Development 44 The Scientific Approach 44 The Path of Empirical Development 45 An Overview of the Trading Strategy Design Process 47 Step 1: Formulate the Trading Strategy 49 Step 2: Translate the Rules into a Definitive Form 49 Step 3: Preliminary Testing 50 Step 4: Optimize the Trading Strategy 51 Step 5: The Walk-Forward Analysis 52 Step 6: Trade the System 53 Step 7: Evaluate Real-Time Performance 54 Step 8: Improving the System 55 CHAPTER 4 The Strategy Development Platform 57 The Scripting Language 58 Diagnostics 59 Reporting 62 Optimization 68 The Objective Function 68 Speed 68 Automation 69 Walk-Forward Analysis 69 Portfolio Analysis 70 In Conclusion 71 CHAPTER 5 The Elements of Strategy Design 73 The Three Principal Components of a Strategy 74 Entry and Exit 74 Risk Management 74 Position Sizing 75

12 Contents ix An Overview of a Typical Trading Strategy 75 A Trade Equals an Entry and an Exit 76 Entry Filters 78 The Management of Risk 79 Trade Risk 80 Strategy Risk 82 Portfolio Risk 84 The Management of Profit 85 The Trailing Stop 85 The Impact of Overnight Change on the Trailing Stop 87 Profit Targets 87 The Impact of Overnight Change on Target Orders 88 Position Sizing 89 Advanced Strategies 91 Summary 92 CHAPTER 6 The Historical Simulation 93 The Essential Reports 94 The Performance Summary 94 The Trade List 94 The Equity Curve 107 Performance by Period 108 The Importance of Accuracy 109 Software Limitations 109 Rounding Issues 109 Phantom Trades 111 Price Orders 112 Realistic Assumptions 113 Price and Trade Slippage 113 Opening Gap Slippage 115 Opening and Closing Range Slippage 116 Slippage Due to Size 117 The Significance of Slippage 117 Limit Moves 118 Major Events and Dates 118 Historical Data 120 Stock Prices 120 Cash Markets 120 Futures Markets 121

13 x CONTENTS The Continuous Contract 124 The Perpetual Contract 124 Adjusted Continuous Contracts 125 The Size of the Test Window 126 Statistical Requirements 127 Sample Size and Statistical Error 127 How Many Trades? 129 Stability 129 Degrees of Freedom 130 Frequency of Trading 131 Types of Markets 132 The Bull Market 132 The Bear Market 134 The Cyclic Market 135 The Congested Market 137 Efficient Markets 137 The Life Cycle of a Trading Strategy 140 Window Size and Model Life 141 CHAPTER 7 Formulation and Specification 145 Formulate the Trading Strategy 145 Specification Translate the Idea into a Testable Strategy 147 Make a Vague Idea Precise 150 CHAPTER 8 Preliminary Testing 157 Verification of Calculations and Trades 158 Calculations 159 Trading Rules 159 In Summary 164 Theoretical Expectations 164 Preliminary Profitability 166 The Multimarket and Multiperiod Test 169 Selecting the Basket 169 Determining the Length of the Test Period 171 Segmenting the Data 171 The Test 172 The Results of the Test 177

14 Contents xi CHAPTER 9 Search and Judgment 179 Search Methods 180 The Grid Search 181 The Prioritized Step Search 185 Hill Climbing Search Algorithms 188 Multipoint Hill Climbing Search 189 Advanced Search Methods 191 Simulated Annealing 193 Genetic Algorithms 195 Particle Swarm Optimization 197 General Problems with Search Methods 198 The Objective Function 201 A Review of a Variety of Evaluation Methods 203 Multiple Evaluation Types 208 CHAPTER 10 Optimization 211 Optimization Contra Overfitting 213 A Simple Optimization 214 The Optimization Framework 216 The Parameters 216 The Scan Range 218 The Historical Sample 220 The Objective Function 222 The Optimization Evaluation 222 A Multimarket and Multiperiod Optimization 223 The Evaluation of the Optimization 225 The Robust Trading Strategy 225 The Robust Optimization 227 The Statistically Significant Optimization Profile 227 The Distribution of the Optimization Profile 229 The Shape of the Optimization Profile 231 How Does the Strategy Respond to Optimization? 235 Does the Strategy Deserve Further Development? 236 CHAPTER 11 Walk-Forward Analysis 237 Is the Trading Strategy Robust? 238 Robustness and Walk-Forward Efficiency 238 The Cure for Overfitting 239

15 xii CONTENTS A More Reliable Measure of Risk and Return 241 Assessing the Impact of Market Changes 241 The Best Parameter Set for Trading 242 The Theory of Relevant Data 243 Peak Performance 243 Statistical Rigor 244 Shifting Markets 245 The Varieties of Market Conditions 245 The Walk-Forward 247 The Role of the Walk-Forward 247 Setting up a Walk-Forward 248 An Example of a Walk-Forward Test 250 The Walk-Forward Analysis 251 The Purpose of the Walk-Forward Analysis 252 An Example of a Walk-Forward Analysis 255 Is the Strategy Robust? 256 What Rate of Profit Should We Expect? 260 What Is the Risk? 261 Walk-Forward Analysis and the Portfolio 261 CHAPTER 12 The Evaluation of Performance 263 The Trading Strategy as an Investment 263 The Dimension of Risk 264 Compare the Strategy to the Alternatives 264 Maximum Drawdown and Trading Risk 265 Maximum Drawdown in Context 267 Maximum Drawdown and the Trader 268 Maximum Run-up and the Trader 269 Trading Capital and Risk 270 Risk Adjusted Return 272 Reward to Risk Ratio 273 Model Efficiency 273 Consistency 276 Patterns of Profit and Loss 277 CHAPTER 13 The Many Faces of Overfitting 281 What Is Overfitting? 282 The Abuse of Hindsight 284

16 Contents xiii The Case of the Overfit Forecasting Model 286 The Case of the Overfit Trading Model 289 The Symptoms of an Overfit Trading Model 290 The Causes of Overfitting 291 Degrees of Freedom 292 Measuring Degrees of Freedom 292 Degrees of Freedom, Sample Size and Startup Overhead 293 Trade Sample Size 295 Optimization Error 1 Overparameterization 296 Optimization Error 2 Overscanning 297 The Big Fish in a Small Pond Syndrome 298 The Walk-Forward Test 298 CHAPTER 14 Trading the Strategy 301 The Mental Aspects of Trading 302 Return on Investment 303 Poor Strategy 303 Market Contraction 303 Unseen Market Conditions 304 Superior Alternatives 305 Maximum Risk 305 Real-time and Evaluation Performance 308 Comparing the Evaluation and Trade Profile 310 Understanding the Test Profile 311 Performance Quirks 313 The Windfall Profit 315 The Losing Run 316 Flat Production 316 In Conclusion 317 Notes 319 Index 323

18 Foreword My relationship with Bob Pardo goes back to 1996 when he approached my firm, DUNN Capital Management, in search of trading capital for his XT99 system. After some extensive system evaluations, we entered into an agreement to help research, develop, and trade Bob s XT99 for Bob, DUNN, and our clients. I am pleased to report that this arrangement has proved beneficial to all parties and that it is still going great guns. When Bob recently asked if I would write the foreword for this second edition I assured him that I would be more than delighted to do so. Because of my scientific background and training we have always viewed system design and development as a diligent application of statistical analysis of the performance of trading models and of their test results. Accordingly, we were very pleased to find that many of these features were used in developing the XT99 model platform and that it was so amenable to further testing and fine-tuning. When my colleagues and I had the opportunity to read the first edition of this work, we were particularly interested in Bob s presentation of the virtues and benefits of using the walk-forward method to guide system development. I am aware that many consider the first edition of this book to be a classic. Generally speaking, it is pretty difficult to improve upon a classic, but in this case it was necessary. As Bob outlines in his preface, to say that our world of computing, trading, and money management has changed since 1991 when the first edition of this book was published, would be a dramatic understatement. Given the vast changes that have occurred since the first edition, a new edition of Bob s book addressing these matters is entirely in line. The good news here is that not only did Bob update the original material; he also reorganized it, explained the material with even greater clarity and insight and added some new insights that he has learned in the intervening years. Did he improve on a classic? You ll have to be the judge of that yourself. xv

19 xvi FOREWORD I have always been impressed with Bob s technical toolbox and his innovative ideas. Bob s focus, dedication, and originality as a researcher and trader are very apparent in this second edition. I think that serious system developers will find this second edition a very interesting and profitable read. Enjoy it. William A. Dunn, PhD Chairman DUNN Capital Management Stuart, Florida May 2007

Preface THERE AND BACK AGAIN The first edition of Design, Testing, and Optimization of Trading Systems (DTOTS, as I always think of it) was published in 1991.

20 Preface THERE AND BACK AGAIN The first edition of Design, Testing, and Optimization of Trading Systems (DTOTS, as I always think of it) was published in It would be an understatement to say that the world has changed dramatically in the 17 years between the 1991 edition and this one. Some would say the markets have changed also. I disagree. The markets do what they always do: incorporate all of the changes in communication, technology, wealth, and trading styles into the instantaneous calculation of their fair value. I have always considered the defining characteristic of markets to be their ability to adapt and alter themselves accordingly to the changing style of market participants. In this introduction, we review the principal changes that have occurred during this time and their impact on the markets and trading. Many will seem obvious. Please bear with me in this walk down memory lane, however, for the sum total of these changes has altered the nature of trading and our industry in ways that directly reflect upon the current art of the design and evaluation of trading strategies. How? you might ask, and that would be a very good question. Let me start by offering my reflections on that subject because it is highly relevant to the topic at hand. The Trading System: From Rock Bottom to Rock Star The first, and perhaps foremost, difference was that in the early 1990s, the argument that raged was about whether trading systems actually worked. For those who are relatively new to the industry, this might come as a bit of a shock. There is now such a widespread, and somewhat unquestioned, belief in the many virtues of algorithmic trading (AT) that it almost borders on religious belief. xvii

A Guide to Creating a Successful Algorithmic Trading Strategy

A Guide to Creating a Successful Algorithmic Trading Strategy The Wiley Trading series features books by traders who have survived the market s ever changing temperament and have prospered some by reinventing