Developing. Neural Models. with Tradecision. A step-by-step tutorial for the development of your own neural network expert. Version 4.

Transcription

1 Developing Neural Models with Tradecision A step-by-step tutorial for the development of your own neural network expert Version 4.7

2 Alyuda Research. All rights reserved. Developing Neural Models with Tradecision No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means: electronic, magnetic tape, mechanical, photocopying, recording, or otherwise, without permission in writing from Alyuda Research Inc. This document is provided for informational purposes only, and the content of this document is subject to change without notice. Alyuda assumes no liability or responsibility for any errors or inaccuracies that may appear in this document. 2

3 Contents Contents... 3 Introduction... 5 About This Guide... 5 Notation Conventions... 6 Benefits of Creating Neural Strategies... 7 Chapter 1: Understanding Neural Networks... 9 Using a Neural Network How Neural Networks Work in Tradecision Selecting Target Selecting the Right Amount of Input Data Selecting Model Test Period (Walk-forward Test) Preventing Curve-Fitting Technical Analysis vs. Neural Nets Some General Aspects in Application of Neural Networks Introduction to Genetic Algorithms Chapter 2: Creating a New Model New Model s Creation New Model Wizard Step 1: Selecting the Model Target and Lookahead Step 2: Selecting the Training and Test Data Ranges Step 3: Selecting Model Inputs Using Templates Searching for the Best Inputs Step 4: Defining the Model-Based Strategy for the Performance Report Optimizing Model Input Parameters (Professional Editions) Optimization Variable Dialog Box Advanced Optimization Options Dialog Box Using the Advanced Optimization Criterion Selecting Additional Stopping Conditions for Genetic Algorithms Understanding the Feature Chapter 3: Model Management Training Model Analyzing Model Understanding Model-Based Strategies Analyzing Model Performance Report Actual vs. Forecasted Tab...59 Summary area Input Importance Tab Equity Tab Trades Tab Strategy Performance Tab Inserting Model Signals into a Chart Price Forecasting Saving a Model Output as.csv file Editing a Model Deleting a Model Import/Export Import/Export Importing a Neural Model Exporting a Neural Model

4 Improving Model Performance Chapter 4: Creating a Model-Based Strategy Chapter 5: Advanced Techniques Committees Using Multiple Models Using Fundamental Data, Market Indices and 3-rd Party Time-Series Chapter 6: Step-by-Step Guide Summary Getting Started Step 1: Selecting the Model Target and Lookahead Step 2: Selecting the Training and Test Data Ranges Step 3: Selecting Model Inputs Step 4: Selecting Strategy Rules for Performance Testing Analyzing Model Creating Model-Based Strategy

5 Introduction About This Guide In this tutorial, you will find the main procedures and techniques that you will need to be able to use and create profitable neural network models and strategies. You will become familiar with the main principles of the neural network theory and its practical application(s). Chapter 1 Understanding Neural Networks contains material that will convey a basic understanding of artificial neural networks and the theoretical fundamentals contributing to neuro-model development. In Chapter 2 Creating New Model you will receive detailed guidance on how to build a neural model and neuro-strategy by using built-in Tradecision Neural Model and Strategy Builders. Chapter 3 Model Management will lead you through the process of model reporting, analysis, editing, and acquaint you with other model-related operations. Chapter 4 Creating a Model-Based Strategy will provide instructions on how to implement neural models as part of a trading strategy. In Chapter 5: Step-by-step Guide you will learn how to develop and apply your first neural model using Tradecision s simple instructions. Note: Examples used in this guide are for informational purposes only. They are provided to enable the reader to get familiar with and understand the most important principles of financial neural network models. This tutorial will not only help you get acquainted with the features and functions embedded in the Tradecision neural tools, but it will also give you an opportunity to develop your own custom neural network experts. By using this newly gained knowledge, you will easily master the contents of this program guide. 5

6 Notation Conventions The following conventions are used in this guide: Element Meaning! Definition Denotes a key definition.! Important Denotes a topic containing important information.! Dialog Denotes a description of the Tradecision dialog boxes. Italic and Bold Indicates emphasis. Note: Refers to significant information inside a topic. 6

7 Benefits of Creating Neural Strategies A neural network trained for a specific security or trading instrument can be used to further develop a trading strategy based on such model forecasts. During recent decades, neural networks have become widely adopted and been used in the financial community to analyze financial markets. Neural networks possess an inherent ability to solve tasks that are otherwise difficult or almost impossible to solve using other, conventional methods. Neural networks are not only indispensable, but they are also most suitable for those tasks whereby it is difficult to describe the underlying problem in a clear logical format or using mathematical dependencies. With the help of artificial neural networks, it is possible to expose and detect most complex hidden patterns by analyzing the actual historical time series data. Neural networks have a number of properties that distinguish them from other mathematical and algorithmic methods. First of all, beyond the memorization of historical information, neural networks have an inherent ability to generalize, i.e. a well-trained network is capable of working successfully not only with the data it memorized, but also with information that was not used during the training or was garbled. Secondly, a network can train itself in upward fashion to improve its ability to forecast as new data arrives. Additionally, neural networks can accept answers from a previously trained network. They can use this information with rapidity and enhance the real-time trading and analysis environment for the end-user. 7

8 ! Important Good neural models (those that received proper amounts of historical data and relevant inputs, forecast with 60-80% accuracy. Out-of-sample accuracy and forecasting errors are reported to the end-user after he/she starts walk-forward testing of his/her model. All other commonly implemented statistical forecasting techniques usually perform more poorly when used for real world stock price forecasting. 8

9 Chapter 1: Understanding Neural Networks Tradecision provides state-of-the-art artificial intelligence technology that enables learning from prior historical data, thus helping the end-user forecast price movements more accurately, and, consequently, enhancing his or her opportunities for buying and selling securities. As a trader, you need no in-depth knowledge of neural network architecture, or training algorithms. Nor do you have to hold a degree in statistics. However, you will need to be able to understand how basic artificial neural networks work. The main principle of neural networks functioning you should understand and remember is the following: neural networks learn to associate values in the output column with the underlying patterns they detect within historical input data. A neural network is trained on past data to find patterns (inter-dependencies) between the input features and desired target. After an artificial neural network is trained, it uses historically detected patterns to make future forecasts. When you update the data in an already trained neural network or feed new data into it, the network starts looking for patterns that are similar to those it has already found to be associated with certain price movements. After that, it makes a forecast based on the existing similarities with the known price patterns. Keep this basic concept in mind while working with neural networks. You will be able to avoid many pitfalls. Neural networks are not a technical indicator, fundamental valuation, regression algorithm or magic super-technology. The main advantage of neural networks is their ability to find subtle non-linear interdependencies and detect patterns within these dependencies, which are otherwise almost impossible to detect visually or using traditional statistical approaches. Neural networks are truly one of the best methodologies ever invented for the prediction and forecasting of historical time series data. 9

10 To make neural networks work for you, you must prepare input data that contains variables that will allow detecting underlying patterns. The rest of the tasks involved in making an accurate prediction (i.e. preprocessing, architecture selection, intelligent control over network training, and so on) will be performed by the Tradecision software. For example, do not select only technical indicators to describe a symbol s price history when you know that your stock is driven primarily by fundamental valuation ratios or factors. Additionally, do not use neural networks with stocks that show changes in their behavior and dramatic changes in the related influencing factors. In general, a neural network consists of an input layer with input units, one (or several) hidden layers with hidden units, and an output layer with one (or several) output units. The underlying neural network architecture determines how the network units are connected and how information is processed in each unit. The training algorithm determines how the network weights (values assigned to each interconnection between the units) are calculated during the network training. Preprocessing and partitioning methods define how the data will be converted into a form suitable for neural networks. And, finally, generalization improvement methods will define how to improve the network s forecasting ability. We do not provide herein the formulas and other technical details of these methods. If you want to learn these details, you can start with ftp://ftp.sas.com/pub/neural/faq.html or send a request to the Tradecision Customer Support (support@tradecision.com). To learn the input data variables, the neural net finds patterns by tuning its weights (and architecture) during the training phase. To tune its weights and architecture, neural network compares its output with the correct value (the actual value from the dataset) during each iteration, and adjusts its structure in such a way that it rewards itself for becoming more precise in predicting the output. In this way, the network trains to the point when further adjustments stop producing lower error rates and the generalization ability can no longer be improved. If there is not enough data for finding the majority of the underlying patterns, or if the old patterns are now obsolete and they will not work with the new data due to a changing market environment, the network will not be able 10

11 to produce good forecasts. However, the following should be mentioned: neural networks excel in their robustness, pattern recognition abilities and capacity to produce good forecasts for new data that has never been analyzed before. 11

12 Using a Neural Network With Tradecision s embedded and fully-automated neural networks, you can create profitable trading strategies, as is being done by a great many other traders. We hope that you will be able to perform this task and beat your competitors who don t have the benefit of using the state-of-the-art Tradecision software. To be successful, you ONLY need to choose the correct inputs, target and look ahead period. And let Tradecision do the rest. The process of using a neural network for forecasting can be divided into two stages: 1. Data preparation and network training using historical data; 2. Forecasting with a trained network using new data (i.e. Real-time stream or EOD update). The first stage can be divided into several tasks: - Input dataset analysis: data type definition (i.e. numerical, categorical, text, data/time, and so on), adequate handling of missing values and outliers, feature input(s) selection and noise reduction. - Data preprocessing: transferring source data into a form suitable for neural network management and utilization. - Network design: selection of network architecture, the training method and training parameters. - Network training. - Performance testing. - Network improvement(s) based on actual test results: the network can be re-trained with other inputs, architecture, training method(s) or parameters. Using a ready network is a less difficult and time-consuming process than its preparation. A data set that has been preprocessed according to the same principles and methods as during the first stage is fed into the network. After 12

13 getting an input signal, the network generates an output signal, being the forecasted value after post-processing. It needs to be mentioned that network preparation and training, generally, require a significant amount of computer/cpu cycle time. The quality of the resulting forecast depends strongly on the training data selected, representation of input features, preprocessing, as well as on the architecture and the training algorithm s fine-tuning. In any case, the amount of the training data should be sufficient to allow solving a problem of higher complexity. 13

14 How Neural Networks Work in Tradecision Tradecision exploits highly adaptable constructive neural networks to create neural models. Constructive nets grow and train themselves during iterative price movement(s) analysis. You don t need profound knowledge of neural networks to successfully create and train a neural network. Technological particulars, such as the neural network architecture and training algorithms, data preprocessing, and so on, are completely automated in Tradecision. You will, however, have to select the necessary data, verify your ideas/concepts, adapt models to the changing market situations, and use your own judgment when assessing overall model or trading strategy performance. You will be required to select the best neural model based on your trading style and risk preferences. Neural networks base their predictions on price data patterns they discover in historical time series data during their training phase. With Tradecision, you can model security prices using neural networks. A neural network trained for a certain security can be used to generate a trading strategy based on model forecasts. This neuro-strategy can be tested, optimized and used to generate buy/sell signals. After you achieve a model you are satisfied with in the Tradecision Model Builder, you need to implement it in Strategy Builder. Here you will use the model output as one of the neuro-strategy rules. You can combine the rule with the indicator-based entry/exit and money management rules. 14

15 Selecting Target! Definition When you create a model, you need to select a market parameter (data column) that should be predicted by the model. This parameter is called target. You can select options for forecasting the Percent change, Close, Open or any of the available technical indicators. By default, Tradecision creates models that forecast the change or change expressed as percentage in the output in order to normalize similar price patterns at different price levels. This simple approach can significantly increase the model performance. For some indicators, normalization is not necessary, but even in that case, Tradecision s current approach can slightly improve the performance. For, example, the fact that a 10% model error in the change value if converted into an absolute value is less significant than the direct 10% error in the absolute values. Tradecision enables forecasting the change/% change in Open/Close, as well as any technical indicator, such as EMA or RSI. By default, Tradecision does not offer the options to forecast the High or Low values, since this data contains a lot of noisy movements and outliers. Therefore, a model targeted toward forecasting High or Low would be of rather poor quality. Selecting a technical indicator as the target is also helpful, especially if you intend to forecast the market volatility, typical price, moving averages, price momentum or market strength. You can also create a composite indicator to use smoothed or de-trended representation of the market. Unfortunately, price data and most indicators contain noise in their values. This is one of the reasons that contribute to the poor model performance. To reduce the noise effect, the end-user can employ a moving average of the Open, Close or indicator as the model target. Note: When you chose the target of a moving average of price or a technical indicator, you should keep the Period parameter of the moving average lower than the model Lookahead parameter. 15

16 Selecting the Right Amount of Input Data The key to success in neural network prediction is choosing the correct combination of predictive inputs. Selecting those variables that are to be included in your neural network is a crucial task which determines whether or not your neural network will work properly. Note: too many inputs will cause the net to over-train. The net will learn the training data very well, but it will perform poorly on out- of- sample examples (your walk-forward data set). From our experience, for the majority of cases the number of inputs that should be used is between 3 and 10. The correct input feature set depends on the trading instrument (i.e. Stock). In the majority of cases, you should use the moving averages or Change/%Change and not Open, High, Low, Close or Volume as inputs. Try to find inputs that characterize the stock from different points of view so that the neural net will have less difficulty in modeling. For example, chose technical indicators from different categories (i.e. volatility indicators, volume indicators, verticality indicators, velocity indicators, cyclical indicators) and hybrids thereof. In many cases, Momentum indicators, such as Stochastics, PercentR, Williams Accumulation-Distribution, CCI and RSI are good choices. Other adequate choices include Volume indicators, such as Accumulation-Distribution, MFI, and OBV. Tradecision requires a set of input features in the form of columns, to be used as training material for the neural network. Selecting appropriate inputs is the most important task the user is faced with. Inputs are the only kind of knowledge about the market that will be provided and made available to the neural network to enable it to determine the best model on which the future forecasts will be based. All the time series data which as you believe significantly affects the value of the target column or represents the current market situation should be used as input data. 16

17 For example, to forecast what the closing price will be three days from now, you could train the model using the price data, lags and other indicators as the inputs and closing prices as the target. This includes the price data (open, high, low, close and volume), technical indicators (moving averages, lags, MACD, RSI, PercentR, and so on), market indices (S&P500, Dow Jones Industrial, NASDAQ, and others), fundamental data (Interest Rates, Unemployment Rates, Consumer Price Indexes, and so on), as well as the relevant correlated stocks and other information, representing the market conditions.! Important It is worth a mention, that irrelevant or insignificant inputs selection may deteriorate the model s performance. Additionally, the more inputs you have, the more historical data you need to provide to train your model. With more historical data, you will need more time to train your model, running a greater risk of curve-fitting or over-optimization. Curve-fitting occurs when a model simply memorizes the training data, which results in the model s lower generalization ability and a larger forecasting error. For example, you should not add Momentum if you are forecasting in a market exhibiting a sideways pattern. In most cases, this will have a minute impact on your model or may even worsen it. However, do not forget to include oscillators (such as Stochastic or RSI), as they perform well on sideway markets. If the number of price patterns in your historical data is too small, your neural network will not have enough information about the market to train itself correctly. For daily bars, it is recommended that one have at least four (better 6+) years of price data ( daily bars) for adequate network training, and at least six (better 12+) months for walk-forward testing. Providing too much data can increase a neural model s training period but it will not improve the quality of the forecasting. Additionally, the old detected price patterns will not simply be valid for the current market situation. Therefore, to reduce the input dataset you need to remove the oldest data. 17

18 Selecting Model Test Period (Walk-forward Test) The current model coupled with your model-based strategy should be tested on out-of-sample data (walk-forward period). Out-of-sample walk-forward testing will achieve the results that simulate real-world trading using this model. Tradecision automatically tests the current model after each training session or after adjustments are made to the model properties settings. The user can select how much data should be used for automatic walk-forward testing. The more data you select for the testing, the more optimal is the number of the patterns that are made available for testing the model, and, therefore, the more confident you can feel about your newly derived test results. At the same time, the more data you select for the testing, the fewer price patterns are available for the model s training. With less data, the model training will be less efficient. It is recommended that one have at least 100 bars of data for testing. 18

19 Preventing Curve-Fitting The biggest mistake made by users of artificial neural networks is that they tend to over-train their networks.! Definition Curve-fitting is a dangerous shortcoming of neural networks, resulting from over-training. When a network overtrains, it does not learn price patterns or develop the ability to generalize, but simply memorizes historical price data without learning any internal dependencies. An over-trained network can produce good results with a training set, but performs unexpectedly or poorly when used with new data. Tradecision uses a special embedded algorithm to clearly identify the trend of over-training. It monitors validation error dynamics and training progress from different points of view. When over-training is detected, Tradecision stops the training process and restores the network with the best-known generalization ability. 19

20 Technical Analysis vs. Neural Nets Neural network approaches are quite different from the classical technical analysis techniques and, therefore, require time and effort on the part of the trader who wants to master them. Neural networks can process huge amounts of data and detect trends and underlying price patterns that cannot be easily seen by humans while observing charts or discovered using traditional technical analysis methods. Neural networks learn from price data and discover complex non-linear dynamic patterns in price movements. Unlike the traditional indicators, neural networks have the ability to learn from the data itself. Neural networks learn the patterns of price data. Routine standard analysis techniques usually assign a model form to data and then test it to see if the data fits the assigned structure. A typical trading system often works for a short while until the market situation changes and/or the system performance deteriorates. A neural network-based trading system can be retrained every time you integrate or update your data. This allows the system to adapt to new market conditions. 20

21 Some General Aspects in Application of Neural Networks 1 Your main task is to select the correct or proper inputs. The rest of the tasks, such as preprocessing, architecture selection, GA optimization, training and others will all be performed by Tradecision automatically. 2 Don't rely on a single model only. Oftentimes, this approach does not due to the market s complexity. Build several models based on different ideas, i.e. whether the trend will continue after a certain condition is met, using different types of market phases, and so on. Combine these models in one neuro-strategy using Strategy Builder. Direct price forecasting works mainly for sideway markets, and often with momentum indicators only. 3 Do not select such inputs for your neural model that contain the same basic underlying information. For example, we do not recommend using the Close and the exponential moving average of the close value EMA(Close, 34) as the neural net regards them as the same type of information. It is much better to expose a neural network to some different aspects of the market. 4 Remember that a neural network analyzes only one row at a time, so if you want the network to consider yesterday s Close value, add Lag(Close, 1) as one more input. Do not assume that the neural network sees the previous data rows. A neural network uses one row at a time to adjust its weights. After the adjustment is made, the network forgets about this row and proceeds to the next one. 5 It s often beneficial to add market and industry indices. At least, add a moving average for S&P500, QQQ or DJI (depending on the kind of the stock you have). 21

22 6 Use only walk-forward test figures to analyze your model. Do not use the model in trading until your strategy performance report shows good figures for the walk-forward data range. 7 Add indicator-based filters to your neural strategy in Strategy Builder. Sometimes, simple techniques can significantly improve advanced mathematical modeling. 8 Prepare to spend significant time on building your neural model. 9 Do not stick to just one favorite stock. It is easy to build a good model for some trading instruments (i.e. Stocks), and, oftentimes, it is almost impossible to model other symbols. In this case, you have not simply taken into account those influential factors that are necessary to build adequate models for these relatively "unpredictable" stocks. 22

23 Introduction to Genetic Algorithms Another AI (Artificial Intelligence) technique that has proven performance gains and excels in optimization tasks is Genetic Algorithms.! Definition Genetic Algorithms are search algorithms based on the mechanics of natural selection and natural genetics. They combine the survival of the fittest rule with a structured yet randomized information exchange. The method uses terms accepted in genetics, such as fitness, population, generation, mutation, gene, and so on. In contrast to random search methods (such as, for example, x Carlo) genetic algorithms are not a simple random walk. These algorithms efficiently use historical information to speculate on new search points with expected improved performance. Their goal is forming or finding a population of trading strategies that will have the best fitness level, or, in other words, the best optimization criteria values. Genetic algorithms possess the best characteristics of the other optimization methods, such as robustness and fast convergence, which does not depend on any of the optimization criteria (for instance, on smoothness). Although genetic algorithms are much faster when a large number of possible options are searched, they are slower than exhaustive search as far as a small search space is concerned. Genetic algorithms are useful in maximizing or minimizing an objective function within a set of constraints. They are especially efficient when the relations are non-linear and/or discontinuous. Tradecision uses Genetic Algorithms to give professional traders greater optimization power for optimizing trading system rules. This approach is also used to optimize neural model inputs and their parameters. 23

24 Chapter 2: Creating a New Model The Tradecision Model Builder assists the user in creating successful neural models. The trader can incorporate them into strategies and thus make better trading systems compared to those built by others. Important features of the Model Builder: Model Builder enables analyzing over 100 model performance figures derived by using a model-based strategy; You can verify the statistical measures of the quality of your model forecasting with the help of the Model Performance Report; Using the Input Importance Chart, you can determine the relative importance of each model input; Actual vs. Forecasted Graph displays a line graph of the actual and forecasted target values for all bars from the training or test periods; Visual analysis of model-based signals can be enabled with just one click by inserting them into the main chart; All models are automatically saved into the internal database. You can edit, re-train or delete a model stored in the database at any time. 24

25 New Model s Creation A new neural model can be easily created using the New Model Wizard. The New Model Wizard will analyze and forecast forthcoming changes in the symbol data within the current chart. To start the New Model Wizard: 1. Open a chart with a symbol you want to predict. 2. From the Tools menu, select Model Builder. Tools menu The Model Builder dialog box will be displayed in a separate window. Model Builder 25

26 3. On the Model Builder toolbar, click New. Model Builder toolbar This command activates the New Model Wizard. 26

27 New Model Wizard The New Model Wizard guides you through the process of creating a neural network model which can then be used to forecast future price or indicator values. It also creates a simple model-based trading strategy to test the model with simple buy/sell rules. Model-based strategies and the buy/sell signals they generate are based solely on model forecasts and, therefore, allow testing model performance only, and not its money management or indicator-based rules. The wizard has four steps: Step 1: Selecting the target and lookahead During this step, you need to define the model target and specify how far into the future you want the model to predict. Step 2: Selecting the training and test data ranges During this step, you need to define how much data will be used for model preparation. You can also manually allocate a portion of this data to be used for walk-forward testing. Step 3: Selecting the model inputs During this step, you need to define model inputs. You need to select those indicators which, as you believe, contain important data that is vital for predicting the model target. New Model Wizard allows finding the optimal parameters for the indicators that will be used by you as inputs. Step 4: Selecting strategy rules for performance testing To test a neural model, you need to define a strategy based on the model forecasts (model-based strategy). Trading strategies produced by the New Model wizard are based solely on model forecasts. These strategies produce buy/sell signals that depend only on model forecasts, and, therefore, allow testing model performance only, and not the money management or indicator-based rules. 27

28 Step 1: Selecting the Model Target and Lookahead During this step, you need to define the model target and specify how far into the future you want the model to predict. Step 1 of the New Model Wizard In the Model name box, you can change the automatically generated model name to a more descriptive one. In the Description box, you can enter the model description that can help you distinguish this model from similar ones created for the same stock. The Target list can be used to define which indicator should be considered as the target one. In other words, you need to define the data that must be predicted by the model. By default, "% change in Close" is selected as the model target. However, you can also select change in Close, % change in Open or change in Open (you may use current built-in or custom indicators). For more information see Selecting Target in Chapter 1. 28

29 Selecting Target In the Lookahead box, you can enter the number of bars for which you want the model to forecast.! Definition The Lookahead parameter specifies how far into the future you would like to forecast. For example: if you are trying to predict a value two trading days in advance using daily price data, this value should be set to 2. The farther into future you want to forecast, the larger the forecasting error you should expect. We recommend forecasting 1-10 bars ahead. However, in many cases, forecasting 4+ bars ahead could produce a lower error than that done for 1-3 bars ahead because of the natural noise smoothing over time. Selecting lookahead 29

30 In the Model retraining area you can define whether to retrain the model with different or the same weights by selecting the corresponding options. These settings define the initial values of the neural network with which the training starts. If the initial weights are the same for both networks, the training results will be the same too. Normally, a neural network trains itself several times using random initial weights to achieve the optimal result. If the Retrain model with different weights option is selected, each training session of the model will start from a random position. If the Retrain model with the same weights option is selected, the weight values will be determined by the Random Seed value. This value is used for initializing pseudo random number generator when the initial network weights are set. As a result, if the Random Seed values are the same for both the models, their initial weights will be the same too. You can select the Define data ranges automatically to skip Step 2. In this case data ranges used for model training and testing will be selected using the default settings. By default, 3000 bars will be used for model preparation, the last 20% being allocated for walk-forward testing. 30

31 Step 2: Selecting the Training and Test Data Ranges During this step you need to define how much data will be used for the model preparation. Step 2 of the New Model Wizard 31

32 Defining the Price Data Range To select the amount and position of the price data: 1. To define the duration of the test period, in the Price data range area, select the Use last option and enter the amount of data that you want to use. Note: The system supports defining the duration of the testing period in bars, years, months or weeks. 2. To precisely specify the start and the end date for the data used for the model preparation, select the Custom date range option. Price data range area Note: This option is especially useful if you want to train or test the model on certain price patterns. 3. Select the Use all available data to use all the data available for this security. 32

33 Defining Walk-Forward Test Data The data that will be used for walk-forward testing will be allocated automatically. The allocated range is displayed in the Walk-forward test area. (See selecting model test period in Chapter 1 for more information.) To change the amount and position of the price data used for model walk-forward testing: 1. In the Walk-forward test area, click Change. Walk-forward test group box 2. In the Test Range dialog box, select the Use % of data from option to specify the test period as a percentage of the data available for the model preparation. The specified amount can be allocated in the beginning or end of the available price data, depending on which is selected by you in the corresponding list. Test Range dialog box 33

34 3. To define the duration of the test period, in the Price data range area, select the Use last option and enter the amount of data that you want to use. Note: The system supports defining the duration of the testing period in bars, years, months or weeks. 4. To precisely specify the start and the end date for the data used for the model preparation, select the Custom date range option. 5. Click OK. Note: This option is especially useful if you want to train or test the model on certain price patterns. The walk-forward data range will be re-allocated. 34

35 Step 3: Selecting Model Inputs During this step, you need to define the model inputs. You need to select the indicators that as you believe contain valuable data to predict the model target. (See Selecting the right amount of input data in Chapter 1 for more information). Step 3 of the New Model Wizard 35

36 To add a model input: 1. During step 3 of the New Model wizard, click Add. The Improvian Editor dialog box will be displayed. 2. In the Categories group, select the function you need or just write it manually in the Expression box, for example, SMA(Close, 9). 3. According to the Improvian syntax, add "return" before the SMA and ";" at the end of the expression. For example: return SMA(Close, 9) ; 4. In the Improvian Editor, click OK. Improvian Editor 36

37 5. The input will be added to the inputs list with as SMA(C, 9). Note: If you use several lines in your code, the name of such an input will be <Complex Input #1>. If you move your pointer to it, a message will be displayed, providing information on the input expression. Note: You can edit or delete the selected input using Edit and Remove buttons. Note: You can easily rename an input using the Rename input button. 37

38 Using Templates You can save your favorite combination of inputs as an input template. Saved templates can be used for new models. Click Save Template to save the input combination and add it to the current list of model inputs as a template. Click Load Template to load a previously saved or built-in template. Loading Model Input templates Note: The built-in templates are simply examples of possible input combinations. 38

39 Searching for the Best Inputs The New Model wizard can find the optimal parameters for the indicators used by you as inputs. To optimize the inputs or, in other words, to run an optimization algorithm that will exclude all insignificant inputs from the model, select the Search for best inputs check box. Then click Advanced. Clear this check box to use all the inputs and optimize only the parameters. The Advanced Optimization Option window will be displayed. You can use one of the two optimization algorithms: Exhaustive Search or Genetic Algorithms. Click OK. Click Next. 39

40 Step 4: Defining the Model-Based Strategy for the Performance Report During this step you can select strategy rules for performance testing. To test a neural model, you need to define a strategy based on your model s forecasts (model-based strategy). Trading strategies produced by the New Model wizard are based solely on model forecasts. These strategies produce buy/sell signals that depend only on model forecasts, and, therefore, allow testing model performance only, and not the money management or indicator-based rules. To define a model-based strategy, you need to specify the entry and exit rules for long and short positions in the Model-based strategy area. To define these rules, you need to select the thresholds for triggering buy/sell signals. The thresholds depend on the volatility of your stock, the model s goal, the minimum profit you are interested in and your risk preferences. The thresholds are specified as percentage or a dollar amount. 40

41 Money Management area In the Initial Account box, you can enter the amount of the starting capital selected to test the current system. The Position size box is used to indicate what amount should be used to calculate the profit/loss of each position generated by the model strategy. The box displays the dollar amount traded during each transaction. By default, the value of the box is set to From the Trade Execution list, select the order execution price for realistic trading simulation. You can specify that orders be executed at the Close of the current bar (Current Bar Close), or at the Open of the next bar (Next Bar Open). 41

42 Model-Based Strategy area To achieve better results, the thresholds should be adopted to your symbol characteristics and current market conditions. You can calculate the standard deviation, average true range, volatility, and other statistical indicators for your symbol and base your thresholds on these calculations. Alternatively, you can simply define your minimum profit target. A strategy enters a long position if the forecasted change in the price exceeds the threshold specified in the Long Entry box. A strategy exits a long position if the forecasted change in the price is less than the threshold specified in the Long Exit box. A strategy enters a short position if the forecasted change in the price is less than the threshold specified in the Short Entry box. A strategy exits a short position if the forecasted change in the price exceeds the threshold specified in the Short Exit box. For example, if the goal of your model is to forecast Open price 2 days ahead, you can trigger the buy signal only if the following conditions are met: - the change in the price is bigger than the average daily price change; - or twice bigger than the average true range; - or the change in the price is simply 30 points bigger (your minimum profit target) Please note, that correctly selected strategy thresholds can significantly improve the performance of a model-based strategy. That is why the Tradecision functionality includes strategy thresholds optimization. Click the Adjust thresholds after training button for the thresholds to be adjusted automatically using the symbol data volatility and selected target. 42

43 To include thresholds optimization in the model building process: 1. Select the Optimize thresholds check box. 2. From the Optimization goal, select the objective for the optimization. You can select from multiple options. 3. In the Optimize thresholds area, click Optimization Range. 4. In the Optimization Variables dialog box, select the limits and step for the threshold optimization and click OK. 43

44 Optimization Variables dialog After the model is selected and trained, its thresholds are optimized to ensure the best performance the model is capable of demonstrating. After you click Finish, a new model will be created, trained and added to the Model Builder database. You will be able to analyze the model and insert its signals into the chart in Model Builder. 44

45 Optimizing Model Input Parameters (Professional Editions) You can use one of the two optimization algorithms: Exhaustive Search or Genetic Algorithms.! Definition Exhaustive search verifies all the possible combinations of the optimized parameters and, therefore, can definitely find the best possible solution. However, the time required for exhaustive search increases rapidly as the number of the parameters is increased. Exhaustive search should be used if you have only several parameters and small limits for them.! Definition Genetic algorithms are search algorithms based on the mechanics of natural selection and natural genetics. They combine survival of the fittest rule with structured yet randomized information exchange. (See also Introduction to Genetic Algorithms in Chapter 1). The New Model wizard can find the optimal parameter value for the indicators that used by you as inputs. Understanding Syntax Optimization of input parameters can be performed using variables. To perform this kind of optimization, you need to declare that you want to use optimization for this input at the beginning of the expression. For example, you want to use RSI and EMA in one input and optimize their parameters. Your expression should be written as follows: var opt1:=#; opt2:=#; end_var return RSI(Close,opt1) + EMA(Open,opt2); Where 'opt1' and 'opt2' are the parameters to be optimized. 45

46 46

47 To add an input with optimized parameters: 1. During step 3 of the New Model dialog box, click Add. The Improvian Editor dialog box, will be displayed. 2. In the Categories group, select a function (or functions) that you need or simply write it manually in the Expression box. 3. With the help of "VAR" declare the optimized parameters. 4. In the Improvian Editor, click OK. The input will be added in the inputs list with a name composed to according to the name mask <Complex Input #1>. If you move your pointer to it, a message will be displayed, providing information on the input expression. 47

48 Optimization Variable Dialog Box Specifying the Range of the Selected Optimization Variable The Optimization Variable dialog is used to specify the range of the selected optimization variable. To specify the range of the selected optimization variable: 1. During step 3 of the New Model dialog box, select an input that contains the optimized parameters. 2. Click Optimization. The Optimization Variable dialog box will be displayed. The Variable Name column of the dialog box shows the variable that you are editing. 48

49 3. In the From column, enter the minimum value for the optimization variable. 4. In the To column, enter the maximum value for the optimization variable. 5. In the Step column, enter the step size by which the optimization variable should be incremented. For example, if the minimum/maximum range is 4 to 10 and the step size is 2, the optimization variable will be replaced with the values 4, 6, 8, and Click OK. 7. Click Advanced. The Advanced Optimization Options dialog box will be displayed. 49

50 50

51 Advanced Optimization Options Dialog Box The Optimization Options dialog box is used to specify the optimization algorithm and its parameters. In the Optimization Method area, select the optimization algorithm. The following options are available: Exhaustive search. This method tests all possible combinations of inputs and selects the best combination. Note: The method may be too time-consuming if you have a large amount of inputs and a big dataset. Genetic algorithms. The method starts with a random population of input configurations. Input configuration determines which inputs are to be ignored during the performance test. During each following step (generation) input configuration uses a process similar to natural selection to select superior configurations and generate a new population. Each step successively produces a better input configuration. During the last step, the best configuration is selected. Note: The Genetic algorithms method is very time-consuming but efficient for determining mutually-required inputs and detecting interdependencies For details, refer to Introduction to Genetic Algorithms. 51

52 To improve the algorithm s efficiency, in the Genetic Algorithm Properties area of the Advanced Optimization Options dialog box, enter the following properties: Population Size. The number of potential configurations used during each step of the algorithm. The larger the population, the bigger is the probability of finding a good configuration and the more time is required to find this solution. Number of Generations. The number of steps of the genetic algorithm. Each generation is equal to the interval during which a new population is created, by a combination of crossover and mutation. Each new generation is produced by modifying the previous generation with a view to selecting those inputs that produced the best results. The larger the Number of Generations, the bigger is the probability of finding a good solution and the more time is required to find this solution. Crossover Probability. This is the probability of crossover for each configuration during any given generation. The higher is the crossover rate, the stronger is the probability of convergence with a similar set of children (configurations). A lower crossover rate requires a bigger number of generations to explore the search space better. Note: Crossover is the process of creating a new configuration using parts of its parent configurations. Mutation Probability. Specifies the probability of mutation for each configuration in each new generation. The higher the mutation rate, the better the search possibilities of the algorithm are and but the bigger the destruction possibility for good configurations. Note: Mutation is the process of altering one or more bits in the inputs configuration (mask). 52

53 Using the Advanced Optimization Criterion To use the optimization criterion: 1. During Step 3 of the New Model wizard, click Advanced. The Advanced Optimization Options dialog box is displayed. 2. In the Advanced Optimization Options dialog box, select the Exhaustive search or Genetic algorithms option. 3. From the Optimization criterion list in the Criterion area, select the criteria for your neural model s optimization. The following options are available: Absolute Error on Test Range; MSE on Test Range; Directional Accuracy on Test Range; Tradability on Test Range; Complex Absolute Error (30% on Training Range and 70% on Test Range); Complex Directional Accuracy (30% on Training Range and 70% on Test Range); Complex MSE; Complex Tradability. 4. Click OK. 53

54 Selecting Additional Stopping Conditions for Genetic Algorithms Understanding the Feature The option can be very helpful when you use Genetic Algorithms as your optimization method. You can use it to prevent excessive use of valuable computer resources. It can be configured in the Genetic Algorithms settings (in the Advanced Optimization Options dialog box) when you are creating a neural model or strategy. This feature has two options: Maximum Time Minutes. The algorithm will be automatically stopped after an indicated time. No improvement for Iterations. The optimization process can be stopped if the fitness of the best model/strategy has not been improved after an indicated number of iterations. 54

55 Chapter 3: Model Management With Model Builder, you can analyze, re-train, edit and delete a model. A model can be inserted into a chart and saved as a text file. In addition, you can import and export models to share them with other Tradecision users. Training Model To train or re-train a model, go to the Model Builder, select your model from the Model List and click Train on the Model Builder toolbar. The training process will be started and the Model training progress bar will be displayed until the training is complete. Model training progress After the training is complete, the model will be automatically tested using the walk-forward test period and you will be able to analyze the model performance and insert its signals into the chart. toolbar Model Builder main 55

56 Analyzing Model To analyze model: 1. From the Model List, select the model that you want to analyze. 2. Click Analyze and select the range you want to analyze from the list. The following options are available: Test range. Select to analyze the model performance during an outof-sample period. Training range. Select to analyze the model performance during the period that was used to train the model. The Model Report dialog box will be displayed. Model Builder dialog Model Report 3. In the Model Report, select the appropriate tab to analyze the modelbased strategy performance. 56

57 Understanding Model-Based Strategies A system must be profitable with both the in-sample and the outof-sample data runs to be used in real-time trading. To test a model, Tradecision applies a simple model-based strategy to historical data and determines whether you would have made a profit if you had followed this strategy, or not. If the strategy turns out to have been profitable, and if it fits your trading personality, you can be confident in using the model to make future trading decisions. A major advantage of model testing using a model-based trading strategy is the possibility to rigorously test the model over different periods of time and with different parameters. To help you thoroughly analyze a model-based strategy, Tradecision creates the Strategy Performance Report. The report provides detailed information about your strategy, including the return on account, drawdowns, risk ratios, number of profitable/losing/outlier trades, as well as timing analysis. To ensure that the figures of the Strategy Performance Report are as close to reality as possible, you can factor in broker's commissions and slippage. In addition to the Strategy Performance Report, you can analyze equity and drawdown curves for your account. For details, refer to Equity and Drawdown curves. Testing against data that was used to train and optimize neural model (or develop the trading system) is referred to as in-sample testing. A trading system SHOULD NOT be tested only against in-sample data. Testing using data that the system was not trained on or optimized against is referred to as out-of-sample or walk-forward testing. The neural network and other system parameters are frozen, and the system is then given data it has not seen before. A system SHOULD be tested against out-of-sample data. When walk-forward testing is done, a portion of the available price data is not used. This portion is used for testing after the model training and optimization are complete. Walk-forward testing represents a truer measure of system performance in real-time trading. In-sample testing results can only outline that the trading system has the potential to yield a profit in walk-forward testing and, consequently, yield a profit in real-time trading. A system must be profitable with both in-sample and out-of-sample data to be used in real-time trading. For trustworthy testing results, you should make sure your test (in-sample or out-of-sample) results in at least 30 trades per signal. Another good approach is to test your neural model and strategy on different periods of data. 57

58 Inspect the stock chart visually to make sure that market behavior during the training period is not dramatically different from that during the test period. For example, if your training period has only small sideway moves but the test one represents a strong bull market, your model will not be able to provide good signals during the walk-forward test period as it was not trained to respond correctly to such market situations. To increase confidence in the system s profitability, you should also test it against other stocks of the same market type. Most likely, the system will not perform as good as with the stock for which the neural model was designed since neural network tends to learn patterns specific for the stock it was trained against. However, in any case you should achieve tolerable results with other similar symbols. 58

59 Analyzing Model Performance Report Important detailed information on model performance can be viewed using the Model Performance Report. The Model Performance Report assists you in analyzing neural model forecasting quality using statistical ratios and graphs. The Report allows you to identify the reason for your trading system s poor performance. If the report ratios are good, you should change your strategy settings. If the report ratios are poor, you should improve your neural model. The Report provides separate sets of figures for the training and walk-forward testing periods. The figures for walk-forward testing are much more important since they represent the model quality you will have when you use the model for new data. Usually, model performance during the training period will be better than during the testing one. A good model should have good performance during both periods. You should remember that the model target is change or % change in price. Therefore, model errors (absolute, relative and MSE) are displayed for changes and not for absolute values. If you are comparing two models for different stocks, you should take into consideration the different price levels of the two stocks.! Dialog Actual vs. Forecasted Tab Actual vs. Forecasted Graph displays a line graph of the actual and forecasted target values for all the bars from the training or test period. Actual vs. Forecasted Graph 59

60 ! Dialog Summary area It describes a general model performance. Summary area Number of bars. The number of bars in the corresponding period. A small number of bars can be the reason for poor model performance. For details, refer to Selecting the Right Amount of Input Data. Tradability ratio. This kind of ratio compares the model forecasting error with a stock s volatility. If this ratio is below 1, the model should not be used for trading, as it's forecasting error is bigger than or comparable with stock volatility. Models with the tradability ratio within the range from 1 to 2 should be considered moderately tradable and should rather be improved before being used in real trading. Models with the tradability ratio above 2 should be considered tradable, at least from this point of view. Directional accuracy. This ratio shows the number of times the model s forecasts and the actual target values moved in the same direction, expressed as percentage. This ratio concentrates only on direction, and not on the size of changes. If your model has good directional accuracy you will be able to make money as you will know the direction for the next market move and even if the size of this move is predicted inaccurately, you will have a profitable position. Average absolute error. The average of the absolute value of the differences between the forecasted and actual target values. If you are forecasting a change in Close, the amount will represent the average error for predicting change in dollars. If you are forecasting a change in an indicator, the value will represent the average error for predicting a change in indicator points. Average relative error. The average of the relative value of the differences between the forecasted and actual target values. Mean Squared Error. The average of the squared value of the difference between the forecasted and actual target values. This type of error is used to indicate models with a lot of forecasting errors bigger than the average. 60

61 Correlation. Correlation is a statistical ratio also known as r coefficient or Pearson's correlation coefficient. It measures the linear correlation between the actual target values and model forecasts. The closer this ratio is to 1, the stronger the positive correlation. The closer the ratio is to -1, the stronger is the negative correlation. Small positive and negative values close to zero indicate a poor model with no correlation between the model forecasts and actual values. r-squared. Squared Correlation. Mean R-Squared. A statistical ratio that compares the model forecasting accuracy with the accuracy of the simplest model that simple uses the mean of all the target values as the forecast for all records. The closer this ratio is to 1, the better the model is. Small positive values close to zero indicate a poor model. Negative values indicate models that are worse than the simple meanbased model. Note: R-squared is not to be confused with r-squared which is only a squared correlation. Last R-Squared. A statistical ratio that compares the model s forecasting accuracy with the accuracy of the model that uses the current values as a forecast for all records. The closer this ratio is to 1, the better the model is. Small positive values close to zero indicate a poor model. 61

62 ! Dialog Input Importance Tab The Input Importance tab can give insights into the usefulness or importance of individual inputs. You can experiment several times with feeding the model with different inputs to achieve good results. Input Importance tab Input Importance Chart shows the relative importance of each input column. This chart can help you decide which of the input columns can be safely removed without affecting the model performance. The chart also helps understand the most important model inputs (price data and indicators) that have had the greatest influence on the model. However, you should be careful while interpreting this chart, as input columns can be interdependent, work only in a combination or encode the same information. Oftentimes, the chart identifies those inputs that can be safely ignored, as well as the key inputs that must always be retained. However, the chart should be used rather carefully because inputs are not, in general, independent and there are interdependencies between them. The Input importance functionality allows rating inputs according to the deterioration in model performance, occurring when an input is no longer available to the neural model. In so doing, it assigns a single rating value to each input. However, the interdependence between the inputs means that no scheme of single ratings per input can ever reflect the subtlety of the true situation. Some of your inputs encode the same or similar information (for example, Price and the Moving Average of this Price). Generally, we try to identify those inputs whose importance values fluctuate more than those of the other inputs and try to remove one of them. In this situation, a specific neural model may wholly depend on one or another input or on some arbitrary combination of these inputs. Similarly, an input that encodes relatively unimportant information, but is the only input to do so, may have higher importance than any number of inputs encoding the same more important information. 62

63 ! Dialog Equity Tab Here you can analyze the Equity and Drawdown curves. Tab Equity The Alyuda system displays Equity and Drawdown curves for both the training and walk-forward test periods. Equity curve displays the account s equity during the strategy testing period. With this graph, you can easily get a general idea about how well the strategy performs. X-axis indicates bars/months, y-axis indicates account equity. Drawdown curve displays the percentage equity dip at the end of each period measured from the previous equity peak. Each black vertical line represents a new equity peak. The curve between the peaks represents the equity dip from the previous equity peak expressed as percentage. X-axis indicates a period of time; y-axis indicates equity drawdown in percentage. Taking into consideration the relative simplicity of equity and drawdown graphs and the informational depth they have, a combination of both graphs provides 63

64 traders with an ideal methodology to compare profits and risks of different systems. 64

65 ! Dialog Trades Tab It allows you to analyze trading results. Trades Tab Trades Report displays details about trades generated by a model during the test period. For each trade the report displays the following values: Trade #. The consecutive number of the trade. Position. Position type: specifies whether the position was long or short. Entry Date. The entry date of the trade. Exit Date. The exit date of the trade. Entry Signal. Entry strategy signal: Buy or Short. Exit Signal. Exit strategy signal: Sell, Cover, Stop-Loss, Trailing Stop or Last Bar. # of units. The number of shares/contracts traded. Entry Price. The security s price when the position was opened. Exit Price. The security s price when the position was closed. Commission. The amount of money paid in commissions for this trade. Slippage. The amount of money lost due to slippage on this trade. Profit/loss. The Profit/loss for this trade. Profit/loss %. The Profit/loss for this trade expressed as percentage. Bars held. The number of bars held in position. Profit/loss per bar. The average profit/loss per bar being the profit/loss divided into the number of bars held in the position. 65

66 Cumulative profit/loss. The cumulative profit/loss for all the preceding trades and current trade. For the first trade, this value is equal to the Profit/loss value. For the second trade, the profit from the first trade and the second trade are combined, and so on. Cumulative profit/loss, %. The percentage of the Cumulative profit/loss. Run-up. Trade run-up. Drawdown. Trade drawdown. 66

67 ! Dialog Strategy Performance Tab Here you can get the detailed model reporting. Strategy Performance Tab The Strategy Performance Report assists you in analyzing a strategy, as well as in evaluating its suitability to your trading style and risk preferences. The report provides over 100 performance parameters that can be used to analyze a strategy. 67

68 The Performance Report contains seven sections: 1. Summary Net Profit. The amount of money that was made or lost trading based on the selected strategy. Return on Initial Capital. A percentage of the profit or loss the system generated based on its initial equity. Gross Profit. The total amount of equity earned during all the profitable trades. Gross Loss. The total amount of equity lost during all the losing trades. Profit/Loss. The dollar amount a strategy has earned for each dollar that was lost as a result of using the strategy. Profit/Loss is calculated by dividing gross profit into gross loss. Commissions paid. The amount of commissions paid to execute all generated trades. Account Size Required. The amount of money you are to have in your account to trade the system. Account Size Required must be used only for those systems that trade futures contracts. It is the main parameter for calculating the Return on Account value. The Account Size Required is calculated using the following formula: Max Drawdown + (Margin x Number of Units), where Margin and Number of Units are taken from the Position settings in the Strategy dialog box. Return on account. A percentage of profit or loss the system has generated based on its Account Size Required. The parameter is calculated by dividing the Net Profit into the Account Size Required. This value is more important than the Net Profit value when you buy or sell on margin. 68

69 2. Trades Total number of trades. The total number of trades that were generated by the strategy. Number of profitable trades. The amount of profitable trades generated by the strategy. Number of losing trades. The amount of losing trades generated by the strategy. % of profitable trades. The percentage of profitable trades generated by the strategy. % of losing trades. The percentage of losing trades generated by the strategy. Largest profitable trade. The amount earned from the largest profitable trade. Largest losing trade. The amount lost during the most unsuccessful trade. Average profitable trade. The average amount earned by one profitable trade. Average losing trade. The average amount lost by one losing trade. Average ratio. A trade effectiveness ratio, calculated as the average profitable trade divided into the average losing trade. Average trade. A trade effectiveness ratio, represent the expected gain from each trade. It is calculated as follows: (% of profitable trades) * (average profitable trade) (% of loosing trade * average losing trade). Max consequent profitable trades. Max consequent losing trades. Average days in profitable trades. Average days in losing trades. Maximum number of days in profitable trade. Maximum number of days in losing trade. 69

70 3. Return Initial Capital. The amount of starting capital selected to test the current system. Final Capital. The amount of the capital that exists at the end of the specified period. Return on Initial Capital. A percentage of the profit or loss the system generated based on its initial capital. Cumulative return. A percentage of the profit or loss calculated as the amount of money you made divided into the amount of money invested in the first trade. Return on Max Drawdown. The Net Profit divided into the maximum drawdown of the current strategy. Buy & Hold return. The percentage difference between the initial equity and the final equity in the buy and hold strategy. The buy and hold strategy assumes that you buy on the first day set in your model and hold the position. The profit is calculated based on the price on the first day and the price on the last day of the trading period. Annual rate of return. The annual rate of return for the whole period. Buy & Hold annual rate of return. The annual rate of return achieved for the same period using the buy and hold strategy. 70

71 4. Risk Gain to Pain Ratio. A reward/risk ratio, calculated as the average annual profit divided into the average annual equity dip. For example, if the Gain to Pain ratio is equal to 3, the average annual profit is three times bigger than the average equity dip measured from the previous equity peak. The higher the ratio, the better the strategy is. Sharpe Ratio. A reward/risk ratio calculated as a measure of an account return relative to the total variability of the account. The Sharpe ratio uses account standard deviation as the measure of risk. The higher the ratio, the better the strategy is. Return Retracement Ratio. A reward/risk ratio that, unlike the Sharpe ratio, distinguishes upside and downside return fluctuations. The higher the ratio, the better the strategy is. Max Equity Drawdown. A risk ratio, calculated as a percentage of the maximum equity dip from the previous equity peak. The lower the ratio, the better the strategy is. Net Profit / Largest Losing Trade. A reward/risk ratio, calculated as the net profit divided into the maximum loss. The higher the ratio, the better the strategy is. Net Profit / Max Drawdown. A reward/risk ratio, calculated as the net profit divided into the maximum drawdown. The higher the ratio, the better the strategy is. 71

72 5. Drawdown and Run-up Max Drawdown. The largest equity dip the system has undergone within a single trade. Drawdown measures the open to the lowest (highest) unrealized low (high) of the trade. Average Drawdown. The average equity dip of all the trades. This amount represents the system s more probable behavior compared to the maximum drawdown. Max Run-up. The largest profit potential (realized or unrealized) experienced by the system on a single trade. Run-up measures the open to the highest (lowest) high (low) of the trade. Average Run-up. The average maximum profit potential of all the trades. This amount represents the system's more probable behavior than the maximum run-up. 72

73 6. Outlier Trades Positive outlier trades. The number of trades and cumulative profit/loss for positive outlier trades. Positive outlier trades are trades that are three standard deviations bigger than the average trade. Negative outlier trades. The number of trades and cumulative profit/loss for negative outlier trades. Negative outlier trades are trades that are three standard deviations lower than the average trade. Total outlier trades. The number of trades and cumulative profit/loss for all outlier trades. 7. Time Analysis Trading period. The trading period in years, months, weeks and days from the start date to the end date. Bars in the market. The number of days during which the strategy had open positions. % in the market. The percentage of the test period during which the strategy had open positions. Bars out of the market. The largest number of bars during which the strategy did not have any open positions. Average time in trades. The average number of bars in profitable/losing/all trades. Average time between trades. The average number of bars between profitable/losing/all trades. 73

74 Inserting Model Signals into a Chart You can display the signals generated by a model-based strategy on the corresponding chart. Thus, you can see exactly where the trades occurred, at what price, with what profit/loss, run-up and drawdown. You can insert signals generated during the training or test period. For analyzing real-world model performance, we recommend that you use only the test period signals. To insert model-based strategy signals into a chart: 1. Open Model Builder. 2. From the Model List, select the model whose signals you want to insert. 3. On the toolbar, click Insert and select Training range or Test range. 74

75 Price Forecasting To insert a forecasted price into a chart: 1. On the Model Builder toolbar, click Insert and then select Forecasted Price. A line will appear in the chart. Now you can see the forecasted direction of the price. 75

76 Saving a Model Output as.csv file You can save your Model Output (forecasts) as.csv file. You can use this possibility to feed model output to other software for analysis or import it into Tradecision as a Custom Time Series and use in Improvian Editor to build indicators, systems, studies and neural models. To save a Model Output as.csv file: 1. In the Model Builder dialog box, click Save Model Output to CSV File. 2. Locate the folder into which you want to save the file on your hard drive. 3. Click OK. 76

77 Editing a Model To edit model properties: 1. In the Model List, select the model that you want to edit. 2. Click Edit. The New Model wizard will be displayed with model properties inserted into the corresponding boxes. 3. Modify the model properties during the appropriate step and click Finish. After the modification of the model, it will be automatically retrained and its performance figures will be automatically updated. 77

78 Deleting a Model To delete a model: 1. From the Model List, select the model that you want to delete. 2. On the Model Builder toolbar, click Delete and confirm your action in the confirmation dialog box. After the confirmation, the model will be deleted from the internal database. 78

79 Import/Export Import/Export With the easy-to-use Import/Export tools, you can manage your ideas more effectively. You can easily exchange any of your trading ideas with your friends. The custom indicators, trading systems, studies and models that you created and saved on your local hard drive can be easily imported, exported and sent by . Importing a Neural Model To import a Neuro-Model: 1. In the Model Builder dialog box, click Import/Export Selected Model and then select Import Models. The Import Models dialog box is displayed. 2. In the Import Models dialog box, click the button to browse for the folder that you want to import the models from on your local hard drive. 3. Choose the models that you want to import by selecting the appropriate check boxes. 4. Click Import. All the imported models will be added to your Models List. 5. Click Close to close the Import Models dialog box. 79

80 Exporting a Neural Model To export a neural model: 1. In the Model Builder dialog box, click Import/Export Selected Model select Export Models. 2. In the Export Models dialog box, click the button to browse the folder on your local hard drive where you want to export the models. 3. Choose the models that you want to export by selecting the appropriate checkboxes. 4. Click Export. The models will be saved in the folder you indicated with the extension *tml. Now you can send them by to your friends or partners. He/she will need to use the Import command to work with them. 5. Click Close to close the Export Models dialog box. 80

81 Improving Model Performance You should be prepared to spend some of your time on making profitable neural models. Building a highly successful neural model is not an easy task and, usually, requires a significant amount of time and persistence on the part of the trader. Tradecision is not a ready-made strategy but rather an advanced set of tools that offers you a significant advantage over other traders. It helps you build accurate forecasts and winning strategies based on the latest artificial intelligence technologies. Neural networks are not a silver bullet. They are excellent pattern recognition devices, but they cannot be expected to do all the work for you. They cannot select input variables among the myriad of factors that influence the financial markets. Different markets and stocks are driven by different factors and, therefore, require different inputs. And it is your task to select the right inputs and amount of data to train and test your neural model. Due to the complexity of the market conditions you should not expect to have a perfect model. You will never be able to create a model that is always profitable. Losing trades will occur in any case. Besides, you have your personal restrictions, such as risk preferences and trading style that will be sure to affect your trading. Therefore, you should aim to build a model that will make money most of the time or, more correctly stated, one that generates an acceptable amount of profit with an acceptable drawdown (and other risk objectives) and allows you to achieve such results more easily than if you tried to do that without this neural model. Additionally, do not expect a good model to be good forever. The market conditions are changing all the time and, from time to time, you will have to rebuild your model or even stop using it at some point. Be prepared to create a new model based on new factors, influencing the price. Do not forget to maintain your profitable models. Be as diligent and active as when you were just beginning to build models and will be sure to always be in profit as many of our clients do. 81

82 Tips on selecting the right model inputs: Provide meaningful information Think about what kind of information you are trying to provide to the neural nets when selecting an input. You need to have a good understanding of the market to consciously select inputs. Provide diverse information Try to avoid redundancy and select inputs that really provide information on the different aspects of the market, such as inter-market inputs, and so on. Use genetic algorithms to find the right inputs and their parameters If you have Tradecision Professional, you can use the genetic algorithms to help you select a few good inputs among those you suggested. Also, be aware that even genetic algorithms-based optimization can take up a great deal of time when too many variables are used. Don t use too many inputs 4-8 inputs is, usually, the optimal number. Very seldom you may need more than 10. Try removing inputs with a low Input Importance Rate. If you have a large set of valuable parameters, try to divide them by category (i.e. fundamental factors, momentum indicators, volatility indicators, price ratios) and make a separate model for each category. Provide relevant inputs Do not use any technical indicators if your stock is driven primarily by fundamental factors and vice versa. 82

83 Tips on how to improve your neural model s performance Once again, analyze and improve your inputs selection. See Tips on selecting the right model inputs. Use cyclic and volatile stocks with similar patterns for the period you are using. Drop stocks with major changes that do not repeat. Do not use too much data. The markets are changing and older patterns are useless while dealing with the current market situation. The amount of historical data depends predominantly on the stock under investigation. However, as a rule, we do not recommend using more than 8 years of data for daily bars. Fine-tune or optimize the Buy/Sell thresholds of your neural model by implementing different Fitness criteria. If you receive a good Model Performance Report but a rather poor Strategy Performance Report on your model-based strategy, you need to apply more money management rules and use thresholds optimization and additional indicator-based filters in your strategy. Build complex neural strategies. Use the full power of Tradecision. Create several models and combine them into a committee, or create cascaded models that use other models forecasts as inputs. Do not use a too short or too long walk-forward test period. A too short test period will not allow you to analyze out-of-sample performance thoroughly. A too long period will use the data that can be used for training and allows for lesser availability of the recent patterns during the training, which decreases drastically your chances for success. Usually, we recommend that you devote from 6 to 12 months to the walk-forward test period. Try to train your neural model several times with the same set of inputs and parameters. If the Model Performance Report figures differ significantly, it is impossible to create a stable model with your input dataset. You will simply receive a random sub-optimal result. Analyze the market conditions. There is a great probability that the market is under the influence of some new, powerful factors or that your stock has entered another life phase. Perhaps, it s time to look for other inputs or drop this stock for the time being. 83

84 Chapter 4: Creating a Model-Based Strategy Before creating a new neural strategy, you need to create an appropriate neural model for that symbol. When you have achieved the desired model quality using the Tradecision Model Builder, you need to go to Strategy Builder and incorporate the model output into the neural strategy rules. You can combine these rules with the indicatorbased entry/exit and money management rules. A neural strategy is a strategy whose rules are based on neural model predictions. A neural strategy can be created only for one symbol. To test a model, Tradecision applies a simple model-based strategy to historical data and determines whether you would have made a profit if you had followed this strategy, or not. If the strategy turns out to have been profitable, and if it fits your trading personality, you can be confident in using the model to make future trading decisions. A major advantage of model testing using a model-based trading strategy is the possibility to rigorously test the model over different periods of time and with different parameters. 84

85 To create a new model-based strategy: 1. In the Strategy Builder dialog box, click New Neuro Strategy. The Symbol List dialog box will be displayed. Select a symbol you need and then click OK. 85

86 The NeuroStrategy dialog will be displayed. The dialog box caption will also contain the name of the strategy symbol for which the strategy is being created. 2. In the Properties tab, select the name of the strategy, its description and Signals, Stops and Trade Price parameters. 3. Select the Entry/Exit Rules tab, and then click Edit to define each rule. 4. In the Categories list in the Improvian Editor, find a new function category called Models. 86

87 The Models category will contain all the models created for the strategy symbol. 5. Select the model that you want to use and click Insert. The name of the model will be inserted into the Expression box. One will be able to use it as a function in the conditions that you want to define. Using Improvian Editor to Define Strategy Rules Each of the models you have created is described as separate function. This function will generate a model forecast. If your target (%Change in Close) and lookahead (7), the function will return the forecasted % change in Close for 7 bars ahead. For example, entry long for DD_Model_3@DD >4 means entering a long position if the model forecasts that the Close will rise for 4% during the next 7 bars. 6. Click OK. A new neural strategy will be created and placed in the Strategy Builder. Now you can insert the strategy into a chart, use to perform a simulation for the corresponding symbol or for optimization purposes. Strategy List 87