Howard Bandy rocks

I just purchased two books by Howard Bandy: Quantitative Trading Systems and Mean Reversion Trading Systems. These are seriously must buy trading system development books. Buy them. There’s another one I haven’t bought, and it’s on top of my buy list: Modeling Trading System Performance.

I recently wrote about another trading system book which gives a great overview of trading system development. But Bandy’s books gives all kinds of examples, source code, formulas, and deep deep technical explanation — the kind of stuff I just live on!

Seriously, if you are developing trading systems of any kind, these books are required reading. The Quantitative Trading Systems book starts with general concepts of development, and goes through entries and exits, indicators, four different kinds of systems, portfolio management, and how to determine if your trading system is still working.

One small thing I learned is that when you are developing a trading system, the first thing you need to do is choose your metric. What makes you happy? What really matters to you in a trading system? If you’re like me, the first thing that comes to mind is net profit, right? You want the system to make money. Ok, fine, but if it loses 12 trades in a row, and draws down half of your account before it turns around and makes a couple good trades, is that ok? Probably not. Those losses are painful. So, maybe your top metric would be minimal drawdown, or even better: the Return to Draw Down ratio. So, if you have a good return to drawdown ratio, but it only makes one trade per month, how about that? Would that make you happy? No? Not so much. I want to see more trades. I want to make sure it’s doing something. Bandy goes through a list of different metrics that might be important to you as a systems trader, and he stresses that it is important to pick the kind of metrics that are important to you.

I’m about one third of the way through the book, and I’ll have a proper book review when I’m done; but I had to get this out there today to tell you how amazing this stuff is. Buy the book. You can’t lose, it will improve your development skills!

Market Direction: The New Third Way

I just read an article in the most recent issue of Futures magazine (the May 2013 issue) called “New Approach in Analyzing Forex Markets“, by Abe Cofnas. The basic premise of the article is that traditional types of analysis aren’t serving us well; both types of analysis are weakening.

What are those two types of analysis? We have Technical Analysis (or “chartists”, as he calls us), and then Fundamental Analysis. Technical analysts use charts, indicators, support and resistance lines, and price movement to determine the movement of the markets; fundamental analysts look at company profit and earnings ratios, government decisions, major banking institutions and world economies to determine market movement.

As programmers, we are mostly into technical analysis; almost every strategy I have seen uses some sort of technical tool to determine entries and exits.

So, the new third type of market analysis is called “sentiment analysis”.

From the article: “Sentiment analysts believe that each week the market resets expectations and fears. Technical patterns of previous weeks become discounted as new concerns and upcoming event risks increase in their relevance. As each week begins, a new re-balance of fears holds the key to more accurate prediction”

What else do we know about sentiment analysis? Well, the traditional sentiment indicators are familiar ones: the VIX (volatility index), the Commitment of Traders reports, put/call ratios, and various other bullish and bearish indicators. All of these give us some indication where the markets are moving, where the big money is, where the “smart money” is.

What really caught my attention in the article were these kinds of sentiment analysis: text mining, word clouds and headline analysis. I did a bit of Googling to see what’s out there, and there isn’t much out there for traders. There are some services and software tools out there, but they seem to focus on customer service sentiment and reputation management. Not much for analyzing the markets. That means this idea is wide open for us as programmers.

What would a sentiment analysis tool look like for an automated trader? How about this: Your program first does a Google news search for the Euro and grabs all of the headlines from Google News. Then using a semantic text analysis engine, it determines if each headline is hopeful, fearful or neutral on the Euro. Maybe it could weight the headlines in terms of financial relevance. It then give you an up, down or flat indicator, which of course you use in your FOREX market entry to buy or sell.

In the semantic analysis tools and APIs that I found, keywords and phrase searching seemed to be the way things were going; searching for keywords and phrases in headlines wouldn’t be too hard: “Greek debt crisis”, “Euro failure” and “weakening of the yen” “sell-off” would be great terms to search for.

Or how about this for another idea: Your program searches twitter hash tags for tags related to the Yen. Again, using some semantic analysis, it would try to determine if the tweets were fearful or hopeful, and would then give you an up or down indicator.

This doesn’t sound like rocket science. I bet if you combine technical indicators with sentiment analysis, you’ll have the best of both worlds, especially if you’re on the daily charts or longer time periods. If this is the next big thing in trading, we’ve got some cool stuff ahead of us as programmers; this is seriously cutting edge stuff.

The article ends with this: “Predicting price direction is predicting where traders are going. Fading or following that crowd is the ultimate decision challenge for the traders…This will become a major third way of understanding markets.”

The Futures Magazine article is here. Abe Cofnas also has some books available: The Forex Trading Course: A Self-Study Guide To Becoming a Successful Currency Trader, Sentiment Indicators – Renko, Price Break, Kagi, Point and Figure: What They Are and How to Use Them to Trade and Trading Binary Options: Strategies and Tactics


Expert programmer, expert trader

How does one become truly great at something?

I recently read the book “Talent Is Overrated: What Really Separates World-Class Performers from Everybody Else” by Geoffrey Colvin. This has been the most inspirational book I have ready in a very long time.

I’ll cut to the chase: instead of seeing some individuals as naturally talented and others as not, Colvin explains that true greatness is a process of dedicated intentional practice, and hard work. Not just any kind of repetitive practice, but a kind of directed effort that he explains in detail. The book is well researched, citing numerous studies in art, sports, music, psychology and business, and relating each situation back to his central theme: they got there by practicing their skills.

Do you know what this means? This means that you and I — mere mortals — have it within ourselves to truly excel at whatever we put our minds to. This is great news!

Colvin spends time explaining much of how deliberate practice works, and other ways that experts differentiate themselves from average performers in various fields. Experts perceive more, they look further ahead, they know more from seeing less, and they make finer discriminations than average performers. Deliberate practice hones these skills.

One other part of the book that struck me was the amount of “domain knowledge” that experts possess. Experts simply know more; in working in their field, and and practicing their craft, they learn more, and use that knowledge more effectively. Additionally, experts are able to remember more: they create a framework of knowledge in their minds that they hang new knowledge on.

So, let me relate this to both trading and programming. Colvin uses the example of traders in the book when he talks about business skills that can be practiced and improved. So, clearly, this is good news for traders: if you put in the effort of dedicated deliberate practice, your trading will improve. And by practicing, you will learn more. Your knowledge will grow. Your ability to discern a good trade will improve.

Now let’s relate this to programming. How do we become better programmers? How do we deliberately practice programming? On Colvin’s point about greater domain knowldege applies here. Programmers who are constantly learning new coding techniques, design patterns, and language features, and trying to put them into practice will improve. I think one way to do this is by learning a new programming technique, and then writing a sample program that uses that technique. Reading through a senior programmer’s code is another way to learn. One of the best techniques I have experienced is pair programming, in which a junior programmer and a senior programmer pair up, and actually sit side by side while they write code. We’ll look into this idea more in a future podcast.

That brings me to a point that Colvin makes is that the right mentor or coach at the right time makes all the difference between an expert performer and an average one. Who better to design the material to practice for a student than an accomplished coach? Colvin makes the obvious point that as a student of a sport or skill such as tennis or martial arts, you do not have the ability to design your own lesson. You get no feedback from “trying to figure it out on your own”, and you will likely flounder in your efforts. I like that especially. To excel, and to practice deliberately requires some guidance. Get some coaching, get a teacher.

So, relating that back to trading, getting a coach or mentor to help you know what to practice, what to look for, and to give you feedback on your efforts is invaluable. And for programmers, finding a mentor or teacher or senior co-worker to help guide your efforts will undoubtedly help you improve. That’s one thing we’re doing over here at Trader Tech Talk – teaching and mentoring the art of programming for traders.

Talent is Overrated” is a must read; it will inspire you to excel, to want to improve. Regardless of your domain, you will find this book helping you on your path to excellence.

Coding automated strategies is iterative

Programming an automated strategy is an iterative process, like any other kind of programming. The first attempt in writing code that trades your strategy will likely not be successful. Ideally, the goal of your first pass at writing your strategy is to create the absolute simplest possible version of your strategy. If you usually trade with 3 indicators, try to code it using only the main one. If you have 4 criteria for entering a trade manually, try to code it using only one or two.

In your next version, you can add in your additional filters, trade criteria, and special cases, but on this first iteration, keep it really simple. You might discover that your strategy is pretty good, even without all the additional indicators and special cases. If you can get this simple version to be profitable in back-testing, the additional indicators and entry or exit criteria will just make it that much better. Be patient. Get the simple version really, really excellent before you make it more complex.

Each iteration should become closer and closer to your goal. And after every version of the software, do crazy amounts of back testing–optimize different parameters, test on long time periods, short time periods, recent months, and time periods from several years ago. Then try your best set of parameters on data you didn’t optimize with (for those in the know, we’re doing walk-forward testing…)


Trading Systems Book Review

Trading Systems by Urban Jaekle and Emilio Tomasini

Trading Systems has changed my life. That’s a stretch, but it’s true. I have been working on writing automated trading systems for a some time now, and this book has given me direction and focus to that task. Previous systems that I have written were done by finding all the details and rules of a manual trading system, and coding each one into the automated system, trying to figure out which rule might take precedence, and whether one rule of the trading system might be more important than another. I’ve come up with what I thought were moderately profitable trading systems, only to see them under-perform in the real world. Enter Trading Systems, which clearly showed me I was doing things backwards, and that there were much better and more reliable ways to test a system’s viability.

The most important thing I learned from this book is that the trading system itself can be be very simple; actually, it must be very simple. The example the authors use throughout the book is the LUXOR system, which is the simplest of moving-average cross-over systems, probably the first trading system anyone learns. Tomasini shows that the system tested against historical data does well; it is profitable with very simple trading rules. Even better, the LUXOR system is free; he tells you where you can download it. The code is presented in EasyLanguage, but it is simple enough that you can translate it into Metatrader or Ninjascript, or whatever your language of choice.

The next important thing I learned is that once you have run some tests to show that the system is basically profitable, there is a correct order of what to do next. In my own previous work, I threw in indicators, trading rules, stops and profit targets, Money Management rules, and everything all at once; but Tomasini is clear that the very first thing you should do is get the trading algorithm to work well on its own. Then, add a filter to two to weed out some bad trades; in the book he adds a time-of-day filter to limit when the system can trade.

Only then do you start thinking about stops, trailing stops and profit targets. In the world of discretionary trading, that’s crazy talk. In my manual trading, I have always been taught that you determine your stop at the same time you determine where you will place the trade. That idea gets drilled into us at every trading class and lecture — Use stops! But in automated trading, the algorithm must be profitable even without stops. In fact, after adding stops to the program, the total profit doesn’t change very much. What does change is the drawdown; using stops, especially trailing stops, drastically reduces drawdown on the account.

The last thing that gets added into the system is money management (or the trademarked term “Position Sizing”); once a good money management system is coded in, the system goes from profitable to uber-profitable. A simple money management rule is that the system will always trade only 2% of the account’s equity. As the equity increases, so does each trade size.

Tomasini spends the majority of this first section of the book on back testing and optimization; dozens of interesting graphs show the testing results after each change is made to the code. His graphing software is making me a bit jealous, and I’ve got it on my list to figure out how he does those graphs so that I can do my own.

One thing that is interesting about optimization is that you have to be clear about what you are optimizing: profits, or percentage of profitable trades, or the average trade, or minimizing drawdown. These various goals need to be clear your mind before you begin optimizing. It’s not enough to say that you want the system to make as much money as quick as possible. If you optimize only for net profit, and ignore drawdown numbers, your system may well wipe you out before you get to profit.

The important lesson of back testing is that over-optimization will make a trading system look good with historical data but perform poorly in the real world. Tomasini spends a lot of time explaining how to not over-optimize. One of those methods is to use walk-forward analysis. What that means is that you use a portion of your historical data to optimize your algorithm’s parameters, and then another portion of the data to test those parameters. In other words, you use half of your data for training the system, and the other half (the “unseen” half) for testing. This is known as “in sample” data and “out of sample data”. These are really important concepts, and make all the difference in the world on creating a profitable system. Walk-forward analysis keeps moving the window of in-sample and out-of sample data forward through time, and each time you get parameter results that are little better than before; by “better”, I mean more robust, and more likely to work well long-term.

Two more concepts Tomasini describes for testing the robustness of a system are timescale changes and Monte Carlo analysis. Timescale changes just means that if you have tested and optimized your system using 1-hour chart data, then you run the tests against 30 minute or 4-hour chart data, and see how it performs. If your system still does moderately well, you likely have a good system.

Monte Carlo analysis sounds so cool that I just like saying the words: “Monte Carlo Analysis”. Monte Carlo Analysis is a tool that checks for the worst case drawdown scenarios. The analysis re-organizes the order of your trades randomly, and shows you various scenarios. Your beginning equity and ending equity are still the same, but the order in which the trades are taken is randomized a number of times. Some of the equity curves will be better, and some will be worse. The analysis shows the worst case scenario on the drawdown on your account for the trading system you are testing.

Finally, Tomasini recommends periodically re-optimizing your system. Markets change over time, so if you re-optimize and re-analyze your system periodically, you can keep it generating good profits.

This is probably one of the best books I have read on my particular niche of trading systems; if you have any thoughts of developing a profitable automated trading system, you need to ready and apply Trading Systems.

(Note: The second part of the book is on portfolio optimization, which is a topic I will cover in a later entry.)

Trading Systems: A New Approach to System Development and Portfolio Optimisation