BPA trading room Q&A: March 17, 2015
An article said backtesting is the first step to becoming a profitable trader. Do you agree with this and how do you advise a beginning trader to start?
Audio duration: 6min 9sec
Audio transcript
Backtesting is more complex than traders think
Every now and then, somebody brings up backtesting, and I can talk for hours about backtesting, and I think that advice is horrible advice because, backtesting is far more complicated than what most traders who backtest understand. There’s really a lot of math in backtesting, and there are all kinds of assumptions. Let me just say that most people who are backtesting and coming up with results, like in TradeStation, and they’re coming up with all these profitable strategies, are fooling themselves. They’re not intentionally fooling themselves, but they don’t understand what they are doing. They think they understand what they are doing. They’re creating stuff that just looks exactly like what TradeStation says, or any other, E-signal or anything else, MetaTrader; it doesn’t matter which one. But you will lose money if you backtest ideas and trade those ideas. You will lose money because backtesting is much more mathematically rigorous than what it appears.
Institutions backtest, yes, but…
If it was so simple that anybody in their first week of trading can come up with a trading idea and backtest it, and then use it to make money, do you think that you are smarter than a physicist or engineer at Goldman Sachs who has been writing algorithms for 20 years? Do you think in the first week or two you’re going to come up with an idea that these guys, who have spent 20 years doing this, have not thought of? You obviously know that anything—any idea that can be backtested has been backtested. Every hedge fund, every high frequency trading firms has a room filled with physicists, mathematicians, engineers, and they get paid $300,000, $500,000 a year to backtest. That’s all they do all day long. And they have horribly boring jobs; they just sit at a desk and do backtesting all day long, but they don’t care because as a mathematician, or an engineer, or a physicist, you have no other place in the world where you’re gonna get paid $300,000 or $400,000 to do that stuff. So, they know they have a crummy job, but they don’t care. They’re getting paid a lot of money to do a crummy job, and so they do it, and they get to live a lifestyle that they could not possibly otherwise live if they were not doing that crummy job.
Back to backtesting, okay? If I’m taking a bet—if someone’s asking me, “Al, I’ll give you even odds. Bet on that physicist with a PhD from MIT coming up with an idea, or bet on this beginner trader who has been trading for six months, and started backtesting a few weeks ago.” I’m being facetious, and I don’t want to be cruel, but I’m trying to make the point that if there is a good workable idea, it’s already been found by a thousand guys on Wall Street who have been doing this for a long, long time. People who run the hedge funds and the high-frequency trading firms, they’re doing all this stuff. And I don’t know if you have any idea how many algorithms a high-frequency trading firm has, but probably most of them have thousands of different things that they use when they trade. And, you know what they do at the end of a week, at the end of three days, at the end of two days? They stop using whatever they’re using, because stuff that works only works for a few days. So if they’re doing stuff that works in a trading range environment, they have programs that measure trading ranginess of the market and trendiness of the market, and they’re constantly switching their algorithms to fit the market. And when a trader is thinking about backtesting, they’re thinking about a Holy Grail; they’re thinking about a goose that’s going to lay golden eggs for them forever, and it ain’t gonna happen. And if you come up with a perfect trade, a golden goose, and it’s working consistently, it will probably take two days before 50 other guys on Wall Street discover it, and they’ll start doing it, and then the reality is your golden goose will die.
Far too many variables
I say that, and I’m a guy who’s done over 10,000 hours of backtesting. So it’s not like, “Oh, Al’s just saying it.” No, I’ve done 10,000 hours of backtesting, so I’ve done all kinds of backtesting, and at the end of it all, after doing it for years, it’s my opinion that, at least for my personality as an active trader, I cannot make money based upon backtested ideas. Theoretically, I could, but, I see too many exceptions. So if I backtest an idea, I see, “Oh, what if the prior two bars are big bear bars closing below their midpoints?” Well, suppose one’s closing above its midpoint. Suppose the range is 50 percent bigger than an average range? Suppose it’s occurring in an environment where the moving averages have a slope of whatever—20 or something?
There are so many variables. And then you have to factor in higher time frames. You don’t have to do any of it, right? You can be just simple. If the open of the day closes above its midpoint, do I buy above the high of the bar? Do I do it for a swing? Do I do it for a scalp?
If we have five consecutive bull bars, do I buy or sell? Five consecutive bull bars, you buy—it went up, Five consecutive bull bars, you buy—it went down!
I’m sorry about going on with this, but I think that backtesting is a terrible idea. I think it makes much more sense to try to understand what the markets are doing. Are you going to understand the kinds of stuff that I’m talking about: about what’s going on – on bar 10, what’s going on – on bar 37? You don’t have to understand that to trade profitably, but if you learn how to draw channel lines and trend lines, I think you’re in really good shape. If you learn how to recognize potential lower low major trend reversals, and either buy them or wait for a breakout and then buy, I think you’re in good shape. But beyond that, I think backtesting—it’s more recreation than it is on an occupation. I don’t think you’re going to make money doing it.
Al Brooks
Information on Al’s Online day trading room
Hi Al,
Using the volatility differentials across varying expiration in futures (crude, natural gas, emini SP) – such as backwardation and contango along with mean reversion trading. Does it have any benefits in your experience?
The only trader who ever mentions the term structure is Gartman. Because I want to be respectful, I will only say that there are many traders on TV whom I believe are good traders, and they never mention it. I have never met a day trader who pays any attention to it. However, if I were regularly holding crude oil or corn positions for weeks at a time, I probably will look at it occasionally for a sign of an impending change.