Algorithmic trading (also called automated trading, black-box trading or robo trading) has taken off in a big way. It is much bigger, I think, than most people realize. By the latest (2009) estimate I could find, algorithmic trading accounts for over 70% of trading volumes in the USA. We have given our wealth into the hands of algorithms, executing strategies not even their creators understand. Is this wise or yet another naive trust that will only result in a repetition of history?
What is algorithmic trading?
Algo-trading is simply the use of computer algorithms to automatically execute trades. In many cases the algorithms will not only decide on what to trade, when and how, but also initiate the trades themselves. The algorithms take as input a range of financial data, such as share prices, or even news articles, analyse it, spotting patterns, and then trade so as to achieve the highest profit (or some other goal).
Why use it?
Any tool in the world of finance has to do one of three things (a) make money, (b) reduce losses, or (c) reduce risk. Algorithmic trading can potentially be used for all three.
One of the simplest (and seemingly benign) uses of algorithmic trading is to break up a large transaction into a number of smaller ones. A large fund may wish to buy (or sell) a large amount of a certain stock. The problem is, should it wish to put in a trade for the whole amount, it will almost certainly move the price against itself. Other traders, seeing that the fund wants to buy (sell) a large amount of stock will increase (lower) the price at which they are willing to trade.
The fact that the fund is trading is a market signal. It indicates that the fund values the stock differently from the market. In order to reduce this signal, to reduce its market impact, (i.e. to hide) the fund will therefore do a number of small trades over a period of time. This can allow the fund to trade with a smaller market impact and thus at a better average price.
Another, very common use of algorithmic trading is known as High Frequency Trading (HFT). Here the algorithms attempt to profit by spotting patterns or price discrepancies in stocks over very short periods. Stocks may only be held for a fraction of a second. The profit made per trade is very small, but multiplied by millions of trades it can add up to large amounts.
The good
Algorithmic trading is claimed (by its proponents) to have increased liquidity in the market liquidity (in this context this means there are more trades and more opportunities to trade), which is mostly seen as positive. This liquidity comes from the fact algo-traders create trading opportunities by offering to buy or sell securities. Many act as market makers, offering to buy at one price and sell at a higher price, profiting from this difference and giving others the opportunity to trade.
Algo-trading has also lowered market
spreads (the cost of trading), which is the difference between the sell and buy prices mentioned above. The gap used to be quite large, but is now very small. With many traders competing to profit from the spread, the gap between the two prices has decreased.
Algorithmic strategies could potentially make markets more efficient (whether they actually have done so is not certain). They allow prices to react more quickly to data. Algorithms can read the news much faster than humans and trade within microseconds. They reduce arbitrage opportunities by actively looking for and taking advantage of them.
Algorithms can reduce market volatility; at least the right algorithms (or rather combination of algorithms) can do so. They can do so via a process of negative feedback. This means that an increase in the price of a stock is an indication that it may fall soon and thus signals traders to sell the stock. Such selling will then result in a reduction (or smaller increase) in the price of the stock. A less volatile market is a safer market, encouraging more investment and giving firms access to finance at a lower cost.
By their natures, computers have certain advantages over humans, which can prove useful in investment. The first is that computers can analyse a lot of data far more quickly than a human can. Algorithms can trade on patterns within thousands of stocks, following strategies humans could not comprehend. Because they act so quickly it is possible to take advantage of opportunities that only exist for seconds and to beat (slower, human) competitors.
Another computer attribute, which I have not seen discussed, is the fact that computers do not have emotions. Investment pundits often warn against emotion in making investment decisions. Algorithms can take the emotion out of investing. Human traders may be tempted to make emotional decisions, but algorithms will (if programmed to) always act rationally.
The bad
Paul Wilmott, an eminent quant whom I have mentioned before, has expressed his concerns about HFT as it is currently practiced. Wilmott worries that algorithmic trading can create a separation between value and price. The HFT algorithms are not concerned with the intrinsic value of a stock, only how the price may move in the next few milliseconds.
Even though algorithmic strategies do often look at news events and try to trade based on the news, there is a danger that fundamental drivers get left behind. With so much trade being driven by algorithms all that really matters is whether you can compete with the other algorithms, not whether demand for coffee or cement is up.
Another of Wilmott’s fears is that HFT can lead to positive feedback (the opposite of the negative feedback I mentioned earlier), which exacerbates market volatility. With positive feedback, an upward movement in price tends to cause further upward movements and similarly for down movements. Ominously, there is an incentive for funds to create such feedback. In volatile markets there is the possibility of making very large profits (if you trade in the right direction); however, should you lose all your client’s money it is your client that loses, not you. In more stable markets less skill is needed to make money and there would be less need for hedge funds and their very high fees.
The liquidity provided by algorithmic traders can easily vanish. Algo-traders are (currently) under no obligation to stay in the market and can choose to withdraw. This would most likely happen in volatile and uncertain market conditions when the traders do not want to risk being active in the market. This happened during the
flash crash, which I analyse later. Algorithmic trading may be good in the good times, but in the bad times, it may make things worse.
The large volumes of orders that algo-traders can generate can overwhelm stock exchanges, forcing them to shut down. There are notable examples of instances where indices had very large lags in being computed, trades were delayed, or exchanges had to shut down trading due to an excess of orders. The majority of the extra orders may well be from algorithmic trading. Algo-traders often place orders that are then cancelled almost immediately, which also contributes to volumes.
It is hard to predict what the effect will be of all these algorithms interacting in the market. They are just too complex. Computer “panic” could erase wealth in seconds. A small bug could cost you (or someone else) their life savings. The very speed for which algorithmic trading is designed also poses much of its danger. The algorithms can spiral out of control very quickly and do a lot of damage before humans can intervene.
The ugly
Algorithmic trading has turned investment into a war. The algorithms compete against each other, each attempting to gain some advantage over the others. Paul Kedrosky calls them “battle bots”. A part of the strategy of some algorithms appears to be to send a large number of quotes into the market, merely to overwhelm and confuse competitor algorithms. Will other traders be the collateral damage of this war that is fought on our markets?
There is also some animosity toward algorithmic traders for having an unfair advantage. They effectively have access to faster and better information than ordinary traders. In some cases they even pay to get quotes a fraction of a second earlier than the rest of the market. Eventually, algorithms may all but drive human traders out of the market.
Lord Myners, a former financial services secretary in the UK, fears that algorithmic trading removes the owner-relationship from shares. You are hardly an owner if you keep a share for only a few milliseconds. The company you buy is hardly accountable to you, then. The algorithms do not care if the underlying business is run well or whether it makes profit.
Another portent is the black-box nature that these strategies can assume. The algorithms can be seen as boxes that take share data as input and output trades. However, what goes on inside the box may not be well understood or may be a mystery even to the programmers. With some strategies, the box chooses the optimal strategy based on patterns it observes in the markets. The strategy it chooses may be very complex (far more complex than humans could hope to understand) and may change frequently. The lack of control we have over the eventual outcomes of our algorithms may be unsettling to some.
Black Monday
On 19 October 1987, known as
Black Monday, stock markets crashed. This was the largest one-day percentage decline in the Dow Jones in history at 22.61% (New Zealand’s market fell by as much as 60%).
Program Trading, an early form of algorithmic trading is oft blamed for the crash. (Remember, though, that causes are very easy to assign retrospectively and there is still no clear consensus as to the actual cause.)
In this case program trading was used for
dynamic portfolio insurance (DPI). The aim of DPI is to protect a portfolio against large drops in the market. As the market falls, the strategy will sell shares, reducing your exposure and thus protecting you against a further drop. The danger in this strategy, if collectively followed, is actually quite obvious.
If the market falls a little and a lot of people sell shares, or rather their DPI strategies sell the shares for them, it will cause the market to fall even further, causing people to sell more shares... and the result is that DPI causes what it is meant to protect against. This is an example of positive feedback causing a crash. Working in the opposite direction it can cause bubbles.
Flash Crash
On 6 May 2010, the Dow Jones lost several percentage points almost instantaneously, and then recovered within minutes. This very rapid anomaly is known as the
Flash Crash (or
The Crash of 2:45). There is still much debate as to the cause and I am certain it is not nearly as simple as most theories would claim, but algorithmic trading may be to blame.
One possibility is that HFTs reacted to a large sell order of futures. This order was being implemented by an algorithm and was to sell the futures without regard for price or timing
1 – this resulted in the futures being dumped far more quickly than expected. The HFTs then also started selling these futures, driving the price down even further in what is called
hot potato trading2 . The HFTs bought the futures then sold them again very quickly, to each other, passing them back and forth, creating a cycle of price declines. This then spilled over into the equity markets. In the latter market, many HFT traders actually withdrew and this caused some shares to sell at very low prices.
What is truly interesting about the flash crash to me, is the quick recovery. Perhaps traders (probably human traders) realised the prices were ridiculously low and started buying, wiping out the crash. One analyst, while giving a television interview and seeing the price of P&G had plummeted, urged people to buy the stock immediately. It is also worth noting that some think the HFTs may actually have played a stabilising role, that is they prevented the crash from being even more severe.
The arms race
HFT requires speed and the ability to trade
before others can act on information that affects the market, to make use opportunities that may only exist for very short periods. Computers need to make millisecond decisions and trades. This is because their real competition is not human traders (who react slowly) but other high frequency traders. This has resulted in a kind of
latency3 race, in which companies try to make their trades faster and faster.
This race has pushed funds to what seem like absurd measures just to gain a few microseconds. Fibre optic cables have been laid across the US just for these strategies; companies have located their trading operations right next to the servers from which internet access is distributed; programmers have reconfigured operating system kernels (notably Linux kernels) for optimal speed.
Algorithmic strategies always need to change. Competition between firms and changes in the market dynamics mean that the strategy employed needs to involve. Competitors reverse engineer the strategies of their competitors, and exploit them. Then their competitors need to change. Those familiar with evolutionary algorithms would know that it is even possible for the algorithms to adapt themselves. This brings us much closer to markets ruled by a Skynet
4 that we do not fully understand.
South Africa
From what I understand algorithmic trading (indeed quantitative strategies in general) are not prevalent in South Africa. The market is not nearly as deep as those of more developed countries and this makes it harder to obtain enough data on enough stocks to find patterns to trade on. As South Africa develops and the market becomes more sophisticated this is likely to change.
Skynet
The equity market is no longer driven by humans. Computers decide the price movements that dictate our wealth. Now, this is not
in itself a bad thing. As I have said, computers have many advantages over humans. The transfer of tasks from humans to machines has fuelled economic growth for the past 200 years.
A bunch of algorithms vying for dominance in the financial markets is not really all that different from human agents doing the same thing (except that the algorithms are so much faster). Human interaction is at least as complex and as difficult to understand. Human error can be as disastrous. At least with computers, there’s an instantaneous off-switch
However, there are signs that this emerging Skynet is making the markets a far more dangerous battlefield. The flash crash was not an isolated incident. Many similar such crashes have occurred in individual stocks and other non-stock markets, but they have not yet been brought to the public eye. I like algorithms. I find the idea of letting a computer do my trading appealing. But it is foolish not to consider the repercussions this could have.
Some references
General
- Duhigg, C. (2006). Artificial intelligence applied heavily to picking stocks. International Herald Tribune. Retrieved from http://www.nytimes.com/2006/11/23/business/worldbusiness/23iht-trading.3647885.html?pagewanted=2
- Duhigg, C. (2009). Stock Traders Find Speed Pays, in Milliseconds. The New York Times. Retrieved from http://www.nytimes.com/2009/07/24/business/24trading.html
- Fiedler, C. (2010). design high frequency system (algorithmic trading system). dbcoretech. Retrieved from http://www.dbcoretech.com/?p=129
- Heires, K. (2009). TRADING ON THE NEWS: Turning Buzz Into Numbers. Securities Technology Monitor. Retrieved from http://www.securitiestechnologymonitor.com/issues/19_104/-23976-1.html?zkPrintable=true
- Der Hovanesian, M. (2005). Cracking The Street’s New Math. Bloomberg Businessweek. Retrieved from http://www.businessweek.com/magazine/content/05_16/b3929113_mz020.htm
- Investopedia. (2011a). Algorithmic trading. Investopedia. Retrieved from http://www.investopedia.com/terms/a/algorithmictrading.asp#axzz1UXV3pHlq
- Investopedia. (2011b). High-Frequency Trading. Investopedia. Retrieved from http://www.investopedia.com/terms/h/high-frequency-trading.asp#axzz1UXV3pHlq
- Jones, H. (2011). Ultra fast trading needs curbs -global regulators. Reuters. Retrieved from http://uk.reuters.com/article/2011/07/07/regulation-trading-idUKN1E7661BX20110707
- Keehner, J. K. (2011). Milliseconds are focus in algorithmic trades. Reuters. Retrieved from http://www.reuters.com/article/2007/05/11/us-exchanges-summit-algorithm-idUSN1046529820070511
- Lati, R. (2009). The Real Story of Trading Software Espionage. Advanced Trading. Retrieved from http://www.advancedtrading.com/algorithms/218401501
- Lekatis, G. (2011). Algorithmic trading. anti-algorithmic trading. Retrieved from http://www.anti-algorithmic-trading.com/
- MacSweeney, G. (2007). Pleasures and Pains of Cutting-Edge Technology. Wall Street & Technology. Retrieved from http://www.wallstreetandtech.com/articles/198001836
- Rogow, G. (2009). Rise of the (Market) Machines. Wall Street Journal Electronic Edition. Retrieved from http://blogs.wsj.com/marketbeat/2009/06/19/rise-of-the-market-machines/
- Salmon, F. (2011). Algorithmic trading and market-structure tail risks. A Slice of lime in the soda. Retrieved from http://blogs.reuters.com/felix-salmon/2011/01/13/algorithmic-trading-and-market-structure-tail-risks/
- Salmon, F., & Stokes, J. (2010). Algorithms Take Control of Wall Street. Wired.
- Sethi, R. (2010). Algorithmic trading and price. Rajiv Sethi. Retrieved from http://rajivsethi.blogspot.com/2010/05/algorithmic-trading-and-price.html
- The Economist. (2007). Ahead of the tape. The Economist. Retrieved from http://www.economist.com/node/9370718?story_id=9370718
- The Economist. (2011). Dodgy tickers. The Economist. Retrieved from http://www.economist.com/node/8829623?story_id=E1_RRNJGNP
- The Telegraph. (2006). Black box traders are on the march. The Telegraph. Retrieved from http://www.telegraph.co.uk/finance/2946240/Black-box-traders-are-on-the-march.html
- Tyrone. (2010). High Frequency Trading & Algorithmic Trading. The High Frequency Trading Review. Retrieved from http://highfrequencytradingreview.com/high-frequency-trading-algorithmic-trading/
- Wikipedia. (2011a). Algorithmic trading. Wikipedia. Retrieved from http://en.wikipedia.org/wiki/Algorithmic_trading
- Wikipedia. (2011f). High-frequency trading. Wikipedia2. Retrieved from http://en.wikipedia.org/wiki/High-frequency_trading
Flash crash
- Bowley, G. (2010). Lone $4.1 Billion Sale Led to “Flash Crash” in May. The New York Times. Retrieved from http://www.nytimes.com/2010/10/02/business/02flash.html?_r=1&scp=1&sq=flash+crash&st=nyt
- Goldfarb, Z. A. (2010). Report examines May’s “flash crash,” expresses concern over high-speed trading. The Washington Post. Retrieved from http://www.washingtonpost.com/wp-dyn/content/article/2010/10/01/AR2010100103969.html?sid=ST2010100107554
- Lauricella, T., Scannel, K., & Strasburg, J. (2011). How a Trading Algorithm Went Awry. The Wall Street Journal. Retrieved from http://online.wsj.com/article/SB10001424052748704029304575526390131916792.html#project=FLASHCRASH_CHART_1007&articleTabs=article
- Mehta, N., & Kisling, W. (2010). Futures Sale Spurred May 6 Panic as Traders Lost Faith in Data. Bloomberg. Retrieved from http://www.bloomberg.com/news/2010-10-01/automatic-trade-of-futures-drove-may-6-stock-crash-report-says.html
- Nanex. (2010). Nanex Flash Crash Summary Report. Retrieved from http://www.nanex.net/FlashCrashFinal/FlashCrashSummary.html
- Spicer, J. (2010). Special report: Globally, the flash crash is no flash in the pan. Reuters. Retrieved from http://www.reuters.com/article/2010/10/15/us-flashcrash-europe-idUSTRE69E1Q520101015
- Spicer, J., & Younglai, R. (2010). UPDATE 4-Single U.S. trade helped spark May’s flash crash. Retrieved from http://www.reuters.com/article/2010/10/01/financial-regulation-flashcrash-idUKN0114164220101001
- Wikipedia. (2011b). 2010 Flash crash. Wikipedia. Retrieved from http://en.wikipedia.org/wiki/Flash_Crash
- Younglai, R. (2010). U.S. probes computer algorithms after “flash crash.” Reuters. Retrieved from http://www.reuters.com/article/2010/10/05/us-flash-crash-idUSTRE6945LH20101005
Black Monday
- Wikipedia. (2011e). Black Monday (1987). Wikipedia. Retrieved from http://en.wikipedia.org/wiki/Black_Monday_(1987)
Paul Wilmott
- Wilmott, P. (2009). Hurrying Into the Next Panic? The New York Times. Retrieved from http://www.nytimes.com/2009/07/29/opinion/29wilmott.html?adxnnl=1&adxnnlx=1313151029-B2ga/zmS0yYo/7PwbwIiLg
- Wilmott, P. (2011). High-frequency Trading: Where are we and how did we get here? Wilmott. Retrieved from http://www.wilmott.com/blogs/paul/index.cfm/2010/6/28/Highfrequency-Trading-Where-are-we-and-how-did-we-get-here
Market impact
- Wikipedia. (2011c). Market impact. Wikipedia. Retrieved from http://en.wikipedia.org/wiki/Market_impact
Market makers
- Wikipedia. (2011d). Market maker. Wikipedia. Retrieved from http://en.wikipedia.org/wiki/Market_maker
Lord Myners
- BBC News. (2009). Myners’ super-fast shares warning. BBC News. Retrieved from http://news.bbc.co.uk/2/hi/business/8338045.stm
Paul Kedrosky
- Kedrosky, P. (2010). The Run on the Shadow Liquidity System. Infectious Greed. Retrieved from http://paul.kedrosky.com/archives/2010/05/run_on_the_shad.html
TED talks on the topic (highly recommended)
- Slavin, K. (2011). How algorithms shape our world. TED. Retrieved from http://www.ted.com/talks/kevin_slavin_how_algorithms_shape_our_world.htm
- Ohayon, Y. (2011). The Impact of Algorithmic Trading. TEDxConcordia. Retrieved from http://www.youtube.com/watch?v=4dEzNMx94s0
1 It appears to have been set to execute so as to take up about 9% of market volume
2 What do you do with a potato burning your hands? You pass it on to someone else as quickly as you can.
3 This is basically the delay between a message being sent and it being received.
4 Reference to the Terminator movies.