High-frequency trading firms rule everything around us. But their current dominance was never inevitable, according to Donald MacKenzie, a sociologist at the University of Edinburgh.

A sociologist might seem like an odd person to ask about HFT, but professor MacKenzie specialises in science and technology, and how they shape the world. In particular, he’s fascinated by the “mundane economics” of things like nuclear missile guidance systems and financial models. Years ago he got deep into ultrafast algorithmic trading outfits, and eventually wrote Trading at the Speed of Light, which was published in 2021.

Riding on the coattails of MainFT’s superb New titans of Wall Street series, FTAV caught up with him to learn more about the technical and physical systems that the likes of Citadel Securities and Jump Trading have used to transform financial markets. The transcript below has been edited for brevity and clarity.

FTAV: In Trading at the Speed of Light you describe high-frequency trading (HFT) as both Einsteinian and mundane. What do you mean?

DM: The Einsteinian feature is because HFT is sufficiently fast that Einstein’s postulate that the maximum speed at which a signal can travel is the speed of light in a vacuum becomes a very important practical concern in HFT. 

The mundane aspect is important when studying finance. You have to pay attention to the everyday ways in which money is made, time and time and time again. In Tom Wolfe’s The Bonfire of the Vanities, someone asks, well what does daddy do? And the child’s mother says when a cake gets passed around at a birthday party from person to person and crumbs drop off the plate . . . that daddy collects those crumbs. A lot of jobs in finance have that kind of characteristic. 

HFT works in much the same way. Most HFT trades are small — sometimes involving only 100 shares, which makes the average profit on each trade absolutely tiny. HFT is only remunerative if very large volumes of trades are made, which explains why the big firms are so focused on fees and costs.

FTAV: Tell me about the infrastructure that underpins HFT and the “speed race” the big firms are engaged in to eke out gains. 

DM: The HFT firms all have to consider closeness to the geodesic, the shortest path on the surface of the earth between the exchanges’ data centres and their own servers. There’s a lot of jostling for microwave towers as close as possible to that geodesic. 

But then there are all sorts of issues here, too. For example, the higher the frequency, the more that links become vulnerable to “rain fade” [“rain droplets are conductive, so they interact with electromagnetic fields — the field expends energy moving electrons inside the droplets, weakening itself,” as one of MacKenzie’s interviewees in the book explains]. 

Then there’s the issue of how you get a signal from one data centre to the other. Because of course the trading in any one data centre is going to be influenced by the trading that’s going on in others. 

And then there’s the issue of what people in the business would call “fibre tail”, which is the last bit of the journey between the microwave antennas and the firm’s server. You want the signal to move as close to the speed of light in a vacuum as possible — fibre optic cables are good but aren’t fast enough because light in the core of the cable is slowed by the refractive index of the glass. 

The exchanges make sure there’s the same physical length of fibre optic cable for each trading firm. If you happen to have your serves closer to the antenna, you don’t get an advantage because your cable will be coiled so it’s the same length as everybody else’s. Even at the heart of today’s high tech markets, something as basic as the coiling of a cable is important to preserving some kind of fairness among participants. 

FTAV: You write about the importance of the futures market for HFT. Even now, prices of share-index futures in Chicago tend to move a tiny fraction of a second before corresponding movements in the prices of the underlying shares in data centres elsewhere. Why do futures move faster, and how do HFT firms leverage this to their advantage?

DM: It’s not 100 per cent clear why futures prices tend to lead the prices of the underlying stocks and particularly, and most interesting of all, the corresponding ETFs. The kind of things that people told me was that leverage is built into futures trading. Whereas in the equities market, leverage is something you have to negotiate. You have to find a prime broker or some sort of firm that will allow you leverage. And that’s more difficult.

But there’s also historically been this kind of stickiness of liquidity. That if you had a view on likely movements in aggregate movements in the US stock market, the natural place to go to try to express that view used to the Chicago Mercantile Exchange’s index futures market, because it was very liquid and liquidity is a kind of self fulfilling prophecy If a venue has got the reputation for high liquidity, then people naturally take their trades there, which makes it more liquid.

Towards the end of the interviewing [for the book], I was getting a sense that maybe liquidity in the Chicago index futures market wasn’t as good as it had been.

FTAV: You write about the differences between “market making” and “liquidity taking”, the latter of which you say is considered somehow less “moral” than the former. What did you mean by that?

DM: What any of these exchanges have is essentially an electronic order book — a file of bids to buy and offers to sell the instruments in question. And what a market-maker does is to place orders into that order book at prices that can’t immediately be traded away. Taking is putting in orders that execute against orders that are already there, that remove liquidity. 

Why some people regard liquidity making as more moral than liquidity taking is there’s a sense in which the liquidity maker, the market-maker — the person who puts the orders into the order book, so that others can execute against them — that they are providing a service to other investors.

Some of the criticism of liquidity taking is basically the argument that at least in some cases, what liquidity takers are doing is looking for stale quotes — quotes that the market-maker has put into the order book and hasn’t cancelled fast enough when circumstances change.

And some people regard that strategy, the picking off the stale quotes, as a somewhat parasitic kind of strategy, which has bad effects, because if you think about the spread between the best bid and the best offer . . . If you’re constantly having your stale quotes picked off, then you’re going to need to broaden the spread to take it.

FTAV: What kinds of risks do HFT firms piling into the same trades pose to broader market stability?

DM: It’s absolutely the case that crowded trades and the unwinding of crowded trades is a persistent risk. 

But it could be said that most HFT firms have fairly limited positions open at any one time. The classic thing that people used to say, and I don’t think this is literally true in any sense, is that you want to go home flat every night. With no net exposure. But the classic HFT firms would typically keep their positions relatively small, because it’s a numbers game: the profit on any one high frequency trade is tiny. You have to do those trades literally hundreds of millions of times to make decent money. It’s a business of very large numbers of very small trades.

One thing that can happen — and did happen to some extent in 2010 [during the Flash Crash] — is that HFT firms rely on the price speed from the exchanges . . . they need to know where they are at all points in time. If weird things start to happen — breakdowns in the price feeds, inconsistencies — the typical reaction often automatically built into the system is to step back from the market.

The trouble is if you are one of the firms that in aggregate is a main provider of liquidity to a market, if those liquidity providers all step back at the same time, that’s likely to have disruptive effects.

FTAV: Have some of the biggest HFT firms become almost too big to fail?

DM: I think too-big-to-fail would be an unfair thing to say of them. Because that is fundamentally about banks and bank-like institutions. If a big high-frequency trading firm failed, I think it would be unlikely that there would be a bailout of that firm. They’re not quite in that kind of category, but they’re part of the infrastructure of markets, so to speak. 

FTAV: What would happen if one of the biggest players did tip over?

DM: I really don’t know because we haven’t seen that happen. The closest, I suppose, was the Knight Capital episode [in 2012]. It was an automated trading firm but not one of the classic high-frequency trading firms. And it was making markets on the New York Stock Exchange. 

It was not exactly a bug, but a software update problem, and they spent 45 minutes desperately trying to find the problem and ended up in effect insolvent. They were bought out by Getco, which wasn’t that prominent. Knight Capital didn’t actually fail, but it could have failed were it not for the emergency takeover. That was an entirely private sector takeover, there was no government involvement. 

The failure of a big market-making firm, if it was an entirely idiosyncratic event, would cause a deterioration of liquidity, but other market-makers would step in. And soon enough, things would be reasonably OK again.

The trouble wouldn’t be a big market-making firm failing for an entirely idiosyncratic reason. Something happening in the wider infrastructure of the market that would affect more than one firm, that would be a much more dangerous situation.

FTAV: For mere mortals, how complicated are the algorithms that the traders design to actually execute trades? And what about the signals the algorithms are responding to? You write that the patterns of data that inform an algorithm’s trading are ridiculously shortlived, usually lasting less than 10 millionths of a second . . . 

DM: It’s a world that’s changed through time. To begin with, the programming of algorithms wasn’t tremendously high-tech, though it’s obviously become more sophisticated. 

I think the thing I found surprising was the simplicity of the most crucial signals that were informing the behaviour of algorithms. When I started doing this work in something like 2009, even back then, your expectation would have been that these would be very sophisticated machine learning-driven things. And I discovered that to a large extent, the signals that [the algorithms] relied on were very straightforward.

There are three main categories, as I understand it. The first is the balance of bids and offers in the order book: if there were a lot more bids to buy than offers to sell, then that probably meant the prices were going to go up.

The second was the trading of the same financial instrument in different exchanges. So if Apple shares go up in one venue, then pretty much certainly they’re going to go up in other venues as well. And thirdly there’s correlated instruments such as futures and corresponding ETFs. Similarly, if one of them moves, the other one will move. 

It’s not too difficult to get your head around those things and everybody in the industry knows them, and everybody knows that everybody knows them. And that’s what made interviewing possible, because no one is going to explain their secret-sauce machine learning algorithm — even if they could explain it to you, they wouldn’t want to.

But it’s the prevalence of those simple signals that give the business the characteristic of a speed race. That is, everybody knows what the signal is, so to speak, so the key to success is to be able to react to the signal faster than the next person.

FTAV: Will speed always be the one area of innovation moving forward? How else do these firms try to outcompete one another? How is AI changing the game?

DM: Large-scale market-making is to a substantial extent an engineering problem. What you’re doing isn’t conceptually that technical, but you absolutely need a robust large-scale technical infrastructure. 

Perhaps it’s most evident in options trading, because the trouble with options is that you have the one share of, say, Apple. But you’ve got puts and calls and different strike prices and different expiries. So a big options market-maker may well be making quotes on something like a million instruments. It’s one thing having one stale price picked off, but if a million stale prices are picked off, you’re in bad, bad financial trouble.

Making sure that doesn’t happen is largely an engineering problem. I would assume that Citadel and Virtu, in particular, have gotten very good at making sure they can solve those large scale infrastructure problems. 

With machine learning, it’s about not just relying on the easy-to-identify big signals, but looking for stuff that’s closer to a signal that scarcely appears above the noise of financial markets. That’s a big, big activity. Those kinds of things are inevitably going to remain very competitive because a classic thing that happens with those kinds of signals is that once the signal starts to be exploited, then their very exploitation causes it to disappear.

What’s interesting is that this hasn’t happened with the biggest signals, perhaps because they’re built into the structure of financial markets — the very fact that equities trading doesn’t take place on just one venue.

In the US, financial futures trading pretty much all takes place on the Chicago Mercantile Exchange. That’s not the case [for cash equities]. There are multiple venues. Something that happens on one will always affect what happens on other venues. People trying to exploit that doesn’t cause it to disappear. 

FTAV: A lot of the big HFT firms are notoriously secretive. Was it hard to get information out of the traders themselves? 

DM: I was helped a little bit by some of the criticism that HFT trading was receiving back then, particularly by Michael Lewis’s book Flash Boys. I think people in the business felt they had got a bit of a bad rap from Lewis so they wanted to put their views across.

But it’s also the case that most of the people in high-frequency trading are people like me: people with some sort of scientific, mathematical background who find financial markets fascinating. 

FTAV: You write about how HFT hastened the demise of the trading floor. What from the old days has been lost?

DM: What you could say is that a unique form of human interaction has been lost. One thing I feel very privileged about is that I got started in work in finance in 1999 or 2000, and visited Chicago when the open outcry trading pits were still at their peak. 

People parading contracts, worth half a million, sometimes a million dollars, trading them by voice, or if it was too noisy, by eye contact and hand signals. That’s quite astonishing. The skill that that takes, to be able to keep a cool head in that environment, with lots of jostling because the pits were crowded, and so on.

A sociologist like me is nostalgic for the loss of a very distinctive form of human interaction that is simply not going to come back. The pits shouldn’t be romanticised, there were implicit cartels and all sorts of other things going on.

But the uniqueness of that interaction . . . It was something to watch!