Originally published in The Financial Times on April 6th, 2011

Tensions in the Middle East and north Africa, we are told, lie behind the recent increase in global fuel prices, which Wednesday hit a 2 ½-year high. Yet while Brent crude this week stayed above $120 a barrel, in Tripoli petrol hovered at around 54 cents a gallon. And that is not a typo. The popular reason for why those closest to the fighting, in this case, suffer less than those farther afield, is Libya’s hefty subsidies. The less popular reason is that world energy markets have been carefully designed to profit from the slightest supply hiccup, even if there is little evidence of actual shortages.

The energy-trading fraternity has never let the facts get in the way of a good supply scare. True, this historically fragile market is vulnerable to price swings as demand threatens to climb faster than production. But there is more to it than that. Indeed, what President Barack Obama did not mention last week in his energy security speech about the faults of the global energy market could fill a Saudi oilfield.

Rising from the ashes of a failed potato exchange in downtown Manhattan, the modern-day oil market came to prominence in the 1980s. Its main architect was Michel Marks, the Paris-born son of a produce merchant, who to this day is none too happy with how his creation turned out. Over time, the market has mushroomed to include futures, options and swaps contracts traded on a handful of exchanges round the world. There is also a thriving private, over-the-counter market, where physical “wet” barrels change hands. Today, the oil market’s global daily value comes to about $600bn, even if limited transparency means exact figures remain elusive. The central market for oil is now part of the Chicago Mercantile Exchange, although it is making inroads into London in an apparent effort to escape new US rules under the Dodd-Frank act.

As is the case with liquid markets subject to swift price changes, the oil market attracted speculators from the start. Their role was, and is, necessary – allowing those who drill for, store and sell oil to manage risks that otherwise would jeopardise their businesses. Oil takes years to pump from the ground, so the long-term viability of the producers is crucial to supply. As governments on both sides of the pond reassess their rules, some have floated the idea of kicking speculators out of the market altogether. That would be a bad idea. Nonetheless, their licence to speculate should run only so far – and much less far than at present.

Since the second Iraq war, for instance, the value of the speculative oil futures market has grown more than threefold. At the same time the amount of oil actually being produced hasn’t risen nearly as much. Speculators this year have also been predicting future rises in prices by a factor of 3 to 1, according to US government data. Yet this comes as global production hit an all-time high in February – and higher supply typically cools prices. Meanwhile, the market’s migration from pit trading to electronic trading has seen a fresh problem associated with high-frequency trading.

Rules governing these markets have not kept pace with this rise in speculation. Speculators can trade with almost no money down, allowing them to influence prices via large bets, while often risking only 5 to 10 per cent of the overall value of the trade. The US Congress has, from time to time, berated “big oil” in public hearings, but has shied away from reducing the traders’ immense use of leverage.

In fact Washington has not only done little to remedy the situation; it has often made it worse. A shining example of this is the “bona fide hedging exemption”, first granted to Goldman Sachs in 1991. It allowed the bank – and then other large speculative players – to make large commodity bets that previously were forbidden without taking delivery of the underlying product. In 2000 the US also passed the Commodity Futures Modernisation Act, which barred regulators from apprehending fraud in high-risk, over-the-counter energy and financial markets. This set the stage for the credit crisis, while off-exchange commodities volumes rose sharply over the next five years to $32,000bn, according to the Bank of International Settlements. In the UK, the Financial Services Authority failed to produce a consistent set of commodities rules, even though it did act quickly to fine one broker for triggering an increase in the price of Brent while drunk.

In short, this is a market that has done what it has pleased for too long. However, the UK, Europe and the US are considering complicated new rules that threaten to cure the disease with more poison – paving the way for regulatory arbitrage among the major global exchanges. Instead, these regions might consider a simpler solution. Rather than limiting speculation outright, they should require that speculators put forward more of their own money when placing bets. Raising margins, or the downpayments on trades, has long been ruled out by Wall Street as the nuclear option. There is a good reason for that: it would work.

The writer is the author of “The Asylum: The Renegades Who Hijacked The World’s Oil Market.” She is a fellow at the Center for Environmental Journalism at the University of Colorado.