Hindsight

I have been cleaning out my office in preparation for moving from DC to Chicago. I came across a small publication by GaveKal Research entitled Our Brave New World, published in 2005. It argued that the outlook for the global economy over the next several years was rosy:

For over ten years now, a wide majority of market strategists and economists from respected investment banks (Morgan Stanley, Dresdner…), a large number of upscale financial publications (The Economist, The Financial Times…), highly respected consulting firms (Lombard Street Research, Grant’s Interest Rate Observer, Gloom Boom Doom…) have drawn on historical parallels to warn us that the expansion of the past decade in US consumption was both unsustainable and likely to end in tears. Real estate all over the Christian civilised world was bound to collapse, along with global equity markets. The world would then enter into an ‘ice age’. So far, despite the strength of the above thought process, and the numerous historical parallels, the dreaded meltdown has completely failed to materialise. So what is the next step?

[…]

The reason so many analysts drag their feet in admitting that history has failed to rhyme this time around is that it would lead one to the dreaded conclusion that ‘things are different this time’.

[…]

And yet, this is exactly what we aim to argue in the following pages.

Arguing that ‘things are different this time’, we freely admit that we might end up drawing the wrong conclusions, say silly things and establish relationships where there are none. We also realise that some of our more cynical clients (say those sitting in Boston or London), might read the coming chapters and conclude that we have really been drinking the Kool-Aid. These are the risks when one ventures into uncharted territory. We accept these risks gladly, for we are convinced that the first step to successful investing is an understanding of the current world.

Unfortunately, History is of little help to this understanding. We have to draw solely on logic, and the help of our friends and clients.

In their catalog, you can see a publication called The End Is Not Nigh, in which “we push some of the themes developed in Our Brave New World a little further and review the reasons that have led us, in recent years, to shy away from prophecies of doom and why we remain positive on global financial markets”. That was in 2007. September 15, 2008, the very day that Lehman Brothers filed for bankruptcy, their research argued “that the US economy will avoid recession and that the Government rescue of Fannie Mae and Freddie Mac would probably mark the low-point of the economic slowdown.”

Then came A Roadmap For Troubling Times in 2009. Something must have scared away the optimism.

It is well-known that few investment and research organizations foresaw the impending crisis, but it is still remarkable to have something in your hands argued so strongly that, only a handful of years later, reads so transparently misguided. Contrary to what they wrote about History, it can be very illuminating to read pre-Recession reports that turned out to be so spectacularly incorrect.

The Value of Forecasting

“The trouble with the world is that the stupid are cocksure and the intelligent full of doubt.”
– Bertrand Russell

Jonah Lehrer asked recently: Do Political Experts Know What They’re Talking About? He talks with Dr. Philip Tetlock, who is well-known for his work assessing the forecasting abilities of professionals paid to predict the future.

After Tetlock tallied up the data, the predictive failures of the pundits became obvious. Although they were paid for their keen insights into world affairs, they often performed worse than random chance. Most of Tetlock’s questions had three possible answers; the pundits, on average, selected the right answer less than 33 percent of the time. In other words, a dart-throwing chimp would have beaten the vast majority of professionals.

Nassim Taleb (of The Black Swan) addresses some of these same questions, and his conclusion is that accurate prediction is very hard if not impossible on a fundamental level, making him one of the radical skeptics mentioned in the Lehrer interview. Very smart people can be “fooled by randomness” into thinking they see predictable patterns where there are none. And the laws of statistics suggest that someone, somewhere, will have to be lucky simply by chance.

Theory plays a disproportionate role in social science, and it’s arguable whether a social science theory can even be disproved. It is vanishingly unlikely that geocentrism will make a comeback for astrophysicists, but can the same be said of mercantilism for policymakers?

This points to a problem inherent in social science. Whereas hard sciences have the benefit of laboratory experimentation, history gives us a sample size of one. We cannot go back and run counterfactuals in order to verify what works and what doesn’t. This is why people can still be arguing about what caused the financial crash three years ago, the Great Depression seventy years ago, and even the fall of Rome, let alone the best methods to avoid a relapse.

Taleb argues for a society that does not make its decisions based on such predictions and that makes itself resilient to the once-in-a-hundred-years events that tend to pop up every decade. What use is a fifty year projection that must be revised yearly?

Dr. Tetlock’s decades of work suggests that forecasters who depend on a top-down theory approach are less accurate than those that are self-critical and apply a bottom-up approach. The question remains open of whether forecasting can be learned or improved. I’m part of the four-year IARPA study mentioned in the article, the most comprehensive experiment on forecasting yet. It will be interesting to see the results.