No two market environments are ever the same. That’s a fact born of the reality of markets – they are complex places where unpredictable things happen all the time for reasons that are elusive and ever-changing. It’s something of an uncomfortable fact for the bards and soothsayers of the airwaves and aether – the commentators and market analysts who take it as their job to break down the market’s inherent complexity into a familiar narrative: the market did “x” today because of “y”, accompanied by photo footage of dot com millionaires or homes in foreclosure or whatever the present Zeitgeist happens to be. But markets don’t only make fools of the clueless commentariat. They also have the ability to embarrass the smartest analysts, portfolio managers and economists – those experts with their quantitative models and spreadsheets and vast store of knowledge about what Citigroup did on Black Monday of 1987 or how GM survived the Great Depression or what role the Knickerbocker Trust Company played in the Panic of 1907.

When we say “those experts” we mean present company included. At MVCM we place a high value on the importance of analysis. Quantitative modeling and qualitative, logical reasoning are constant features of our approach to investment management. In naming this essay “The Limits of Analysis” we are speaking very much as insiders who believe in this practice and also who know that analysis is not the same thing as prophecy – it has limits as do all humans who practice it.

The DNA of Investment Analysis

Analysis is the intellectual fuel that drives investment management. The DNA of this analysis, as it is with many other economic and social sciences, is the science of probability and statistics. The language of this science is a bit arcane, but once you achieve mastery of the basic concepts it is a powerful tool through which to make informed decisions. Unfortunately, it is a tool which has limitations, and in the past two months we have seen the limits of statistical analysis in their full colors.

Here’s what we mean by that. At its heart, statistical analysis is about inferring the likelihood of a range of possible outcomes for any given event – the direction of the stock market, the Presidential election or the performance of a mutual fund manager against his benchmark. Here’s an example. If you were closely following the recent US Presidential election you may have heard the political pundits frequently describe the importance of the state of Missouri to the outcome. Not just because Missouri was one of the hotly contested battleground states – it certainly was that – but because in every election since 1948 with one exception (1956) the popular vote in Missouri and hence its Electoral College votes went to the candidate who eventually won the election. Political commentators boiled this down to a simple phrase “As goes Missouri, so goes the nation” and repeated it endlessly as the Obama-McCain contest drew to its final days. As it turned out Missouri almost kept its winning streak alive this year – it was the last state to be called – but Senator McCain appears to have just eked out a win and thus Missouri falls to the losing side of this year’s outcome.

Here’s another example: in no calendar decade since the 1930s has the Dow Jones Industrial Average shown a negative annual average return for more than three years out of that decade (it returned negative performance for exactly three years during the 1940s and also the 1970s). At the beginning of this year many investment analysts looked at the 2000s, with three negative years already (2000-02) and concluded it was statistically improbable that we would show a negative result for 2008. After all, they reasoned, the 1930s was the decade of the Great Depression! We just have a little subprime mess on our hands (which the Fed and a chorus of economists at the beginning of this year still insisted was “contained”), not even a certain recession let alone depression! And yet…here we are in November and barring some direct intervention by angelic hosts we are about a month and a half away from this statistically improbable “four-peat”. In fact the Dow’s trading chart for the 1930s, shown below, looks eerily similar to that of the current decade. Note the three consecutive years of decline at the beginning of the decade, followed by a recovery year in 1933, stabilization and then a second leg of the growth spurt, followed by another collapse. How statistically probable is that similarity?

[chart]

Here’s the problem with all those statistical prognostications the analysts and their talking head mouthpieces on TV make: any statistical analysis comes with a whole bunch of caveats that are largely ignored by the commentators who like to present their analysis as somehow predictive. One of the most important caveats is sample size – and here is where making too much of the role of Missouri in elections or the DJIA in any given calendar decade falls short. By “sample size” we mean the total number of times an outcome is recorded in any given measurement exercise. For example the total number of calendar decades from the 1930s to the 2000s (including each of these endpoints) is eight. This is our sample size. Likewise the number of Presidential elections since Harry Truman shocked the punditocracy with his upset win in 1948 is 16 – so that’s the sample size for the beloved “Missouri Barometer”.

In the world of probability & statistics the smaller the sample size the less statistically significant the results. 8 or 16 data points are simply insufficient to draw any meaningful conclusions. Those of us who practice investment analysis can compensate for this deficiency to some extent by using “rolling windows” in the place of simple calendar years. A rolling window is simply an extension of a particular interval (e.g. 12 months) for each incremental ensuing time period. For example the two calendar years of 2007-2008 contain 13 complete 12-month rolling windows: 1/1/07-12/31/07, 2/1/07-1/31/08, 3/1/07-2/29/08,…,1/31/08-12/31/08. 13 is a better statistical sample than 2 – the higher the number of discrete observations, the more meaningful the conclusions.

Still, even a higher sample size doesn’t solve all of the problems with statistical analysis. Another is deviation. Think again of the DJIA. We can go back and collect the index’s annual return for each year since its launch in1896 (or, for the statistical purists among us, the performance for each 12 month rolling window since that time). Adding all of the observations together and dividing them by the number of observations gives us the mean – the average over the period. With enough observations in the sample the mean can have increased statistical significance as a meaningful predictor. For example investment advisors often tell their clients (and we are certainly among them) that “in the long run stocks outperform bonds” – meaning that if you take the 100 years and change of observed returns for the DJIA or some other stock index you see that the average return over very long periods for stocks indeed outperforms that for bonds. But how risky is that investment in shorter time periods? That’s where the concept of deviation, or variance, comes into play.

The most common measure of risk used in the investment profession is standard deviation. Standard deviation is a measure of tendency around a statistical average – how much fluctuation up or down we can expect, within a given level of confidence, based on the sample of observations we are observing. For example if a hypothetical long-term stock index produces an annual average 12% and its standard deviation is also 12% then we would expect the return for any given year to fall between 0% and 24% 67% of the time (based on the statistical properties of standard deviation in normal data series distributions). We would further expect the return to fall between -12% and 36% 95% of the time (or “two standard deviations” in statistical parlance) and between -24% and 48% (three standard deviations) 99% of the time.

“99% of the time” is a pretty convincing phrase – sounds like firm ground underfoot, right? Unfortunately that firm ground is in reality a rather deep abyss between 99% and 100%, and that is where we find ourselves today. As of this writing the DJIA is down about 35% for the year to date. It has been down as much as 40% and conceivably could be back at or below those levels again between now and the end of the year. “99% of the time” is cold comfort to someone whose portfolio has just diminished by over 30%.

Enter the Black Swan

This gives rise to the third stumbling block in statistical analysis that we will discuss in this paper (there are others but in the interests of time and keeping our readers interested we will refrain from delving into them). Any sample of data is subject to the influence of unpredicted variables that result in improbable outcomes. A very useful example of this is provided by mathematician Nissim Talleb in his book “The Black Swan”. Talleb notes the fact that for many centuries in Europe the common prevailing assumption was that all swans in the world are white, because that was the only variety indigenous to the region. Hence one could predict – with apparent 100% certainty – that “all swans are white” with no variance at all. But in the late 17th century an exploration to Australia discovered a species of swan that was in fact…black. That one unpredicted variable obviously changed the entire calculus.

Black swans in fact abound in our study of stock market performance. In the fall of 1987 the US Congress brought up a bill for discussion that would effectively put a punitive tax on the so- called “greenmail” tactics of corporate raiders in the popular hostile takeover climate of the times. On October 14 of that year this bill managed to come onto the radar screens of Wall Street’s merger arbitrage traders who at that time were profiting handsomely from making bets on the outcomes of these hostile takeover situations. The arbs, fearing that the good times were coming to an end, started liquidating massive volumes of their positions. This by itself would have probably caused little more than a few days of selling pressure on the markets, if not for interacting with another variable to cause an unexpected outcome: the popularity at the time of portfolio insurance, a strategy for institutions to hedge their portfolios in the event of a decline in the stock market. In brief, the sudden selling pressure from the merger arbs reacting to the bill in Congress triggered massive sell decisions by the computers running the (unrelated) portfolio insurance positions. As the market fell further the amounts required for hedging increased. Selling fed more selling – a tipping point was reached. That tipping point melted into Monday, October 22 – “Black Monday” – when the Dow fell over 23% in one trading day (most of which, we will not be surprised to learn, happened in the final two hours of the trading day).

The message from the world of black swans is: outliers happen. No amount of confidence based on statistical likelihood can assuage the fact that when outliers happen the effects can be devastating (they can also be spectacular – the performance of the Nasdaq Stock Index from June 1999 to March 2000 evidences the fact that black swans happen on the upside as well). But there is one additional worrying feature about the black swans of today – they seem to happen more frequently and with a greater magnitude than the long-term statistical analysis would expect. Black Monday 1987 ushered in awareness of what happens when technology takes over Wall Street. The demise of Long Term Capital Management in 1998, where the black swan of the day was the government of Russia defaulting on its sovereign debt obligations, brought financial markets to another precipice. Then-Fed Chairman Alan Greenspan, Treasury Secretary Lawrence Summers and ex-Treasury Secretary Robert Rubin feared the systemic risk of all the collateral fallout from LTCM’s failure could bankrupt the entire system, and they corralled the rest of Wall Street into a bailout.

And now, ten years later, we are in the midst of a much larger, much more serious maelstrom. The credit markets froze in September when the extent of AIG’s problems with its credit default swap portfolio and the bankruptcy of Lehman Brothers convinced banks that there was no way to properly gauge the risks in lending, even in the most seemingly benign, traditionally creditworthy markets. Concerted intervention by government regulators and policymakers around the world has at least stabilized this situation over the past three weeks – but that stability sits on top of a precarious global economy whose main engine of growth for the past 20 years – the seemingly indefatigable US consumer – has buckled under the strain of rising debt, falling asset values and falling real household income. No doubt there is some natural degree of correlation between the weakening position of consumers and the faltering credit markets – but the fact that both consumers and credit markets are undergoing a painful deleveraging at the same time is compounding that natural correlation and making the results all the worse.

So what to do? Is the lesson from this piece that statistical analysis is a hopeless errand, a useless exercise in fiddling while Rome burns? Not at all, at least not in our opinion. The only prudent way to make informed investment decisions is to use every tool in the arsenal to understand what strategies and tactics may be most appropriate to achieve the best potential returns for any client in accordance with that client’s tolerance for risk and specific financial circumstances. That is our fiduciary duty. Rigorous statistical analysis cannot predict the timing and magnitude of the unpredictable – but it gives us a level of clarity and insight as we take actions to safeguard our clients’ interests. We always wish that we could see into the future with absolute certainty – but in the absence of that we can at least work through the difficult times with patience, discipline and unrelenting attention to every detail we deem material to our clients’ success and their financial well-being.

Next in the “Markets in Crisis” series: the perils and pitfalls of active money manager strategies in market crash environments.

Download Article