Efficient markets, data dredging and the wisdom of crowds
I've been commenting on some blogs on topics touching tangentially on the psychology of investing, data-dredging and the Efficient Market Theory.
First the oft repeated argument that long term market outperformance by an investor somehow violates the Efficient Markets hypothesis. I have seen this repeated often and have never quite understood the underlying logic. This came up again recently in the light of Anthony Bolton's retirement from the Fidelity Special Situations fund. Does his long term outperformance contradict the EMT ?
If you take an arbitrarily long time period and measure average market performance over it, clearly some investors' records would be superior to that average as they can't all be the same. Looking back it is easy to identify many such patterns and instances and the fact that it happens has nothing to do with the EMT. This will happen simply due to the nature of averages and the fact that the sum total of all investors is the market.
This outperformance could well be explained simply by chance rather than any particular skill.
The more relevant question is whether you can predict outperformance in advance rather than by creative mining of past data. Could Anthony Bolton or Buffett's performance have been predicted when he started out ? Even here it gets tricky as the number of holdings is also significant. Investors taking increased risk and holding a smaller number of stocks have a greater probability of long term outperformance ( effectively a reward for the increased risk) without contradicting the EMT. People like Buffett, Munger etc. fit into this bracket.
So I believe that long term outperformance is totally consistent with the EMT. That doesn't mean that I am convinced about the EMT though.
Which brings us to the issue of data dredging, something that even pollutes a large amount of scientific research but is especially noticeable in economics. A great ( if not entirely serious) example is here . In fact Chris seems to be a data dredger extraordinaire, and a lot of his analysis fits the definition of the term to a T. That still makes him one of the most original and intellectually stimulating bloggers out there. The key point he, perhaps deliberately, never mentions is that just because you have uncover correlations and patterns in data from the past does not mean that the relationships weren't just due to chance and have zero predictive value going forward.
Finally, some discussion on cyclical markets and rationality is here . I think people underestimate the effectiveness of decision making by the 'average' person in aggregate , as part of a large, diverse group. Admittedly, my views may have something to do with one of the books I am currently reading.
posted by Piyush Pant at 4:52 pm
3 Comments:
I've never been too upset about the accusation of data dredging.
The alternative to data-dredging, more often than not, is simply a blind adherence to one's prejudice.
I never claim that past relationships will continue into the future; I really hate the utilitarian assumption that the interest in equations lies only in their money-making potential.
In showing that link between the S&P and industrial production (which only exists since the mid-90s), I was merely raising an interesting point. No more, no less. Of course, you could claim that the in-sample link is just an accident - but how would you verify that claim?
So many topics in one post!
I have slight misapprehension about EMT.
Profit-maximisation and utility maximisation are in my mind quite different. An individual may intend to be profit maximising, but given a personal situation (e.g. needs to sell shares to pay off something else), the individual may sell at less than market value, but by having the much needed cash in hand, becomes utility maximising. This may not be seen as rational to the market, but to the individual this is maximising the benefit to them. This information is not released to the market.
Lets take another example. If we assume that there is a difference in profit maximisation and utility maximisation, a fund manager will make movements for totally different reasons (spotted better opportunity ,other). From my understanding of EMT ,it doesn't ever addresses the issue of size in the market. As a fund manager often controls significantly more stock than a private investor, The impact on the market is non-uniform, leading to the messages and interpretations on the reasons why becoming skewed.
On the topic of interpreting information have a look at Memes (Wikipedia) This has a lot I believe to do with information transfer, message transfer and information interpretation. Word of caution: There are quite a lot of different thoughts on memes - the wikipedia entry is only one. Susan Blackmore's article published in Scientific American is also worth reading
Long term outperformance. You mention investors taking increased risk and holding a smaller number of stocks have greater probability of LT outperformance. Robert Kiyosaki (Rich Dad Poor Dad) has the view that if you invest large amounts in a small number of stocks (not diversifying), an individual is more likely to think much harder about why they are buying, what they are buying and how they manage their investment.
Wisdom of crowds. This deserves a write-up on its own Piyush!
Chris,
Don't get me wrong - data snooping is a valuable source of insight and I didn't refer to it in a perjorative sense. I think your blog demonstrates that it can be a source of many original ways of thinking about common problems.
My only ( slight) reservations are around the certainty with which you can draw conclusions from the results. Re: verifying the in sample claim, of course there isn't an easy answer. You could partition your sample but that won't always work.
Paul, the EMT does not address the individual preferences of the investor - it focuses on a market made up of a large number of interacting individuals. Whatever the preferences of any one person, you could argue that there are also individuals with a counteracting set of preferences in the market. The key issue in the EMT is whether a person has the ability to use new information to generate disproportionate profit.
Any market in which this isn't the case is not efficient. I haven't come across any evidence that this happens in the financial markets. In fact, there is some evidence to suggest that fund managers as a group are worse at benefitting from new information than the average individual ! So regardless of the amount of stock controlled, as long as it does not give a participant the ability to capitalise disproportionately on new information, the market should be efficient.
Interesting links. The whole area of "focus investing" as source of risk reduction by increased research is actually the bedrock of Warren Buffet's strategy too.
Post a Comment
<< Home