Chart Reading, Data Mining, biased reports and other potential investor traps.
The Human mind has an amazing capability for finding predictive power in random stock data and trend. Pareidollia describes a psychological phenomenon involving a vague and random stimulus being mistakenly perceived as recognizable.
Novice and professional investors alike essentially vote more strongly in favor of recent price trends as an influence on their decision making. In finance, pareidollia creep into chart reading and technical analysis. As computing power has increased in recent years, correlations can be sought and found in previously inaccessible places. Yet, generally brokers nurture such fallacies as it induces trading activity with little brain tasking analysis.
A systematic bias in information processing is inherent in the human brain…where we almost fanatically and unconsciously look for trends, correlations and assimilations. Then when we find one, often by chance, we assume it’s a rational correlation. But in reality its often just pure chance.
Searching for predictive patterns in financial data is called data mining. This leads to the discovery of often false conclusions. Just pure random coincidences, where careless statisticians and numerologists besides investors can easily be fooled. Massive computing power over the past decade has exponentially increased the problem of statistical coincidence. Alas, using computers for data analysis creates thousand so many new ways for self deception.
Professor David Leinweber holds a doctorate in Math form Harvard University. He has worked in trading technology for many years and for many exclusive Wall Street firms. During the last decade he noticed so many of his investor colleagues using computers to find spurious correlations in data. He so decided to put some needed light on it all. He wrote a paper called “Stupid Data mining Tricks: Over fitting the S&P 500”. The professor analyzed the database of the United Nations’ commodity productions figures, looking for correlations. He came up with statistical evidence that butter production in Bangladesh had a 75% predictive power over the S&P500. And when sheep production in New Zealand was factored into his model the accuracy jumped to over 90%. All showing how relatively easy it is to find highly predictive relationships in random data.
The media does not help unfortunately as they often promote spurious relationships in market date or truths in the past which however changed. USA’s CNBC TV is full of market talk in recent weeks that if the US stock market now is on hard times, surely it is not wise to invest in more risky emerging markets. Yet recently just the opposite has been true. The risks have been in the US, and many emerging markets have and keep doing reasonably well, despite the US fallout.
“Under scientific scrutiny, chart-reading must share a pedestal with alchemy” By Burton Malkiel in his 1996 book “Random Walk down Wall Street”.
Quantitative tools for forecasting markets have an allure of certainty which is enhance through charts so be viewed even more graphically enhancing.
Professor Andrew LO at MIT (Massachusetts Institut of Technology) quantified a study on this very topic of chart reading.
Chart patterns also called technical analysis (TA) such as the head and shoulders and double bottom patterns were tested for their predictive power. US stock prices from 1962 to 1996 were used in their analysis. The research found that some technical patterns appeared to provide incremental information during intraday price movements. While not affirming that TA can be reliably used for profits, Prof. Lo and his colleagues then concluded: “It does raise the possibility that technical analysis can add value to the investment process…TA has survived through the years perhaps because its visual mode of analysis is more conducive to human cognition…” The “possibility”? Yes it is possible a stock goes up, unless it reverses -and then it does not.
Biased -and yet unaware of the chart effect.
A large sample of investors (professionals and others) were given some company information along with one chart, to compare with a similar company and information but with a different looking chart. The first chart showed sort of a bottoming process and a recent trend on the way back up. This, as compared to the second chart, which showed a recent peak and now recently trending down. Results: 3.5 times more people liked the company/stock with the chart historically showing the price recently going up. Yet when asked less then a third said it was due to that chart. People/investors like charts even though they have very little if any proven predictive power -and they do so most of the time not even realizing they bias their decision based on the chart. When given more information about the companies involved they liked the charts even more…evidently because they got information overload. *
Trading and basing investment decisions on charts is too often just a brain cognition fallacy. The mind loves patterns, trends and reversals etc.. and yes the brokers love the trading commissions. Trading on charts is mostly a speculative activity too often used by lazy investors/analysts/newsletter writers.
Data mining is dangerous because it can find correlations where there are none. It can also come up with at first view interesting companies’ based on some numbers -but not so to the on location expert. Thailand for example has many “wall flower” type companies. They look good by the numbers and on mining data from far away, but in practice and over the years most these stock prices never go anywhere. Indeed, a bargain which remains a bargain is no bargain.
In the end, long term investors are wiser to chose companies to invest in where they have added value insights besides unbiased and updated opinions based on qualitative research, not charts or data mining. And this based on proven experience and on-location analysis which among other tools includes in person company visits.
Results here show this professional endeavor produce average investment results well beyond just broker research, data mining and chart reading.
Best Regards,
Paul A. Renaud.
* “What comes up must come down. How charts influence buy/sell decisions". Journal of Behavioral Finance (121) Year 2003.
