How accurate are stock pickers?

Ran into this article from the CXO Advisory group today and thought it was interesting.  I'm sure everyone who reads about the market comes across various newsletters are experts that proclaim they know what the market will do in the future (hey I do this too but I don't get paid for it) and wondered how accurate it is.  CXO looks at their predictions and determines how accurate their advice is.  Turns around, most predictions are about a coin flip at ~45-50% (note that this is different than predicting what one company out of hundreds will do well, it merely says if the stock they chose did well).  I'm interested to see if they have a figure for what % of companies do well.

This info is not intended for you to only follow the accurate callers as past performance is not indicative of future results.

See full link

Grading Methodology:

The essential grading methodology is to compare forecasts for the U.S. stock market (whether quantified or qualitative) to S&P 500 index returns over the future interval(s) most relevant to the forecast horizon. However, many forecasts contain ambiguities about degree and timing, equivocations and/or conditions. In general, we:

• Exclude forecasts that are too vague to grade and forecasts that include conditions requiring consideration of data other than stock market returns.
• Match the frequency of a guru’s commentaries (such as weekly or monthly) to the forecast horizon, unless the forecast specifies some other timing.
• Detrend forecasts by considering the long-run empirical behavior of the S&P 500 Index, which indicates that future returns over the next week, month, three months, six months and year are “normally” about 0.1%, 0.6%, 2%, 4% and 8%, respectively. For example, if a guru says investors should be bullish on U.S. stocks over the next six months, and the S&P 500 Index is up by only 1% over that interval, we would judge the call incorrect.
• Grade complex forecasts with elements proving both correct and incorrect as both right and wrong (not half right and half wrong).

Weaknesses in the methodology include:

• Some forecasts may be more important than others, but all are comparably weighted. In other words, measuring forecast accuracy is unlike measuring portfolio returns.
• Consecutive forecasts by a given guru often are not independent, in that the forecast publishing interval is shorter than the forecast horizon (suggesting that the guru repetitively uses similar information to generate forecasts). This serial correlation of forecasts effectively reduces sample size.
• In a few cases, for gurus with small samples, we include forecasts not explicitly tied to future stock market returns. There are not enough of these exceptions to affect aggregate findings.
• Grading vague forecasts requires judgment. Random judgment errors tend to cancel over time, but judgment biases could affect findings. Detailed grades are available via links below to individual guru records. Within those records are further links to source commentaries and articles (some links are defunct). Readers can therefore inspect forecast grades and (in many cases) forecast selection/context.
• S&P 500 Index return measurements for grading commence at the close on forecast publication dates, resulting in some looseness in grading because forecast publication may be before the open or after the close. Very few forecast grades are sensitive to a one-day return, and we try to take looseness into account in grading any forecasts that focus on the very short term.

List:

Guru	Forecasts	Accuracy	Guru	Forecasts	Accuracy
David Nassar	44	68.2%	Doug Kass	178	47.6%
Jack Schannep	61	66.7%	Jeremy Grantham	37	47.5%
Ken Fisher	115	65.0%	Don Hays	85	47.1%
David Dreman	44	63.6%	James Stewart	115	47.0%
James Oberweis	35	62.9%	Marc Faber	154	47.0%
Louis Navellier	149	59.9%	Richard Band	31	46.9%
Jason Kelly	125	59.4%	Jim Cramer	62	46.8%
Dan Sullivan	115	59.1%	Gary D. Halbert	93	46.4%
John Buckingham	17	58.8%	Dennis Slothower	144	45.3%
Cabot Market Letter	48	58.8%	Jim Jubak	141	44.2%
Steve Sjuggerud	49	58.5%	Bill Cara	198	44.2%
Richard Moroney	55	56.4%	Tim Wood	182	43.8%
Carl Swenlin	125	55.4%	Martin Goldberg	109	43.1%
Jon Markman	36	55.3%	Price Headley	348	42.0%
Aden Sisters	39	54.8%	Gary Savage	119	41.9%
Bob Doll	161	54.7%	Linda Schurman	57	41.4%
Paul Tracy	52	53.8%	Donald Rowe	69	40.6%
Bob Brinker	44	53.3%	Igor Greenwald	37	40.5%
Mark Arbeter	230	53.2%	Bob Hoye	54	40.4%
Gary Kaltbaum	144	53.1%	John Mauldin	211	39.9%
Robert Drach	19	52.6%	Nadeem Walayat	66	39.7%
Don Luskin	201	52.0%	Jim Puplava	43	39.5%
Laszlo Birinyi	27	51.9%	Comstock Partners	203	39.3%
Tobin Smith	281	50.2%	Gary Shilling	40	37.5%
James Dines	39	50.0%	Bill Fleckenstein	148	37.3%
Ben Zacks	32	50.0%	Richard Russell	166	36.9%
Richard Rhodes	42	48.8%	Mike Paulenoff	12	35.7%
Bernie Schaeffer	81	48.8%	Abby Joseph Cohen	56	35.1%
Stephen Leeb	27	48.3%	Peter Eliades	29	34.5%
S&P Outlook	145	48.3%	Curt Hesler	92	33.0%
Carl Futia	96	48.1%	Steven Jon Kaplan	99	31.4%
Clif Droke	97	48.1%	Robert McHugh	128	28.9%
Charles Biderman	48	47.9%	Steve Saville	35	23.7%
Trading Wire	69	47.8%	Robert Prechter	23	21.7%