AAPL analysts: pros' telescopic aim low, blogs' eyeball aim true

Tomorrow Apple will report its financial results for Fiscal Q4 2011. My estimates were published a couple weeks ago. Most savvy AAPL investors already know what to expect thanks to the excellent work of Apple 2.0's Philip Elmer-DeWitt, in compiling and contrasting all estimates made by professional analysts working for the biggest banks, brokerages, and financial research institutions, as well as those made by independent (aka bloggers or "amateurs") AAPL analysts.

PED - Our Apple whisper numbers

These days the gap between these two types of analysts has grown immense. According to Yahoo! Finance, Wall Street analysts are expecting EPS of $7.28 on $29.45b in revenue, on average, while PED's compilation of pros shows an EPS average of $7.31. But the 16 independent bloggers/amateurs he polled are expecting EPS of $9.07 (24% higher than pros) on $33.47b in revenue (a cool $4b higher), on average. For the last few quarterly "smackdowns" pitting bloggers vs. pros, PED's been using a "two billion gap" mantra in his titles, which already seemed preposterous. Well the gap has now widened to $4b and it seems to me this trend will continue.

PED - Wall Street still doesn't understand Apple
PED - Bloggers vs. Pros on Apple's Q3: Yet another $2 billion gap

Most financial websites which provide stocks' forward-looking valuation information do so based on Wall Street's professional analysts data. This means widely quoted forward P/E multiples you'll see in almost every article panning (or even recommending) AAPL as a potential investment, are blindly following the blind. Fortunately we have PED's work and the couple dozen independent analysts trying to get a better sense of AAPL's financial performance, and most importantly, sharing and publishing the results.

FY2008: Q2, Q3, Q4
FY2009: Q1, Q2, Q3, Q4
FY2010: Q1, Q2, Q3, Q4
FY2011: Q1, Q2, Q3


This has gone on for more than 3 years now, and it's amazing to me that the mainstream investment community and media don't appear to have caught on at all and still rely on Wall Street's "experts" on Apple and AAPL. The good news for the small investor is the opportunity remains there, and armed with these publicly available data we can take advantage of the information asymmetry (to borrow from one of asymco's ideas) and continue reaping the rewards. Despite all the naysayers along the way, look what us small AAPL investors have achieved:

Apple Stock Chart by YCharts

Back in April I did some extensive analysis on this dichotomy in performance between pro and blogger analysts, and it's time for an update. Since then, I've seen several arguments attempting to invalidate the obvious conclusions:

1. This is a fluke based on just one quarter. Let's see what happens in the long term.

False. There's now more than 13 quarters worth of reliable data (I have more than 6 years worth of estimates but that's just one blogger). There's data for 48 different pro analysts and 16 different bloggers (keeps expanding every quarter), and estimates for up to 7 categories (EPS, Revenue, unit forecasts, and GM%). A total of 2155 estimates including 546 made by bloggers. This is plenty of data.

2. Bloggers only seem better because they're more bullish, and since Apple has performed well recently, it's a coincidence that bloggers nail it. It's a fluke.

Perhaps. But it doesn't excuse pro analysts from being unjustifiably bearish. Besides, there's dispersion patterns in the data (see below) that show bloggers reliably aim at the bullseye (even when they have bad aim), while pros systematically aim low (and with false precision).

3. Nobody relies on WS estimates as we all know it's a rigged game. AAPL is not undervalued based on a misperception of it's financial performance. Everyone knows AAPL smashes WS expectations, yet all players always have much higher expectations to begin with.

False. Not everyone knows, and what gets reported is usually misleading. Most investors don't have the time to follow all these undercurrents in perception. Most people invested in AAPL don't even know they are (through pension/ETF funds and such).

PED:

We've taken the average estimates of the six analysts with the best records over the past year to calculate our "whisper numbers" -- defined by Investopedia as "the unofficial and unpublished earnings per share (EPS) forecasts that circulate among professionals on Wall Street ... generally reserved for the favored (wealthy) clients of a brokerage."

We're going to do Wall Street one better and offer not just one whisper number for Apple's last fiscal quarter of 2011, but seven of them. Moreover, we're not going to whisper them to some favored (wealthy) clients, we're going to publish them.

You tell them Philip!

Alright, enough rousing preamble. This post is more about revealing such data asymmetry and less about justifying its presence or convenience. So let's look at some charts!

Let's begin with a measure of estimate precision and accuracy combined, which ignores the sign of the error and thus bypasses the bullish/bearish argument, penalizing equally if you're consistently low/high or sometimes low and sometimes high. This is called mean absolute percentage error (MAPE) and here's a revealing chart (click to enlarge):

I've decided to focus on EPS and Revenue estimates because those are the most relevant for investors. Only analysts with at least 4 estimates under their belt get their own error bars, with all one-timers or new-comers with 3 or less estimates aggregated under "Other Blogs" or "Other Pros". Notice I sorted from left to right by the combined error for EPS and Revenue, not by analyst type. All the top spots are taken by bloggers. Overall errors for bloggers and pros as a group are highlighted: on average, blogger estimates are 2.5 times better than pros.

Next, let's see how all these individual, pairwise (EPS/Revenue for each quarter) estimates look in a cool scatter chart where we're able to keep their signs and thus see the bullish/bearish stance on the two dimensions (also gives a sense of profit margin expectations along the diagonal). For this one I'll ignore all individual analysts and only discriminate those made by pros and blogs through color:

Lotsa dots! But you can see how most red dots fall below and to the left implying a consistent bearish stance, while most blue dots are scattered around the bullseye at the intersection of the axes. You can see how Apple almost always beats even the most bullish of pros, particularly in EPS where it fell short of only 4 of the pros EPS expectations in the last 3 years (out of 250 instances). One of those (see Brian Marshal in Q1 2010 here) is clearly an outlier with an outlandish (30% higher) assumption for iPhone units. I've included the group averages with different symbols, this time not taking the absolute values but instead including every sign (i.e. positives cancel negatives), since here the bullish/bearish stance is the main feature and I want to reveal the group's bias or lack thereof. The contrast is striking.

Now I'll aggregate all these by averaging them out for each individual analyst (again, only those who have at least 4 estimates). As mentioned above, since now we can fully show the bearish/bullish dimensions, the averages are not of absolute values but instead include all their signs. Therefore, an analyst consistently low or high will get penalized and appear far from the bullseye, while one who is sometimes high and sometimes low will average out to be near the bullseye. This I call "Bias" which reveals (here in two dimensions) who is consistently lowballing and who's aim is true. The only drawback is that it loses the dispersion information (precision or even false precision when their aim is off) for each analyst.

Only the top 5 and worst 5 analysts from each group get their dots tagged with their last name and the number of estimates shown in parenthesis. As it turned out, there were precisely 10 bloggers with 4 or more estimates, so all blue dots are identified. On the other hand there remain 20 red dots untagged, which at least alleviates clutter. Averages for all Bloggers and all Pros are also highlighted with a different symbol as in the previous chart.

This shows some individuals' mastery (and blunder!). It's quite easy to see that the 5 worst pros (and several others who will go unmentioned) are not to be taken seriously at all. One should be extremely choosy with pros estimates, and only consider those that approach the most conservative of the bloggers as somewhat valuable.

On the other hand, it's quite hard to pick favorites among bloggers. For example, it's not readily apparent which are the 5 best or the 5 worst (except for a couple obvious ones). In fact these blogger positions changed quite a bit since April, and only will stabilize somewhat after a few more quarters. Therefore, one should be open minded with bloggers and try to consider all their views.

PED:

[...]a group of independent analysts we've been calling -- for lack of a better term -- "the bloggers," have consistently out-performed the analysts who work for the big banks and brokerage firms ("the pros").

But even within the blogger group there are individuals who stand out as particularly prescient.

I'd like to slightly depart from this last conclusion, at least for the time being. Perhaps everyone has their personal favorites (which is good) most likely based on the quality of analysis and content on each individual's blog. But the data is still relatively sparse to conclude any advantage from picking and choosing among bloggers. However, the data is quite conclusive regarding the divergence between bloggers and pros.

With that in mind, I'd like to go back to studying more contrasting aspects along the bloggers/pros dimension, by going back to the set of raw estimates only discerning the analyst type.

I've already mentioned that it looks like most blogger estimates revolve around the bullseye representing Apple's actual reported numbers, while pros seem to drift downwards and to the left, clustering around a point 15% below actual EPS and 7% below actual Revenue. But what about those few blue dots also near this area, or those few red dots near the bullseye? I wondered how to measure the importance of these relative outliers within their group, and what I came up with is definitely not the only way to reveal a pattern for these non-representative estimates, but it's interesting nonetheless.

What I chose to do is to rank all the estimates within each group based on their distance from the origin. So we have a ranked list of 83 blogger estimates and 250 pro estimates. Then I decided to aggregate different slices of these two lists of ranked estimates, and looking at the mean bias "location" on the chart for these slices as I moved the slice from the best to the worst estimates from each group. I chose a 20% moving "window" or "slice", moving it by adding the next best estimate and dropping the first, and thus we get these quintiles at every 20% mark: the top fifth, the second best fifth, middle fifth, next to worst fifth, and worst fifth, for each group. I still liked the almost continuous movement in between these quintiles, so I decided to graph it all as it moved along the bullseye chart.

It's remarkable how the bloggers estimates remain unbiased as a group, all the way down to the bad estimates, except the bottom 20% worst estimates, when aggregated in 20% slices. Those estimates may be bad, relatively far from the bullseye, but when averaged the "center of mass" or "point of aim" remains very near the bullseye (because they're all over the place instead of all clustered near a "safe" conservative zone where it would show false precision, like pros do), and only veer off down and to the left for the worst few (15% or about a dozen) estimates. These are in fact the few times when Apple's numbers have put almost everyone, blogger or pro, in ridicule, including myself. Like last quarter (except Nicu and a few other bloggers), or this (embarrassing) one from a couple years ago.

Instead, pro estimates already start biased on EPS at the best 20% slice of estimates, already worse than bloggers second-worst quintile. And they don't stick around as you move down to the lesser ones. Immediately the center of mass or aim moves downward and further to the left. Soon after their mid quintile (similar to the median) they're already further off than the bloggers worst 20%. Yes, the ones I just described as bad and embarrassing for us bloggers, turn out to be the pros "par for the course" performance. Then they keep on moving further and further (even accelerating for the last quintile), ending up at a point below 25% in EPS and below 12% in Revenue.

My goal with all these quintiles was to arrive at some sort of 80-20 rule, and I think there's something like that going on: Only the top 20% of pro estimates (and those are probably flukes) are as good as the most mediocre blogger estimates. Similarly, only the worst 20% of blogger estimates (and again these are most likely all flukes) are as bad as the great majority of pros estimates.

Well those "center of mass" lines and low-aiming revelations are all fine and pretty looking. But what about that other aspect we lost when we started averaging out estimates in big slices, the dispersion? There must be a reason why those near the bottom blogger estimates are, well, near the bottom, despite the overall aim as a group being accurate. Well of course the reason they're bad is they're farther away from the bullseye. Clearly, those estimates are no good individually, because they're all dispersed, so I want to see how wide they're dispersed, how bad is the precision. And I want to compare it to the pros, to corroborate what I intuitively sensed, that pros exhibit false precision, tunnel vision, or group-think as they all cluster around their extremely conservative center of mass. For that, I decided to look into the standard deviation of the same 20% slices of ranked estimates, and see what that revealed.

If you could visualize a circle or bubble, centered on the center of mass traveling along the path revealed in the previous chart, and growing or shrinking in size as the estimates get more or less dispersed (higher or lower standard deviation for the corresponding 20% slice) then that would be ideal. But my visualization techniques are a bit limited and I doubt my initial intuitive vision of superimposing 300+ circles of varying sizes and color intensities would be of any practical use (a project worthy of asymco's data visualization animations, perhaps). The compromise I reached was drawing just 3 circles for each analyst group: the best 20%, the mid 60%, and the worst 20%. These correspond with the first quintile, the next 3 quintiles combined, and the last quintile. The width and height of the circles are derived from adding and subtracting two standard deviations from their center (the mean bias as shown on the previous chart). Here goes that:

Finally (yay this post was getting too damn long, although I hope it was interesting), as I wasn't satisfied with these discrete circles and still had the continuous data for the inter-slices between the quintiles, here's a chart where I've dropped the convenience of the 2-dimensional EPS/Revenue bullseye and resorted to a conventional line chart. Picture the two sets of ranked estimates lying ordered and in parallel (percentage-wise) along the horizontal axis, and the measure of dispersion or imprecision (standard deviation) for these 20% slices plotted vertically.

Here, despite the obvious outlier mentioned before throwing off the pros dispersion between the second and third quintile, one can clearly see how they engage in this "tunnel vision" or group think, clumping together the estimates even as they get worse. The blogger estimates instead naturally get more and more dispersed as they grow further from the bullseye yet still centered around it, except for the worst few estimates.


Back to where we started, we have Wall Street consensus of about $7.30. We now know the majority of these should simply be thrown out. Heck, according to Yahoo! Finance (and I'm sure Reuters and Fact-Set or whatever its current name is), there's some bozo analyst still with a $24.30 estimate for the fiscal year when the first 3 quarters cumulative EPS is already $20.62 (most likely hasn't updated his model for a couple of quarters). These institutional services are still showing someone with a stock price target for AAPL of $210! As one commenter wondered on one of PED's articles, how can Gauna still be employed?

Anyway, armed with these data now we know perhaps a much better way to gauge AAPL's performance against realistic expectations would be to pick the top 20% of pro analyst estimates (just to have some conservative component), and average those with the top 80% of bloggers estimates (wait I sec, I think that would exclude me! hmmm).

Ok, I'll better leave that simple exercise to the reader. Good luck to all!

PS: two very interesting articles I hadn't read (from working on all this stuff over the weekend) that approach the subject from different angles:

Posts At Eventide - The Apple Analyst Sandbag Factor
Seeking Alpha - Apple's Annual Earnings Estimates Are The Ultimate Sandbag And The Ultimate Opportunity