Over the past year or so I've been tracking all of us analyst performance in forecasting Apple's financial metrics, and ranking us based on the 6 or 7 categories of estimates compiled by Fortune's Apple 2.0 blogger Philip Elmer-DeWitt, and the outcome has always been a consistent underperformance by pros (see here, here, here and here). The comparison and friendly "competition" has almost become laughable, if it weren't for the serious amount of capital that these "professional" analysts look over, and thus the effect of their cluelessness on Apple's share price gets felt.
However, all this time I've been applying a somewhat forgiving methodology on my rankings. By averaging out all the categories with equal weights, the resulting score improperly reflects the relative importance and sensitivity on the stock price of these variables. It should be clear to all investors that forecasting earnings and revenue is most critical, while the number of iPods sold has very little effect (for quite a few years now) on Apple's financial performance. Yet by applying the same weight to these, the effect of the most important metrics gets watered-down by the less important ones so the analyst scores and thus the rankings do not reflect what investors should be focusing on out of all the stuff analysts throw out there.
To correct this I've now decided to concentrate on those two most important metrics, EPS and revenue, when calculating the scores. This is what investors really want to see if Apple beats the expectations of, and by how much. Granted, gross margins and the iPhone units metrics are almost as important, but these already have such a big effect on the EPS and revenue forecasts that including them in the score would be like counting them twice towards the final outcome. If either of these are a blowout or a disappointment, it will certainly show in the revenue and EPS results, so they're already being accounted for. So there.
Also, for the cumulative rankings I've only used the latest 4 quarters worth of data compiled by PED, mostly because that's when the blogger analysts have come out in force with their estimates and truly made this a representative competition. Still, I feel like the work done by us three pioneering blogger-analysts, Bullish Cross' Andy Zaky, Financial Alchemist's Turley Muller, and myself, as well as the gracious opportunity given to us by PED over more than 2 years now, should not go to waste. Therefore I've decided to expand the scope of these scores and rankings to cover the whole period of smackdowns, going back to the early "analysing the analysts" posts by PED (and Andy just once before that).
One thing I'd like to make crystal clear is that this reassessing of the scores and rankings is in no way motivated to make my estimates, or those of us bloggers as a group, look better or rank higher, even if the actual result is as such. We certainly don't need that to come out on top. Rather, this is just me trying to get the most relevance out of the data for investors. Neither does it mean I'm discarding all the other variables. All that rich history of unit estimates comes handy when assessing the speculative musings of any of these analysts in the future (e.g. Gauna's recent shenanigans). Keeping tabs on for example their iPhone or iPad units accuracy should give pause to anyone thinking about taking those speculations too seriously. But I'll leave that to each investor, and as I said, I'll build my ranked list based on the EPS and revenue figures.
For this, I've taken the trouble of recompiling all the data going back three years, as published by PED (and one early article by Andy):
FY2011: Q1
I've loaded all these estimates as well as the actual results as reported by Apple on these 6 or 7 categories over these 11 quarters and up to 51 different analysts into a nifty database model, so I can now conveniently query, slice and dice the data at will (Excel was already starting to show its limitations). More than 1500 data points in all. Let's take a look.
First, here's an overall picture of the accuracy of bloggers and pros by category, showing an interesting pattern:
Average Accuracy ( 1 - abs.error% ) by Category
Category Bloggers Pros Err. reduction -------- -------- -------- -------------- EPS 94.1% 85.0% 60.9% Revenue 96.7% 93.8% 46.5% iPhones 89.9% 88.0% 16.1% iPods 95.1% 93.2% 28.2% Macs 95.3% 94.1% 19.1% iPads 86.3% 86.0% 2.1% GM% 96.8% 96.5% 8.2% -------- -------- -------- -------------- All Cats 93.2% 92.0% 15.2% EPS+Rev 95.4% 89.4% 56.8%
Bloggers are better than pros in every category, but we're way, way better at EPS and revenue. Here's a bar chart of that (click to enlarge):
I'm aware from visual information rules and guidelines that the green continuous line (the error reduction) shouldn't have been a line but bars, as there's no continuity between categories. Yet, I liked how it looks as a line, I need it to stand in contrast with the bars behind so I'm sticking to it. *sosumi*
Next is a similar idea but this time I'm breaking it down by quarters, showing the whole history of available estimates and their changes over time:
Accuracy Over Time (all categories) Period Bloggers Pros Err. reduction ------ -------- -------- -------------- F08 Q2 95.8% 91.3% 52.2% F08 Q3 F08 Q4 94.9% 90.5% 46.2% F09 Q1 85.1% 87.9% -23.4% F09 Q2 91.8% 89.4% 21.9% F09 Q3 97.3% 93.0% 60.7% F09 Q4 95.1% 91.3% 44.1% F10 Q1 97.5% 93.0% 63.5% F10 Q2 92.4% 88.5% 33.5% F10 Q3 95.7% 93.9% 28.3% F10 Q4 90.1% 90.1% 0.6% F11 Q1 96.1% 91.0% 56.4%
Notice only one period in which we did worse than pros, and one which was a tie. All the others we were significantly better. Here's a chart (click to enlarge):
And now I'll focus on the critical EPS and revenue metrics. Here's that over time:
Accuracy Over Time (EPS+Revenue) Period Bloggers Pros Err. reduction ------ -------- -------- -------------- F08Q2 95.4% 93.3% 31.2% F08Q3 F08Q4 96.4% 93.8% 41.3% F09Q1 96.3% 87.4% 71.0% F09Q2 96.3% 87.5% 70.0% F09Q3 98.5% 93.5% 77.4% F09Q4 89.9% 86.5% 25.6% F10Q1 96.8% 92.9% 54.2% F10Q2 88.0% 82.3% 31.9% F10Q3 96.2% 92.2% 50.9% F10Q4 97.2% 91.2% 68.2% F11Q1 96.9% 87.2% 75.7%
The chart:
Cool! Feels good to get within 5% and even 3% with such consistency, but even better to cut the pros error in half and sometimes even down to less than a fourth. But doing this on the critically relevant EPS and revenue figures is what smart investors care for.
The only thing about these figures is that because they're expressed as percentages in order to be able to compare and combine the different metrics each with different units (EPS in dollars, revenue in $billions, units in millions of units, GM is a ratio), then the actual size of the errors by pro analysts kind of gets lost. Another distinction is that since these are all intended as measures of accuracy, it's not shown in which direction is the miss, it's just the average percentage amount of a miss, be that positive or negative, they all add up. But it's quite revealing to also consider the net size and direction of the average miss, and to do that I'll now keep the sign of each error and thus the positives will cancel some of the negatives, and what's left is a measure of how consistently skewed (or not) in whichever direction (always down!) are the estimates, which I call bias (you can see a more detailed accuracy/bias analysis on my own estimates going back more than 6 years here). Also, I'd like to start looking at individual performances now.
So here's the actual net average dollar amount of EPS by quarter, broken and sorted by analyst, that they tend to miss including the direction (all of us aim low but pros are extremists). Notice I include the number of estimates involved in each analyst's score to show that these are not one or two-times flukes, but persistent sandbagging. The last column really puts this in context, showing the net amount algebraically added over all the estimates made by each analyst. I've aggregated those analysts that have not made enough estimates for a consistent bias evaluation as "other" (includes most of the new bloggers like Asymco's Horace Dediu and the AFB gang who as a group show a remarkably neutral bias). So here we go:
It's amazing to me that such a highly regarded analyst as Gene Munster, and considered by many the most bullish on Apple, has managed to underestimate EPS by a whole 5 bucks and a half over less than 3 years. That's a whole $100 worth on AAPL if you price it at an 18x multiple. And that's our bullish guy. Mr. McCourt on the other hand, managed to misplace almost 5 bucks in only a year and a half. Give it another year and he'll easily be 8 bucks behind, or near $150 on the stock by a conservative multiple. But he'll go on giving interviews on CNBC and such.
The chart is a bit dizzying:
The only thing about these figures is that because they're expressed as percentages in order to be able to compare and combine the different metrics each with different units (EPS in dollars, revenue in $billions, units in millions of units, GM is a ratio), then the actual size of the errors by pro analysts kind of gets lost. Another distinction is that since these are all intended as measures of accuracy, it's not shown in which direction is the miss, it's just the average percentage amount of a miss, be that positive or negative, they all add up. But it's quite revealing to also consider the net size and direction of the average miss, and to do that I'll now keep the sign of each error and thus the positives will cancel some of the negatives, and what's left is a measure of how consistently skewed (or not) in whichever direction (always down!) are the estimates, which I call bias (you can see a more detailed accuracy/bias analysis on my own estimates going back more than 6 years here). Also, I'd like to start looking at individual performances now.
Bullish/Bearish EPS Bias (listed analysts with at least 4 EPS estimates) Rank Analyst AvgBias($) No.ofEsts AccumBias($) ---- ----------- ---------- --------- ------------ - Other blogs -0.04 15 -0.53 1 Zaky -0.05 11 -0.54 2 Tello -0.09 11 -0.94 3 Muller -0.09 10 -0.94 4 Leitao -0.11 4 -0.43 5 Cabot -0.15 4 -0.59 6 Abramsky -0.31 11 -3.37 7 Marshall -0.33 7 -2.29 8 Reiner -0.35 9 -3.19 9 Craig -0.37 9 -3.33 10 Sacconaghi -0.40 10 -3.99 11 Rakesh -0.41 5 -2.04 12 Wu -0.41 9 -3.72 13 Gardner -0.42 8 -3.39 14 Chokshi -0.43 4 -1.71 15 Reitzes -0.48 11 -5.26 16 Munster -0.50 11 -5.49 17 Huberty -0.50 10 -5.01 18 Shope -0.51 5 -2.56 19 Reid -0.53 7 -3.68 20 Fidacaro -0.53 7 -3.68 21 Fearnley -0.55 4 -2.19 22 Moskowitz -0.56 5 -2.81 23 Um -0.61 4 -2.45 24 Whitmore -0.63 4 -2.51 25 Bachman -0.64 6 -3.85 26 McCourt -0.81 6 -4.83 - Other pros -0.65 31 -20.26 -------------- ------- --------- ------------ All blogs -0.07 55 -3.97 All pros -0.50 183 -91.61 -------------- ------- Bias reduction 85.6%
It's amazing to me that such a highly regarded analyst as Gene Munster, and considered by many the most bullish on Apple, has managed to underestimate EPS by a whole 5 bucks and a half over less than 3 years. That's a whole $100 worth on AAPL if you price it at an 18x multiple. And that's our bullish guy. Mr. McCourt on the other hand, managed to misplace almost 5 bucks in only a year and a half. Give it another year and he'll easily be 8 bucks behind, or near $150 on the stock by a conservative multiple. But he'll go on giving interviews on CNBC and such.
The chart is a bit dizzying:
Same thing with revenue sandbagging, no end to their pessimism:
Bullish/Bearish Revenue Bias (listed analysts with at least 4 revenue estimates) Rank Analyst AvgBias($b) No.ofEsts AccumBias($b) ---- ----------- ----------- --------- ------------- - Other blogs -0.19 15 -2.83 1 Leitao +0.22 4 0.87 2 Tello -0.12 11 -1.35 3 Muller -0.16 10 -1.63 4 Cabot -0.18 4 -0.73 5 Zaky -0.20 11 -2.22 6 Abramsky -0.44 11 -4.87 7 Reiner -0.49 9 -4.45 8 Munster -0.81 11 -8.95 9 Craig -0.83 9 -7.50 10 Wu -0.86 9 -7.78 11 Reitzes -0.87 11 -9.62 12 Sacconaghi -0.89 10 -8.93 13 Gardner -0.90 8 -7.23 14 Marshall -0.97 7 -6.78 15 Shope -0.98 4 -3.92 16 Huberty -1.02 10 -10.23 17 Fidacaro -1.06 7 -7.39 18 Moskowitz -1.10 5 -5.50 19 Chokshi -1.21 4 -4.82 20 Bachman -1.21 6 -7.27 21 Rakesh -1.21 5 -6.06 22 Fearnley -1.30 4 -5.19 23 Whitmore -1.36 4 -5.42 24 Reid -1.38 5 -6.89 25 Um -1.42 4 -5.66 26 McCourt -1.65 6 -9.87 - Other pros -1.32 31 -40.90 -------------- -------- --------- ------------- All blogs -0.14 55 -7.89 All pros -1.03 180 -185.23 -------------- -------- Bias reduction 86.1%
One billion dollars short, each and every quarter, on average for the last 2-3 years. It may not seem much right now compared to these $25b quarters, but in 2008 quarterly revenue was only around $7.5-8b. And because recently sales have accelerated yet the analyst's percentage error remains constant or keeps increasing, that means in absolute dollars amount it's getting worse every year and every quarter. Once again Mr. McCourt is exemplary with his $1.65b average bias every quarter for the last year and a half. I suspect the pro average quarterly revenue miss for this year and the next will be nearing well over $2b.
PED (Jan 2011 post): Bloggers vs. pros: The $2 billion gap
PED (Apr 2011 post): Bloggers v. Pros: A $2 billion gap
PED (Jan 2011 post): Bloggers vs. pros: The $2 billion gap
PED (Apr 2011 post): Bloggers v. Pros: A $2 billion gap
On the other hand, well deserved kudos to Robert Paul Leitao for his courage in taking up the sole but oh-so-very-slightly bullish position over the last 4 quarters. And here's the chart:
Makes one wonder how far off them Wall Street consensus estimates for 18% revenue growth and 15% EPS growth in 2012 will turn out. If they haven't been able to get within 10% of the most recent quarter, how will they fare on stuff that's more than a year away? For now, they're 25% short of my 2012 EPS estimate of $35.76. If I'm roughly right and their past-quarter performance is to remain around 10% short, then over the next few quarters they'll need to raise the 2012 estimates by about 20%, putting further pressure on the declining forward P/E if the current stock price was maintained.
Finally, the whole list of all 51 analysts, including some obscure one-timers, ranked by the combination of EPS and revenue accuracy, which is the one I'll keep tracking regularly (although I'll keep an eye on the other things too):
Finally, the whole list of all 51 analysts, including some obscure one-timers, ranked by the combination of EPS and revenue accuracy, which is the one I'll keep tracking regularly (although I'll keep an eye on the other things too):
EPS and Revenue Combined Accuracy ( 1 - abs.error% ) Rank Analyst Accuracy No.ofEsts. ---- ---------- -------- --------- 1 Muller 96.3% 20 2 Fosberg 96.2% 6 3 Hildebrand 96.2% 6 4 Tello 95.7% 22 5 Mihalache 95.4% 6 6 Dediu 95.2% 6 7 Zaky 95.1% 22 8 Cabot 94.9% 8 9 Leitao 94.7% 8 10 Smellie 93.1% 4 11 Abramsky 92.5% 22 12 Chokshi 92.3% 8 13 Klugman 92.2% 1 14 Beauch 92.0% 2 15 Cihra 92.0% 6 16 Kumar 91.8% 6 17 Reiner 91.5% 18 18 Gillis 91.4% 2 19 Rakesh 91.3% 10 20 Gauna 91.2% 2 21 Craig 90.8% 18 22 Schackart 90.5% 4 23 Sacconaghi 90.4% 20 24 Abramowitz 90.4% 2 25 Marshall 90.4% 14 26 Ghai 90.2% 6 27 Fearnley 90.0% 8 28 Wu 90.0% 18 29 Moskowitz 89.8% 10 30 Hoffman 89.5% 6 31 Gardner 89.5% 16 32 Fidacaro 89.4% 14 33 Whitmore 89.3% 8 34 Haley 88.8% 2 35 Walkley 88.5% 4 36 Susanto 88.4% 2 37 Shope 88.2% 9 38 Ernst 88.2% 4 39 Huberty 88.2% 20 40 Um 88.0% 8 41 Munster 88.0% 22 42 Reid 87.9% 12 43 Reitzes 87.8% 22 44 Bachman 87.6% 12 45 White 86.8% 2 46 Sutherland 86.7% 2 47 McCourt 85.5% 12 48 Power 84.6% 2 49 Misek 84.3% 4 50 Wolf 81.7% 4 51 Bailey 80.2% 1 ------------ ------ --------- All bloggers 95.4% 110 All pros 89.4% 363 ------------ ------ Err. reductn 56.8%
A well deserved congrats to Turley, Jeff, and Dennis (told ya!), and to all bloggers for consistently getting it right where it matters most: the revenue and EPS forecasts. And a huge thank you to Philip Elmer-DeWitt and Fortune (as well as their readers) for making all this possible. Keep it up!
Posts
10 comments:
Of course, this begs the question why Wall Street firms aren't beating down your doors. Thanks for providing this, and keep up the good work :-)
@ David - I don't know. I ask that f'ing question every day. I'll take Huberty's job! Morgan Stanley can do so much better!
To Daniel - DAMN good article. WOW! What an effort. Thanks for this! Really. Thank you!
Great article. You put a lot of effort into this .
Thanks,
Well done for illuminating what we have all felt for the last few years..
Now, the next step is for Apple to recognise this and invite you three leaders to ask questions at the earnings calls instead of the daft professionals.
Great work
Yep we should nominate you as the representative of the afb. Poll a question and have you ask it to tc.
Daniel,
Just curious what is your take on the AAPL stock price in the next 2 years. I am long AAPL stock that I got in the ~ 200's, so I am good there and I am long term bullish on AAPL - of course!
The reason for my question is more with the 2013 way out of the money leaps that I own.
What are your thoughts. As much as I hate/ care less about technical analysis or even thinking about technicals, I just can't understand what is holding back AAPL stock for the last 3+ months?
Suddy, I'm not too fond of TA either (and long-term is irrelevant) but I see AAPL trading over $600 by Jan 2013, and during 2012 can't see it ever dipping under $450. Last 3 months is just AAPL trading silly, which sometimes happens in the short term. This silliness only affects those of us holding May-Jul 2011 options. Fortunately you're long shares and LEAPS13 (any strike currently available will gain nicely). If you're too scared you can always throw OTM puts for some protection. Or try on a collar.
Of course this is just my opinion and highly subjective guess, not guaranteed nor intended as financial advise. Good luck!
Excellent work Daniel.. nice consolidation of all the info.. Your hard work is most appreciated. Yeah - you told me alright!
Thanks,
Dennis
...and the ultimate question remains....why are these professional analysts moving in a herd and consistently below the amateurs? It all looks a bit staged. Perhaps the pros take the mindset that being (ultra)conservative and low-balling revenue/eps etc, etc every quarter for AAPL, they are being robustly cautious reflecting a mature and trustworthy mirror to their investors!
Thus, why the consistent discrepancy? We assume the pros have access to the same figures/data/parameters that the amateurs work with. Or is this being too simplistic on my part?
What a massive project. Thanks form all of us novices... JWP
Post a Comment