| ||||||||||||
| ||||||||||||
|
![]() Portfolio Opportunity Distributions (PODs) comprise all of the portfolios that could have been held by an investment manager adhering to a specified mandate, such as the S&P500 or the Russell 1000 Growth. These comprehensive, scientifically designed peer groups solve the serious problems associated with traditional peer groups and indexes by combining the better characteristics of each in a manner that eliminates the shortcomings of both. Unlike traditional peer groups, PODs are timely, unbiased and accurate. Unlike traditional indexes, PODs determine statistical significance for periods much shorter than decades.
Popular Index Portfolio Opportunity Distributions (PIPODs) unify the better aspects of peer groups with those of benchmarks to create a performance evaluation background that is fairer, more accurate, much more timely than current approaches, and it provides indications of significance that are unattainable elsewhere. Please read the following article for more details. Unifying Best Practices to Attain Best-of-Breed Investment Performance Evaluation How do you evaluate investment performance? Good chance that you use both peer groups and benchmarks. A recent survey of 700 consultants and investors, conducted by the Investment Management Consultants Association (IMCA), found that 90% of respondents use peer groups to evaluate investment performance and 95% use benchmarks. In other words, most evaluators use both peer groups and benchmarks.
Benchmarks solve most of the problems with peer groups, but come with a unique and serious problem of their own it takes decades to determine with confidence whether the manager is actually skillful or not. If youre using a benchmark, sooner or later youre going to want to know if that 2% return above the benchmark is a big deal or not.
The median of a PIPOD peer group is the return on the index, and the percentiles around the median are indications of the significance of success or failure. A ranking in the top decile of a PIPOD universe says that there is a 90% probability that skill, not luck, was involved, regardless of the time period. And the answer to the question What portfolios are in a PIPOD universe? is All of them., so you are assured of a fair and accurate evaluation. Some examples will demonstrate the benefits of PIPODs. Lets start with significance. If your manager underperformed his/her benchmark by 5% in the 4th quarter of 2002, would that be really bad or just sort of bad? How significant would it be? Well youll probably say that it depends on the volatility in the managers style, as represented by his/her benchmark. 5% underperformance in a very conservative style would be more disappointing than the same underperformance in a very aggressive, volatile approach, where theres an implied greater tolerance for risk. But where do the lines get drawn? The following exhibit delivers the answer.
A return of 2.5% lagged the Russell 2000 Small Cap Growth index by 5%, but because this is a volatile mandate, this underperformance is not significant, ranking in the 75th %ile. By contrast, underperforming the more conservative, large company, S&P500 by 5% is significant at the 94% confidence level a big deal, indicating a significant management mistake, not bad luck. Dont try this at home kids, unless you have PIPODs. PIPODs solve the waiting-time problem inherent to benchmarks. In the example above we determine significance at a very high confidence level for a short period of time, namely one quarter. Benchmarks require decades to draw similar inferences. Now heres another example: choosing the right benchmark. Consider the manager shown in the next exhibit, who is being benchmarked against the S&P500, everyones favorite benchmark. In the IMCA survey, 96% said they use the S&P500, making it the most popular choice.
For periods of 3 quarters or longer it looks like this manager is sensational, off the tops. But wait a second. If PIPODs are all of the portfolios that could have been held using stocks in the index, this manager must have held stocks outside the index, and plenty of them the manager is not managing to the S&P500. Its the only way the manager could be off the tops. Well heres the reason. We fabricated this example to make a point, by using the median returns for small cap value, as shown in the next exhibit. The point is that a mediocre manager can look really good, or bad, when compared to the wrong index.
As you can see, if it looks too good (or bad) to be true, it probably is. The important thing is that you get the most accurate look that you can. PIPODs deliver fairness and accuracy by letting you select the best benchmark for your manager, and by feeding back some reasonableness checks for your consideration.
Ron Surz (Ron@PPCA-Inc.com) *The analogy that's frequently used to describe survivorship bias is the marathon with 1000 runners and 100 finishers. Is the 100th finisher dead last, or in the top decile? He's in the top decile.
John O'Brien originated this approach at A.G. Becker in the 1970s, and recently suggested the adaptation to popular indexes. John is the executive director of the Master's Program in Financial Engineering, and the adjunct professor at the Haas School of Business, U.C. California at Berkeley. | |||||||||||
![]() |