What are mutual fund star ratings (in plain English)?

Last Updated on December 29, 2021 at 11:55 am

This is a simple, non-technical description of what are mutual fund star ratings and how they are calculated with the help of an analogy – employee performance appraisal. This is meant for beginners and newbies and I have avoided all jargon. It is only when we understand how star ratings are calculated, can we understand their limitations and why they should not be taken seriously.

Imagine that you are working in a company of about 500 people. When the results of the last performance appraisal showed itself in the next month’s paycheck, there was anger throughout the office. Almost everyone felt that the appraisal was unfair – they did not get their due and/or people who were not as efficient as they got a higher pay hike etc.

So the management decides to change the way it rates performance and put you in charge (maybe because you cribbed the loudest!). How would you go about it? Yes, mutual funds ratings are calculated in a similar manner to employee performance. In this post, I shall avoid the fancy metrics that the ratings use and use plain English. In the next post, we shall discuss how the metrics are used.

What are mutual fund star ratings? An analogy with employee appraisal

Step 1: Group them into categories: Accounting, human resources, front-end services, back-end services, administration, custodian, security etc.  The appraisal of each category is considered to be independent of the other (although in real life this is not true)

Step 2:  Group each category into sub-categories. For example, the nature of each service performed each forms a sub-category. Each sub-category is also evaluated independently (again an over-simplification). The assumption here is that the employees that are part of a sub-category all perform the same service (also an over-simplification)

Step 3: If there are very few employees in each sub-category, less than say 15, you tell the management that meaningful evaluation is not possible (so everyone here gets the same raise yay!)

Step 4: You need a performance benchmark for each sub-category. Only the paymasters can decide this regardless of how fair or unfair it is. So you ask management to provide you with this in consultation with an employee committee. The benchmark tells you the no of reasonable hours necessary to complete any service associated with a sub-category. Delays allowed due to human errors and system breakdowns are factored in. The expenditure per hour the company is willing to pay for each job and the revenue per hour that it expects.

Step 5:  Now that you have a benchmark, your job becomes easy. You consider each sub-category and list the no of hours it takes for each service to be completed. Arrange this in ascending order.

Step 6:  You find the top 10% of employees who take the shortest time to complete a job, give them 5-stars. The next 22.5% get 4-stars, the next 35% get 3-stars, the next 22.5% get 2-stars and the bottom 10% get 1-star. Like this.

There you have it! Your star ratings! Wait a minute, an employee who does something fast does not mean an employee who does something right. Someone who finishes a task in a hurry only for clients to complain it is filled with errors is not what a company wants. Delays will cost money, reputation and profit. So the above ranking ignores the associated risk.

So you find out a way to rank employees in terms of those who satisfied clients the fastest (for the sake of our objective, let us assume that this is possible). This, you argue implies lower expenses for the company and potentially higher revenue as the client is likely to return for more. You call this risk-adjusted rating.

This is done in two steps. First, you rank employees in terms of speed of completion. Call this the speed rank. Then you rank employees in terms of mistakes detected in the review stage, mistakes detected by the client, delays that they are principally responsible for etc. Call this the risk rank. 

risk-adjusted performance rank = speed rank – risk rank.

You rank employees in terms of their risk-adjusted performance rating and offer 5-stars to the top 10%, 4-starts to next 22.5% and so on as above. This is essentially how mutual fund star ratings work. The point here is a slow worker who finishes without mistakes will be rewarded more than a fast worker who had to re-check his work because the client was not satisfied.

How mutual fund star ratings work

Now let us complete the analogy with mutual funds.

Step 1:  Separate mutual funds into categories.

Step 2:  Divide each category into sub-categories. This is the most crucial step. If this is wrong everything else is wrong. Unfortunately, this is always wrong!!  It is impossible to group mutual funds in such a manner. There will always be differences in their investing styles. Similarly, it is also impossible to bin employees this way, even if they perform similar tasks.

In all fairness, after the SEBI categorization rules, this has become a lot easier, but it can never be perfect.

Step 3:  The risk-adjusted performance rank as described above is also carried out for mutual funds (more details in the next post). The rating is calculated once a month for the last 3 years, 5 years, 10 years etc.

Step 4:  The problem is that if the fund has changed investment style in the last 6 months, then the previous 2.5-year performance data is of no use anymore! Star rating portals routinely ignore this and lazy investors don’t care. It is like an employee being re-assigned to a new division and being judged by a set of rules applicable only to the old division.

The analogy mentioned above is rather crude and modified to suit the mutual fund rating itself. It could well be incorrect as I have never been the subject of such appraisals myself (thankfully!). In terms of the math used, performance evaluations are similar to fund star ratings and that is the only point I wish to convey here (plus the idea of risk adjustment).

The point here is that although these ratings use math, they begin with crude assumptions and arbitrary binning.  One of the many reasons why investors should not be using them. If mutual funds could speak, they would be cribbing just as much as humans do after seeing their appraisal ratings.

Glossary

Company performance benchmark  <==> mutual fund benchmark

speed rank <==> mutual fund return rank

risk rank <==> mutual fund risk (well, technically volatility) rank

risk-adjusted performance rank <==> mutual fund star rating

Further reading on mutual fund star ratings

I have complained long and hard about ratings and as usual, they fall on deaf ears

Here is why you should ignore mutual fund star ratings

Part II: Here is why you should ignore mutual fund star ratings

The Trouble With Mutual Fund Star Ratings

Mutual Fund Star Ratings are Flawed, but Investors are to blame for taking them at face value

Dangers of relying on mutual fund star ratings and past performance