How a market crash affects index funds: A better way to measure tracking error

Published: November 3, 2020 at 12:45 pm

Last Updated on December 29, 2021 at 5:49 pm

In this article, we show how the NAV of an index fund or the price of an ETF is affected by a market crash and discuss a simple, effective way to measure tracking error. That is how well the fund or ETF was able to track the underlying total returns index.

Textbooks define tracking error as the standard deviation of “difference between the returns of an investment and its benchmark”. This is always a positive number and higher the value, more the deviation. What many investors fail to appreciate is, the tracking error depends on the time window chosen. That is tracking error over the last one year is different from that over the last two years.

This definition does not distinguish between underperformance  (index fund return less than benchmark return say over a month) and outperformance (index fund return higher than the benchmark). As pointed by Siva from Facebook group, Asan Ideas for Wealth (privately), expenses do not result in tracking error!

That is if you remove say, 1% annual fee from the Nifty TRI data (distributed daily), the return difference bet the bare data and the processed data is 1%, but the tracking error is zero. This is because there is no variation in return between the index data and our imaginary index fund.

Join over 32,000 readers and get free money management solutions delivered to your inbox! Subscribe to get posts via email!
🔥Enjoy massive discounts on our robo-advisory tool & courses! 🔥

This is because the tracking error is measured as a standard deviation. If you invest in a fixed deposit, the monthly return is the same each month. So the standard deviation is zero. The reason here is the same if the daily expense ratio is the same and there are no other factors that affect the fund management.

Thus the main reason for tracking error in an index fund is fund in and outflows and the difficulty in trying to copy changes to the index. These, in turn, depend on how easy or difficult it is to buy or sell stocks. Siva also showed that it is possible to construct an index fund NAV that has the same return as that of the index but with significant tracking error. The tracking error is non-intuitive and unsuitable for use by retail investors.

A fund with a low expense ratio can have a high tracking error: This has been established in multiple articles:

To make things worse, the return and tracking error of an ETF are measuring using its NAV data, while investors buy and sell at market price. This price movement can deviate significantly from the NAV movement especially during a market crash when there is a sudden drop in demand.

We plot below the rolling monthly return difference between Nifty 50 TRI and Nippon India Nifty Bees price (blue) and NAV (red). The price data is so huge that we had to multiply the NAV data by ten and it is still small! Stay away from ETFs unless you want to trade in real-time!

How a market crash affects index funds

Monthly return of Nifty TRI minus the monthly return of Nippon India Nifty Bees ETF NAV and Price (multiplied by 10 for clarity)
Monthly return of Nifty TRI minus the monthly return of Nippon India Nifty Bees ETF NAV and Price (multiplied by 10 for clarity)

Notice how much the tracking error measured with the NAV data misrepresents reality even far away from the crash! Every return that is mentioned online is based on NAV and not on the price! The huge upward and downward spike correspond to the March 2020 crash.

Ideally Nifty 50 TRI monthly return – Nippon India Nifty Bees ETF NAV monthly return should be positive. That is over a month, the index will always have a higher return primarily because of the expenses associated with the ETF (or any index fund). Out of the 469 monthly return differences studied, the ETF NAV return was higher than that of the index 196 times! This is clear evidence that difficulty in aligning the portfolio with that of the index is the main reason for this return difference – so much that it negates the loss due to expenses!

In our opinion, studying this return difference is a simpler and superior way than computing tracking error as a standard deviation. The ETF price return is expected to oscillate above and below that of the index return due to demand and supply fluctuations. This is welcome as long as the swings are small and bi-directional (both ways).

Let us now consider one of the worst index fund performers: Tata Nifty Index fund. Currently, this has the same expense ratio as that of Nippon Nifty Bees.  The negative blue dots represent monthly returns of fund which were higher than the index.

Monthly return of Nifty TRI minus the monthly return of Nippon India Nifty Bees ETF NAV and Tata Nifty Index Fund NAV
Monthly return of Nifty TRI minus the monthly return of Nippon India Nifty Bees ETF NAV and Tata Nifty Index Fund NAV

During a market crash, the negative deviation is sharp, but such deviations can occur on any given day.  As many as 148 such negative return differences were seen for the Tata fund and 196 for Nifty Bees ETF (with NAV).

A better way to measure tracking error

Instead of reporting a single always positive number as tracking error, we can quantify deviations from the index in the following ways:

  • no of times fund return or etf return (with price) was higher than then index for a month. This means the index – fund monthly return difference is negative.
  • The gap between the max return difference and min return difference
  • These along with few other quantities are tabulated below.
QuantityNippon India Nifty Bees PriceNippon India Nifty Bees NAVTata Index Fund-Nifty Plan(G)-Direct PlanFranklin India Index Fund-NSE Nifty(G)-Direct Plan
No of negative monthly return difference24419614887
Total monthly returns474469469469
median monthly return difference-0.01%0.00%0.03%0.05%
stdev of monthly return differnces0.72%0.03%0.25%0.17%
avg monthly return difference-0.13%0.00%-0.02%0.10%
max monthly return difference7.63%0.24%2.13%1.22%
min monthly return difference-4.86%-0.04%-1.17%-0.66%

If someone were to study the ETF NAV alone, they would be impressed with the gap between the max and min monthly return differences: 0.24% – (-0.04%) = 0.28%. Only if they bother to look at this difference based on the price they would appreciate it is as high as 12.49%.

Compared to this, the Franklin fund has fared reasonably well with a spread of 1.89% and only 87/469 instances of fund outperformance (negative return difference).  These quantities are intuitive and easier to understand than the tracking error. As one can see from the above table, the median and average of the return differences are quite different. This makes the standard deviation (conventional tracking error) an incorrect measure unrepresentative of the actual spread.

A monthly tracking error data sheet with this data will soon be made available. The top 2-3 funds in this list are not names you would see commonly in DIY forums 🙂

Do share this article with your friends using the buttons below.

🔥Enjoy massive discounts on our courses, robo-advisory tool and exclusive investor circle! 🔥& join our community of 5000+ users!
Use our Robo-advisory Tool for a start-to-finish financial plan! More than 1,000 investors and advisors use this!
New Tool! => Track your mutual funds and stock investments with this Google Sheet!
We also publish monthly equity mutual funds, debt and hybrid mutual funds, index funds and ETF screeners and momentum, low-volatility stock screeners.
Follow Freefincal on Google News
Follow Freefincal on Google News
Subscribe to the freefincal Youtube Channel. Subscribe button courtesy: Vecteezy.
Subscribe to the freefincal Youtube Channel.
Follow freefincal on WhatsApp Channel
Follow freefincal on WhatsApp
Podcast: Let's Get RICH With PATTU! Every single Indian CAN grow their wealth! 
Listen to the Lets Get Rich with Pattu Podcast
Listen to the Let's Get Rich with Pattu Podcast
You can watch podcast episodes on the OfSpin Media Friends YouTube Channel.
Lets Get RICH With PATTU podcast on YouTube
Let's Get RICH With PATTU podcast on YouTube.
🔥Now Watch Let's Get Rich With Pattu தமிழில் (in Tamil)! 🔥
  • Do you have a comment about the above article? Reach out to us on Twitter: @freefincal or @pattufreefincal
  • Have a question? Subscribe to our newsletter using the form below.
  • Hit 'reply' to any email from us! We do not offer personalized investment advice. We can write a detailed article without mentioning your name if you have a generic question.

Join over 32,000 readers and get free money management solutions delivered to your inbox! Subscribe to get posts via email!

About The Author

Pattabiraman editor freefincalDr M. Pattabiraman(PhD) is the founder, managing editor and primary author of freefincal. He is an associate professor at the Indian Institute of Technology, Madras. He has over ten years of experience publishing news analysis, research and financial product development. Connect with him via Twitter(X), Linkedin, or YouTube. Pattabiraman has co-authored three print books: (1) You can be rich too with goal-based investing (CNBC TV18) for DIY investors. (2) Gamechanger for young earners. (3) Chinchu Gets a Superpower! for kids. He has also written seven other free e-books on various money management topics. He is a patron and co-founder of “Fee-only India,” an organisation promoting unbiased, commission-free investment advice.
Our flagship course! Learn to manage your portfolio like a pro to achieve your goals regardless of market conditions! More than 3,000 investors and advisors are part of our exclusive community! Get clarity on how to plan for your goals and achieve the necessary corpus no matter the market condition is!! Watch the first lecture for free!  One-time payment! No recurring fees! Life-long access to videos! Reduce fear, uncertainty and doubt while investing! Learn how to plan for your goals before and after retirement with confidence.
Our new course!  Increase your income by getting people to pay for your skills! More than 700 salaried employees, entrepreneurs and financial advisors are part of our exclusive community! Learn how to get people to pay for your skills! Whether you are a professional or small business owner who wants more clients via online visibility or a salaried person wanting a side income or passive income, we will show you how to achieve this by showcasing your skills and building a community that trusts and pays you! (watch 1st lecture for free). One-time payment! No recurring fees! Life-long access to videos!   
Our new book for kids: “Chinchu Gets a Superpower!” is now available!
Both boy and girl version covers of Chinchu gets a superpower
Both the boy and girl-version covers of "Chinchu Gets a superpower".
Most investor problems can be traced to a lack of informed decision-making. We made bad decisions and money mistakes when we started earning and spent years undoing these mistakes. Why should our children go through the same pain? What is this book about? As parents, what would it be if we had to groom one ability in our children that is key not only to money management and investing but to any aspect of life? My answer: Sound Decision Making. So, in this book, we meet Chinchu, who is about to turn 10. What he wants for his birthday and how his parents plan for it, as well as teaching him several key ideas of decision-making and money management, is the narrative. What readers say!
Feedback from a young reader after reading Chinchu gets a Superpower (small version)
Feedback from a young reader after reading Chinchu gets a Superpower!
Must-read book even for adults! This is something that every parent should teach their kids right from their young age. The importance of money management and decision making based on their wants and needs. Very nicely written in simple terms. - Arun.
Buy the book: Chinchu gets a superpower for your child!
How to profit from content writing: Our new ebook is for those interested in getting side income via content writing. It is available at a 50% discount for Rs. 500 only!
Do you want to check if the market is overvalued or undervalued? Use our market valuation tool (it will work with any index!), or get the Tactical Buy/Sell timing tool!
We publish monthly mutual fund screeners and momentum, low-volatility stock screeners.
About freefincal & its content policy. Freefincal is a News Media Organization dedicated to providing original analysis, reports, reviews and insights on mutual funds, stocks, investing, retirement and personal finance developments. We do so without conflict of interest and bias. Follow us on Google News. Freefincal serves more than three million readers a year (5 million page views) with articles based only on factual information and detailed analysis by its authors. All statements made will be verified with credible and knowledgeable sources before publication. Freefincal does not publish paid articles, promotions, PR, satire or opinions without data. All opinions will be inferences backed by verifiable, reproducible evidence/data. Contact information: letters {at} freefincal {dot} com (sponsored posts or paid collaborations will not be entertained)
Connect with us on social media
Our publications

You Can Be Rich Too with Goal-Based Investing

You can be rich too with goal based investingPublished by CNBC TV18, this book is meant to help you ask the right questions and seek the correct answers, and since it comes with nine online calculators, you can also create custom solutions for your lifestyle! Get it now.
Gamechanger: Forget Startups, Join Corporate & Still Live the Rich Life You Want Gamechanger: Forget Start-ups, Join Corporate and Still Live the Rich Life you wantThis book is meant for young earners to get their basics right from day one! It will also help you travel to exotic places at a low cost! Get it or gift it to a young earner.

Your Ultimate Guide to Travel

Travel-Training-Kit-Cover-new This is an in-depth dive into vacation planning, finding cheap flights, budget accommodation, what to do when travelling, and how travelling slowly is better financially and psychologically, with links to the web pages and hand-holding at every step. Get the pdf for Rs 300 (instant download)