Understanding the Nature of Stock Market Returns

Since we cannot change the way the stock market operates all we can do is to make sense of its returns and invest wisely.In this post I use Sensex data to understand the nature of its fluctuating returns (including dividends!). The Excel sheet attached which will be of interest and utility to all investors regardless of experience/expertise.

To say that many investors are sacred of the stock market is an understatement. Many already invested in stock and mutual funds are worried about seeing their investment value in the red for months together. Many who consider equity investments stay away when they see the Sensex plunge a few hundred points every other day.

The main reason for this fear and apprehension is fluctuating stock returns. This fear haunts everyone, from newbie investors to even ones holding good stocks or mutual funds. Understanding the nature of these fluctuating returns (or volatility) is crucial for disciplined investing. Volatility is a double-edged sword. It is essential to produced returns that beat inflation over a long period of time. Unfortunately too much of it too often will kill the power of compounding.

Before we proceed, the difference between risk and volatility must be emphasized. Risk either refers toloss of capital or loss of value (due to inflation). Volatility refers to fluctuating returns. When you invest for a short term goal you cannot afford volatility in your portfolio leading to loss of capital. When you invest for a long term goal you have a choice:

  • If you do not want loss of value risk then you MUST embrace volatility in return. This by its  very nature will lead to short term loss of capital. You will have to get used to it!
  • If you cannot stomach short term loss of capital risk you stay away from volatility in return and therefore you embrace loss of value risk. This is even more worse!
  • In plain English: If you don't want inflation to erode the value of your money invest in stocks (or MFs), get used to fluctuating returns and stay invested.

The first step: quantifying fluctuating returns:  Consider a good old FD offering a return of 10% (compounded annually). If I invest Rs. 100 in this for 3 years my money will compound in the following way:

End of year 1: I get 100 x (1+10%) = 110

End of year 2: I get 110 x (1+10%) = 121

End of year 3: I get 121 x (1+10%) = 133.1

Fantastic! Now what is the ‘average’ rate of return? The answer that begs to be heard is 10%! Our mind automatically calculates (10%+10%+10%)/3 =10%. To check if this answer is correct we calculate: 100 x (1+r)3 = 133.1. This is the simple compound interest formula. Here r =10%, the ‘average’ return we have calculated. Thus our answer matches with the final obtained after 3 years and is no doubt correct. However this method of calculating average return is not applicable for all situations.

I invest Rs. 100 in an equity mutual fund and get the following sequence of returns: +10%, -10% and 25%. My money would have compound the following way:

End of year 1: I get 100 x (1+10%) = 110

End of year 2: I get 110 x (1-10%) = 99

End of year 3: I get 99 x (1+25%) = 123.75

If I calculate the ‘average’ return it is: (10%-10%+25%)/3 =8.33%. When I cross check the answer: 100 x (1+8.33%)3 = 127.14.

This answer is quite different from the actual value of 123.75. The key takeaway is: the moment returns vary from year to year the usual formula for average (r1+r2+r3)/3 (also known as arithmetic average or AM) does not work. We need an alternative way to compute the average.

The alternative average known as the geometric average is given by

r = [(1+r1)  x (1+r2) x (1+r3)] (1/3) -1.

Here r1,r2, and r3 are interest rates for each year. When compounding is involved, the geometric average (GM) is referred to as CAGR (compounded annual growth rate). For the 3 year equity investment, r(GM) = 7.361%. When I plug this into the compound interest formula, I get, 100 x (1+7.361)3 = 123.75,  the correct value.

The GM formula will work for all situations that involve compounding. For the FD investment, r1 =r2 = r3 =10% with GM of 10%. When returns are constant each year, AM = GM. When returns vary year AM will always be greater than GM. Therefore the difference between AM and GM can be used as a measure of volatility (for a more precise definition you could consult this resource). So let us define volatility = AM – GM. Higher the difference, higher the volatility.

To get a sense of stock market volatility, we compute the difference between AM and GM with Sensex returns over say, 3, 5, 7,10,12,15,20 and 25 year periods.  Since there are many 3 year periods from 1978-79 (base year) to 2012-13, we roll over the 3 year period. That is we first calculate the 3 year AM and GM for financial years (FY) 1, 2 and 3 and move onto FY 2,3,4 and then to FY 3,4,5 and so on. We then average (arithmetically!) the AM and GM values over all 3 year periods. This is also done for other durations.

The average volatility (AM – GM) for 10,12,15,20 and 25 year periods is approximately 10%. The volatility is 9% for 7 years, 8.5% for 5 years and 7.3% for 3 years. Thus regardless of when we measure and regardless of how long we measure it for, the volatility of the market is nearly constant! It is incorrect to think that the market has behaved differently for last few years only. It has always behaved the same way! So there is no right or wrong time to invest – only the right or wrong investment.

The 15 year average rolling return of the Sensex is a number used by financial experts to convince people that time in the market is important. They say that the stock market has given impressive returns over 15 year periods and that the probability of losing money is zero. What do these statements really mean?

To understand this, we first look at the 3 year average GM. This is 18.4%, a fantastic rate of return. Unfortunately the standard deviation (a measure of how much individual returns that make up the average deviate from the average) is 20.9%. This means that if I consider 100 stock marker investors about 68 of them will have returns ranging from 18.4% –20.9% to 18.4% + 20.9%. A fancy way of saying that if you are in the market for only 3 year your returns could basically be anything!

Now consider the 15 year average GM(CAGR). This is 15.8%. The standard deviation is 5.4%. So now 68 out of 100 investors who have stayed invested for 15 years will have returns ranging from 15.8% – 5.4% to 15.8% +5.4%.

Notice that because of a lower standard deviation the fluctuation in return is much, much lower.  In other words, when it comes to the stock market time is return! The longer you stay the lower your returns are likely to fluctuate. If you don’t say invested it is very likely that you will loose not only money but also the opportunity to invest when the market hits a low.

Standard_deviation_diagram.svg
The Normal distribution Here μ refers to the average and σ to the standard deviation. Read through the text and then see the connection to this picture.

Okay 68% of investors are likely to get decent returns (15.8% ±5.4%). What about the others? About 14% of investors may get luckier than the 68% and get returns ranging from 15.8% + 5.4% to 15.8% +10.8% (10.8% is twice the standard deviation). Another 14% may fare worse than the 68% and get returns ranging from 15.8% – 10.8% to 15.8% – 5.4%. These statements are subject to certain conditions (see below).

No one is likely to loose money (wow! What great comfort!). About 90% of investors are likely to get returns that beat inflation. This is not bad at all. It is important to keep in mind that just because you stay invested in the stock market for long periods of time you are not guaranteed returns that beat inflation. The only thing guaranteed about the market is its volatility!

Ghosts of Scandals Past In FY 1991-92 the Sensex returned an astonishing 267% due to the Harshad Mehta Scam. It corrected the next year by -47% (thanks to Subra for pointing out these facts). Even for the stock market such a large positive return is an aberration. What do you think will the long term CAGR of the market be if the scam did not occur? We can’t turn the clock back and avert the scam but we can certainly see how the long term average will fare if the 267% is replaced by something tamer, say 15% (let us not worry about -47%. Although related to the scam such negative returns are not uncommon!). Why do this?  The impact of the scam will provide a natural answer.

The 15 year average CAGR is 15.8%. If I replace the 267% return by 15%, the average CAGR reduces to 10.1% (a 36% decrease!) with a standard deviation of 3.59% (a 34% decrease!). Is this good or bad? The way I see it, if no such scan occurs in future the stock market will be less volatile than it was in the past. Yes returns will be significantly lower but it is more important that volatility is lower. How will you interpret this?

The Excel sheet which has the above mentioned analysis can be downloaded from the link below.

  • The annual Sensex returns listed (source: RBI) do not factor in dividends. You can enter a nominal dividend (Subra likes 2%) to see how it will influence the CAGR. Including dividends is Subra's idea.
  • By including a 2% dividend the 15 Y average CAGR increases from 15.8% to 18.1% (the stand deviation increases marginally). How big a change do you think this is?
  • Conditions Apply! The above analysis assumes that the Sensex returns follow a Normal Distribution. The normal distribution is God’s own distribution. Any measurement if performed often enough and long enough is expected to follow a normal distribution. Is this true for the Sensex? Quite debatable! I would say it is about 70% true!!
Download the Stock Market Returns Analyzer
  • I would love to hear your thoughts on stock market volatility and how you cope with it.
  • Important: Did you find this post too mathematical and difficult to understand? If yes, then you have two choices: (a) find something more appealing to read and/or (b) please drop everything and seek help from a financial planner if you haven’t done so. Why?
  • Why? Because mathematics rules the world. From the small movement of your index finger which got you to this page to the rotation of our galaxy. That is the way God made it! So read up or ….

Install Financial Freedom App! (Google Play Store)

Install Freefincal Retirement Planner App! (Google Play Store)

book-footer

Buy our New Book!

You Can Be Rich With Goal-based Investing A book by  P V Subramanyam (subramoney.com) & M Pattabiraman. Hard bound. Price: Rs. 399/- and Kindle Rs. 349/-. Read more about the book and pre-order now!
Practical advice + calculators for you to develop personalised investment solutions

Thank you for reading. You may also like

About Freefincal

Freefincal has open-source, comprehensive Excel spreadsheets, tools, analysis and unbiased, conflict of interest-free commentary on different aspects of personal finance and investing. If you find the content useful, please consider supporting us by (1) sharing our articles and (2) disabling ad-blockers for our site if you are using one. We do not accept sponsored posts, links or guest posts request from content writers and agencies.

Blog Comment Policy

Your thoughts are vital to the health of this blog and are the driving force behind the analysis and calculators that you see here. We welcome criticism and differing opinions. I will do my very best to respond to all comments asap. Please do not include hyperlinks or email ids in the comment body. Such comments will be moderated and I reserve the right to delete  the entire comment or remove the links before approving them.

50 thoughts on “Understanding the Nature of Stock Market Returns

  1. k s

    I agree that the only thing to expect in the market is volatility. I disagree that it is normally distributed. It has been repeatedly proven (by the periodic occurences of 1 in 100 year events every 5 years) that market behaviour is not normal. I would strongly suggest you read mandelbrots - (mis) behavior of markets.

    Reply
    1. pattu

      Of course I know it is not normal. When I write I assume I am not preaching to the choir. A non-zero skew is statistics 102 not 101.

      I think the Indian market is still way too young to be pegged down to one particular distribution.

      For the non-expert understanding volatility in terms of a Normal distribution is more than enough. The idea is to encourage someone to stay invested. Nothing more nothing less.

      Thanks for your response.

      Reply
  2. bemoneyaware

    Very interesting and number crunching post. Reminded me of my college days 🙂
    How do I deal with market volatility
    by deciding how much I am comfortable investing in stock market directly or indirectly(mutual funds)
    by investing only that part of money which I need for long term(10 years).
    Not following the stock market daily.

    Reply
    1. pattu

      Thanks. Your comfort level is often secondary to your goals comfort level. For example if a 22 year old starts investing for retirement at 60 he can afford to stay away from equity even at 8% inflation. A 32 year old starting out needs quite a bit of equity exposure and cannot afford to say I will not touch equity.

      First one needs to determine how much one can invest. Then determine average rate of return for the corpus reqd. The fix the debt return and assign a comfortable debt allocation %. This fixes the equity return. If this unreasonable (to me anything more than 10% is unreasonable) I will adjust the proportions until I get some equity return expectation I am comfortable with and proceed with investing.

      I follow the stock market daily. I just don't act on it. Once you understand the difference bet risk and volatility, the daily ups and downs will not bother you.

      Reply
  3. Rakesh

    Pattu,

    Very informative article. People don't realize that equity is for a long term, they want quick returns. If one would see MF SIP returns for the last two years they would have hardly beaten FD returns. And then people think they have lost interest for two years and they were better off investing in FD's.

    Rakesh

    Reply
  4. sreekant

    Great post. Ben Graham reportedly said "You understand value investing in the first 5 mins or you would never". Similarly, one understands equity investing after reading your post or he would never. Great work... keep them coming.

    Reply
  5. Arun

    Me thinks its extremely risky to predict long term returns from equity markets (stocks for the long run, Jeremy Siegel, not withstanding). Imagine you were a long term investor in 1980's in Japanese market (Nikkei 225). It reached a peak of 39K. Well, its been 25 years and you will still be waiting to get your money back. Or you will be dead (in the long run we are all dead)!. Or for that matter if you invested in Nasdaq during the dot com bubble. Here, I am specifically talking about the 'buy and hold' strategy. So I think its important to reduce the equity exposure and allocate a good chunk in debt too (whatever your age, young or old). Anyways, its a major predicament and I always remember this quote from Harry Markowitz (father of modern portfolio theory): "“I visualized my grief if the stock market went way up and I wasn’t in it — or if it went way down and I was completely in it. So I split my contributions 50/50 between stocks and bonds.”

    Good post on "the permanent portfolio" too!

    Reply
    1. pattu

      Hi Arun, Thanks for your observations. I fully agree with you. Asset allocation must be carefully chosen depending on age, goal time frame, time for monitoring and rebalancing, aptitude, appetite etc.

      Reply
    2. abvblogger

      One has to look at multiples & valuation when entering. The Japan example is oft-quoted. It ignores both valuation and diversification responsibility. Who would enter equities only in Japan and in a lump-sum with a peak valuation? If the answer is, a Japanese investor with patriotism and optimism for his country, then one must knock on the doors of the psychology department of universities, not the finance department, to answer for the mess that ensues 🙂 Too many investors are convinced that their country is somehow magical or 'safer' and will do much better than the rest of the world. This is availability bias. I never tire of saying that mathematically, India is just 1 country.

      Reply
      1. pattu

        Could not agree more. Unfortunately, I realised this only recently. If had international equity exposure in 2009, my portfolio would have been much more healthier now. May I trouble with a request to write another guest post on, "how to build a well diversified equity portfolio"?

        Reply
  6. bharat shah

    i also read about Nikkie's movement. but then, as a layman, i am confused , how Japan created wealth all these long yrs to be one of wealthy nations? similarly for USA , as i understand, its markets are not performed that great for some times , but investors like Warren Buffet made a fortune. even in India, in last 5 f.ys(07-12) some 100 companies' shares made wealth creation at average of @20% CAGR for the share holders v/s 6% CAGR of sensex companies. so there seems more than ------. as regard to the post, i am just refreshing my maths to understand it.

    Reply
    1. pattu

      The Nikkie is given as a scary example to the long term index investor. Intelligent stock picking has and will always given profits. Unfortunately, I know nothing about it.

      Reply
      1. bharat shah

        thank you for your honest reply. i heard people saying :picking good stocks for investment needs not rocket science!

        Reply
        1. pattu

          I also agree that stock picking is not difficult. However,you need to understand the business of the stock and where it is heading. I have never been able to do this properly. So I stay away from direct equity. I have a small exposure to BHEL and nothing else.

          Reply
    2. abvblogger

      Markets price in expected earnings. If you expect a 100cr profit with 10% annual growth over 20 years, and what you actually get over the next 20 years is 50cr with 5% annual growth, you will see the price go down, down and down.

      Whereas, 50cr might be excellent performance. It might represent a very high return on equity. It might even be better than all the other companies out there. But it didn't meet the expectations embedded in the price. A country can grow rich, and its employees be paid high wages, and its companies perform well, without making stock market investors rich. That is because standard of living and wealth are absolute measures, whereas stock returns are relative measures: it is performance relative to expectation which is rewarded, not performance per se.

      Markets move to the tune of positive surprise. And people with high expectations are very difficult to surprise positively. People with depressed expectations are easy to delight. Yet another reason to look at multiple values before entering.

      Reply
        1. abvblogger

          I don't pick stocks either, at least not qualitatively. I use quantitative screens, which are basically passive investing with a slightly active bent. With respect to looking at earnings expectations, one can do so by looking at the index PE (price earnings ratio). This is a pretty good indicator of the market's value and the daily data is available free for public download into excel from BSE & NSE websites.

          Reply
          1. pattu

            Yes I know. I am working on a PE correlation illustration. However I still have some kind of decision phobia when it comes to stocks.

          2. pattu

            Yes. I have quite a bit of large cap exposure via MFs. So I was thinking of buying a small-cap or mid-cap stock. So I have trouble picking it in the first place!

            I like the way you use the index to buy stocks. i think it is fantastic. It would be great if you could share here in more detail.

          3. bharat shah

            interesting talks, but i think , ABVBLOGGER seems settled for index funds at timing by PE , and you seem to seek the details for entering mid small cap equity!

          4. abvblogger

            @bharat shah I am not a pure indexer yet because I don't mind implementing the academic research from capital markets. Value investing is a proven strategy, so the EV-EBITDA weight is my own tweak on a conventional index (I got the idea from Joel Greenblatt). But yes, I don't pick individual stocks.

          5. abvblogger

            I pick CNX Midcap and CNX Smallcap because they are broad (100 stocks) and liquid. I used to filter midcaps and smallcaps based on my own criteria and screens but this was very frustrating because after all that work, the stock might turn out to be illiquid. I'm not the sort of person who really wants to pick stocks. I like holding broad diversified baskets because they are basically like index investments.

            So I take CNX Midcap & Smallcap, weight them equally, and buy. I will rebalance the weights annually to avoid tax. It is that simple. I also implement an EV-EBITDA weight index for CNX midcap & smallcap stocks which have low debt. This borrows from the philosophy of value-investing.

            I only plan on doing this until Indian institutions offer good low cost ETFs / index funds which are liquid.

            For my large-cap needs I use CNX 500. It is 75% large cap, about 25% the rest (due to free float market cap weight). I believe a large-cap fund should capture not just existing large caps but also potential large caps, and the best place to look for the next 50 to 100 large companies of the future is CNX 500. As a company grows, the index will reward it with greater weight. CNX 500 has higher risk than Nifty, but I think it is a much better long-term bet. If India grows to even half of what macro-economists are predicting for the next 30 years, then we will not be having just 50 large caps.

          6. abvblogger

            Btw, I don't buy all CNX 500 stocks! That would be so much hard work... No, I just buy Goldman CNX 500 Direct Plan. It's my core portfolio holding, and the one I have the maximum long-term faith in, though I know it will be subject to short-term volatility.

          7. abvblogger

            As for timing, well, I think as long as PE is not far above the long-term average, I am happy to buy. Today, for the Sensex, that is anything between 17,000 to 19,000. But even if the PE were slightly higher, and the sensex were at 20,000, I would keep buying, just in smaller quantities and focusing more on the value-investing style, until I hit my target asset allocation. Simply because it is mathematically important to be invested sooner rather than later. It is only the emotion which makes it difficult to see dips. I would only consider stopping buying if PE is above 22 (which would approx be Sensex at 25,000 today).

  7. bharat shah

    thank you. for sake of argument, the fund managers, where we invest, do really understand all businesses of the stocks of the funds and where they are heading, and doing it properly, or do they decide from the published accounts of the companies? is it not shy away to take a solace or fear in case of our portfolio not doing that great? of course individual compared to mf amc can not gather the data economically for picking and tracking , but still it is feasible due to internet now compared to 10 yrs. ago particularly for person of your caliber and knowledge , as i think. at least one who is investing equity mf for some time can enter-prune with part of fund for direct equity , and then proceed further , if found satisfactory. this is not advice , but self talk to myself!

    Reply
    1. pattu

      Yes of course I understand what you are saying. I think fund managers must have a formal training in stock picking. Yes once we have equity MF exposure for a few years then direct equity is a good option provided one makes the effort to read and learn. Just too lazy to do this!

      Reply
  8. AyushP306

    Very nice and helpful post. It is very well explained. Till now i too used to calculate average returns of the fund by seeing its yearly returns in Value Research with arithmetic average method. I had a gut feeling that it is wrong but didn't knew the right way to do it. But now i know it does not work when returns vary from year to year. The right way to do it is to calculate the geometric average otherwise known as CAGR.

    Reply
    1. pattu

      Yes Ayush. The right way to do is GM with geometric standard deviation. Unfortunately, G-stdev is a product and difficult to make sense of. So I have to resort GM and arithmetic stddev. Easier to understand but technically wrong!

      Reply
      1. mono

        GEOMETRIC MEAN
        TO MAKE SENSE OF IT...
        THINK LIKE YOU HAVE 2 CHILDREN
        AFTER 10 GENERATIONS
        WHAT WILL BE THE AVERAGE POPULATION
        THAT IS GEOMETRIC MEAN

        Reply
      1. mono

        NORTH INDIANS MAKES MAZAK OF YOU..
        CALL YOU BLACKY'S
        WHY DO YOU EVEN BOTHER TO REPLY THEM
        FIRST GENERATE SOEM SELF RESPECT IN YOURSELF
        THEN GENERATE YOUR INTELLIGENCE

        Reply
  9. mono

    rubbish
    INFLATION ..IS DUE TO CORRUPTIONS..
    STOCK MARKET LOOSES ARE ACTUALY DUE TO FRAUDS..
    WHICH IS DUE TO CORRUPTION AGAIN...
    SO DON'T INVEST IN STOCKS UNTIL CORRUPTION IS THERE
    YOU CAN DO NOTHING ABOUT INFLATION EXCEPT TO
    CURB CONSUMPTION
    OR BUY FROM WHOLESALE FOR NECESSARY THINGS

    Reply
  10. mono

    IF YOU WANT TO EARN USING STOCKS
    THEN
    instead of becoming BULLISH
    BECOME BEARISH...

    sell and buy
    instead of buy and sell

    Reply
  11. mono

    how to earn using stock markets

    think like this that your stocks is like an investment..
    ( like you have bought a shop for selling ladoos )
    and sell and buy them many times....
    when price falls....

    if the price rises don't sell them...
    until that price shoots beyond your initial investment
    ie your first price...

    problem only arises
    when price become stagnant
    and for a long time
    which i have never seen

    Reply
  12. Paresh

    Dear Pattu,
    Very well written as usual. The intent obviously being to educate and not impress. I have used some of your calculators , followed your advice and
    built up my goal based portfolio accordingly.

    A lot of what you write resonates with people like me . I would not hesitate to contribute if required . Please let me know if i can.

    Thanks again for all your efforts.
    Sincerely
    Paresh

    Reply

Do let us know what you think about the article