Data sources for DIY investors (updated May 2024)

Last Updated on September 24, 2024 at 3:39 pm

A reader says,  “Where do you get the data for your analysis discussed in this video? Index Investing In India TNIA Talk Part 2. I also want to make a similar analysis. Where can I get it? Can you please share the website?”

This list of free data sources can be used for DIY investing and analysis. It is the culmination of years of searching – the fun and frustration of hunting for reliable information online far exceed that of using a readymade list, but it might hasten better analysis.

Some of the resources can be auto-retrieved via macros. You can get these (and many other financial planning and analysis tools) by joining the freefincal investor circle. Some other ways of extraction can also be found in our Google Sheet-based MF and stock portfolio tracker and associated modules.

1. Historical price data of BSE indices, index ratios (PE, PB, DY), stock price history and a whole lot of other information can be found here: https://www.bseindia.com/market_data.html Please have the patience to scroll down and investigate each section. Total returns index information is not available here.
2. BSE indices total returns data plus data on any S&P index worldwide can be obtained here: https://www.spglobal.com/spdji/en/indices/equity/sp-bse-sensex/. You can add multiple indices to the chart and download them together. Please note the data is normalized for the last 10Y (max). You will not get absolute TRI values but only data rebased to a date 10Y ago. I use paid sources to get absolute Sensex TRI data.
3. NSE offers price, total returns and index ratios all at the same place (use the selector above the dates), and their new site allows you to download these from inception in one go: https://www.niftyindices.com/reports/historical-data
4. S&P 500 Total Returns can be obtained from Jan 1988 at Yahoo Finance: https://finance.yahoo.com/quote/%5ESP500TR/history.
   • Prof Schiller provides monthly data (inflation-adjusted S&P TR data, but this can easily be changed to unadjusted TR data) from the 1870s! Excel will be able to handle dates from 1900: http://www.econ.yale.edu/~shiller/data.htm
   • Limited data can also be found at https://ycharts.com/indicators/sp_500_monthly_total_return
   • Historical Returns on US Stocks, Bonds and Bills: 1928-2023 by Aswanth Damodaran. Thanks to Sekhar for the link.
5. Nasdaq 100 Total Returns from https://in.investing.com/indices/nasdaq-100-tr-historical-data. Google Finance has this data, too. You can retrieve it on a Google sheet via
   • =GOOGLEFINANCE(“XNDX”, “price”, “01/11/2006”, “9/07/2021”, “DAILY”)
6. Historical data of a wide duration of Indian govt bond yields can be found here: https://in.investing.com/rates-bonds/india-government-bonds. Click on the individual bonds and use the historical data tab.
7. Gold price from 1978 in USD, INR and other currencies: https://www.gold.org/goldhub/data/gold-prices. Free registration is required for download.
   • Sov Gold Bond Prices: https://www.icicidirect.com/fd-and-bonds/sovereign-gold-bonds-listing And https://sgbanalyzer.com/home
8. USD to INR exchange rate from 1978. The gold data in INR and USD can be used to obtain this.
9. Mutual fund NAV. For historical data (from April 3rd 2006), you can use https://www.mfapi.in/ or get it via AMFI using Excel. For daily NAV updates, you can use Excel to get it from AMFI or via Google Finance. See, for example, Google spreadsheet portfolio tracker for stocks and mutual funds.
10. Stock and ETF price history from moneycontrol: https://www.moneycontrol.com/stocks/histstock.php. I was informed earlier that the close price is adjusted for bonuses and stock splits but not for dividends. If you choose Yahoo Finance, the close price is adjusted for dividends and splits but not for bonuses!
11. Charts today are often interactive. That is, they are run with a script. You may be able to get this data from the XHR section of the Network tab in your browser console (inspect or ctrl + shift + I in Chrome). However, the legality of this will depend on the terms and conditions of usage of each particular site. So do check that before proceeding!
12. Data handled by scripts are often in JSON format. You can easily convert this CSV format using this site https://data.page/json/csv. It operates on a freemium model.
13. All sorts of macro-economic indicators can be found in the RBI’s database on the Indian economy: https://www.rbi.org.in/scripts/Statistics.aspx
14. Worldwide economic information is available at https://tradingeconomics.com/. Only a limited dataset is free, though.
15. https://www.ishares.com/ is a place to get international index and fund NAV data.
16. Moneycontrol for
    • daily NPS NAV (module available for users of the freefincal portfolio tracker),
    • mutual fund portfolio for overlap detection (module available for users of the freefincal portfolio tracker),
    • Stock balance sheets (updated tools coming soon)
17. NPS API for daily and historical NAV.
    • New easy to use source: npsnav.in
    • Not all schemes are updated daily, and the history is often incomplete. Use cautiously. https://purifiedbytes.com/nps/

If you can add more data sources, do send an email to freefincal [at] Gmail [dot]  com, or you can tweet to @freefincal or @pattufreefincal