Does Yahoo/Google no longer support web-scraping of FOREX data?

External question — Quantitative Finance Stack Exchange Author: ddm-j Original post: https://quant.stackexchange.com/questions/34249 License: CC BY-SA 3.0 — https://creativecommons.org/licenses/by-sa/3.0/ Adaptation: HTML converted to plain text; contact email addresses removed. I use Python 3.6 to develop quantitative FOREX trading strategies. I used to use Yahoo or Google in conjunction with pandas.datareader in order to collect forex data for day to day strategy development. However, lately, these the pandas API hasn't been working at all. I'm not sure if they changed the way FOREX data is accessed... Using something like EURUSD=X' or EUR=X`usually works as an argument for the 'name' of the security. I've also tried using ffn for python. It is a package that makes downloading data/backtesting very simple. However, I've found that even this has stopped functioning. Did yahoo and google change their data policies surrounding FOREX data? Or am I doing something completely wrong? The datareader and ffn are both trying to access this URL to no avail: Yahoo: http://ichart.finance.yahoo.com/table.csv

External answer — Quantitative Finance Stack Exchange Author: siegel Original post: https://quant.stackexchange.com/a/34253 License: CC BY-SA 3.0 — https://creativecommons.org/licenses/by-sa/3.0/ Adaptation: HTML converted to plain text; contact email addresses removed. They are trying to make this more difficult. In matlab, you can bypass it by pretending to be a chrome user using weboptions, and pretending to access from Chrome: https://www.mathworks.com/help/matlab/ref/weboptions.html options.UserAgent = 'Chrome'; Maybe something similar is possible in python?

External answer — Quantitative Finance Stack Exchange Author: Brian from QuantRocket Original post: https://quant.stackexchange.com/a/34260 License: CC BY-SA 3.0 — https://creativecommons.org/licenses/by-sa/3.0/ Adaptation: HTML converted to plain text; contact email addresses removed. Yahoo has changed their site structure. The new download URLs look like this: https://query1.finance.yahoo.com/v7/finance/download/MSFT?period1=1463461200&period2=1494910800&interval=1d&events=history&crumb=lHxk.yfuuzZ These links originate from pages like this: https://finance.yahoo.com/quote/MSFT/history?period1=1463461200&period2=1494910800&interval=1d&filter=history&frequency=1d They've switched to Unix timestamps for the date format, and the download URL now validates a cookie (the crumb param in the query string), which means that anyone who wants to scrape the data will need to deal with cookies. There's a fuller discussion at Elite Trader: https://www.elitetrader.com/et/threads/yahoo-historical-data-did-they-change-the-url-recently.309554/

External answer — Quantitative Finance Stack Exchange Author: Denis Alaev Original post: https://quant.stackexchange.com/a/34751 License: CC BY-SA 3.0 — https://creativecommons.org/licenses/by-sa/3.0/ Adaptation: HTML converted to plain text; contact email addresses removed. It seems that after several changes Yahoo Finance closed their API forever. The API was closed on May 15, one month ago already. I've tested several alternatives and found that  https://eodhistoricaldata.com  the best one for those who used Yahoo Finance. They provide raw data, adjusted closes and splits/dividends. They also have CSV output, with very similar format for Yahoo Finance users. Also there is a  https://intrinio.com/  data provider, looks good, but they much more expensive, have no data for Mutual Funds and API is very different in compare to Yahoo Finance. Then you need to significantly change your code.