Feature selection for time series prediction

External question — Data Science Stack Exchange Author: Leandro Ercoli Original post: https://datascience.stackexchange.com/questions/36823 License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/ Adaptation: HTML converted to plain text; contact email addresses removed. I'm working on an LSTM-based stock market forecasting problem and trying to figure out a way to select input variables. When calculating correlation between variables (e.g. Close price of Tesla vs Close price of Microsoft), would differentiating the curves give a more accurate (or correct) correlation index ? I'm finding values in the range 0.7-0.9 for non-differentiated variables, and lower values after differentiation. Once I have a correlation matrix of all my variables, is there a way to figure out which ones would add information to the neural net and which ones would just add noise ?

External answer — Data Science Stack Exchange Author: Ryan Ghorbandoost Original post: https://datascience.stackexchange.com/a/36832 License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/ Adaptation: HTML converted to plain text; contact email addresses removed. You don’t need to select variables for feeding to network, deep neural networks (DNN) will do this automatically. Actually DNN gives more importance to relevant variables by setting its weights. After setting the weights, some of the hidden nodes take 0 and some of them take 1 (because of sigmoid function). You can think of this 1 and 0’s as choosing relevant variables, too. By the way, correlation matrix can not be used to select relevant variables directly. If you want to reduce the number of variables that are fed to DNN, you can use PCA. Actually PCA components are calculated by getting the Eigen-vectors of correlation matrix.