Multivariate, multistep forecasting with LSTM

External question — Data Science Stack Exchange Author: Aurast Original post: https://datascience.stackexchange.com/questions/37581 License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/ Adaptation: HTML converted to plain text; contact email addresses removed. I want to use an RNN with LSTM to forecast multiple steps into the future, based on multiple inputs. I have some ideas for different ways to approach this, but I'm afraid I'm missing the "right way" to do it. Please let me know if any of these approaches are generally better or worse than others, or if I'm missing any. More specifically, I have ~1,000,000 periods of financial data to train on: a stock closing price (X), the value of stock indicator 1 (Y), and the value of stock indicator 2 (Z). I want to train on that data and predict the closing price (X) up to and including 10 periods in the future. Here's how I can imagine approaching this: 1. Separate models for predicting each input value Have one model that predicts the next X based on the previous X, Y and Z. Have a second model that predicts the next Y based on the previous X, Y, and Z. Have a third model that predicts the next Z based on the previous X, Y, and Z. Use the predictions from these three models as the inputs for the subsequent step. 2. Separate models with different period width Train one model on every period in the training data, use it to predict one period into the future. Train a second model on every second period in the training data, use it to predict two periods into the future. Train a third model on every third period in the training data, use it to predict three periods into the future. Etc 3. Train one model that predicts a vector of ten values, each representing the predicted X at each period up to ten periods into the future. 4. Train one model that predicts a vector of three values, each representing the predicted next X, Y, and Z. Use them as inputs for the subsequent step.

External answer — Data Science Stack Exchange Author: petr Original post: https://datascience.stackexchange.com/a/134076 License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/ Adaptation: HTML converted to plain text; contact email addresses removed. One can build a prediction model with all of the four base architectures, but most options might get unwieldy in handling and unaffordable in training costs quickly. Here is a brief commenting on the options: Separated Iterative Prediction Model (Separate models for predicting each input value) The Correlation Structure of predicted X,Y,Z may be less utilized, compared to a model that predicts the complete Vector (X,Y,Z) Training might be more complex to set up, since ideally, data that is actual input data, and data that is a product of (iterated) intermediate prediction is marked as such (and generated for the training process) having 3 different models (greatly) complicates performance and confidence interval estimations Predicting a horizon of Length N by having separate models that build on each other, one further extending the prediction of the last, (which it takes as an input), by another period (Separate models with different period width) Usually performing weaker than just having a series of separate models for predicting each target value period, given there is an 1, 2, 3,... N period gap before the predicted value. (bc this implicitly separates synthetic from actual informations.) but also can perform better, if the input data window is narrow (in terms of temporal extension), because the prediction iteration now functions as information transmitter. Any ways! this is extremely un-wieldy because 1 model for each gap pattern easily explodes effort and compute Generating, managing and evaluating model training and performance gets extremely unwieldy, beceause predictions of one model get training data of the next one. Predict vector of k values into the future. (Train one model that predicts a vector of ten values, each representing the predicted X at each period up to ten periods into the future.) Thats the common approach/architecture (also with having the model predict all three of the variable throughout the whole prediction horizon) Varying with model complexity, this will introduce averaging over the predicted vector. This is best adressed through weights assignments Model Prediction of Complete Vector (X,Y,Z) one step ahead (Train one model that predicts a vector of three values, each representing the predicted next X, Y, and Z. Use them as inputs for the subsequent step.) Same as (1), but without the disadvantage of loosing X,Y,Z correlation in the prediction. With Careful Training data compilation through augmentation by generation of intermediary predictions, it may be the best of all the options