How to deal with different amounts of data every day?
How to deal with different amounts of data every day?
Loading saved threads...
user900476 · External communityPost link
External question — Data Science Stack Exchange
Author: user900476
Original post: https://datascience.stackexchange.com/questions/111743
License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/
Adaptation: HTML converted to plain text; contact email addresses removed.
I am doing a time series prediction task. There are different amounts of news headlines every day, and the goal is a binary prediction task to predict next day's stock movement.
The amount of headlines varies everyday. There might be 5 headlines, 6 headlines or more for one day. I am planning to embed each headline into a vector space of, for example, 300 dimensions.
How shall I deal with it? As far as I know, neural networks require a fixed size of input. Should I pad my data? For example, there are at most 10 headlines everyday, so should I pad my data into size of [10, 300] for every day?
PS: I don't want to compute the average of the embeddings because I want to know the impact of each news healine later.
Quote
Report
alexmolas · External communityPost link
External answer — Data Science Stack Exchange
Author: alexmolas
Original post: https://datascience.stackexchange.com/a/111744
License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/
Adaptation: HTML converted to plain text; contact email addresses removed.
One common option is to aggregate the embeddings of all the headlines. For example, you can compute the average of the embeddings and use the resulting 300-dim vector as input of the model.
Quote
Report
Post Reply
Quoted from Forex.com.bd-Editorial External question — Data Science Stack Exchange Author: user900476 Source score (net votes, not local likes): 0 Original post: https://datascience.stackexchange.com/questions/111743 License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/ Adaptation: HTML converted to plain text; contact email addresses removed. I am doing a time series prediction task. There are different amounts of news headlines every day, and the goal is a binary prediction task to predict next day's stock movement. The amount of headlines varies everyday. There might be 5 headlines, 6 headlines or more for one day. I am planning to embed each headline into a vector space of, for example, 300 dimensions. How shall I deal with it? As far as I know, neural networks require a fixed size of input. Should I pad my data? For example, there are at most 10 headlines everyday, so should I pad my data into size of [10, 300] for every day? PS: I don't want to compute the average of the embeddings because I want to know the impact of each news healine later.
Checking account access…