why MSE on test set is very low and doesn't seem to evolve (not increasing after increasing epochs)


1 weeks ago


8 time


I am working on a problem of predicting stock values using LSTMs.

My work is based on the following project . I use a data set (time series of stock prices) of total length 12075 that I split into train and test set (almost 10%). It is the same used in the link project.

train_data.shape (11000,)

test_data.shape (1075,)enter image description here

The predictions at the end seem not bad. However, I just don't understand why the training error decreases dramatically and the test error is always very very low (though it keeps decreasing by very little). I know that normally the test error should also start to increase after some number of epochs because of overfitting. I have tested with a simpler code and with a different dataset and I have encountered relatively similar MSE graphs.

Could this be normal ? should I just increase the size of test set ? Is there any thing done wrong ? any ideas please on how to test/investigate that ?

0 answers