Training and evaluating accuracies different in keras LSTM model: why does evaluation not produce same results as during training?

Refresh

March 2019

Views

12 time

2

We are building an LSTM for modeling a physical optical procees.

So far, I have produced the following code in python using Keras with Tensorflow backend.

#Define model
model = Sequential()
model.add(LSTM(128, batch_size=BATCH_SIZE, input_shape=(train_x.shape[1],train_x.shape[2]), return_sequences=True, stateful=False ))#,,return_sequences=Tru# stateful=True 
model.add(Dense(2, activation='softmax'))
opt = tf.keras.optimizers.Adam(lr=0.01, decay=1e-6)

#Compile model
model.compile(
    loss='sparse_categorical_crossentropy',
    optimizer=opt,
    metrics=['accuracy']
)

model.fit(
    train_x, train_y,
    batch_size=BATCH_SIZE,
    epochs=EPOCHS,#,
    verbose=1)

#Now I want to make sure that the we can predict the training set (using evaluate) and that it is the same result as during training
score = model.evaluate(train_x, train_y, batch_size=BATCH_SIZE, verbose=0)
print(' Train accuracy:', score[1])

The Output of the code is

Epoch 1/10 5872/5872 [==============================] - 0s 81us/sample - loss: 0.6954 - acc: 0.4997
Epoch 2/10 5872/5872 [==============================] - 0s 13us/sample - loss: 0.6924 - acc: 0.5229 
Epoch 3/10 5872/5872 [==============================] - 0s 14us/sample - loss: 0.6910 - acc: 0.5256
Epoch 4/10 5872/5872 [==============================] - 0s 13us/sample - loss: 0.6906 - acc: 0.5243 
Epoch 5/10 5872/5872 [==============================] - 0s 13us/sample - loss: 0.6908 - acc: 0.5238

Train accuracy: 0.52480716

So the problem is that the final modeling accuracy (0.5238) should be equal (evaluation) accuracy (0.52480716) which it is not. What have I done wrong here any help highly appreciated

M B

1 answers

0

"Because your model is changing over time, the loss over the first batches of an epoch is generally higher than over the last batches."

https://keras.io/getting-started/faq/#why-is-the-training-loss-much-higher-than-the-testing-loss

For evaluation, since the trained model is taken, the accuracy is higher.