# Questions tagged [lstm]

723 questions

1

votes

1

answer

2.2k

Views

### what is the principle of readout and teacher forcing?

These days I study something about RNN and teacher forcing. But there is one point that I can't figure out. What is the principle of readout and teacher forcing? How can we feeding the output(or ground truth) of RNN from the previous time step back to the current time step, by using the output as fe...

1

votes

0

answer

12

Views

### Hierarchical LSTM in Keras

I want to predict the next element of a sequence using Lstm in Keras. I want to use hierarchical Lstm but not standard stacking. I want to implement two-layer lstm like below image (in the first layer there is time_step lstm unroll and in the second row only exist for example 3 lstm unroll)
how can...

0

votes

0

answer

7

Views

### How to use Keras to create LSTM time series model with many input and many output

I have training data like this:
train_x = np.random.randint(1, 20, (5, 4))
train_x
array([[ 4, 19, 5, 4],
[ 5, 2, 2, 8],
[11, 9, 17, 16],
[18, 18, 7, 10],
[ 2, 1, 1, 4]])
train_y = np.random.randint(1, 10, (5, 2))
train_y
array([[2, 7],
[2, 9],
[4, 5],
[7, 8],
[2, 8]])
And also validation...

1

votes

0

answer

19

Views

### Pytorch LSTM batch training and evaluating

I'm trying to figure out/learn, how to correctly train and evaluate pytorch LSTM by using batches and also what are other possible approaches.
Datasets
I have datasets from different datetime parts of single timeseries. Every dataset has variable length(1k-5k rows) and same amoun...

-2

votes

0

answer

12

Views

### Keras: CNN + LSTM for video recognition

I am trying to implement the Model shown in the above picture that basically consists of time-distributed CNNs followed by a sequence of LSTMs using Keras with TF. I have divided two types of class, and extract 10 frames from each video captured.
I have extracted 10 frames for each video and stored...

0

votes

0

answer

2

Views

### Comparing the result of LSTM and a deep neural network

Can I use sequenceInputLayer(n_features) when applying deep neural net and NOT LSTM model in Matlab?
I applied LSTM on time-series data in Matlab 2018b. Now, I want to show that considering temporal relationships in LSTM improves performance. To do that, I need to apply a deep neural net with the sa...

2

votes

0

answer

118

Views

### How to extract forecasting errors from all training sets into a single data frame in R?

By forecasting errors, I mean the differences between predicted and actual values.
I am doing a time series analysis using a deep learning model called the long-short term memory (LSTM) based on this great article. The author distributed the data set into 11 samples to train the model and then make...

0

votes

0

answer

3

Views

### Blas GEMM launch failed [Tensorflow-GPU]

I'm trying to train LSTM network for sequence to sequence task using tensorflow-gpu and no other library like keras on top of it. Whenever begin the training process i get this annoying error.
Blas GEMM launch failed : a.shape=(128, 532), b.shape=(532, 1024), m=128, n=1024, k=532
[[{{node rnn/while/...

1

votes

0

answer

8

Views

### why MSE on test set is very low and doesn't seem to evolve (not increasing after increasing epochs)

I am working on a problem of predicting stock values using LSTMs.
My work is based on the following project .
I use a data set (time series of stock prices) of total length 12075 that I split into train and test set (almost 10%). It is the same used in the link project.
train_data.shape
(11000,)
te...

1

votes

2

answer

1.2k

Views

### How to generate sequence using LSTM?

I want to generate a sequence when a particular input is activated. I want to generate odd or even sequence according to its corresponding input neuron activation. I am trying to create a model using LSTM because it can remember the short term order.
I tried this way
import numpy as np
from keras.mo...

1

votes

1

answer

2.8k

Views

### LSTM with CRF in Keras

I don't really understand how to combine sklearn_crfsuite and Keras.
I have to made a classic LSTM and insteed of the last Activation, I use sklearn_crfsuite?
Someone have an example?
Thx,

1

votes

2

answer

332

Views

### Keras expected dense_13 to have 2 dimensions

I am getting very confusing error messages from Keras. I use the following model and pass to it input with shape (num_examples, n, 1).
def create_model():
model = Sequential()
model.add(LSTM(64, input_shape=(n,1), return_sequences=False))
model.add(Dense(units=n, activation='linear'))
return model
I...

1

votes

1

answer

95

Views

### Define custom LSTM Cell in Keras?

I use Keras with TensorFlow as back-end. If I want to make a modification to an LSTM cell, such as "removing" the output gate, how can I do it? It is a multiplicative gate, so somehow I will have to set it to fixed values so that whatever multiplies it, has no effect.

0

votes

1

answer

28

Views

### expected ndim=3, found ndim=2

I'm new with Keras and I'm trying to implement a Sequence to Sequence LSTM.
Particularly, I have a dataset with 9 features and I want to predict 5 continuous values.
I split the training and the test set and their shape are respectively:
X TRAIN (59010, 9)
X TEST (25291, 9)
Y TRAIN (59010, 5)
Y TE...

1

votes

0

answer

16

Views

### Train/test set for LSTM on multivariate Time series with varying length sampes

I have a time series dataset with (30 seconds) time-step and 20 features. Each observation/sample has a length between 188 to 200 time-steps. I have just over 2000 samples collected from the past three years.
I want to implement LSTM to make a prediction at time t=1. I.e first 30 sec for all (20) fe...

0

votes

0

answer

5

Views

### LSTM, Keras - Plot of Loss on the Train and Test Datasets

I'm using LSTM to train a multivariate time-series model to forecast value.
after I train/test split and reshape the data, I train the model and the "Loss on the Train and Test" having very big gap which mean the error is a lot.
Training Network 1
`model = Sequential()
model.add(LSTM(5278, input_sha...

0

votes

1

answer

46

Views

### CONVLSTM2D to predict the second image from the first image

I have sequences of images (2 images in each sequence). I am trying to use CONVLSTM2D to train on this sequence.
Question:
Can I train LSTM model on just 2 images per sequence? The goal would be, prediction of second image from the first image.
Thanks!

1

votes

1

answer

341

Views

### What is the fastest way to prepare data for RNN with numpy?

I currently have a (1631160,78) np array as my input to a neural network. I would like to try something with LSTM which requires a 3D structure as input data. I'm currently using the following code to generate the 3D structure needed but it is super slow (ETA > 1day). Is there a better way to do thi...

2

votes

0

answer

434

Views

### output of bidirectional_dynamic_rnn?

I am using bidirectional_dynamic_rnn to deal with classification task for variable length of sentences. Here is a part of my code:
lstm_fw_cell=rnn.BasicLSTMCell(hidden_size)
lstm_bw_cell=rnn.BasicLSTMCell(hidden_size)
lstm_fw_cell=rnn.DropoutWrapper(lstm_fw_cell,output_keep_prob=self.dropout_rate)...

0

votes

0

answer

4

Views

### Concatenate an input of 27 fields to the output of the LSTM layer using Keras in Python

I have an existing LSTM model that looks as follows:
model_glove1 = Sequential()
model_glove1.add(Embedding(vocabulary_size, 25, input_length=50, weights=[embedding_matrix25],trainable=False))
model_glove1.add(LSTM(32))
model_glove1.add(Dense(128, activation='relu'))
model_glove1.add(Dense(64, activ...

2

votes

0

answer

271

Views

### Multilayer Seq2Seq model with LSTM in Keras

I was making a seq2seq model in keras. I had built single layer encoder and decoder and they were working fine. But now I want to extend it to multi layer encoder and decoder.
I am building it using Keras Functional API.
Training:-
Code for encoder:-
encoder_input=Input(shape=(None,vec_dimension))...

2

votes

1

answer

39

Views

### Why is import cntk as C not working in google colab

I installed opencv version 3.4.4, installed cntk,Importing into google collab gives the following results.
import cntk as C
/usr/local/lib/python3.6/dist-packages/cntk/cntk_py_init.py:56: UserWarning: Unsupported Linux distribution (ubuntu-18.04). CNTK supports Ubuntu 16.04 and above, only.
warnin...

0

votes

1

answer

14

Views

### How do I shape multivariate data for input to LSTM

What I am trying to achieve.
I am trying to predict opening price of Natural Gas ("NG Open") from multiple input parameters per table below. I have followed some tutorials but they don't explain the reason behind a particular format.The code is working after multiple trial and error but need to have...

2

votes

1

answer

39

Views

### Having incompatible issue when build LSTM VAE model

I try to build a VAE LSTM model with keras. Input shape is (sample_number,20,31)
While, there are some incompatible issue happening.
I'm not sure which part of my code being wrong, forgive me for posting all of them.
My import:
from keras.models import Sequential, Model
from keras.objectives import...

2

votes

1

answer

29

Views

### tf.data API cannot print all the batches

I am self-teaching myself about tf.data API. I am using MNIST dataset for binary classification. The training x and y data is zipped together in the full train_dataset. Chained along together with this zip method is first the batch() dataset method. the data is batched with a batch size of 30. Since...

1

votes

1

answer

3k

Views

### Keras LSTM predicting only 1 category, in multi-category classification - how to fix?

I have a text dataset that has equal number of labels - 0,1,2,3,4. I ran the Keras binary classification example LSTM (imdb example) on their website with my dataset and the compile line changed to
"model.compile(loss='categorical_crossentropy', optimizer='adam', class_mode="categorical")"
But the...

1

votes

2

answer

1.4k

Views

### Creating sequence vector from text in Python

I am now trying to prepare the input data for LSTM-based NN.
I have some big number of text documents and what i want is to make sequence vectors for each document so i am able to feed them as train data to LSTM RNN.
My poor approach:
import re
import numpy as np
#raw data
train_docs = ['this is tex...

17

votes

4

answer

3.8k

Views

### Use LSTM tutorial code to predict next word in a sentence?

I've been trying to understand the sample code with https://www.tensorflow.org/tutorials/recurrent
which you can find at https://github.com/tensorflow/models/blob/master/tutorials/rnn/ptb/ptb_word_lm.py
(Using tensorflow 1.3.0.)
I've summarized (what I think are) the key parts, for my question, belo...

17

votes

3

answer

10.4k

Views

### What's the difference between a bidirectional LSTM and an LSTM?

Can someone please explain this? I know bidirectional LSTMs have a forward and backward pass but what is the advantage of this over a unidirectional LSTM?
What is each of them better suited for?

2

votes

0

answer

26

Views

### Getting an error InvalidArgumentError: indices[10,0] = 92379 is not in [0, 92379) because of vocab size mismatching

I am trying to learn LSTM for the first time, Basically concatenating two LSTM layers. Below is model architecture.
Layer (type) Output Shape Param # Connected to
=====================================================================================...

3

votes

1

answer

94

Views

### Understanding word embeddings, convolutional layer and max pooling layer in LSTMs and RNNs for NLP Text Classification

Here is my input data:
data['text'].head()
0 process however afforded means ascertaining di...
1 never occurred fumbling might mere mistake
2 left hand gold snuff box which capered hill cu...
3 lovely spring looked windsor terrace sixteen f...
4 finding nothing else even gold s...

18

votes

1

answer

4.9k

Views

### In Keras, what exactly am I configuring when I create a stateful `LSTM` layer with N `units`?

The first arguments in a normal Dense layer is also units, and is the number of neurons/nodes in that layer. A standard LSTM unit however looks like the following:
(This is a reworked version of "Understanding LSTM Networks")
In Keras, when I create an LSTM object like this LSTM(units=N, ...), am I...

0

votes

0

answer

2

Views

### Visualize Hidden states LSTM

I have seen this visualisation in differents blog/website such as:
http://blog.echen.me/2017/05/30/exploring-lstms/ or
http://karpathy.github.io/2015/05/21/rnn-effectiveness/
But even if I understand the terms of hidden states I do not understand how link this with the data. Anyone got an clue ?

2

votes

1

answer

42

Views

### Should The Gradients For The Output Layer of an RNN Clipped?

I am currently training an LSTM RNN for time-series forecasting. I understand that it is common practice to clip the gradients of the RNN when it crosses a certain threshold. However, I am not completely clear on whether or not this includes the output layer.
If we call the hidden layer of an RNN h...

19

votes

3

answer

11k

Views

### How to train a RNN with LSTM cells for time series prediction

I'm currently trying to build a simple model for predicting time series. The goal would be to train the model with a sequence so that the model is able to predict future values.
I'm using tensorflow and lstm cells to do so. The model is trained with truncated backpropagation through time. My questio...

2

votes

1

answer

3.2k

Views

### How to get the output shape of a layer in Keras?

I have the following code in Keras (Basically I am modifying this code for my use) and I get this error:
'ValueError: Error when checking target: expected conv3d_3 to have 5 dimensions, but got array with shape (10, 4096)'
Code:
from keras.models import Sequential
from keras.layers.convolutional imp...

2

votes

1

answer

1.3k

Views

### What is Sequence length in LSTM?

The dimensions for the input data for LSTM are [Batch Size, Sequence Length, Input Dimension] in tensorflow.
What is the meaning of Sequence Length & Input Dimension ?
How do we assign the values to them if my input data is of the form :
[[[1.23] [2.24] [5.68] [9.54] [6.90] [7.74] [3.26]]] ?

3

votes

2

answer

998

Views

### Tensorflow tf.constant_initializer is very slow

Trying to use pre trained word2vec embeddings of 100 dim for training a LSTM
@staticmethod
def load_embeddings(pre_trained_embeddings_path, word_embed_size):
embd = []
import time
start_time = time.time()
cnt = 4
with codecs.open(pre_trained_embeddings_path, mode="r", encoding='utf-8') as f:
for li...

2

votes

1

answer

1.5k

Views

### Tensorflow ValueError: Shapes (?, 1) and (?,) are incompatible

I'm facing this error when running my code with 3 lstm layers. Not sure how to fix it. Can anyone help. Here MAX_SEQUENCE_LENGTH=250. After running the cost function, i get the error 'ValueError: Shapes (?, 1) and (?,) are incompatible'
# Generate a Tensorflow Graph
tf.reset_default_graph()
batch_si...

2

votes

1

answer

406

Views

### How is the output h_n of an RNN (nn.LSTM, nn.GRU, etc.) in PyTorch structured?

The docs say
h_n of shape (num_layers * num_directions, batch, hidden_size): tensor containing the hidden state for t = seq_len
Now, the batch and hidden_size dimensions are pretty much self-explanatory. The first dimension remains a mystery, though.
I assume, that the hidden states of all "last cel...