Questions tagged [lstm]

1

votes
1

answer
2.2k

Views

what is the principle of readout and teacher forcing?

These days I study something about RNN and teacher forcing. But there is one point that I can't figure out. What is the principle of readout and teacher forcing? How can we feeding the output(or ground truth) of RNN from the previous time step back to the current time step, by using the output as fe...
slkingxr
1

votes
0

answer
12

Views

Hierarchical LSTM in Keras

I want to predict the next element of a sequence using Lstm in Keras. I want to use hierarchical Lstm but not standard stacking. I want to implement two-layer lstm like below image (in the first layer there is time_step lstm unroll and in the second row only exist for example 3 lstm unroll) how can...
Arezoo Torkaman
0

votes
0

answer
7

Views

How to use Keras to create LSTM time series model with many input and many output

I have training data like this: train_x = np.random.randint(1, 20, (5, 4)) train_x array([[ 4, 19, 5, 4], [ 5, 2, 2, 8], [11, 9, 17, 16], [18, 18, 7, 10], [ 2, 1, 1, 4]]) train_y = np.random.randint(1, 10, (5, 2)) train_y array([[2, 7], [2, 9], [4, 5], [7, 8], [2, 8]]) And also validation...
wangmyde
1

votes
0

answer
19

Views

Pytorch LSTM batch training and evaluating

I'm trying to figure out/learn, how to correctly train and evaluate pytorch LSTM by using batches and also what are other possible approaches. Datasets I have datasets from different datetime parts of single timeseries. Every dataset has variable length(1k-5k rows) and same amoun...
Tomas Trdla
-2

votes
0

answer
12

Views

Keras: CNN + LSTM for video recognition

I am trying to implement the Model shown in the above picture that basically consists of time-distributed CNNs followed by a sequence of LSTMs using Keras with TF. I have divided two types of class, and extract 10 frames from each video captured. I have extracted 10 frames for each video and stored...
lai hang
0

votes
0

answer
2

Views

Comparing the result of LSTM and a deep neural network

Can I use sequenceInputLayer(n_features) when applying deep neural net and NOT LSTM model in Matlab? I applied LSTM on time-series data in Matlab 2018b. Now, I want to show that considering temporal relationships in LSTM improves performance. To do that, I need to apply a deep neural net with the sa...
user9439906
2

votes
0

answer
118

Views

How to extract forecasting errors from all training sets into a single data frame in R?

By forecasting errors, I mean the differences between predicted and actual values. I am doing a time series analysis using a deep learning model called the long-short term memory (LSTM) based on this great article. The author distributed the data set into 11 samples to train the model and then make...
T-T
0

votes
0

answer
3

Views

Blas GEMM launch failed [Tensorflow-GPU]

I'm trying to train LSTM network for sequence to sequence task using tensorflow-gpu and no other library like keras on top of it. Whenever begin the training process i get this annoying error. Blas GEMM launch failed : a.shape=(128, 532), b.shape=(532, 1024), m=128, n=1024, k=532 [[{{node rnn/while/...
deepak nandwani
1

votes
0

answer
8

Views

why MSE on test set is very low and doesn't seem to evolve (not increasing after increasing epochs)

I am working on a problem of predicting stock values using LSTMs. My work is based on the following project . I use a data set (time series of stock prices) of total length 12075 that I split into train and test set (almost 10%). It is the same used in the link project. train_data.shape (11000,) te...
Othmane
1

votes
2

answer
1.2k

Views

How to generate sequence using LSTM?

I want to generate a sequence when a particular input is activated. I want to generate odd or even sequence according to its corresponding input neuron activation. I am trying to create a model using LSTM because it can remember the short term order. I tried this way import numpy as np from keras.mo...
Eka
1

votes
1

answer
2.8k

Views

LSTM with CRF in Keras

I don't really understand how to combine sklearn_crfsuite and Keras. I have to made a classic LSTM and insteed of the last Activation, I use sklearn_crfsuite? Someone have an example? Thx,
Williamben
1

votes
2

answer
332

Views

Keras expected dense_13 to have 2 dimensions

I am getting very confusing error messages from Keras. I use the following model and pass to it input with shape (num_examples, n, 1). def create_model(): model = Sequential() model.add(LSTM(64, input_shape=(n,1), return_sequences=False)) model.add(Dense(units=n, activation='linear')) return model I...
Valeria
1

votes
1

answer
95

Views

Define custom LSTM Cell in Keras?

I use Keras with TensorFlow as back-end. If I want to make a modification to an LSTM cell, such as "removing" the output gate, how can I do it? It is a multiplicative gate, so somehow I will have to set it to fixed values so that whatever multiplies it, has no effect.
0

votes
1

answer
28

Views

expected ndim=3, found ndim=2

I'm new with Keras and I'm trying to implement a Sequence to Sequence LSTM. Particularly, I have a dataset with 9 features and I want to predict 5 continuous values. I split the training and the test set and their shape are respectively: X TRAIN (59010, 9) X TEST (25291, 9) Y TRAIN (59010, 5) Y TE...
mht
1

votes
0

answer
16

Views

Train/test set for LSTM on multivariate Time series with varying length sampes

I have a time series dataset with (30 seconds) time-step and 20 features. Each observation/sample has a length between 188 to 200 time-steps. I have just over 2000 samples collected from the past three years. I want to implement LSTM to make a prediction at time t=1. I.e first 30 sec for all (20) fe...
MatN
0

votes
0

answer
5

Views

LSTM, Keras - Plot of Loss on the Train and Test Datasets

I'm using LSTM to train a multivariate time-series model to forecast value. after I train/test split and reshape the data, I train the model and the "Loss on the Train and Test" having very big gap which mean the error is a lot. Training Network 1 `model = Sequential() model.add(LSTM(5278, input_sha...
B.Choo
0

votes
1

answer
46

Views

CONVLSTM2D to predict the second image from the first image

I have sequences of images (2 images in each sequence). I am trying to use CONVLSTM2D to train on this sequence. Question: Can I train LSTM model on just 2 images per sequence? The goal would be, prediction of second image from the first image. Thanks!
AKSHAYAA VAIDYANATHAN
1

votes
1

answer
341

Views

What is the fastest way to prepare data for RNN with numpy?

I currently have a (1631160,78) np array as my input to a neural network. I would like to try something with LSTM which requires a 3D structure as input data. I'm currently using the following code to generate the 3D structure needed but it is super slow (ETA > 1day). Is there a better way to do thi...
Tengyu Liu
2

votes
0

answer
434

Views

output of bidirectional_dynamic_rnn?

I am using bidirectional_dynamic_rnn to deal with classification task for variable length of sentences. Here is a part of my code: lstm_fw_cell=rnn.BasicLSTMCell(hidden_size) lstm_bw_cell=rnn.BasicLSTMCell(hidden_size) lstm_fw_cell=rnn.DropoutWrapper(lstm_fw_cell,output_keep_prob=self.dropout_rate)...
HAO CHEN
0

votes
0

answer
4

Views

Concatenate an input of 27 fields to the output of the LSTM layer using Keras in Python

I have an existing LSTM model that looks as follows: model_glove1 = Sequential() model_glove1.add(Embedding(vocabulary_size, 25, input_length=50, weights=[embedding_matrix25],trainable=False)) model_glove1.add(LSTM(32)) model_glove1.add(Dense(128, activation='relu')) model_glove1.add(Dense(64, activ...
aastha
2

votes
0

answer
271

Views

Multilayer Seq2Seq model with LSTM in Keras

I was making a seq2seq model in keras. I had built single layer encoder and decoder and they were working fine. But now I want to extend it to multi layer encoder and decoder. I am building it using Keras Functional API. Training:- Code for encoder:- encoder_input=Input(shape=(None,vec_dimension))...
SAGAR
2

votes
1

answer
39

Views

Why is import cntk as C not working in google colab

I installed opencv version 3.4.4, installed cntk,Importing into google collab gives the following results. import cntk as C /usr/local/lib/python3.6/dist-packages/cntk/cntk_py_init.py:56: UserWarning: Unsupported Linux distribution (ubuntu-18.04). CNTK supports Ubuntu 16.04 and above, only. warnin...
Malathi
0

votes
1

answer
14

Views

How do I shape multivariate data for input to LSTM

What I am trying to achieve. I am trying to predict opening price of Natural Gas ("NG Open") from multiple input parameters per table below. I have followed some tutorials but they don't explain the reason behind a particular format.The code is working after multiple trial and error but need to have...
Siddharth Kulkarni
2

votes
1

answer
39

Views

Having incompatible issue when build LSTM VAE model

I try to build a VAE LSTM model with keras. Input shape is (sample_number,20,31) While, there are some incompatible issue happening. I'm not sure which part of my code being wrong, forgive me for posting all of them. My import: from keras.models import Sequential, Model from keras.objectives import...
Jacky Tsai
2

votes
1

answer
29

Views

tf.data API cannot print all the batches

I am self-teaching myself about tf.data API. I am using MNIST dataset for binary classification. The training x and y data is zipped together in the full train_dataset. Chained along together with this zip method is first the batch() dataset method. the data is batched with a batch size of 30. Since...
ARAT
1

votes
1

answer
3k

Views

Keras LSTM predicting only 1 category, in multi-category classification - how to fix?

I have a text dataset that has equal number of labels - 0,1,2,3,4. I ran the Keras binary classification example LSTM (imdb example) on their website with my dataset and the compile line changed to "model.compile(loss='categorical_crossentropy', optimizer='adam', class_mode="categorical")" But the...
ganga rocks
1

votes
2

answer
1.4k

Views

Creating sequence vector from text in Python

I am now trying to prepare the input data for LSTM-based NN. I have some big number of text documents and what i want is to make sequence vectors for each document so i am able to feed them as train data to LSTM RNN. My poor approach: import re import numpy as np #raw data train_docs = ['this is tex...
Alexey Trofimov
17

votes
4

answer
3.8k

Views

Use LSTM tutorial code to predict next word in a sentence?

I've been trying to understand the sample code with https://www.tensorflow.org/tutorials/recurrent which you can find at https://github.com/tensorflow/models/blob/master/tutorials/rnn/ptb/ptb_word_lm.py (Using tensorflow 1.3.0.) I've summarized (what I think are) the key parts, for my question, belo...
Darren Cook
17

votes
3

answer
10.4k

Views

What's the difference between a bidirectional LSTM and an LSTM?

Can someone please explain this? I know bidirectional LSTMs have a forward and backward pass but what is the advantage of this over a unidirectional LSTM? What is each of them better suited for?
shekit
2

votes
0

answer
26

Views

Getting an error InvalidArgumentError: indices[10,0] = 92379 is not in [0, 92379) because of vocab size mismatching

I am trying to learn LSTM for the first time, Basically concatenating two LSTM layers. Below is model architecture. Layer (type) Output Shape Param # Connected to =====================================================================================...
Yog
3

votes
1

answer
94

Views

Understanding word embeddings, convolutional layer and max pooling layer in LSTMs and RNNs for NLP Text Classification

Here is my input data: data['text'].head() 0 process however afforded means ascertaining di... 1 never occurred fumbling might mere mistake 2 left hand gold snuff box which capered hill cu... 3 lovely spring looked windsor terrace sixteen f... 4 finding nothing else even gold s...
Abhishek
18

votes
1

answer
4.9k

Views

In Keras, what exactly am I configuring when I create a stateful `LSTM` layer with N `units`?

The first arguments in a normal Dense layer is also units, and is the number of neurons/nodes in that layer. A standard LSTM unit however looks like the following: (This is a reworked version of "Understanding LSTM Networks") In Keras, when I create an LSTM object like this LSTM(units=N, ...), am I...
0

votes
0

answer
2

Views

Visualize Hidden states LSTM

I have seen this visualisation in differents blog/website such as: http://blog.echen.me/2017/05/30/exploring-lstms/ or http://karpathy.github.io/2015/05/21/rnn-effectiveness/ But even if I understand the terms of hidden states I do not understand how link this with the data. Anyone got an clue ?
Boat
2

votes
1

answer
42

Views

Should The Gradients For The Output Layer of an RNN Clipped?

I am currently training an LSTM RNN for time-series forecasting. I understand that it is common practice to clip the gradients of the RNN when it crosses a certain threshold. However, I am not completely clear on whether or not this includes the output layer. If we call the hidden layer of an RNN h...
Rehaan Ahmad
19

votes
3

answer
11k

Views

How to train a RNN with LSTM cells for time series prediction

I'm currently trying to build a simple model for predicting time series. The goal would be to train the model with a sequence so that the model is able to predict future values. I'm using tensorflow and lstm cells to do so. The model is trained with truncated backpropagation through time. My questio...
Jakob
2

votes
1

answer
3.2k

Views

How to get the output shape of a layer in Keras?

I have the following code in Keras (Basically I am modifying this code for my use) and I get this error: 'ValueError: Error when checking target: expected conv3d_3 to have 5 dimensions, but got array with shape (10, 4096)' Code: from keras.models import Sequential from keras.layers.convolutional imp...
2

votes
1

answer
1.3k

Views

What is Sequence length in LSTM?

The dimensions for the input data for LSTM are [Batch Size, Sequence Length, Input Dimension] in tensorflow. What is the meaning of Sequence Length & Input Dimension ? How do we assign the values to them if my input data is of the form : [[[1.23] [2.24] [5.68] [9.54] [6.90] [7.74] [3.26]]] ?
Stuti Kalra
3

votes
2

answer
998

Views

Tensorflow tf.constant_initializer is very slow

Trying to use pre trained word2vec embeddings of 100 dim for training a LSTM @staticmethod def load_embeddings(pre_trained_embeddings_path, word_embed_size): embd = [] import time start_time = time.time() cnt = 4 with codecs.open(pre_trained_embeddings_path, mode="r", encoding='utf-8') as f: for li...
jknair
2

votes
1

answer
1.5k

Views

Tensorflow ValueError: Shapes (?, 1) and (?,) are incompatible

I'm facing this error when running my code with 3 lstm layers. Not sure how to fix it. Can anyone help. Here MAX_SEQUENCE_LENGTH=250. After running the cost function, i get the error 'ValueError: Shapes (?, 1) and (?,) are incompatible' # Generate a Tensorflow Graph tf.reset_default_graph() batch_si...
Lavan Reddy
2

votes
1

answer
406

Views

How is the output h_n of an RNN (nn.LSTM, nn.GRU, etc.) in PyTorch structured?

The docs say h_n of shape (num_layers * num_directions, batch, hidden_size): tensor containing the hidden state for t = seq_len Now, the batch and hidden_size dimensions are pretty much self-explanatory. The first dimension remains a mystery, though. I assume, that the hidden states of all "last cel...
esBeee

View additional questions