Questions tagged [lstm]

1

votes
1

answer
311

Views

Why return sequences in stacked RNNs?

When stacking RNNs, it is mandatory to set return_sequences parameter as True in Keras. For instance in Keras, lstm1 = LSTM(1, return_sequences=True)(inputs1) lstm2 = LSTM(1)(lstm1) It is somewhat intuitive to preserve the dimensionality of input space for each stacked RNN layer, however, I am not...
Buomsoo Kim
1

votes
1

answer
3.9k

Views

KeyError : The tensor variable , Refer to the tensor which does not exists

Using LSTMCell i trained a model to do text generation . I started the tensorflow session and save all the tensorflow varibles using tf.global_variables_initializer() . import tensorflow as tf sess = tf.Session() //code blocks run_init_op = tf.global_variables_intializer() sess.run(run_init_op) sa...
Madhi
0

votes
0

answer
5

Views

How to use RNN cell in a network?

I am trying to use a customized RNN cell in my network. I started with the RNN cell example of Keras where the RNN cell is defined as MinimalRNNCell. When I am trying to use the defined cell in my recurrent network, by replacing a simpleRNN that I was using previously with the customized RNN cell, b...
omid
1

votes
1

answer
51

Views

Make LSTM learn from the correlation of 3 variables

I have a dataset of 3 variables x,y,z. and they are the readings of 3 different sensors. These will be the inputs. When these sensors find a specific object, the corresponding output of their readings should 1. Otherwise the corresponding output of there readings should be 0. This is example when t...
Emad
1

votes
0

answer
10

Views

LSTM Encoder-Decoder Inference Model

Many tutorials for seq2seq encoder-decoder architecture based on LSTM, (for example English-French translation), define the model as follow: encoder_inputs = Input(shape=(None,)) en_x= Embedding(num_encoder_tokens, embedding_size)(encoder_inputs) # Encoder lstm encoder = LSTM(50, return_state=True...
userInThisWorld
0

votes
0

answer
4

Views

How should the input shape of Keras LSTM layer looks like

I've been reading for a while about training LSTM models using tf.keras, where i did use the same framework for regression problems using simple feedforward NN architectures and i highly understand how should i prepare the input data for such models, however when it comes for training LSTM, i feel...
peter bence
1

votes
1

answer
216

Views

LSTM with Numpy, can't find a definitive algorithm

I am doing LSTM from scratch and am following this guide, but the loss is not decreasing but increasing. This is the best guide I have found thus far but that is not saying much as even this one is incomplete. Aside from spotting the problem specific to my code I would appreciate any sources showing...
Grant Swalwell
1

votes
1

answer
693

Views

Time series classification using LSTM - How to approach?

I am working on an experiment with LSTM for time series classification and I have been going through several HOWTOs, but still, I am struggling with some very basic questions: Is the main idea for learning the LSTM to take a same sample from every time series? E.g. if I have time series A (with sam...
Andre444
1

votes
0

answer
57

Views

Erorr using keras 2.0 in R

I am trying to replicate Siraj's code for predicting stock prices in R (https://github.com/llSourcell/How-to-Predict-Stock-Prices-Easily-Demo). This is my code: url
Mislav
1

votes
0

answer
315

Views

Attention mechanism in spelling correction model

I'm trying to test attention mechanism in this code (based on the work of MajorTal): def generate_model(output_len, chars=None): '''Generate the model''' print('Build model...') chars = chars or CHARS model = Sequential() # 'Encode' the input sequence using an RNN, producing an output of hidden_size...
1

votes
0

answer
32

Views

Keras RNN - ValueError when checking Input

I recently got a Nvidia Card and wanted to try LSTM-Models with the new GPU-Support. Sadly I do not know much about LSTMs. And I build this little model to test it: import pandas as pd from keras.models import Sequential from keras.layers import Dense, LSTM, Dropout from sklearn.model_selection impo...
Patrick.H
1

votes
1

answer
93

Views

Error: Value rank (4) should be larger than the Variable rank (1) at most by number of dynamic axes (2)

I would like to know How to provide input varibales to CNTK Network Modelling and training the model like given here Since i have two independent variable and 3 target variables, X and Y are of shape: X['train'].shape Out[60]: (34567, 10, 2, 1) Y['train'].shape Out[61]: (34567, 3, 1) i gave time_...
Jonreyan
1

votes
0

answer
531

Views

Keras LSTM model has very low accuracy

I am trying to build a model that takes a sentence as input, takes each word and tries to predict the next word. My input and output both are a 3D matrix with (number of sentences, number of words per sentence, dimension of word embedding). Input is e. g. 'I like green apples' while the output is 'l...
1

votes
2

answer
346

Views

How to modify the initial value in BasicLSTMcell in tensorflow

I want to initial the value for weight and bias in BasicLSTMcell in Tensorflow with my pre-trained value (I get them by .npy). But as I use get_tensor_by_name to get the tensor, it seems that it just returns a copy for me and the raw value is never changed. I need your help!
KazuhiraDZ
1

votes
0

answer
35

Views

Python- LSTM Based RNN error in input?

i am trying to build a deep learning network based on LSTM RNN here is what is tried from keras.models import Sequential from keras.layers import Dense, Dropout, Activation from keras.layers import Embedding from keras.layers import LSTM import numpy as np train = np.loadtxt('TrainDatasetFinal.txt...
Hadeer El-Zayat
1

votes
1

answer
130

Views

Why do tensorflow Variables in outputs of dynamic (or static) rnn with lstm cell has plus-one dimensionality?

My question is about the elemental dynamic or static rnn outputs's dimensionality. nlu_input = tf.placeholder(tf.float32, shape=[4,1607,1]) cell = tf.nn.rnn_cell.BasicLSTMCell(80) outts, states = tf.nn.dynamic_rnn(cell=cell, inputs=nlu_input, dtype=tf.float32) Then tf.gloabal_valiables() returns the...
Atsu
1

votes
1

answer
160

Views

lstm dimension not match by tensorflow

I construct a LSTM network, and my input's dimension is 100*100*83 ( batch_size=100, steps = 100, char_vector = 83). I build a two LSTM layers which has 512 hidden units. # coding: utf-8 from __future__ import print_function import tensorflow as tf import numpy as np import time class CharRNN: def...
THANK FLY
1

votes
0

answer
273

Views

Confusion about many-to-one, many-to-many LSTM architectures in Keras

My task is the following: I have a (black box) method which computes a sequence starting from an initial element. At each step, my method reads an input from an external source of memory and outputs an action which potentially changes this memory. You can think of this method as a function f: (exter...
1

votes
1

answer
183

Views

LSTM parity generator

I am trying to learn deep learning, I have stumbled on one exercise here It is first warm-up exercise. I am stuck. For constant sequence of small lengths(2,3) it solves it no problem. However when I try whole sequence of 50. it stops at 50% accuracy, which is basically random guess. According to he...
Filip Rak
1

votes
1

answer
458

Views

How to reuse LSTM layer and variables in variable scope (attention mechanism)

I have an issue in my code where I would like to share weights in my lstm_decoder (so essentially just use one LSTM). I know there are a few resources online on that but I am still unable to understand why the following does not share weights: initial_input = tf.unstack(tf.zeros(shape=(1,1,hidden_si...
Tom
1

votes
1

answer
2.1k

Views

Pytorch tutorial LSTM

I was trying to implement the exercise about Sequence Models and Long-Short Term Memory Networks with Pytorch. The idea is to add an LSTM part-of-speech tagger character-level features but I can't seem to work it out. They gave as a hint that there should be two LSTMs involved, one that will output...
David
1

votes
0

answer
151

Views

Input dimension mismatch in Keras

Hi Can anyone help me out with the error, I have seemed to search through the documentation but to no avail. The aim is to predict a time series. I have used a dummy data shape = (N, timesteps, features). I wish to predict x_2 from x_1, x_3 from x_2 and so on till x_11 from x_10 using LSTM. (Any sug...
Nilesh Kumar Jha
1

votes
0

answer
371

Views

Training on multiple time-series of various length using recurrent layers in Keras

TL;DR - I have a couple of thousand speed-profiles (time-series where the speed of a car has been sampled) and I am unsure how to configure my models such that I can perform arbitrary forecasting (i.e. predict t+n samples given a sample t). I have read numerous explanations (1, 2, 3, 4, 5) about how...
GLaDER
1

votes
1

answer
477

Views

Inputting both Word-level and Char-level embedding in LSTM for PoS tagging

I am referring to this research paper 'Learning Character-level Representations for Part-of-Speech Tagging', where the author says: 'The proposed neural network uses a convolutional layer that allows effective feature extraction from words of any size. At tagging time, the convolutional layer gener...
Grimlock
1

votes
0

answer
158

Views

How to train a LSTM Neural Network to forecast a full “cycle” of a Time Series?

I have the below data whose 'cycles' grow in period over time: My goal is to feed some fraction of this into a LSTM network in order to predict or forecast the remaining points (at least enough to make up one full cycle). I've been trying to follow this tutorial: https://machinelearningmastery.com/...
Sam
1

votes
0

answer
185

Views

Keras LSTM Error: ValueError: setting an array element with a sequence

I'm loading sentence vectors embeddings (Facebook's InferSent) in my neural network along with other metadata. Here is a glimpse of my dataset: My dataset Endpoint Vector: Sentence embedding ndarray. Shape: (4096,1) Description Vector: Sentence embedding ndarray. Shape: (4096,1) HTTP Path: Categoric...
Pranav Makhijani
1

votes
0

answer
403

Views

LSTM tensorflow shape error?

I have a Class for RNN that can classify the sentences into positive or negative for sentiment analysis. Because the length of sentences are different, so I used placeholder named text_len to be a vector of sentence lengths in one batch when feeding the training batch data: class TextRNN: def __init...
HAO CHEN
1

votes
0

answer
173

Views

Examples on how to declare Caffe LSTM?

Caffe seems to have an LSTM layer now: http://caffe.berkeleyvision.org/tutorial/layers/lstm.html However, there is no documentation on how to use it. How would I declare the following LSTM layer with the new API? layer { name: 'lstm1' type: 'Lstm' bottom: 'data' bottom: 'clip' top: 'lstm1' Recurrent...
raaj
1

votes
0

answer
232

Views

Increasing Label Error Rate (Edit Distance) and Fluctuating Loss?

I am training a handwriting recognition model of this architecture: { 'network': [ { 'layer_type': 'l2_normalize' }, { 'layer_type': 'conv2d', 'num_filters': 16, 'kernel_size': 5, 'stride': 1, 'padding': 'same' }, { 'layer_type': 'max_pool2d', 'pool_size': 2, 'stride': 2, 'padding': 'same' }, { 'la...
Rocket Pingu
1

votes
0

answer
20

Views

Implemented LSTM inference code shows different test loss compared to using lasagne.

I have difficulty implementing LSTM inference code using matlab. I extraced weight from trained model using theano and lasagne. And then, i coded LSTM inference(test) code using extraced weights and biases. The loss of implemented inference code shows about 5, but the test loss of theano and lasagn...
hishin
1

votes
1

answer
132

Views

Time series prediction with LSTM

I'm currently learning on LSTMs and time series prediction with LSTMs. And I try to predict speeds for a road segment. from sklearn.preprocessing import MinMaxScaler sc = MinMaxScaler() training_set = sc.fit_transform(training_set) X_train = training_set[0:1257] //speed at (t) y_train = training_set...
Klaus
1

votes
1

answer
698

Views

LSTM overfitting but validation accuracy not improving

The task I am trying to do is to classify EEG signals into 4 possible classes. The data is divided up into trials. Subjects were asked to think about doing 1 of four actions, and the classification task is to predict what they were thinking based on the EEG signals. I have ~2500 trials. For each tr...
jonnyd42
1

votes
0

answer
42

Views

Multiple Sequence to Sequence RNN with Various Time Scales

I'm trying to forecast a time series based on several influencing factors. For example, forecasting how many shoppers per hour there will be in my shop tomorrow will depend on three things: Yesterday's weather and shoppers/hour Tomorrow's weather How many tourists there are in town The first two are...
fishstix44
1

votes
1

answer
127

Views

How to use LSTM to make prediction with both feature from the past and the current ones?

Suppose I have a data frame like the following, Discount is calculated from selling price/list price, unit_sales is the number of items sold on that day. If I am going to use LSTM to make sales prediction for the next day (data in the green box), based on the past 3 days of sales and discount (data...
Aaron_Geng
1

votes
0

answer
29

Views

Predicting the next value of data points using LSTM and Keras having tensorflow as a backend

I have a data with 65 samples. Example: 2697,2825,2136,2824,3473,2513,2538,3051,2737.9805,3133.849,2350.8695,6000,3121.225 I've divided the data in training and testing (train and test), scaled it and also supervised it. I have to predict what would be the 66th sample value. 2697,2825,2136,2824,347...
Akshay Khurana
1

votes
1

answer
31

Views

How can we define an RNN - LSTM neural network with multiple output for the input at time “t”?

I am trying to construct a RNN to predict the possibility of a player playing the match along with the runs score and wickets taken by the player.I would use a LSTM so that performance in current match would influence player's future selection. Architecture summary: Input features: Match details -...
Sethu N
1

votes
1

answer
251

Views

Tensorflow: gradients are zero for LSTM and GradientDescentOptimizer

Gradients which are computed by GradientDescentOptimizer for LSTM network are always zero. They are zero even on the first step, so, I think it is not vanishing gradient problem. The same issue happens for AdamOptimizer. I have reduced input to one point of time series and label (expected output) to...
Danil Nemirovsky
1

votes
1

answer
550

Views

Keras show MemoryError at model.fit()

I keep getting MemoryError without additional explanation from Keras at model.fit(), no matter how small the number of neurons or batch size. Does anyone have any idea what error does this error refer to or how to fix this? Error: Using TensorFlow backend. Traceback (most recent call last): File 'C:...
Erland Devona
1

votes
1

answer
234

Views

Error when fitting LSTM on Time Series Data

I have daily time series data of 14 features. I am interested in predicting a one-step ahead forecast for feature 1 (binary) using all features 1 to 14. In order to do so I implement a LSTM model in R using Keras. To work with the LSTM layer I convert my train and validation data matrix in a 3D arr...
hokage555
1

votes
1

answer
78

Views

Implement modified architecture of LSTM cell in Python

I wish to modify the internal cell structure and equations of the standard LSTM cell representation(for example in the implementation of Keras). How do I do the same(which functions/modules to overwrite and in which file of the Keras implementation)? Any other suggestions in terms of libraries or fr...
humble_me

View additional questions