Questions tagged [recurrent-neural-network]

0

votes
0

answer
5

Views

How to use RNN cell in a network?

I am trying to use a customized RNN cell in my network. I started with the RNN cell example of Keras where the RNN cell is defined as MinimalRNNCell. When I am trying to use the defined cell in my recurrent network, by replacing a simpleRNN that I was using previously with the customized RNN cell, b...
omid
1

votes
0

answer
13

Views

Inconsistency between GRU and RNN implementation

I'm trying to implement some custom GRU cells using Tensorflow. I need to stack those cells, and I wanted to inherit from tensorflow.keras.layers.GRU. However, when looking at the source code, I noticed that you can only pass a units argument to the __init__ of GRU, while RNN has an argument that is...
csej
0

votes
0

answer
4

Views

How should the input shape of Keras LSTM layer looks like

I've been reading for a while about training LSTM models using tf.keras, where i did use the same framework for regression problems using simple feedforward NN architectures and i highly understand how should i prepare the input data for such models, however when it comes for training LSTM, i feel...
peter bence
1

votes
1

answer
693

Views

Time series classification using LSTM - How to approach?

I am working on an experiment with LSTM for time series classification and I have been going through several HOWTOs, but still, I am struggling with some very basic questions: Is the main idea for learning the LSTM to take a same sample from every time series? E.g. if I have time series A (with sam...
Andre444
1

votes
1

answer
497

Views

Tensorflow 1.4 Bidirectional RNN not working as expected

I am trying to use Bidirectional RNN and pass the output through a CNN for text classification. However, I am getting all sorts of shape errors with bidirectional RNN. Although, If I use two dynamic rnn with reverse op in the second layer, it appears to work fine: Here is bidirectional RNN code that...
MLNINJA
1

votes
0

answer
83

Views

Tensorflow error with input to dynamic_rnn ValueError: Cannot feed value of shape (2,) for Tensor 'Placeholder:0', which has shape '(1, ?)'

I'm a newer to tensorflow and python. I would like to use dynamic RNN to construct sentence embeddings. I want to use variable-length sequences. I would have for each sentence its embedding. I really need your help. Here is a part of my code. graph = tf.Graph() with graph.as_default(): x_data = tf.p...
AmalB
1

votes
0

answer
75

Views

Neural Network error is constant

I am building a Recurrent Neural Network based on the one at this link: intro to RNN in Tensorflow But when I try to run it, the accuracy is stuck at 19.1% for the test set and similar values for the train and validation set. From my understanding, it looks like the network is not updating the weigh...
Nadni
1

votes
2

answer
346

Views

How to modify the initial value in BasicLSTMcell in tensorflow

I want to initial the value for weight and bias in BasicLSTMcell in Tensorflow with my pre-trained value (I get them by .npy). But as I use get_tensor_by_name to get the tensor, it seems that it just returns a copy for me and the raw value is never changed. I need your help!
KazuhiraDZ
1

votes
0

answer
68

Views

Indices and Slicing of TensorFlow's global variables of kernels for hidden weights and recurrent states of LSTM

I have a question about detailed indices of global variables which are generated by LSTM cells. placeholders = {'inputs':tf.placeholder(tf.float32, shape=[None, None, 1000])} cell = tf.nn.rnn_cell.BasicLSTMCell(80) outs, states = tf.nn.dynamic_rnn(cell=cell, inputs=placeholders['inputs'], dtype=tf.f...
Atsu
1

votes
0

answer
273

Views

Confusion about many-to-one, many-to-many LSTM architectures in Keras

My task is the following: I have a (black box) method which computes a sequence starting from an initial element. At each step, my method reads an input from an external source of memory and outputs an action which potentially changes this memory. You can think of this method as a function f: (exter...
1

votes
1

answer
477

Views

Inputting both Word-level and Char-level embedding in LSTM for PoS tagging

I am referring to this research paper 'Learning Character-level Representations for Part-of-Speech Tagging', where the author says: 'The proposed neural network uses a convolutional layer that allows effective feature extraction from words of any size. At tagging time, the convolutional layer gener...
Grimlock
1

votes
0

answer
51

Views

How to fit cubic equation on pybrain

I am trying to fit a Neural Network on a cubic Equation, but after many tries changing the number of the neurons on the hidden layer and increasing the number of epochs I could only get this: Could you guys help me on this? from pybrain.datasets import SupervisedDataSet from pybrain.tools.shortcuts...
Gustavo Braga
1

votes
0

answer
249

Views

Implementing Hierarchical Attention for Classification

I am trying to implement the Hierarchical Attention paper for text classification. One of the challenges that I am finding is how to manage batching and updates to the weights of the network by the optimizer. The architecture of the network is made of two encoders stacked one after the other: a sen...
Jadiel de Armas
1

votes
0

answer
70

Views

How to get results from all Tensorflow's while_loop() iterations as a list of not specified length?

I want to get results from all while_loop() iterations, so that tf.shape(result) == [batch_size, input_length]. batch_size and input_length may differ from mini-batch to mini-batch, so I evaluate their shapes dynamically. I created this code: batch_size = tf.shape(x)[0] input_length = tf.shape(x)[1...
Ziemo
1

votes
1

answer
48

Views

Artificial life simulator not producing any results

I have been experimenting with evolving artificial creatures, but so far all creatures just die. To initialize the creatures that do not result from asexual reproduction; I create around 8 random neurons which both have a connection in and a connection out. I'm using mutation to get a set of weights...
Sam
1

votes
1

answer
132

Views

Time series prediction with LSTM

I'm currently learning on LSTMs and time series prediction with LSTMs. And I try to predict speeds for a road segment. from sklearn.preprocessing import MinMaxScaler sc = MinMaxScaler() training_set = sc.fit_transform(training_set) X_train = training_set[0:1257] //speed at (t) y_train = training_set...
Klaus
1

votes
0

answer
42

Views

Multiple Sequence to Sequence RNN with Various Time Scales

I'm trying to forecast a time series based on several influencing factors. For example, forecasting how many shoppers per hour there will be in my shop tomorrow will depend on three things: Yesterday's weather and shoppers/hour Tomorrow's weather How many tourists there are in town The first two are...
fishstix44
1

votes
1

answer
34

Views

What is the difference of using the same cell or two different cells for backward and forward cells in bidirectional RNN in tensorflow?

I have this question that I couldn't figured out. What is the difference between a bidirectional RNN which have two different cells (one for forward and one for backward) and a bidirectional RNN that share the same cell (same cell for backward and forward) ? here the MWE of the two codes : 1) Using...
Clement Viricel
1

votes
1

answer
251

Views

Tensorflow: gradients are zero for LSTM and GradientDescentOptimizer

Gradients which are computed by GradientDescentOptimizer for LSTM network are always zero. They are zero even on the first step, so, I think it is not vanishing gradient problem. The same issue happens for AdamOptimizer. I have reduced input to one point of time series and label (expected output) to...
Danil Nemirovsky
1

votes
0

answer
255

Views

How do I combine features like word embeddings and sentiment polarity for text classification using LSTM neural networks?

Embeddings layer of LSTM is fed with the weights=embedding_matrix from the vocab, and model.fit has X_train which is the tokenized text data. My X_train has shape (12,000 , 100) and embeddings_matrix has shape (34613, 300) where 34613 are the number of tokens(vocab from complete data ~15000 sentence...
RAFIYA SHEIKH
1

votes
0

answer
53

Views

Many to one recurrent network for episodic patterns in Keras

I'm trying to build a recurrent neural network using Keras. I'm using as base the discussion presented here. However, in the solution proposed on the original discussion, as far as I understand, there is no concept of an 'episode'. Let me explain what I mean by that. Imagine you have 6 instances x1...
arnaldocan
1

votes
0

answer
212

Views

Using pytorch cuda for RNNs on google colaboratory

I have a code (a code we saw in a class) of a recurrent neural network that reads a given text and tries to produce its own text similar to the example. The code is written in python and uses the pytorch library. I wanted to modify to see whether I could increase its speed by using GPU instead of CP...
Sina
1

votes
0

answer
39

Views

Understanding distributing deep learning with tensorflow

I have written an RNN with tensorflow that trains on some time series data. I am training the model on moving window taking in 2 time slices and predicting for the third. I batch my inputs and labels and run the entire set of batches over 30 epochs. The model runs fine as a single TF application....
clicky
1

votes
1

answer
358

Views

Feed stacked RNN output into fully connected layer

I'm trying to solve a regression problem with a stacked RNN in tensorflow. The RNN output should be fed into a fully connected layer for the final prediction. Currently I'm struggeling on how to feed the RNN output into the final fully_connected layer. My input is of shape [batch_size, max_sequence...
Nico Lindmeyer
1

votes
0

answer
426

Views

combining DropoutWrapper and ResidualWrapper with variational_recurrent=True

I'm trying to create a MultiRNNCell of LSTM cells wrapped with both DropoutWrapper and ResidualWrapper. For using variational_recurrent=True, we must provide input_size parameter to DropoutWrapper. I'm not able figure out what input_size should be passed to each LSTM layer, since ResidualWrapper als...
devin
1

votes
1

answer
502

Views

confusion about sequence length in dynamic rnn version of tensor flow

i am training a RNN to classify a sequence of input (13 values at each time step) as a particular class. Here is relevant part of my code (line numbers prefixed) 50 data = tf.placeholder(tf.float32, [None, 40,13]) #Number of examples, number of input, dimension of each input 51 target = tf.placehold...
user1371666
1

votes
1

answer
52

Views

What is the error term in backpropagation through time if I only have one output?

In this question, RNN: Back-propagation through time when output is taken only at final timestep I've seen that if I only have one output at final time step T, which is y(T), then the error at earlier time step is unneeded. Then, is the loss function term E = sum(E(t)) instead the value of E = E(T)...
林彥良
1

votes
1

answer
437

Views

How to interpret Tensorflow Histograms and Distribution plots

I am implementing a two-layer RNN. When I run the code, I have the following plots for weights and biases for the two layers of RNN. (Sorry not enough reputation to include images on SO, please click on links.) Plot for RNN cell 0 Plot for RNN cell 1 By looking at these plots, I don't understand if...
Mohit Kumar
1

votes
0

answer
108

Views

validation set get higher accuracy than training set for LSTM RNN as learning rate gets smaller

I'm currently trying to test the differences of behavior between a LSTM RNN and a GRU RNN on the prediction of time series ( classification 1/0 if time series goes up or down). I use the fit_generator method (as in Keras François Chollet book's) I feed 30 point to the network and the next point ha...
hariboy
1

votes
0

answer
35

Views

How to get the input word as output word in neural network

I am trying to get the input word as output word with LSTM(copy a word). For that, I have seen some snippets. In those examples, I observed that we need to convert characters of a word into integers. But Those examples were written to predict the next characters. an example of what I am trying input...
prasanna
1

votes
0

answer
220

Views

How to implement a Many-to-Many RNN in tensorflow?

The below code gives me all the hidden state values of the unrolled RNN. hidden_states,final_hidden_state = tf.nn.dyanamic_rnn(...) How do I multiply each of the hidden_states with the weight 'Why' shown in the figure. I find this confusing because the shape of hidden_states is [mini_batch_size,max_...
1

votes
1

answer
67

Views

Running RNN in Tensorflow

If I have an array of 20 elements of type float. Based on the values of the first ten elements I want a RNN to predict what the value of the last ten elements are. Using various online resources and books I have gotten a RNN built that reads the first 10 elements and processes them. However I don't...
Damisco
1

votes
0

answer
272

Views

Concatenate encoder hidden states/cells/outputs from different sources for attention calculation - issues?

I am using Pytorch for an LSTM encoder-decoder sequence-to-sequence prediction problem. As a first step, I would like to forecast 2D trajectories (trajectory x, trajectory y) from multivariate input - 2-D or more (trajectory x, trajectory y, speed, rotation, etc.) I am following the below notebook (...
user3530347
1

votes
0

answer
156

Views

Keras embedding layer with LSTM

My original model is: input = Input(shape=(1, 6)) # 1 time step, 6 features LSTM_layer = LSTM(self.lstm_units, return_sequences=False, return_state=True) lstm_output, out_h, out_c = LSTM_layer(embedded, initial_state=[h, c]) The input is a one hot vector, eg [0, 1, 0, 0, 0, 0] Now I've read that it'...
Andy Wei
1

votes
0

answer
44

Views

Using incremental integer in Neural Network

I am using Brain.js to develop a neural network for a project of mine. The idea is to have something like this: { input: [day, month, hour, minute], output: some categoryID } Basically, I will train the network to learn user's habits over time, so that later the network can predict a user's favourit...
Ola Alsaker
1

votes
0

answer
127

Views

Prediction of Words from the corpus

I am trying to build a model where, as the user types, my model should predict the word (Name of a medicine in this case). I have a corpus of all the words that are going to be typed. When I tried to use rnn with lstm to predict these words, I'm getting words that are similar to the words present in...
Geeth Govind S
1

votes
0

answer
47

Views

How to print (or see) i, j, f and o gates values and dimensions while using BasicLSTMCell tensorflow for sequence classification?

I am using this tensorflow example written by Aymeric Damien to classify some sequences using BasicLSTMCell. How can I change this example to print i, j, f and o (gates) values and shapes at each time step during the training? Thanks!
khemedi
1

votes
0

answer
50

Views

Learning a non-linear mapping using LSTM units: encountering overfitting

I am trying to use a recurrent neural network to perform sensor fusion for an inertial measurement unit. IMUs are commonly used in conjunction with a Kalman filter (KF), which performs both fusion of accelerometer and gyroscope data along with 'smoothing', which results in a filtered, final output o...
HighVoltage
1

votes
0

answer
115

Views

Linear Regression using RNN tensorflow python

Can anyone please site an example on how linear regression can be implemented using RNN in pure tensorflow other than using keras API.. Eg : x_train = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] I ll divide them into 5 batches of 2 element each so x_train will be of shape [5, 2] [1, 2] [3, 4] [5, 6].. y_train...
Kevin Toms
1

votes
0

answer
26

Views

Passing the context of one RNN to another

I have 'n' number of RNNs and I wish to pass the context of first RNN along with second input to second RNN, then the context of this(second)RNN and third input to third RNN and so on. How can I achieve this in Keras. I wish to use keras.layers.SimpleRNN. Is there a way to fetch the context of previ...
yamini goel

View additional questions