# Questions tagged [recurrent-neural-network]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

**1**

vote

**0**answers

40 views

### OutputProjectionWrapper vs fully connected layer on top of RNN

I'm reading the 14th chapter of Hands-On Machine Learning with Scikit-Learn and TensorFlow. It says:
Although using an OutputProjectionWrapper is the simplest solution to reduce the dimensionality ...

**-2**

votes

**0**answers

22 views

### I want Python Code for rainfall prediction using RNN algorthim [on hold]

Here I attached Dataset for rainfall prediction
Kindly Please Help!!

**0**

votes

**3**answers

31 views

### why set return_sequences=True and stateful=True for tf.keras.layers.LSTM?

I am learning tensorflow2.0 and follow the tutorial. In the rnn example, I found the code:
def build_model(vocab_size, embedding_dim, rnn_units, batch_size):
model = tf.keras.Sequential([
tf....

**0**

votes

**1**answer

21 views

### using rnn_cell inside of tf.while getting ValueError: The two structures don't have the same number of elements

Given data = tf.placeholder(tf.float32, [2, None, 3]) (batch_size * time_step * feature_size), Ideally I want do tf.unstack(data, axis = 1) to get a number of tensors each of which has the shape of [2,...

**0**

votes

**0**answers

7 views

### in nmt_with_attention,the gru layer in decoder confuse me

Tensorflow have an example of seq2seq translate model using attention, in github link
In the Decoder class, the gru layer is defined ,and it is used in call() function, the code is:
output, state = ...

**1**

vote

**0**answers

17 views

### Why are the gradients added instead of averaged in backpropagation through time?

In the following implementation of a backward pass of an RNN, the gradients for Wh, Wx, and b are calculated by adding the calculated gradients at each time step. Intuitively, what does this do and ...

**-2**

votes

**0**answers

17 views

### Tune input controls based on output quality degradation through machine learning

i am having a case where if there is degradation in the output quality, i need to change the controls available in the input to maintain the quality. i have historic data where based on the output ...

**-1**

votes

**0**answers

24 views

### How to feed spatio-temporal data files to Neural Networks? [on hold]

I am trying to use Recurrent Neural Networks to predict temporal results. However, my training data contains location statistics as well, each time-step has one csv file, each file has location, ...

**0**

votes

**1**answer

32 views

### Pytorch RNN HTML Generation

I’ m stuck for a couple of days trying to make and RNN network to learn a basic HTML template.
I tried different approaches and I even overfit on the following data:
<!DOCTYPE html>
<html>...

**2**

votes

**0**answers

16 views

### Custom RNN Layer with previous input and output time steps as Inputs

How can I extend the SimpleRNN or create a custom layer that could use not just the previous output y[n-1] but also the previous output y[n-2] and the previous inputs x[n-1], x[n-2]?
def call(self, ...

**1**

vote

**0**answers

37 views

### pytorch LSTM does not overfit single sample

I try to overfit a single time series. Meaning, I try to perform the training on a single (X,Y) pair over and over again. I do this to get an impression of the capabilities of the hyperparameters. But ...

**0**

votes

**0**answers

18 views

### Tensorflow dynamic_rnn deprecation

It seems that the tf.nn.dynamic_rnn has been deprecated:
Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Please use keras.layers.RNN(cell), ...

**4**

votes

**1**answer

52 views

### Understanding GRU Architecture - Keras

I am using the Mycroft AI wake word detection and I am trying to understand the dimensions of the network. The following lines show the model in Keras:
model = Sequential()
model.add(GRU(
...

**0**

votes

**0**answers

4 views

### Error with cudnnRNNForwardTraining standalone

ERROR: CUDNN_BAD_PARAM
I am trying to write a standalone for cudnnRNNForwardTraining. Getting an bad param. These are the descriptor values before the cudnnRNNForwardTraining call. What could be the ...

**0**

votes

**0**answers

15 views

### How to visualize CNN + RNN in tensorflow?

I build a model first use a CNN to extract features in each time step input, and then feed these features to a RNN's time step input, and at last feed each time step output to a FC layer to classify ...

**1**

vote

**1**answer

22 views

### signal to signal pediction using RNN and Keras

I am trying to reproduce the nice work here and adapte it so that it reads real data from a file.
I started by generating random signals (instead of the generating methods provided in the above link). ...

**0**

votes

**0**answers

11 views

### Keras: set initial states of a simple recurrent layer keeping stateful=False

I need to set the initial states of my RNN units with keras to study the network response (pre-training) to different initial values but I am confused with syntaxis of the methods. This is my model:
...

**0**

votes

**0**answers

10 views

### Use tf.nn.dynamic_rnn() multi-times, Dimensions must be equal Error occurs

When I use tf.nn.dynamic_rnn() multi-times in my code,
ValueError: Dimensions must be equal, but are 80 and 90 for
'..scope1/rnn/while/gru_cell/MatMul_4' (op: 'MatMul') with input shapes:
[50,...

**0**

votes

**1**answer

21 views

### Simple Data recall RNN in Pytorch

I am learning Pytorch and am trying to make a network that can remember previous inputs.
I have tried 2 different input/output structures(see below) but haven't gotten anything to work the way I would ...

**1**

vote

**1**answer

18 views

### Can you make an LSTM forget context manually?

I am very new to machine learning and was wandering if it’s possible to manually empty an LSTM’s short term memory. Say, for instance, I wanted to train an LSTM on the sentence
“Jack and Jill went up ...

**-1**

votes

**0**answers

11 views

### Queries regarding Deepspeech-2 neural network model? [closed]

Deep speech-2 neural network was developed by mozilla or baidu-research?
Can i get trained deep-speech model with weights?
Please share the available links(github etc) and published papers if any ...

**-2**

votes

**0**answers

12 views

### 2D CNN requires input in the form of matrix, then how to feed MFCCs features into the network of CNN+LSTM?

I am working on speech recognition and trying to use MFCCs for input of CNN + LSTM network. But, i am not understanding how to make the MFCCs data structure to give input to CNN ?

**0**

votes

**2**answers

24 views

### RNN LSTM Sentiment analysis model with low accuracy

I have a dataset with 200000 samples.
I am using the train_test_split from Sklearn.
model = Sequential()
model.add(Embedding(50000,128, input_length=14))
model.add(LSTM(16, return_sequences=True, ...

**0**

votes

**1**answer

27 views

### Different time_step input for LSTM in Keras

I'm trying to build a encoder-decoder network to classify video data. Reading the Keras documentation for LSTM cells, it expects a fixed number of time_step to the cell. However, the data that I'm ...

**0**

votes

**0**answers

18 views

### LSTMs passing multiple times series as input

I have a few, perhaps basic, questions about LSTMs and how they work. I spent some time trying to find these answers but I have not quite found what I am looking for. As I am relatively new to neural ...

**-2**

votes

**0**answers

24 views

### Binary Function Estimator using Neural Network

I have 100 data with 3 columns, all of them are binary and 8 bits, for example :
Raw 1 --> col1: 00000001, col2: 10101010, col3 : 00000010
Raw 2 --> col1: 10000011, col2: 10101100, col3 : 00000100
......

**3**

votes

**1**answer

37 views

### Tensorflow: my rnn always output same value, weights of rnn are not trained

I used tensorflow to implement a simple RNN model to learn possible trends of time series data and predict future values. However, the model always produces same values after training. Actually, the ...

**0**

votes

**0**answers

4 views

### tensor-flow changing datatype for large tensors

I am creating an rnn which inherit most of its properties for a basicRNN in Tensorflow, however I have 4D state tensors of batch x d x n x m , each of which are in the order of 100-200, except d. ...

**-2**

votes

**0**answers

15 views

### matlab neural network for change point detection in data stream (on-line detection)

I'm working on a project, and I need your help. I'm working with time series and step change detection. The goal is to realize an artificial neural network, that properly trained, is able to identify ...

**0**

votes

**1**answer

30 views

### input data shapes & sizes, RNN Keras, regression

Im having trouble sorting my data into the correct format for RNN with Keras.
I have a csv file with 22 columns, 1344 rows.
My data is continuous variables recorded at 30min intervals, over a number ...

**-2**

votes

**1**answer

21 views

### what is the best way to read file containing handwritten text along with typed text?

what is the best way to read pdf file containing handwritten text along with typed text?
Like file started with typed text then some blank spaces are filled with handwritten text followed by typed ...

**0**

votes

**0**answers

14 views

### Make RNN model without tf.nn.dynamic_rnn, tf.nn.basic_cell , ProjectionWithoutWrapper but model is not working

it's my RNN code
this model predict future value
for example
when input is
x = [x1, x2, x3, x4],
model's output is
pred = [x2, x3, x4, x5]
when using high level api (ex, Wrapper) , it's work ...

**0**

votes

**0**answers

16 views

### Graph disconnect in inference in Keras RNN + Encoder/Decoder + Attention

I've successfully trained a model in Keras using an encoder/decoder structure + attention + glove following several examples, most notably this one and this one. It's based on a modification of ...

**0**

votes

**1**answer

24 views

### How to implement “do something when meeting <EOS>”

The token <EOS> is ubiquitously used in NLP. As I haven't used it, the implementation of conditioning on it is a bit unclear to me. Could anyone provide a snippet of Python code. (If statements ...

**0**

votes

**0**answers

20 views

### How can make early-stopping for simple RNN or LSTM before overfitting?

I'm dealing with time series RNN and LSTM models and looking forward to providing an EarlyStopping algorithm as a Model-Checkpoint so that I can avoid overfitting when the loss(MSE) can't be improved ...

**0**

votes

**2**answers

41 views

### GRU Language Model not Training Properly

I’ve tried reimplementing a simple GRU language model using just a GRU and a linear layer (the full code is also at https://www.kaggle.com/alvations/gru-language-model-not-training-properly):
class ...

**1**

vote

**2**answers

51 views

### Output from LSTM not changing for different inputs

I have the an LSTM implemented in PyTorch as below.
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable
class LSTM(nn.Module):
...

**-2**

votes

**1**answer

41 views

### Unstable behavior of RNN

I am training LSTM model with some financial data. Can not disclose details of data as it is real trade data. The issues which I am facing is that while training Keras prints out the logs with info ...

**5**

votes

**0**answers

52 views

### Forcing a validation step in an Unreasonably Effective RNN

Reading through Andrej Karpathy's 2015 post, "The Unreasonable Effectiveness of Recurrent Neural Networks," there's a section where LaTeX code is generated:
Algebraic Geometry (Latex)
The ...

**1**

vote

**1**answer

18 views

### ValueError: Input 0 is incompatible with layer lstm_14: expected ndim=3, found ndim=2

I am building a cnn_rnn network for image classification. I am getting an error while running the following python code in my jupyter notebook.
# model
model1 = Sequential()
# first ...

**0**

votes

**0**answers

14 views

### TensorFlow will not save Recurrent Neural Network

I'm writing a recurrent neural network in Tensor Flow for Python. This is the code to train it:
saver = tf.train.Saver()
init = tf.global_variables_initializer()
with tf.Session() as sess:
...

**1**

vote

**1**answer

30 views

### LSTM Network not learning from sequences. Underfiting or Overfitting using Keras, TF backend

Thanks in advance for your help.
I am working in a problem with sequences of 4 characters. I have around 18.000 sequences in the training set. Working with Keras+TensorFlow backend. The total number ...

**0**

votes

**0**answers

31 views

### Tensorflow: “not a valid checkpoint”

I am trying to restore a recurrent neural network from a .cpkt file. My code to restore the network is:
graph = tf.Graph()
with graph.as_default():
X = tf.placeholder(tf.float32, [1, n_steps, ...

**-2**

votes

**0**answers

35 views

### Neural Network predictions too large

I am trying to predict stock returns. I am training two models. One is LSTM the other one is a simple neural network. However, the simple neural network predicts values like 1500%-2000% for some ...

**0**

votes

**0**answers

25 views

### What is the easiest way to fix outliers in output of RNN or LSTM prediction in Keras?

I'm dealing with RNN and LSTM models by normalized data in range of [-1,+1] and reshaped data for each time sequence from 3 individual matrices A,B,C to long row includes elements of all 3 matrices ...

**0**

votes

**1**answer

18 views

### Keras: SimpleRNN - how to feed in new inputs at each time step (instead of using the output)

In the documentation for SimpleRNN (https://keras.io/layers/recurrent/#simplernncell), it is described as a "Fully-connected RNN where the output is to be fed back to input".
Is there a ...

**0**

votes

**3**answers

53 views

### Input_size error in LSTM of PyTorch: RuntimeError: shape ‘[10, 30, 1]’ is invalid for input of size 150

everyone,
I am using LSTM to predict the stock index of someday using the ones of 30 days before it as the input only. I think in this example, the size of LSTM input should be [10,30,1],so I use t_x=...

**-1**

votes

**1**answer

22 views

### If a small neural network were used as a scoring function for Attention model, what label/value it is trained against?

I am reading up a paper on Attention mechanism of encoder-decoder architecture for machine translation. There were several proposals for the scoring function for decoding step, such as cosine ...

**-2**

votes

**1**answer

18 views

### Recurrent Neural Network: Different numpy reshape in these two articles?

The First: https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/
The above says that we have a sequence of like 96 samples, 1 time-step, and 1 feature....

**0**

votes

**0**answers

35 views

### ValueError: cannot reshape array of size 9575706 into shape (9575706,137,1)

I am trying to build a generative RNN-LSTM model, based off this repo: https://github.com/flawnson/Molecule_generating_RNN/blob/master/final_model.py
When trying to re-shape input X to be [samples, ...