# Questions tagged [deep-learning]

3154 questions

1

votes

3

answer

1.2k

Views

### Cannot set attribute. Group with name “keras_version” exists

I have updated the example program in the keras
cifar10_resnet to work on the cifar 100 instead of cifar 10 data set. I am able to run the code only for the first epoch. When i try to save the model the programs breaks with the below error -
'KeyError: 'Cannot set attribute. Group with name "ke...

0

votes

0

answer

4

Views

### Vanishing Gradient and Diverging problems in machine learning using tensorflow and python3

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
#feature [height, weight, foot size]
#label [0 = woman, 1 = man]
#Goal : Predict man or woman for x_test
x_test = np.array([[162,50,240],#woman
[156,49,225],
[163,53,255],
[167,65,260],
[175,68,270],
[180,70,275]
], dty...

0

votes

0

answer

3

Views

### Load of Transfer Learning on GPU (Matlab)

I've been trying the transfer learning examples provided by mathworks with my own datasete (105 different labels) on the pretrained models matlab provided.
When training, I noticed that the Training Progress window dows say 'Hardware Resourse : Single GPU' and I've seen the 'Cuda library needs to be...

0

votes

1

answer

22

Views

### Train and validation score is high but very Poor Test Accuracy

I am working on multi-label image classification, i am using inception net as my base architecture.
after the complete training i am getting, training accuracy > 90% and validation accuracy > 85% but i am getting 17% accuracy on test data.
Model training -->
model = Model(pre_trained_model.input, x)...

7

votes

0

answer

123

Views

### Tensorflow-Deeplearning - Correlation between input and output

I'm experimenting with tensorflow for speech recognition.
I have inputs as waveforms and words as output.
The waveform would look like this
[0,0,0,-2,3,-4,-1,7,0,0,0...0,0,0,20,-11,4,0,0,1,...]
The words would be an array of numbers while each number represents a word:
[12,4,2,3]
After training I a...

0

votes

0

answer

2

Views

### Comparing the result of LSTM and a deep neural network

Can I use sequenceInputLayer(n_features) when applying deep neural net and NOT LSTM model in Matlab?
I applied LSTM on time-series data in Matlab 2018b. Now, I want to show that considering temporal relationships in LSTM improves performance. To do that, I need to apply a deep neural net with the sa...

1

votes

2

answer

670

Views

### How to know Tensorflow Lite model's input/output feature info?

I'm mobile developer. And I want to use various Tensorflow Lite models(.tflite) with MLKit.
But there are some issues, I have no idea of how to know .tflite model's input/output feature info(these will be parameters for setup).
Is there any way to know that?
Sorry for bad English and thanks.
Update...

1

votes

1

answer

419

Views

### How does upsampling in Fully Connected Convolutional network work?

I read several posts / articles and have some doubts on the mechanism of upsampling after the CNN downsampling.
I took the 1st answer from this question:
https://www.quora.com/How-do-fully-convolutional-networks-upsample-their-coarse-output
I understood that similar to normal convolution operation,...

1

votes

1

answer

43

Views

### The input layer disappears from the structure of a deep learning model

I used the following code to create a CNN model using VGG16 but after creating the model, the input layer of the model disappears from the structure (see the image).
Why the input layer disappears from the structure?
vgg16_model = keras.applications.vgg16.VGG16()
model = Sequential([])
for layer in...

0

votes

1

answer

11

Views

### ModuleNotFoundError: No module named 'ioc' in TFlearn Image Classification

I am designing an image classifier using CNN in python, pycharm. I want to plot an accuracy graph at the end. Following is my code to plot the graph:
hist = {
'Accuracy' : [x.value for x in ea.Scalars('Accuracy')],
'Validation Accuracy' : [x.value for x in ea.Scalars('Accuracy/Validation')],
'L...

4

votes

1

answer

226

Views

### Tensorflow: Simple 3D Convnet not learning

I am trying to create a simple 3D U-net for image segmentation, just to learn how to use the layers. Therefore I do a 3D convolution with stride 2 and then a transpose deconvolution to get back the same image size. I am also overfitting to a small set (test set) just to see if my network is learning...

0

votes

0

answer

3

Views

### How to set lr_mult for convolutional layer in pytorch?

In caffe, it has the option to set the learning multiple for convolution as follows
layer {
name: "conv1a"
type: "Convolution"
bottom: "data"
top: "conv1a"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 64
kernel_size: 3
pad: 1
stride: 1
bias_filler {
type: "constant"...

1

votes

1

answer

4.4k

Views

### What does it mean by deconvolution (backwards convolution)?

What does it mean by deconvolution or backwards convolution in convolutional neural nets?
I understand convolution, if we consider a 3x3 window W and a kernel k of same size the result of the convolution W*K will be one value. Here k is a matix with 3x3 elements.
In my understanding deconvolution tr...

1

votes

2

answer

95

Views

### TensorFlow Graph to Keras Model?

Is it possible to define a graph in native TensorFlow and then convert this graph to a Keras model?
My intention is simply combining (for me) the best of the two worlds.
I really like the Keras model API for prototyping and new experiments, i.e. using the awesome multi_gpu_model(model, gpus=4) for t...

1

votes

1

answer

153

Views

### Understanding caffe library

I am trying to understand the caffe library. For that I run through step by step for feature_extraction.cpp and classification.cpp.
In those cpp files, I found out layers, prototxt file, caffemodel, net.cpp, caffe.pb.cc, caffe.pb.hfiles.
I know caffe is formed using different layers. So those layer...

1

votes

2

answer

339

Views

### How do I get Bokeh to update a plot displaying some measure vs epoch when using keras fit_generator?

I have seen it once before briefly in a presentation, someone was plotting:
loss/validation_loss vs epoch
accuracy/validation_accuracy vs epoch
The special thing was that this person was using a Bokeh plot that udpates itself after each epoch. How is such a feat accomplished?

1

votes

1

answer

2k

Views

### How to implement RGB images as tensors in tensorflow?

I'm new to tensorflow and I'm trying to create a model of Stacked Sparse Denoising Auto-encoders. I have found a way on how to load my training ( and testing) set through examples from here and github but I cannot use them as a tensor to perform the required multiplications etc. (this code is only f...

1

votes

0

answer

13

Views

### AttributeError: 'NoneType' object has no attribute '_inbound_nodes' (custom layer)

Introduction:
What I'm trying to do (briefly):
I'm implementing a custom layer in Keras called ROIPoolingLayer (I know there is some other implementation for this specific layer, but it is different). It is specifically implemented for tensorflow backend.
ROIPoolingLayer is one of the most important...

1

votes

1

answer

298

Views

### Erase layer of tensorflow

How could I erase layer of tensorflow?
Example:
net = tf.reshape(input, [-1, 128*128])
net = tf.layers.dense(inputs = net, units = 16384, activation = tf.nn.relu)
net = tf.layers.dropout(net, training = is_training, name ="erase_later")
net = tf.layers.dense(inputs = net, units = 8, name = 'regr...

1

votes

1

answer

91

Views

### What is the learning rate status when applying keras model fit() iteratively?

I am applying keras model fitting iteratively (within a for loop) due to a large dataset. My goal is to split the dataset into 100 parts, read each part at once and apply the fit() method.
My Question: In each iteration, does the fit() method begins from the initial learning rate (lr=0.1) which I se...

1

votes

2

answer

3.2k

Views

### Training Loss and Validation Loss in Deep Learning

Would you please guide me how to interpret the following results?
1) loss < validation_loss
2) loss > validation_loss
It seems that the training loss always should be less than validation loss. But, both of these cases happen when training a model.

1

votes

1

answer

586

Views

### What is the difference between a tensor and a multi-d matrix in Tensorflow?

I am following the tensorflow CNN tutorial and bumped into the question of what programatically is the difference between a 'tensor' and a multi-dimensional matrix in Tensorflow and in general as well.
I tried to research on my own what a tensor is and what I have found out is:
it it can be of ord...

1

votes

3

answer

559

Views

### batch size in model.fit and model.predict

In keras, both model.fit and model.predict has a parameter of batch_size. My understanding is that batch size in model.fit is related to batch optimization, what's the physical meaning of batch_size in model_predict? Does it need to be equal to the one used by model.fit?

1

votes

2

answer

1.5k

Views

### Difference between tensor.permute and tensor.view in PyTorch?

What is the difference between tensor.permute() and tensor.view()?
They seem to do the same thing.

1

votes

2

answer

916

Views

### Tensorflow model saving and loading

How can save a tensorflow model with model graph like we do in do keras.
Instead of defining the whole graph again in prediction file, can we save whole model ( weight and graph) and import it later
In Keras:
checkpoint = ModelCheckpoint('RightLane-{epoch:03d}.h5',monitor='val_loss', verbose=0, sav...

1

votes

1

answer

631

Views

### Use pretrained model with different input shape and class model

I am working on a classification problem using CNN where my input image size is 64X64 and I want to use pretrained model such as VGG16,COCO or any other. But the problem is input image size of pretrained model is 224X224. How do I sort this issue. Is there any data augmentation way for input image s...

1

votes

1

answer

317

Views

### PyTorch NotImplementedError in forward

import torch
import torch.nn as nn
device = torch.device('cuda' if torch.cuda.is_available() else
'cpu')
class Model(nn.Module):
def __init__(self):
super(Model, self).__init__()
self.layer = nn.Sequential(
nn.Conv2d(1, 16, kernel_size=3, stride=1, padding=1),
nn.BatchNorm2d(16),
nn.ReLU(),
nn.MaxP...

1

votes

1

answer

208

Views

### Google Colab KeyError: 'COLAB_TPU_ADDR'

I'm trying to run a simple MNIST classifier on Google Colab using the TPU option. After creating the model using Keras, I am trying to convert it into TPU by:
import tensorflow as tf
import os
tpu_model = tf.contrib.tpu.keras_to_tpu_model(
model,
strategy=tf.contrib.tpu.TPUDistributionStrategy(
tf.c...

1

votes

1

answer

330

Views

### Keras with Lambda-Layer

I'm relatively new to Keras and I am about to build a Dueling Q-Network to train a KI. I found a code snippet to build a model that surprisingly seems to work. I just have no idea why, because I'm not too familiar with lambda expressions in Keras. Can anybody explain me how exactly the creation of t...

1

votes

1

answer

121

Views

### Global GPU quota needed but can't request increase

I'm trying to use GCloud's deep learning VM image. My request for 8 Tesla K80s was approved. But when I try to create an instance with even a single GPU, I get an error saying the Global GPU limit of 0 is exceeded.
The error statement in specific:
ERROR: (gcloud.compute.instances.create) Could not f...

1

votes

2

answer

25

Views

### TensorFlow not recognising feed_dict input

I am running a simple neural network for linear regression. However TensorFlow is complaining that my feed_dict placeholder(s) are not an element of the graph. However my placeholders and my model are all defined within my graph as can be seen below:
import numpy as np
import tensorflow as tf
from t...

1

votes

1

answer

25

Views

### Randomized Search Get param not implemented

I am training my cnn model on some images and want to add randomized search for hyper parameter optimization but I am having trouble in using randomized search of hyper parameters. I am sharing my model and some code and Error I am having.
I have tried sklearn documentation example and other articl...

1

votes

1

answer

40

Views

### Should I use conv 3x3 in last convolutional layer for semantic segmentation?

In semantic segmentation, the convolutional 1x1 often use to replace fully connected layer to maintain spatial information. Should I use larger kernel size, for example 3x3, instead of 1x1. Because 3x3 kernel size will have larger view information to make the final decision. Thanks

1

votes

2

answer

34

Views

### How to train my own custom word embedding on web pages?

I have tons of text data on multiple web pages about the product I am interested to sell to customers. I tried using pre-trained fasttext word embedding trained on Wikipedia and it didn't give me good results for the classification task. Probably because the text data on the website contains lots of...

0

votes

0

answer

5

Views

### why the channel I set for conda didn't work?

System:windows 10
conda Version:4.6.1
location：China
when I tried to install pytorch on my computer, I used the following command :
conda install pytorch torchvision -c pytorch
but the speed was so slow, then I cancelled the installation and set the conda channel used the following command:
conda...

0

votes

0

answer

7

Views

### How to fix there input related errors in using pretrained inception_v4 for image classification

i followed this project that uses ineption_v3 and modified the retrain.py code to work with inception_v4. this is a link to my project. i manged to retrain the model properly and it had a very good accuracy too. testing in the same code works fine.
when i tried to use use the test.py code to test, i...

1

votes

0

answer

13

Views

### Why we need to normalize input as zero mean and unit variance before feed to network?

In deep learning, I saw many papers apply the pre-processing step as normalization step. It normalizes the input as zero mean and unit variance before feeding to the convolutional network (has BatchNorm). Why not use original intensity? What is the benefit of the normalization step? If I used histo...

2

votes

0

answer

99

Views

### Best way to import data in google-colaboratory for fast computing and training?

I am running a simple deep learning model on google's colab what my problem is it's running slower than my MacBook air with no GPU.
I read this and found out it's problem because of dataset importing over internet. But unable to figure out proper step to fasten up the process.
Google Colab is very s...

1

votes

1

answer

359

Views

### Where can I find fully trained deep networks for download?

I'm trying to examine a hypothesis about the statistics of trained "deep" networks. There have been quite a few impressive results published in recent years (most recently, state of the art state detection based on multi-layer neural networks). It's proved to be surprisingly difficult to find code t...

1

votes

2

answer

728

Views

### How to get word vector representation when using Deep Learning in NLP

How to get word vector representation when using Deep Learning in NLP ? The words are represented by a fixed length vector, see http://machinelearning.wustl.edu/mlpapers/paper_files/BengioDVJ03.pdf for more details.