# Questions tagged [pytorch]

733 questions

0

votes

0

answer

6

Views

### ValueError: operands could not be broadcast together with shapes (50,50,512) (3,) (50,50,512) while converting tensor to image in pytorch

I'm doing a neural style transfer. I'm trying to reconstruct the output of the convolutional layer conv4_2 of the VGG19 network.
def get_features(image, model):
layers = {'0': 'conv1_1', '5': 'conv2_1', '10': 'conv3_1',
'19': 'conv4_1', '21': 'conv4_2', '28': 'conv5_1'}
x = image
features = {}
fo...

4

votes

1

answer

139

Views

### How to transfer the follow tensorflow into pytorch

I want to re-implement the word embedding here
here is the original tensorflow code (version: 0.12.1)
import tensorflow as tf
class Network(object):
def __init__(
self, user_length,item_length, num_classes, user_vocab_size,item_vocab_size,fm_k,n_latent,user_num,item_num,
embedding_size, filter_size...

0

votes

0

answer

3

Views

### PyTorch implementation: Transformer not generating correct sentences with test data

I've implemented Transformer model with pytorch (the github repo is here), and the validation cost seems decreasing while training.
But when feeding test data, the model only generates ..., so there must be bugs somewhere in the code. Could someone help me find the bug?
(I've also merged all related...

1

votes

2

answer

33

Views

### PyTorch conversion between tensor and numpy array: the addition operation

I am following the 60-minute blitz on PyTorch but have a question about conversion of a numpy array to a tensor. Tutorial example here.
This piece of code:
import numpy as np
a = np.ones(5)
b = torch.from_numpy(a)
np.add(a, 1, out=a)
print(a)
print(b)
yields
[2. 2. 2. 2. 2.]
tensor([2., 2., 2., 2.,...

1

votes

1

answer

336

Views

### Order of CUDA devices [duplicate]

This question already has an answer here:
How does CUDA assign device IDs to GPUs?
3 answers
I saw this solution, but it doesn't quite answer my question; it's also quite old so I'm not sure how relevant it is.
I keep getting conflicting outputs for the order of GPU units. There are two of them: T...

0

votes

0

answer

5

Views

### Display examples of augmented images in PyTorch

I want to display some samples of augmented training images.
My transform includes the standard ImageNet transforms.Normalize like this:
train_transforms = transforms.Compose([transforms.RandomRotation(30),
transforms.RandomResizedCrop(224),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),...

0

votes

0

answer

4

Views

### Need very different learning rate for manual updates vs. using model

I am currently just trying to write some pedagogical material, in which I borrow from some common examples that have been reworked numerous times on the web.
I have a simple bit of code where I manually create tensors for layers, and update them within a loop. E.g.:
w1 = torch.randn(D_in, H, dtype=...

1

votes

1

answer

1.1k

Views

### Pytorch hidden state LSTM

Why do we need to initialize the hidden state h0 in LSTM in pytorch. As h0 will anyways be calculated and get overwritten ? Isn't it like
int a
a = 0
a = 4
Even if we do not do a=0, it should be fine..

1

votes

1

answer

1.1k

Views

### PyTorch: Add validation error in training

I am using PyTorch to train a cnn model. Here is my Network architecture:
import torch
from torch.autograd import Variable
import torch.nn as nn
import torch.nn.functional as F
import torch.nn.init as I
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.conv1 = nn.Conv2d(1, 3...

1

votes

1

answer

320

Views

### PyTorch : How to properly create a list of nn.Linear()

I have created a class that has nn.Module as subclass.
In my class, I have to create N number of linear transformation, where N is given as class parameters.
I therefore proceed as follow :
self.list_1 = []
for i in range(N):
self.list_1.append(nn.Linear(self.x, 1, bias=mlp_bias))
In the forward met...

1

votes

1

answer

526

Views

### ‘DataParallel’ object has no attribute ‘init_hidden’

What I want to do is using DataParallel in my custom RNN class.
It seems like I initialized hidden_0 in a wrong way...
class RNN(nn.Module):
def __init__(self, input_size, hidden_size, output_size, n_layers=1):
super(RNN, self).__init__()
self.input_size = input_size
self.hidden_size = hidden_size
s...

1

votes

1

answer

925

Views

### Where do I get a CPU-only version of PyTorch?

I'm trying to get a basic app running with Flask + PyTorch, and host it on Heroku. However, I run into the issue that the maximum slug size is 500mb on the free version, and PyTorch itself is ~500mb.
After some google searching, someone wrote about finding a cpu-only version of PyTorch, and using t...

0

votes

0

answer

2

Views

### Pytorch problem testing CNN: RuntimeError: shape '[64, 1]' is invalid for input of size 1920

Testing network:
def test(args, model, device, test_loader):
model.eval()
total = 0
test_loss = 0
correct = 0
with torch.no_grad():
for batch_idx, batch in enumerate(test_loader):
data = batch['image']
target = batch['key_points']
data, target = data.to(device), target.to(device)
# data, target = Va...

1

votes

0

answer

13

Views

### How to transfer the follow Embedding code in tensorflow to pytorch?

I have an Embedding code in Tensorflow as follow
self.input_u = tf.placeholder(tf.int32, [None, user_length], name='input_u')
with tf.name_scope('user_embedding'):
self.W1 = tf.Variable(
tf.random_uniform([user_vocab_size, embedding_size], -1.0, 1.0),
name='W')
self.embedded_user = tf.nn.embedding_l...

1

votes

1

answer

197

Views

### Standard deviation in batches

I am using this function to compute the standard deviation of my output. I am evaluating my model on 2000 examples, in batches of 4. Can I just take the standard deviation per batch and then divide by 2000 ?

1

votes

1

answer

2k

Views

### subtraction of scalar from tensor yields 'inconsistent tensor size' in pytorch

I'm using pytorch and my variables are
x = [torch.FloatTensor of size 1x3x32x32]
mean = Variable containing:
1.00000e-02 *
2.0518
[torch.FloatTensor of size 1]
what I want to do is subtract the scalar mean from x by doing
x = x - mean
However, I'm getting this error:
RuntimeError: inconsistent tenso...

1

votes

1

answer

275

Views

### Backpropagation without updating weights (DCGAN)

The basic idea is, that I load a trained model (DCGAN) and make a anomaly detection with it on images. For the anomaly detection I have to do some iterations on the test phase to evaluate it, if it is a anomaly or not.
For that I have two Loss-Functions in the test setup, which should be calculatin...

1

votes

3

answer

1.4k

Views

### Error importing PyTorch - Python

I have tried to install PyTorch using the following command in terminal:
pip install http://download.pytorch.org/whl/torch-0.2.0.post1-cp27-none-macosx_10_7_x86_64.whl
I then run the following code in python:
import torch
torch.__file__
and get the following error:
File '/Users/brian/anaconda/lib/p...

1

votes

1

answer

608

Views

### Pytorch load model

I used tensorflow some days ago. To build conv layers with fixed weights is easy, just pass the weight kernel to conv2d(). And it is convenient to load pretrained models such as VGG19. But I found it did't work that way using pytorch, because conv2d() doesn't accept an explicit kernel but a kernel s...

1

votes

1

answer

630

Views

### When to use individual optimizers in PyTorch?

The example given here uses two optimizers for encoder and decoder individually. Why? And when to do like that?

1

votes

1

answer

416

Views

### Do I need to define the backward() in custom loss function?

I have already to define my own loss function. It does work. The feedforward may not have problem. But I am not sure whether it is correct because I don't define the backward().
class _Loss(nn.Module):
def __init__(self, size_average=True):
super(_Loss, self).__init__()
self.size_average = size_aver...

1

votes

2

answer

358

Views

### NumPy/PyTorch extract subsets of images

In Numpy, given a stack of large images A of size(N,hl,wl), and coordinates x of size(N) and y of size(N) I want to get smaller images of size (N,16,16)
In a for loop it would look like this:
B=numpy.zeros((N,16,16))
for i in range(0,N):
B[i,:,:]=A[i,y[i]:y[i]+16,x[i]:x[i]+16]
But can I do this just...

1

votes

1

answer

41

Views

### How to generate a new Tensor with different vectors in PyTorch?

I want to generate new a○b vector with a and b (○ means element wise multiply). My code is below, but the performance looks bad because of for. Are there any efficient way?
a = torch.rand(batch_size, a_len, hid_dim)
b = torch.rand(batch_size, b_len, hid_dim)
# a_elmwise_mul_b = torch.zeros(batch...

1

votes

1

answer

553

Views

### torch.pow does not work

I'm trying to create a custom loss function using PyTorch, and am running into a simple error.
When I try to use torch.pow to take the exponent of a PyTorch Variable, I get the following error message:
AttributeError: 'torch.LongTensor' object has no attribute 'pow'
In the python terminal, I create...

1

votes

1

answer

322

Views

### Does tensorflow have the function similar to pytorch's “masked_fill_”

I want to set INF value to matrix by mask matrix, just like pytorch code:
scores.data.masked_fill_(y_mask.data, -float('inf'))
I try to use tf.map_fn to implement that, but the performance is too slow. So does tensorflow have any efficient function to implement that?

1

votes

1

answer

608

Views

### How can I give a name to each module in ModuleList?

I have the following component in my model:
feedfnn = []
for task_name, num_class in self.tasks:
if self.config.nonlinear_fc:
ffnn = nn.Sequential(OrderedDict([
('dropout1', nn.Dropout(self.config.dropout_fc)),
('dense1', nn.Linear(self.config.nhid * self.num_directions * 8, self.config.fc_dim)),
('...

1

votes

1

answer

1.7k

Views

### PyTorch ValueError: Target and input must have the same number of elements

I am kinda new to PyTorch, but I am trying to understand how the sizes of target and input work in torch.nn.BCELoss() when computing loss function.
import torch
import torch.nn as nn
from torch.autograd import Variable
time_steps = 15
batch_size = 3
embeddings_size = 100
num_classes = 2
model = nn.L...

1

votes

2

answer

786

Views

### What is the function in TensorFlow that is equivalent to expand() in PyTorch?

Let's say I have a 2 x 3 matrix and I want to create a 6 x 2 x 3 matrix where each element in the first dimension is the original 2 x 3 matrix.
In PyTorch, I can do this:
import torch
from torch.autograd import Variable
import numpy as np
x = np.array([[1, 2, 3], [4, 5, 6]])
x = Variable(torch.from_...

1

votes

1

answer

89

Views

### Pytorch: Updating numpy array not updating the corresponding tensor

When I run the following code,
import numpy as np
a = np.ones(3)
b = torch.from_numpy(a)
np.add(a, 1, out=a)
print(a)
print(b)
Both a and b are 2s.
However, When I run:
import numpy as np
a = np.ones(3)
b = torch.from_numpy(a)
a = a+1
print(a)
print(b)
b remains as 1s while a has been updated to 2s....

1

votes

1

answer

289

Views

### Pytorch transfer learning predictions

I have been following the pytorch transfer learning tutorial,and following the tutorial to my own dataset i have arrived at this
model_conv = train_model(model_conv, criterion, optimizer_conv,
exp_lr_scheduler, num_epochs=25)
Epoch 1/1
......
......
......
Epoch 24/24
train Loss: 0.8674 Acc: 0.5784...

1

votes

1

answer

1.4k

Views

### Simple Pytorch Example - Loss on training doesnt decrease

I am just starting to try and learn pytorch and am finding it frustrating regardless of how it is advertised :)
Here I am running a simple regression as an experiment but since the loss doesn't seem to be decreasing with each epoch (on the training) I must be doing something wrong -- either in train...

1

votes

2

answer

1.1k

Views

### Why VGG-16 takes input size 512 * 7 * 7?

According to https://github.com/pytorch/vision/blob/master/torchvision/models/vgg.py
I don`t understand why VGG models take 512 * 7 * 7 input_size of fully-connected layer.
Last convolution layer is
nn.Conv2d(512, 512, kernel_size=3, padding=1),
nn.ReLU(True),
nn.MaxPool2d(kernel_size=2, stri...

1

votes

1

answer

1.1k

Views

### Is a torch.FloatTensor not a Tensor?

Although this example has no training, this is an adapted part of a larger program where training does occur. I simply want the generator network to spew out a random image in this case:
import torch
from torch.autograd import Variable
import torch.nn as nn
import torch.nn.functional as F
from torch...

1

votes

1

answer

317

Views

### Image Captioning Example input size of Decoder LSTM Pytorch

I'm new to Pytorch, there is a doubt that am having in the Image Captioning example code . In DcoderRNN class the lstm is defined as ,
self.lstm = nn.LSTM(embed_size, hidden_size, num_layers, batch_first=True)
in the forward function ,
embeddings = self.embed(captions)
embeddings = torch.cat((feat...

1

votes

1

answer

101

Views

### How do you change require_grad to false for each parameters in your model in pytorch?

My code is below which I thought would do what I want but the output shows require_grad didn't change to false.
import torch
import torch.nn as nn
encoder = nn.Sequential( nn.Conv2d(1, 4, 1), nn.Sigmoid())
for params in encoder.parameters():
params.require_grad = False
print(params.requires_grad) #...

1

votes

1

answer

274

Views

### Problems about torch.nn.DataParallel

I am new in deep learning area. Now I am reproducing a paper’s codes. since they use several GPUs, there is a command torch.nn.DataParallel(model, device_ids= args.gpus).cuda() in codes. But I only have one GPU, what
should I change this code to match up my GPU?
Thank you!

1

votes

2

answer

236

Views

### Pytorch DataLoader multiple data source

I am trying to use Pytorch dataloader to define my own dataset, but I am not sure how to load multiple data source:
My current code:
class MultipleSourceDataSet(Dataset):
def __init__ (self, json_file, root_dir, transform = None):
with open(root_dir + 'block0.json') as f:
self.result = torch.Tensor(...

1

votes

1

answer

249

Views

### pytorch Crossentropy error in simple example of NN

H1, I am try to make NN model that satisfy simple formula.
y = X1^2 + X2^2
But when i use CrossEntropyLoss for loss function, i get two different error message.
First, when i set code like this
x = torch.randn(batch_size, 2)
y_hat = model(x)
y = answer(x).long()
optimizer.zero_grad()
loss = loss_f...

1

votes

3

answer

519

Views

### TypeError: object of type 'numpy.int64' has no len()

I am making a DataLoader from DataSet in PyTorch.
Start from loading the DataFrame with all dtype as an np.float64
result = pd.read_csv('dummy.csv', header=0, dtype=DTYPE_CLEANED_DF)
Here is my dataset classes.
from torch.utils.data import Dataset, DataLoader
class MyDataset(Dataset):
def __init__(...

1

votes

1

answer

35

Views

### What is loss_cls and loss_bbox and why are they always zero in training

I'm trying to train a custom dataset on using faster_rcnn using the Pytorch implementation of Detectron here. I have made changes to the dataset and configuration according to the guidelines in the repo.
The training process is carried out successfully, but the loss_cls and loss_bbox values are 0 fr...