Questions tagged [pytorch]

0

votes
0

answer
6

Views

ValueError: operands could not be broadcast together with shapes (50,50,512) (3,) (50,50,512) while converting tensor to image in pytorch

I'm doing a neural style transfer. I'm trying to reconstruct the output of the convolutional layer conv4_2 of the VGG19 network. def get_features(image, model): layers = {'0': 'conv1_1', '5': 'conv2_1', '10': 'conv3_1', '19': 'conv4_1', '21': 'conv4_2', '28': 'conv5_1'} x = image features = {} fo...
Judy T Raj
4

votes
1

answer
139

Views

How to transfer the follow tensorflow into pytorch

I want to re-implement the word embedding here here is the original tensorflow code (version: 0.12.1) import tensorflow as tf class Network(object): def __init__( self, user_length,item_length, num_classes, user_vocab_size,item_vocab_size,fm_k,n_latent,user_num,item_num, embedding_size, filter_size...
jason
0

votes
0

answer
3

Views

PyTorch implementation: Transformer not generating correct sentences with test data

I've implemented Transformer model with pytorch (the github repo is here), and the validation cost seems decreasing while training. But when feeding test data, the model only generates ..., so there must be bugs somewhere in the code. Could someone help me find the bug? (I've also merged all related...
yusugomori
1

votes
2

answer
33

Views

PyTorch conversion between tensor and numpy array: the addition operation

I am following the 60-minute blitz on PyTorch but have a question about conversion of a numpy array to a tensor. Tutorial example here. This piece of code: import numpy as np a = np.ones(5) b = torch.from_numpy(a) np.add(a, 1, out=a) print(a) print(b) yields [2. 2. 2. 2. 2.] tensor([2., 2., 2., 2.,...
syltruong
1

votes
1

answer
336

Views

Order of CUDA devices [duplicate]

This question already has an answer here: How does CUDA assign device IDs to GPUs? 3 answers I saw this solution, but it doesn't quite answer my question; it's also quite old so I'm not sure how relevant it is. I keep getting conflicting outputs for the order of GPU units. There are two of them: T...
Alex
0

votes
0

answer
5

Views

Display examples of augmented images in PyTorch

I want to display some samples of augmented training images. My transform includes the standard ImageNet transforms.Normalize like this: train_transforms = transforms.Compose([transforms.RandomRotation(30), transforms.RandomResizedCrop(224), transforms.RandomHorizontalFlip(), transforms.ToTensor(),...
Tom Hale
0

votes
0

answer
4

Views

Need very different learning rate for manual updates vs. using model

I am currently just trying to write some pedagogical material, in which I borrow from some common examples that have been reworked numerous times on the web. I have a simple bit of code where I manually create tensors for layers, and update them within a loop. E.g.: w1 = torch.randn(D_in, H, dtype=...
David Mertz
1

votes
1

answer
1.1k

Views

Pytorch hidden state LSTM

Why do we need to initialize the hidden state h0 in LSTM in pytorch. As h0 will anyways be calculated and get overwritten ? Isn't it like int a a = 0 a = 4 Even if we do not do a=0, it should be fine..
sww
1

votes
1

answer
1.1k

Views

PyTorch: Add validation error in training

I am using PyTorch to train a cnn model. Here is my Network architecture: import torch from torch.autograd import Variable import torch.nn as nn import torch.nn.functional as F import torch.nn.init as I class Net(nn.Module): def __init__(self): super(Net, self).__init__() self.conv1 = nn.Conv2d(1, 3...
Edamame
1

votes
1

answer
320

Views

PyTorch : How to properly create a list of nn.Linear()

I have created a class that has nn.Module as subclass. In my class, I have to create N number of linear transformation, where N is given as class parameters. I therefore proceed as follow : self.list_1 = [] for i in range(N): self.list_1.append(nn.Linear(self.x, 1, bias=mlp_bias)) In the forward met...
Mickey
1

votes
1

answer
526

Views

‘DataParallel’ object has no attribute ‘init_hidden’

What I want to do is using DataParallel in my custom RNN class. It seems like I initialized hidden_0 in a wrong way... class RNN(nn.Module): def __init__(self, input_size, hidden_size, output_size, n_layers=1): super(RNN, self).__init__() self.input_size = input_size self.hidden_size = hidden_size s...
Weimin Chan
1

votes
1

answer
925

Views

Where do I get a CPU-only version of PyTorch?

I'm trying to get a basic app running with Flask + PyTorch, and host it on Heroku. However, I run into the issue that the maximum slug size is 500mb on the free version, and PyTorch itself is ~500mb. After some google searching, someone wrote about finding a cpu-only version of PyTorch, and using t...
Matt
0

votes
0

answer
2

Views

Pytorch problem testing CNN: RuntimeError: shape '[64, 1]' is invalid for input of size 1920

Testing network: def test(args, model, device, test_loader): model.eval() total = 0 test_loss = 0 correct = 0 with torch.no_grad(): for batch_idx, batch in enumerate(test_loader): data = batch['image'] target = batch['key_points'] data, target = data.to(device), target.to(device) # data, target = Va...
wantToCode
1

votes
0

answer
13

Views

How to transfer the follow Embedding code in tensorflow to pytorch?

I have an Embedding code in Tensorflow as follow self.input_u = tf.placeholder(tf.int32, [None, user_length], name='input_u') with tf.name_scope('user_embedding'): self.W1 = tf.Variable( tf.random_uniform([user_vocab_size, embedding_size], -1.0, 1.0), name='W') self.embedded_user = tf.nn.embedding_l...
jason
1

votes
1

answer
197

Views

Standard deviation in batches

I am using this function to compute the standard deviation of my output. I am evaluating my model on 2000 examples, in batches of 4. Can I just take the standard deviation per batch and then divide by 2000 ?
Qubix
1

votes
1

answer
2k

Views

subtraction of scalar from tensor yields 'inconsistent tensor size' in pytorch

I'm using pytorch and my variables are x = [torch.FloatTensor of size 1x3x32x32] mean = Variable containing: 1.00000e-02 * 2.0518 [torch.FloatTensor of size 1] what I want to do is subtract the scalar mean from x by doing x = x - mean However, I'm getting this error: RuntimeError: inconsistent tenso...
SumakuTension
1

votes
1

answer
275

Views

Backpropagation without updating weights (DCGAN)

The basic idea is, that I load a trained model (DCGAN) and make a anomaly detection with it on images. For the anomaly detection I have to do some iterations on the test phase to evaluate it, if it is a anomaly or not. For that I have two Loss-Functions in the test setup, which should be calculatin...
loose11
1

votes
3

answer
1.4k

Views

Error importing PyTorch - Python

I have tried to install PyTorch using the following command in terminal: pip install http://download.pytorch.org/whl/torch-0.2.0.post1-cp27-none-macosx_10_7_x86_64.whl I then run the following code in python: import torch torch.__file__ and get the following error: File '/Users/brian/anaconda/lib/p...
brian4342
1

votes
1

answer
608

Views

Pytorch load model

I used tensorflow some days ago. To build conv layers with fixed weights is easy, just pass the weight kernel to conv2d(). And it is convenient to load pretrained models such as VGG19. But I found it did't work that way using pytorch, because conv2d() doesn't accept an explicit kernel but a kernel s...
Sun Chuanneng
1

votes
1

answer
630

Views

When to use individual optimizers in PyTorch?

The example given here uses two optimizers for encoder and decoder individually. Why? And when to do like that?
Edityouprofile
1

votes
1

answer
416

Views

Do I need to define the backward() in custom loss function?

I have already to define my own loss function. It does work. The feedforward may not have problem. But I am not sure whether it is correct because I don't define the backward(). class _Loss(nn.Module): def __init__(self, size_average=True): super(_Loss, self).__init__() self.size_average = size_aver...
Ruijian
1

votes
2

answer
358

Views

NumPy/PyTorch extract subsets of images

In Numpy, given a stack of large images A of size(N,hl,wl), and coordinates x of size(N) and y of size(N) I want to get smaller images of size (N,16,16) In a for loop it would look like this: B=numpy.zeros((N,16,16)) for i in range(0,N): B[i,:,:]=A[i,y[i]:y[i]+16,x[i]:x[i]+16] But can I do this just...
Aleksk89
1

votes
1

answer
41

Views

How to generate a new Tensor with different vectors in PyTorch?

I want to generate new a○b vector with a and b (○ means element wise multiply). My code is below, but the performance looks bad because of for. Are there any efficient way? a = torch.rand(batch_size, a_len, hid_dim) b = torch.rand(batch_size, b_len, hid_dim) # a_elmwise_mul_b = torch.zeros(batch...
jef
1

votes
1

answer
553

Views

torch.pow does not work

I'm trying to create a custom loss function using PyTorch, and am running into a simple error. When I try to use torch.pow to take the exponent of a PyTorch Variable, I get the following error message: AttributeError: 'torch.LongTensor' object has no attribute 'pow' In the python terminal, I create...
bgenchel
1

votes
1

answer
322

Views

Does tensorflow have the function similar to pytorch's “masked_fill_”

I want to set INF value to matrix by mask matrix, just like pytorch code: scores.data.masked_fill_(y_mask.data, -float('inf')) I try to use tf.map_fn to implement that, but the performance is too slow. So does tensorflow have any efficient function to implement that?
Jack.Liu
1

votes
1

answer
608

Views

How can I give a name to each module in ModuleList?

I have the following component in my model: feedfnn = [] for task_name, num_class in self.tasks: if self.config.nonlinear_fc: ffnn = nn.Sequential(OrderedDict([ ('dropout1', nn.Dropout(self.config.dropout_fc)), ('dense1', nn.Linear(self.config.nhid * self.num_directions * 8, self.config.fc_dim)), ('...
Wasi Ahmad
1

votes
1

answer
1.7k

Views

PyTorch ValueError: Target and input must have the same number of elements

I am kinda new to PyTorch, but I am trying to understand how the sizes of target and input work in torch.nn.BCELoss() when computing loss function. import torch import torch.nn as nn from torch.autograd import Variable time_steps = 15 batch_size = 3 embeddings_size = 100 num_classes = 2 model = nn.L...
Daniel Le
1

votes
2

answer
786

Views

What is the function in TensorFlow that is equivalent to expand() in PyTorch?

Let's say I have a 2 x 3 matrix and I want to create a 6 x 2 x 3 matrix where each element in the first dimension is the original 2 x 3 matrix. In PyTorch, I can do this: import torch from torch.autograd import Variable import numpy as np x = np.array([[1, 2, 3], [4, 5, 6]]) x = Variable(torch.from_...
mauna
1

votes
1

answer
89

Views

Pytorch: Updating numpy array not updating the corresponding tensor

When I run the following code, import numpy as np a = np.ones(3) b = torch.from_numpy(a) np.add(a, 1, out=a) print(a) print(b) Both a and b are 2s. However, When I run: import numpy as np a = np.ones(3) b = torch.from_numpy(a) a = a+1 print(a) print(b) b remains as 1s while a has been updated to 2s....
Allen
1

votes
1

answer
289

Views

Pytorch transfer learning predictions

I have been following the pytorch transfer learning tutorial,and following the tutorial to my own dataset i have arrived at this model_conv = train_model(model_conv, criterion, optimizer_conv, exp_lr_scheduler, num_epochs=25) Epoch 1/1 ...... ...... ...... Epoch 24/24 train Loss: 0.8674 Acc: 0.5784...
Ryan
1

votes
1

answer
1.4k

Views

Simple Pytorch Example - Loss on training doesnt decrease

I am just starting to try and learn pytorch and am finding it frustrating regardless of how it is advertised :) Here I am running a simple regression as an experiment but since the loss doesn't seem to be decreasing with each epoch (on the training) I must be doing something wrong -- either in train...
B_Miner
1

votes
2

answer
1.1k

Views

Why VGG-16 takes input size 512 * 7 * 7?

According to https://github.com/pytorch/vision/blob/master/torchvision/models/vgg.py I don`t understand why VGG models take 512 * 7 * 7 input_size of fully-connected layer. Last convolution layer is nn.Conv2d(512, 512, kernel_size=3, padding=1), nn.ReLU(True), nn.MaxPool2d(kernel_size=2, stri...
Nazzzz
1

votes
1

answer
1.1k

Views

Is a torch.FloatTensor not a Tensor?

Although this example has no training, this is an adapted part of a larger program where training does occur. I simply want the generator network to spew out a random image in this case: import torch from torch.autograd import Variable import torch.nn as nn import torch.nn.functional as F from torch...
Dair
1

votes
1

answer
317

Views

Image Captioning Example input size of Decoder LSTM Pytorch

I'm new to Pytorch, there is a doubt that am having in the Image Captioning example code . In DcoderRNN class the lstm is defined as , self.lstm = nn.LSTM(embed_size, hidden_size, num_layers, batch_first=True) in the forward function , embeddings = self.embed(captions) embeddings = torch.cat((feat...
Arijit
1

votes
1

answer
101

Views

How do you change require_grad to false for each parameters in your model in pytorch?

My code is below which I thought would do what I want but the output shows require_grad didn't change to false. import torch import torch.nn as nn encoder = nn.Sequential( nn.Conv2d(1, 4, 1), nn.Sigmoid()) for params in encoder.parameters(): params.require_grad = False print(params.requires_grad) #...
Inkplay_
1

votes
1

answer
274

Views

Problems about torch.nn.DataParallel

I am new in deep learning area. Now I am reproducing a paper’s codes. since they use several GPUs, there is a command torch.nn.DataParallel(model, device_ids= args.gpus).cuda() in codes. But I only have one GPU, what should I change this code to match up my GPU? Thank you!
Jiangang yang
1

votes
2

answer
236

Views

Pytorch DataLoader multiple data source

I am trying to use Pytorch dataloader to define my own dataset, but I am not sure how to load multiple data source: My current code: class MultipleSourceDataSet(Dataset): def __init__ (self, json_file, root_dir, transform = None): with open(root_dir + 'block0.json') as f: self.result = torch.Tensor(...
sealpuppy
1

votes
1

answer
249

Views

pytorch Crossentropy error in simple example of NN

H1, I am try to make NN model that satisfy simple formula. y = X1^2 + X2^2 But when i use CrossEntropyLoss for loss function, i get two different error message. First, when i set code like this x = torch.randn(batch_size, 2) y_hat = model(x) y = answer(x).long() optimizer.zero_grad() loss = loss_f...
justin_sakong
1

votes
3

answer
519

Views

TypeError: object of type 'numpy.int64' has no len()

I am making a DataLoader from DataSet in PyTorch. Start from loading the DataFrame with all dtype as an np.float64 result = pd.read_csv('dummy.csv', header=0, dtype=DTYPE_CLEANED_DF) Here is my dataset classes. from torch.utils.data import Dataset, DataLoader class MyDataset(Dataset): def __init__(...
Sarit
1

votes
1

answer
35

Views

What is loss_cls and loss_bbox and why are they always zero in training

I'm trying to train a custom dataset on using faster_rcnn using the Pytorch implementation of Detectron here. I have made changes to the dataset and configuration according to the guidelines in the repo. The training process is carried out successfully, but the loss_cls and loss_bbox values are 0 fr...
RakihthaRR

View additional questions