Questions tagged [vectorization]

1

votes
1

answer
203

Views

GCC Auto Vectorization

In gcc compiler is there a way to enable auto vectorization only? I do know that -ftree-vectorize flag enables auto vectorization. But it requires at least -O2 optimization level. Is there a way to enable auto vectorization without using the -O2 optimization flag? Thanks in advance.
Pasindu
1

votes
4

answer
76

Views

Increasing the speed of finding an index of an item in list

I'm trying to speed up a program I've written, and after importing cProfile, I see that one function takes up a massive bit of computation time. It's this, which finds an numpy.ndarray in a list: def locate(arr, l ): for i in range(len(l)): if np.all(l[i] == arr): return i return -1 As the list can'...
cucumber
1

votes
1

answer
34

Views

Vector double-double floating point arithmetic

There exist workloads for which double precision floating point is not quite adequate, hence a need for quad precision. This is rarely supplied in hardware, so a workaround is to use double-double, where a 128-bit number is represented by a pair of 64-bit numbers. It's not true IEEE-754 quad precisi...
rwallace
1

votes
2

answer
45

Views

Scalar-Vector multiplication with meshgrid (Matlab)

I have an expression such as s=aU+bV, where a and b are scalars and U and V 3-component vectors. The output s is clearly a 3-component vector. Let's assume I want to plot the first component of s find out how this varies when I change a and b. In order to plot I have to use surf, which takes matrice...
usumdelphini
1

votes
2

answer
39

Views

How to vectorize calculation of homogenous transformation matrix/tensor?

For my simulation I need to calculate many transformation matrices therefore I would like to vectorize a for-loop that I'm using right now. Is there a way to vectorize the existing for-loop or do I probably need another approach in calculating the vectors and matrices before? I prepared a little wor...
Berger
1

votes
5

answer
105

Views

Python, iterating and modifying DataFrames in a dictionary of df's

Coming from a C# background (years ago) and being very new to Python I'm struggling to optimise my code. Literally just learned that for loops are very slow. In the code below the loop that adds a calculated column to each DataFrame in the Dict appears to be a huge bottleneck. I've read up on ways t...
DublinMeUp
1

votes
0

answer
36

Views

Vectorize quintuple for loop in numpy, with non-trivial indexing

I have a series of nested for loops, used to populate a rank-3 array as follows: S = ... # Some number N = ... # Some other number an_array = np.zeros((S, N * N, N * N)) for s in range(S): for i in range(N): for a in range(N): for j in range(N): for b in range(N): an_array[s, i * N + a, j * N + b]...
Daniele Grattarola
1

votes
0

answer
249

Views

Elementwise bitwise-XOR numpy matrix with numpy vector

Suppose that we are given a two dimensional matrix A of dtype=uint8 with N rows and M columns and a uint8 vector of size N called x. We need to bit-wise XOR each row of A, e.g. A[i], with the corresponding element in x, i.e. x[i]. Currently, I am doing this as follows, but think that there are more...
Ali Shakiba
1

votes
0

answer
176

Views

What is the expected return type for tokenizer that is passed as parameter into Tfidfvectorizer

I am looking at: http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html It just says: tokenizer : callable or None (default) Override the string tokenization step while preserving the preprocessing and n-grams generation steps. Only applies if analyzer...
Kaizer Sozay
1

votes
0

answer
61

Views

Performance of using a matrix as vector index

In my code I have a slow part of which the idea can be summarized in the following short example: A = randi(10,5); %Random 5×5 matrix containing integers ranging from 0 to 10 B = rand(10,1); %Random 10×1 vector containing values ranging from 0 to 1 C = B(A); %New 5×5 matrix consisting of...
Knippie
1

votes
1

answer
55

Views

Does OpenMP hints bypass the vectorisation legality check in llvm

I am currently looking into how '#pragma omp for simd' is actually recognised in llvm. To my knowledge, clang will parse it and set metadata in IR to indicate this force-vectorisation hint and later optimisation passes would read it and vectorise the marked loop. Therefore, the loop should be vector...
PST
1

votes
0

answer
290

Views

Vectorize Conway's Game of Life in pure numpy?

I'm wondering if there is a way to implement Conway's game of life without resorting to for loops, if statements and other control structures typical of programming. It should be pretty easy to vectorize for loops, but how would you convert the checks on the neighborhood to a matrix operation? The...
Daniele Grattarola
1

votes
0

answer
123

Views

Is there a way to vectorize linalg.expm

To follow up on this question I am new to python and I am trying to calculate the exponential of a product Matrix - Scalar using vectorization (if possible) What I did: n=10 t_ = np.arange(1,n+1)*5*np.pi/n a_11,a_12, a_21, a_22=0,1,-1,-1 x_0,v_0=1,1 A = np.array([[a_11,a_12], [a_21, a_22]]) A_ =...
PerelMan
1

votes
0

answer
64

Views

np.vectorize local variable (?) referenced before assignment

I am working on a python code where I want to make some modifications to datetime.datetime instances. I am capable of making the code work, yet I do not understand why, I just gave what the error asked for. def ts_round(ts,lev,p): sum = False hour = 0 minute = 0 second = 0 if 'hour' is lev: a = ts.h...
Mass
1

votes
0

answer
32

Views

Vectorized addition of vectors with sparse differences

I represent vectors with a lot of repeated elements by a index-vector and a value vector so that the index-vector contains the indices of value-changes in the vector and the value-vector contains the value at those indices. For instance: [1, 1, 1, 7, 7, 4, 4] is represented by index-vector [0, 3, 5...
kuppern87
1

votes
1

answer
66

Views

column name in column -> copy specified column's value to single column

I have a dataset where I have column names in a column, and I'd like to copy the value of that column to a specific column. While I can do this with purrrly::by_row() or apply(), this seems inefficient and I'm looking for a more vectorized solution if possible. Here's a setup where I have column na...
Dekan
1

votes
0

answer
36

Views

When is np.vecotrize useful

I am wondering when should we use np.vectorize over lambda, for example suppose I want to do the following operation over each element of numpy array for which I have lambda like f1 = lambda x: x / 12.92 if x < .03928 else ((x + .055) / 1.055) ** 2.4 and I vectorize it as f2 = np.vectorize(f1) My ob...
pradystar
1

votes
1

answer
43

Views

How to use vectorization with if conditional logic

Question I am new to programming and am experiencing an issue with conditional logic. In my function the condition has length > 1 and thus only the first case of it being true will populate. How would I go about implementing the correct functionality? Output I get incorrect information for the Base...
datanalyst
1

votes
1

answer
94

Views

vectorizing a point to line/segment/ray distance function (matlab)

I am using the function from here in order to calculate the distances of multiple rays to multiple points in the space. The way I am doing it now is by forming two for loops as you can notice in the code below since the distancePoint2Line() function is not vectorized. However, when I have to through...
ThT
1

votes
0

answer
483

Views

Is it possible to vectorize this python function to run on gpu?

I have this function (see below) which takes a tuple of point coordinates and returns a bool if the point is within a shapely polygon (global variable). I want to vectorize it to take an array of tuples and return the equivalent array of bool. It works nicely with numpy.vectorize, but it's slow (6-...
Radusaurus
1

votes
1

answer
38

Views

using apply function based on a previous row/ Converting a looping statement to an apply function

I am currently working on a file which has multiple users profile changes based on every single time their profile was updated for a particular instance. I would like to make the column a 1 if there was a change from the previous profile in that particular column and keep it a zero if there wasn't a...
Shikhar Sharma
1

votes
0

answer
186

Views

Vectorizing merge/union of two sorted arrays

I have recently started looking into opportunities to speed up my code by using vector instructions. My code heavily relies on operations with sets - for simplicity let us assume that these are represented as sorted arrays of 16bit unsigned integers. The operations I need to perform are: Intersectio...
Karel Horak
1

votes
1

answer
34

Views

Is there a way to vectorize selection of columns (with repetition) from a matrix?

I have a matrix L of size n x k and a vector Z of size p. Z is composed of integers which represent the column indices of L. I want to create a matrix X of size n x p which is the aggregation of the corresponding columns of L selected based on the values in Z. Z = c(1, 3, 1, 2) L = matrix(c(73,50,4,...
user5054
1

votes
0

answer
17

Views

How can I find when the vector values become zero?

I have a series of dates along with a numeric vector, for example: dates
Gabriel Gajardo
1

votes
2

answer
66

Views

Function not finding an argument when vectorized in R?

I'm wondering why my vectorized function below works fine when I use cii(peta = c(.3, .4), N = 120, df1 = 3, df2 = 116) BUT when I use cii(F.value = c(30, 40), N = 120, df1 = 3, df2 = 116) the function gives 'peta' is missing? P.S. I have made it clear in my function that when F.value is NA, functio...
rnorouzian
1

votes
0

answer
76

Views

Vectorizing conditional count in Pandas

I have a Pandas script that counts the number of readmissions to hospital within 30 days based on a few conditions. I wonder if it could be vectorized to improve performance. I've experimented with df.rolling().apply, but so far without luck. Here's a table with contrived data to illustrate: ID VIS...
gherka
1

votes
0

answer
44

Views

Conditional slicing of multi dimensional numpy arrays

I have a five dimensional matrix e.g.: matrix = np.random.random([3,3,3,10,2]) I think of this as a 3x3x3 cube, each voxel of which stores a 10x2 matrix. At each voxel I want to e.g. get the mean of a subset items in the first column of this 10x2 matrix based on values in the second column of this 5...
RobMcC
1

votes
0

answer
35

Views

Vectorizing for loops in python when read some images

I'm new in python and i wrote a code to read some images in 2 for Loops and compute some values and then added them into a dictionary but because of huge number of images , it doesn't efficient at all. I want to help me for vectorizing this code . Thanks Testimages = [f for f in glob(f'{test_dir}/*....
hdiz
1

votes
0

answer
27

Views

How do I cache vectorized calls that take user input in R?

I am trying to calculate a field for all rows of a large dataset. The function to calculate it is from the package taxize, and uses an HTTP request to query an external site for the right ID number. It is searching by scientific name, and often there are multiple results, in which case this function...
choff
1

votes
0

answer
64

Views

Error When Checking Target (Keras)

Hi I am following this code along from Deep learning with python Book. I am trying to train a model with pretrained word embeddings on raw IMDB dataset. I get this error how to fix this. I followed this process 1. Processing the labels of the raw IMDB data 2. Tokenizing the text of the raw IMDB data...
1

votes
1

answer
100

Views

speed up R code - vectorize?

I am struggling to make my code faster. Currently I am getting somewhere using doParallel, but I am wondering if it could be made faster though clever programming instead of faster hardware. Here is a stylized version of what I want to do: library(dplyr) library(doParallel) library(data.table) cl
bjvca
1

votes
2

answer
40

Views

Vectorization of a nested for-loop that inputs all paired combinations

I thought that the following problem must have been answered or a function must exist to do it, but I was unable to find an answer. I have a nested loop that takes a row from one 3-col. data frame and copies it next to each of the other rows, to form a 6-col. data frame (with all possible combinatio...
Amos
1

votes
0

answer
63

Views

Numpy: populating arrays to vectorize nested summations

I would like to vectorize nested sums in Python, in order to speed up the process. At the moment I have nested for loops. for ja in np.arange(0,Na): for jb in np.arange(0,Nb): for ma in np.arange(-ja,ja+1): ... The end result is the sum across 2x2 matrices, each with entries dependent on the values...
maor
1

votes
0

answer
80

Views

Taking specific 2d array from 3d in numpy

Is there a way to avoid using the for loop and get the result just by calling arr with some indexing? Potentially dim1 will be equal to 50 000, dim2 up to 1000, dim3 fixed to 3. import numpy as np dim1 = 10 dim2 = 2 dim3 = 3 arr = np.arange(60).reshape(dim1,dim2,dim3) arr2 = np.arange(dim1*dim2).res...
strikal
1

votes
0

answer
98

Views

Vectorization of pandas groupby using conditional filters

Let's say I have a sample df, such as: df = pd.DataFrame([['William', 1, 0, 'T', 0, 1],['James', 0, 1, 'R', 1, 1],['James', 1, 0, 'S', 0, 1],['Dean', 1, 0, 'R', 1, 0],['William', 0, 1, 'S', 0, 0],['James', 0, 0, 'S', 0, 1]],columns=['Name','x1','x2','x3','x4','x5']) Which gives: Name x1 x2 x3 x4...
rahlf23
1

votes
1

answer
56

Views

Correct Way to Parallelize Pandas on Data Frame Slices

let's assume I have a data frame with N multi valued categorical columns and I want to encode them as fast as possible using Pandas. This is what i achieved so far, not sure if it is the best way to parallelize Pandas though (I would prefer a vectorized approach where possible) : def encode_single_c...
1

votes
1

answer
25

Views

How to get formulas of multiple regressions by vectorizing

Suppose I have the following code that makes multiple regressions and stores the lm and lm with stepwise selection models in tibbles: library(dplyr) library(tibble) library(MASS) set.seed(1) df
kkz
1

votes
2

answer
37

Views

Mixing 3D arrays into a bigger 3D array

I wish to fill a N x M x W matrix ‘S’ with the data from matrices ‘P’ and ‘Q’. They are defined below and illustrated in the attached image. Also, we know for sure that n_1 + n_2 = N, m < M, so all the data may fit in the ‘S’ matrix. S = zeros(M,N,W); P = rand(m,n_1,W); Q = rand(m,n_...
Fennec
1

votes
2

answer
60

Views

Move index to the left, according to grid

Let's say I have a grid that contains permissible and forbidden zones. import numpy as np forbidden = np.array([False, False, False, False, False, False, True, True, True, True, True, True, True, True, True, True, True, True, True, True, True, True, True, True, True, True, True, T...
FooBar
1

votes
1

answer
165

Views

x86_64 Dot Vector Product Intrinsic to ARM64

I am porting a small C routine that includes x86_64 intrinsics to a ARM64 platform. I am unable to find an equivalent ARM64 intrinsic for _mm_dp_pd. I do have access to the arm neon intrinsics. I am not sure how to replace x86_64 intrinsic with a ARM64 equivalent. Any help would be much appreciate...
weegz

View additional questions