# Questions tagged [vectorization]

2749 questions

1

votes

1

answer

203

Views

### GCC Auto Vectorization

In gcc compiler is there a way to enable auto vectorization only? I do know that -ftree-vectorize flag enables auto vectorization. But it requires at least -O2 optimization level. Is there a way to enable auto vectorization without using the -O2 optimization flag?
Thanks in advance.

1

votes

4

answer

76

Views

### Increasing the speed of finding an index of an item in list

I'm trying to speed up a program I've written, and after importing cProfile, I see that one function takes up a massive bit of computation time.
It's this, which finds an numpy.ndarray in a list:
def locate(arr, l ):
for i in range(len(l)):
if np.all(l[i] == arr):
return i
return -1
As the list can'...

1

votes

1

answer

34

Views

### Vector double-double floating point arithmetic

There exist workloads for which double precision floating point is not quite adequate, hence a need for quad precision. This is rarely supplied in hardware, so a workaround is to use double-double, where a 128-bit number is represented by a pair of 64-bit numbers. It's not true IEEE-754 quad precisi...

1

votes

2

answer

45

Views

### Scalar-Vector multiplication with meshgrid (Matlab)

I have an expression such as s=aU+bV, where a and b are scalars and U and V 3-component vectors. The output s is clearly a 3-component vector. Let's assume I want to plot the first component of s find out how this varies when I change a and b.
In order to plot I have to use surf, which takes matrice...

1

votes

2

answer

39

Views

### How to vectorize calculation of homogenous transformation matrix/tensor?

For my simulation I need to calculate many transformation matrices therefore I would like to vectorize a for-loop that I'm using right now.
Is there a way to vectorize the existing for-loop or do I probably need another approach in calculating the vectors and matrices before?
I prepared a little wor...

1

votes

5

answer

105

Views

### Python, iterating and modifying DataFrames in a dictionary of df's

Coming from a C# background (years ago) and being very new to Python I'm struggling to optimise my code. Literally just learned that for loops are very slow.
In the code below the loop that adds a calculated column to each DataFrame in the Dict appears to be a huge bottleneck.
I've read up on ways t...

1

votes

0

answer

36

Views

### Vectorize quintuple for loop in numpy, with non-trivial indexing

I have a series of nested for loops, used to populate a rank-3 array as follows:
S = ... # Some number
N = ... # Some other number
an_array = np.zeros((S, N * N, N * N))
for s in range(S):
for i in range(N):
for a in range(N):
for j in range(N):
for b in range(N):
an_array[s, i * N + a, j * N + b]...

1

votes

0

answer

249

Views

### Elementwise bitwise-XOR numpy matrix with numpy vector

Suppose that we are given a two dimensional matrix A of dtype=uint8 with N rows and M columns and a uint8 vector of size N called x. We need to bit-wise XOR each row of A, e.g. A[i], with the corresponding element in x, i.e. x[i].
Currently, I am doing this as follows, but think that there are more...

1

votes

0

answer

176

Views

### What is the expected return type for tokenizer that is passed as parameter into Tfidfvectorizer

I am looking at:
http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html
It just says:
tokenizer : callable or None (default) Override the string
tokenization step while preserving the preprocessing and n-grams
generation steps. Only applies if analyzer...

1

votes

0

answer

61

Views

### Performance of using a matrix as vector index

In my code I have a slow part of which the idea can be summarized in the following short example:
A = randi(10,5); %Random 5×5 matrix containing integers ranging from 0 to 10
B = rand(10,1); %Random 10×1 vector containing values ranging from 0 to 1
C = B(A); %New 5×5 matrix consisting of...

1

votes

1

answer

55

Views

### Does OpenMP hints bypass the vectorisation legality check in llvm

I am currently looking into how '#pragma omp for simd' is actually recognised in llvm. To my knowledge, clang will parse it and set metadata in IR to indicate this force-vectorisation hint and later optimisation passes would read it and vectorise the marked loop. Therefore, the loop should be vector...

1

votes

0

answer

290

Views

### Vectorize Conway's Game of Life in pure numpy?

I'm wondering if there is a way to implement Conway's game of life without resorting to for loops, if statements and other control structures typical of programming.
It should be pretty easy to vectorize for loops, but how would you convert the checks on the neighborhood to a matrix operation?
The...

1

votes

0

answer

123

Views

### Is there a way to vectorize linalg.expm

To follow up on this question
I am new to python and I am trying to calculate the exponential of
a product Matrix - Scalar using vectorization (if possible)
What I did:
n=10
t_ = np.arange(1,n+1)*5*np.pi/n
a_11,a_12, a_21, a_22=0,1,-1,-1
x_0,v_0=1,1
A = np.array([[a_11,a_12], [a_21, a_22]])
A_ =...

1

votes

0

answer

64

Views

### np.vectorize local variable (?) referenced before assignment

I am working on a python code where I want to make some modifications to datetime.datetime instances. I am capable of making the code work, yet I do not understand why, I just gave what the error asked for.
def ts_round(ts,lev,p):
sum = False
hour = 0
minute = 0
second = 0
if 'hour' is lev:
a = ts.h...

1

votes

0

answer

32

Views

### Vectorized addition of vectors with sparse differences

I represent vectors with a lot of repeated elements by a index-vector and a value vector so that the index-vector contains the indices of value-changes in the vector and the value-vector contains the value at those indices.
For instance: [1, 1, 1, 7, 7, 4, 4] is represented by index-vector [0, 3, 5...

1

votes

1

answer

66

Views

### column name in column -> copy specified column's value to single column

I have a dataset where I have column names in a column, and I'd like to copy the value of that column to a specific column. While I can do this with purrrly::by_row() or apply(), this seems inefficient and I'm looking for a more vectorized solution if possible.
Here's a setup where I have column na...

1

votes

0

answer

36

Views

### When is np.vecotrize useful

I am wondering when should we use np.vectorize over lambda, for example suppose I want to do the following operation over each element of numpy array for which I have lambda like
f1 = lambda x: x / 12.92 if x < .03928 else ((x + .055) / 1.055) ** 2.4
and I vectorize it as
f2 = np.vectorize(f1)
My ob...

1

votes

1

answer

43

Views

### How to use vectorization with if conditional logic

Question
I am new to programming and am experiencing an issue with conditional logic. In my function the condition has length > 1 and thus only the first case of it being true will populate. How would I go about implementing the correct functionality?
Output
I get incorrect information for the Base...

1

votes

1

answer

94

Views

### vectorizing a point to line/segment/ray distance function (matlab)

I am using the function from here in order to calculate the distances of multiple rays to multiple points in the space. The way I am doing it now is by forming two for loops as you can notice in the code below since the distancePoint2Line() function is not vectorized. However, when I have to through...

1

votes

0

answer

483

Views

### Is it possible to vectorize this python function to run on gpu?

I have this function (see below) which takes a tuple of point coordinates and returns a bool if the point is within a shapely polygon (global variable). I want to vectorize it to take an array of tuples and return the equivalent array of bool.
It works nicely with numpy.vectorize, but it's slow (6-...

1

votes

1

answer

38

Views

### using apply function based on a previous row/ Converting a looping statement to an apply function

I am currently working on a file which has multiple users profile changes based on every single time their profile was updated for a particular instance. I would like to make the column a 1 if there was a change from the previous profile in that particular column and keep it a zero if there wasn't a...

1

votes

0

answer

186

Views

### Vectorizing merge/union of two sorted arrays

I have recently started looking into opportunities to speed up my code by using vector instructions. My code heavily relies on operations with sets - for simplicity let us assume that these are represented as sorted arrays of 16bit unsigned integers. The operations I need to perform are:
Intersectio...

1

votes

1

answer

34

Views

### Is there a way to vectorize selection of columns (with repetition) from a matrix?

I have a matrix L of size n x k and a vector Z of size p. Z is composed of integers which represent the column indices of L. I want to create a matrix X of size n x p which is the aggregation of the corresponding columns of L selected based on the values in Z.
Z = c(1, 3, 1, 2)
L = matrix(c(73,50,4,...

1

votes

0

answer

17

Views

### How can I find when the vector values become zero?

I have a series of dates along with a numeric vector, for example:
dates

1

votes

2

answer

66

Views

### Function not finding an argument when vectorized in R?

I'm wondering why my vectorized function below works fine when I use cii(peta = c(.3, .4), N = 120, df1 = 3, df2 = 116) BUT when I use cii(F.value = c(30, 40), N = 120, df1 = 3, df2 = 116) the function gives 'peta' is missing?
P.S. I have made it clear in my function that when F.value is NA, functio...

1

votes

0

answer

76

Views

### Vectorizing conditional count in Pandas

I have a Pandas script that counts the number of readmissions to hospital within 30 days based on a few conditions. I wonder if it could be vectorized to improve performance. I've experimented with df.rolling().apply, but so far without luck.
Here's a table with contrived data to illustrate:
ID VIS...

1

votes

0

answer

44

Views

### Conditional slicing of multi dimensional numpy arrays

I have a five dimensional matrix e.g.:
matrix = np.random.random([3,3,3,10,2])
I think of this as a 3x3x3 cube, each voxel of which stores a 10x2 matrix.
At each voxel I want to e.g. get the mean of a subset items in the first column of this 10x2 matrix based on values in the second column of this 5...

1

votes

0

answer

35

Views

### Vectorizing for loops in python when read some images

I'm new in python and i wrote a code to read some images in 2 for Loops and compute some values and then added them into a dictionary but because of huge number of images , it doesn't efficient at all. I want to help me for vectorizing this code . Thanks
Testimages = [f for f in glob(f'{test_dir}/*....

1

votes

0

answer

27

Views

### How do I cache vectorized calls that take user input in R?

I am trying to calculate a field for all rows of a large dataset. The function to calculate it is from the package taxize, and uses an HTTP request to query an external site for the right ID number. It is searching by scientific name, and often there are multiple results, in which case this function...

1

votes

0

answer

64

Views

### Error When Checking Target (Keras)

Hi I am following this code along from Deep learning with python Book. I am trying to train a model with pretrained word embeddings on raw IMDB dataset. I get this error how to fix this.
I followed this process
1. Processing the labels of the raw IMDB data
2. Tokenizing the text of the raw IMDB data...

1

votes

1

answer

100

Views

### speed up R code - vectorize?

I am struggling to make my code faster. Currently I am getting somewhere using doParallel, but I am wondering if it could be made faster though clever programming instead of faster hardware. Here is a stylized version of what I want to do:
library(dplyr)
library(doParallel)
library(data.table)
cl

1

votes

2

answer

40

Views

### Vectorization of a nested for-loop that inputs all paired combinations

I thought that the following problem must have been answered or a function must exist to do it, but I was unable to find an answer.
I have a nested loop that takes a row from one 3-col. data frame and copies it next to each of the other rows, to form a 6-col. data frame (with all possible combinatio...

1

votes

0

answer

63

Views

### Numpy: populating arrays to vectorize nested summations

I would like to vectorize nested sums in Python, in order to speed up the process. At the moment I have nested for loops.
for ja in np.arange(0,Na):
for jb in np.arange(0,Nb):
for ma in np.arange(-ja,ja+1):
...
The end result is the sum across 2x2 matrices, each with entries dependent on the values...

1

votes

0

answer

80

Views

### Taking specific 2d array from 3d in numpy

Is there a way to avoid using the for loop and get the result just by calling arr with some indexing? Potentially dim1 will be equal to 50 000, dim2 up to 1000, dim3 fixed to 3.
import numpy as np
dim1 = 10
dim2 = 2
dim3 = 3
arr = np.arange(60).reshape(dim1,dim2,dim3)
arr2 = np.arange(dim1*dim2).res...

1

votes

0

answer

98

Views

### Vectorization of pandas groupby using conditional filters

Let's say I have a sample df, such as:
df = pd.DataFrame([['William', 1, 0, 'T', 0, 1],['James', 0, 1, 'R', 1, 1],['James', 1, 0, 'S', 0, 1],['Dean', 1, 0, 'R', 1, 0],['William', 0, 1, 'S', 0, 0],['James', 0, 0, 'S', 0, 1]],columns=['Name','x1','x2','x3','x4','x5'])
Which gives:
Name x1 x2 x3 x4...

1

votes

1

answer

56

Views

### Correct Way to Parallelize Pandas on Data Frame Slices

let's assume I have a data frame with N multi valued categorical columns and I want to encode them as fast as possible using Pandas.
This is what i achieved so far, not sure if it is the best way to parallelize Pandas though (I would prefer a vectorized approach where possible) :
def encode_single_c...

1

votes

1

answer

25

Views

### How to get formulas of multiple regressions by vectorizing

Suppose I have the following code that makes multiple regressions and stores the lm and lm with stepwise selection models in tibbles:
library(dplyr)
library(tibble)
library(MASS)
set.seed(1)
df

1

votes

2

answer

37

Views

### Mixing 3D arrays into a bigger 3D array

I wish to fill a N x M x W matrix ‘S’ with the data from matrices ‘P’ and ‘Q’. They are defined below and illustrated in the attached image. Also, we know for sure that n_1 + n_2 = N, m < M, so all the data may fit in the ‘S’ matrix.
S = zeros(M,N,W);
P = rand(m,n_1,W);
Q = rand(m,n_...

1

votes

2

answer

60

Views

### Move index to the left, according to grid

Let's say I have a grid that contains permissible and forbidden zones.
import numpy as np
forbidden = np.array([False, False, False, False, False, False, True, True, True,
True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True, True, True, True,
T...

1

votes

1

answer

165

Views

### x86_64 Dot Vector Product Intrinsic to ARM64

I am porting a small C routine that includes x86_64 intrinsics to a ARM64 platform. I am unable to find an equivalent ARM64 intrinsic for _mm_dp_pd.
I do have access to the arm neon intrinsics.
I am not sure how to replace x86_64 intrinsic with a ARM64 equivalent.
Any help would be much appreciate...