# Questions tagged [numpy]

32264 questions

1

votes

2

answer

1.2k

Views

### Hamming Window, python 2.7

Hi I have a FFT which is quite noisy. How to apply to my code Hamming window to make it less noisy. Look at my code:
plt.subplot(212)
plt.title('Fast Fourier Transform')
plt.ylabel('Power [a.u.]')
plt.xlabel('Frequency Hz')
fft1 = (Bx[51:-14])
fft2 = (By[1:-14])
for dataset in [fft1]:
dataset = np.a...

-2

votes

0

answer

14

Views

### Efficient way to calculate matrix cell distance from arbitrary coordinate in numpy

I am looking for an efficient numpy solution to the following problem:
I have a N by N numpy matrix. Given an arbitrary i,j coordinate (can be decimal) of the matrix and an arbitrary range, I need to calculate the value of each cell of the matrix divided by its euclidian distance to the coordinate w...

1

votes

3

answer

106

Views

### Why does np.argwhere's result shape not match it's input?

Suppose I pass a 1D array:
>>> np.arange(0,20)
array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
17, 18, 19])
>>> np.arange(0,20).shape
(20,)
into argwhere:
>>> np.argwhere(np.arange(0,20)>> np.argwhere(np.arange(0,20)

0

votes

0

answer

19

Views

### how to handle with continuous values in array

I would like to create a submission file to the problem, but my predictions got continuous values in the array, please help me how to solve.
I have array values like this:
predictions
array([[5.5161709e-01, 4.4297403e-01, 5.3959554e-03, 1.2935511e-05],
[5.5161709e-01, 4.4297403e-01, 5.3959554e-03, 1...

1

votes

1

answer

2k

Views

### converty numpy array of arrays to 2d array

I have a pandas series features that has the following values (features.values)
array([array([0, 0, 0, ..., 0, 0, 0]), array([0, 0, 0, ..., 0, 0, 0]),
array([0, 0, 0, ..., 0, 0, 0]), ...,
array([0, 0, 0, ..., 0, 0, 0]), array([0, 0, 0, ..., 0, 0, 0]),
array([0, 0, 0, ..., 0, 0, 0])], dtype=object)
N...

1

votes

1

answer

182

Views

### Adding an array to a numpy array

I'd like to have a numpy array that looks something like this:
X = np.array([[10, 20], [20, 25], [30, 16], [40, 18], [50, 90], [60, 87]])
I currently have dictionary values that I retrieve from firestore:
doc_ref = db.collection('CPU Logs')
query_ref = doc_ref.where(u'testData', u'==', True).order_b...

1

votes

4

answer

154

Views

### What is difference between parenthesis and bracket in Numpy?

Could somebody explain the difference between () and [] operations in Numpy?
For example, I have run the following codes:
import numpy as np
x1 = np.array(([2, 9], [1, 5], [3, 6]), dtype=float)
print(x1)
print(type(x1))
x2 = np.array([[2, 9], [1, 5], [3, 6]], dtype=float)
print(x2)
print(type(x2))
y...

1

votes

3

answer

123

Views

### Slicing a pattern in a large 1-d NumPy array

I have got a 1-d array where I have a pattern in the enteries. I will give an example. In the array arr, I have
first 4 enteries with single digits, next 4 enteries with two digits and then the next 6 enteries with 3 digits.
(This single, double, triple digit thing is just to highlight the pattern...

1

votes

1

answer

22

Views

### Pandas: How to find number of unique elements for one column coming from another column?

I have a dataframe like this:
import numpy as np
import pandas as pd
df = pd.DataFrame({'carrier': ['c1','c1','c1','c2','c2','c2','c3','c4','c5','c5'],
'airport': ['a1','a3','a1','a1','a2','a2','a3','a4','a4','a1'],
})
df
carrier airport
0 c1 a1
1 c1 a3
2 c1 a1...

0

votes

0

answer

5

Views

### LOGNORM.INV function of excel in python scipy

I need to implement the below excel function in python and need to create 10k such calculations. How can I implement this logic in scipy ? Can I use lognormal api of scipy to get lognormal inverse ?
B10 - Probability event will happen
C10 - Lower bound
D10 - Upper bound
=IF(RAND()

1

votes

2

answer

41

Views

### How to feed OpenCV image to a trained CNN model(add a new dimension) in Python?

I'm getting this error Error when checking input: expected conv2d_11_input to have 4 dimensions, but got array with shape (300, 300, 3)
How can I pass the RGB image to a CNN? How to enumerate samples to create a 4D image?

1

votes

3

answer

92

Views

### Subtracting minimum of row from the row

I know that
a - a.min(axis=0)
will subtract the minimum of each column from every element in the column. I want to subtract the minimum in each row from every element in the row. I know that
a.min(axis=1)
specifies the minimum within a row, but how do I tell the subtraction to go by rows instead...

1

votes

1

answer

40

Views

### Pandas, groupby and counting data in others columns

I have data with four columns, that includes: Id, CreationDate, Score and ViewCount.
The CreationDate has a next format, for example: 2011-11-30 19:41:14.960.
I need to groupby the years of CreationDate, count them, summing Score and ViewCount also, and to add to additional columns.
I want to use wi...

1

votes

1

answer

117

Views

### Fastest way to store a numpy array in redis

I'm using redis on an AI project.
The idea is to have multiple environment simulators running policies on a lot of cpu cores. The simulators write experience (a list of state/action/reward tuples) to a redis server (replay buffer). Then a training process reads the experience as a dataset to gener...

1

votes

2

answer

288

Views

### Numpy: Combine list of arrays by another array (np.choose alternative)

I have a list of numpy arrays, each of the same shape. Let's say:
a = [np.array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]]),
np.array([[11, 12, 13],
[14, 15, 16],
[17, 18, 19]]),
np.array([[99, 98, 97],
[96, 95, 94],
[93, 92, 91]])]
And I have another array of the same shape that gives the list indices I wan...

1

votes

2

answer

77

Views

### Resample pandas dataframe and interpolate missing values for timeseries data

I need to resample timeseries data and interpolate missing values in 15 min intervals over the course of an hour. Each ID should have four rows of data per hour.
In:
ID Time Value
1 1/1/2019 12:17 3
1 1/1/2019 12:44 2
2 1/1/2019 12:02 5
2 1/1/2019 12:28 7
Out:...

1

votes

3

answer

41

Views

### Max value of a Series in a single Dataframe cell

I have a Dataframe where each row contains a Series in a single column.
col1
row1 [34,55,11,8]
row2 [36,76,69,6]
row3 [77,31,40,55]
row4 [51,41,26,30]
I want to get the max of each value in the series and produce that in a new column.
col1 max
row1 [34,55,1...

1

votes

2

answer

45

Views

### Using np.where to find index of element in 2D array gives ValueError

I'm trying to use np.where to find the index of an element in an array, specifically row number
I have an array of say size 1000 x 6, named 'table'. The first element in each row is a 2 x 2 string array, and the rest are 0s. Eg. a 5 x 6 example of elements in 'table':
[['s',' '] 0 0 0 0 0
[' ',' '...

1

votes

4

answer

76

Views

### Increasing the speed of finding an index of an item in list

I'm trying to speed up a program I've written, and after importing cProfile, I see that one function takes up a massive bit of computation time.
It's this, which finds an numpy.ndarray in a list:
def locate(arr, l ):
for i in range(len(l)):
if np.all(l[i] == arr):
return i
return -1
As the list can'...

1

votes

3

answer

38

Views

### Number of 2D array to 3D array, extending third dimension

I have 10 different matrix of size (60, 100). I want to put them along the third dimension inside a for loop, so that the final shape is (10, 60, 100).
I tried with concatenate and end up with size (600, 100).

1

votes

1

answer

26

Views

### Update QpushButton icon from an numpy array don't work

I'm trying to make an icon from a numpy array displayed with matplotlib imshow.
I succeed in getting colors from matplotlib into a numpy array of dimension (n*n*4)
I then convert this numpy array to an Qimage then into a Qpixmap in order to update the Icon of a Qpushbutton.
However the icon of the b...

1

votes

2

answer

35

Views

### How to find the index of the not-null item which not in a list?

My system is python3.6, with numpy 1.16.2,scipy 1.2.1,matplotlib 3.0.3
import pandas as pd
import numpy
df=pd.DataFrame({'col1':['a','b','c'],'col2':['d',numpy.NaN,'c'],'col3':['c','b','b']})
df = df.astype({'col2': 'category'})
print(df)
output of above script is:
col1 col2 col3
0 a d...

1

votes

3

answer

47

Views

### What's the difference between shape(150,) and shape (150,1)?

What's the difference between shape(150,) and shape (150,1)?
I think they are the same, I mean they both represent a column vector.

1

votes

2

answer

44

Views

### Summing selected elements of a matrix in Python

I have a [n x n] matrix containing values belonging to different groups, and a [1 x n] vector defining to which group each element belongs.
(n usually ~1E4, in this example n=4)
I want to calculate a matrix obtained by summing together all the elements belonging to the same group.
I use np.where() t...

1

votes

2

answer

130

Views

### Performing calculations on a numpy array and adding them to a pandas dataframe

Let's say I have an array such as this:
a = np.array([[1, 2, 3, 4, 5, 6, 7], [20, 25, 30, 35, 40, 45, 50], [2, 4, 6, 8, 10, 12, 14]])
and a dataframe such as this:
num letter
0 1 a
1 2 b
2 3 c
What I would then like to do is to calculate the difference between the first and last...

1

votes

2

answer

25

Views

### Blockwise reshape in numpy?

I have a numpy array with columns that are in blocks. I want to transpose the blocks. It's conceptually simple, and I guess one can do it simply, but I dont know how.
Given a numpy array on block form np.hstack(list_of_blocks), I want to get np.vstack(list_of_blocks).
To make it more precis, I want...

1

votes

2

answer

38

Views

### Faster way to modify a numpy array

I have a large numpy array with 4 million rows and 4 columns (shape = (4000000,4))
I need to modify/ decrease the number of rows, based on the value in fourth column.
For example few of my rows in my data set look like the following:
a = np.array([[1.32, 24.42, 224.21312, 0],[1.32, 24.42, 224.21312...

1

votes

1

answer

22

Views

### How to delete images with the low pixel value in a folder

I have some problems with removing images in a folder
The followings are what I have done.
import os,glob
from PIL import Image
from skimage import io
import numpy as np
path = '/Users/Xin/Desktop/SVM-Image-Classification-master/Folder'
# Delete images with the low pixel value
for filename in os.lis...

1

votes

1

answer

21

Views

### “Rank” DataFrame columns per row

Given a Time Series DataFrame is it possible to create a new DataFrame with the same dimensions but the values are the ranking for each row compared to other columns (ordered smallest value first)?
Example:
ABC DEFG HIJK XYZ
date
2018-01-...

1

votes

1

answer

47

Views

### How to select the smallest number within one row in python?

I have data with multiple numbers in each row like this:
gene numbers
P53 11 11 11 7 13 8 8
ACE 6 5 4 5 3
BRCA 2 2 1
I want the numbers column to only contain the smallest number that is in that row, with output being for example:
gene numbers
P53 8
ACE 3
BRCA 1
I have lo...

1

votes

2

answer

34

Views

### Numpy: Pick elements based on bool array

I've got an array and a boolean array (as one hot encoding)
a = np.arange(12).reshape(4,3)
b = np.array([
[1,0,0],
[0,1,0],
[0,0,1],
[0,0,1],
], dtype=bool)
print(a)
print(b)
# [[ 0 1 2]
# [ 3 4 5]
# [ 6 7 8]
# [ 9 10 11]]
# [[ True False False]
# [False True False]
# [False False True]...

1

votes

1

answer

33

Views

### Value Error Converting the datatype of elements in an array

Im trying to put all matching elements in two different arrays into a singular one. However I'm running into a type error that I'm not entirely sure about.
This is what I initially tried to do:
IRS_zips = AGI.zipcode.unique() # np array of type int
medi_zips = df.nppes_provider_zip.unique() # np ar...

1

votes

4

answer

56

Views

### How to sum up for each distinct value c in array X all elements Y[i] where X[i] = k efficiently?

Given an 1D array X of length n in Numpy with k distinct values, I want to sum up for each of these distinct values c in another 1D array Y with the same length, all those elements Y[idx] where X[idx] == c in the most efficient way.
Example:
X = [1, 3, 2, 1, 2] and Y = [0.1, 0.2, 0.5, 2.0, 0.3]. The...

1

votes

0

answer

26

Views

### How does Python convert date value from excel

I am reading a csv file with a CDATE column. The structure of the column is:
|CDATE |
|08/28/2018|
|08/28/2018|
|08/29/2018|
|08/30/2018|
|09/02/2018|
|09/04/2018|
...
|04/10/2019|
As you can see there is duplicate date as well as missing dates in this column, and I would like to find the missin...

0

votes

0

answer

21

Views

### Python ValueError. Don't understand error or how to fix

I am following the tutorial here; https://www.analyticsvidhya.com/blog/2018/10/predicting-stock-price-machine-learningnd-deep-learning-techniques-python/#comment-155692
Instead of using the provided dataset I am using one needed for my assignment.
The code used is
#import packages
import pandas as p...

0

votes

0

answer

25

Views

### columns as a paramater in .at[] in python

I want to fill a dataframe's row proper values for proper columns
I am trying to reach a column by a parameter but I can not. If I send the name as string it works fine. The dataframe has the column that I am trying the reach.
list = key_queue.get()
column_name = column_name_queue.get()
print(column...

0

votes

1

answer

6

Views

### How to remove duplicates and force elements in a numpy array to be unique using a given range?

In case of differential evolution, during mutation, the formula that is used most often is
arr[a] = (arr[b] + M * (arr[c] - arr[d])) % arr.shape[1]
Where
arr is a 2d array consisting of non-negative integers such that all elements in each row are unique,
a represents each row of arr,
M is the mutat...

0

votes

0

answer

3

Views

### How to add magnitude or value to a vector in Python?

I am using this function to calculate distance between 2 vectors a,b, of size 300, word2vec, I get the distance between 'hot' and 'cold' to be equal 1.
How to add this value (1) to a vector, becz i thought simply new_vec=model['hot']+1, but when I do the calc dist(new_vec,model['hot'])=17?
import...

0

votes

1

answer

16

Views

### Iterating through data frame to buy or reject stocks

I have created the following dummy data for (1 year of stock prices)
df = pd.DataFrame(np.random.uniform(100,200, size = (365,1)), index=list_dates, columns=list('A'))
df.head()
Following is the head from the data frame.
A
2018-01-01 151.034052
2018-01-02 157.682393
2018-01-03 134.194843
2018-01-...

1

votes

0

answer

12

Views

### Numpy outer equality comparison with more than one element

I have two numpy arrays with which I'm doing an outer equality comparison
m = np.random.randint(4,size=(4,4,4))
ar = np.array([1,2,3])
x = np.equal.outer(ar,m)
This works well if I'm interested in testing every single element of ar in m. But what if I'd like to do more than one comparison in one go,...