# Questions tagged [numpy]

24173 questions

1

votes

1

answer

18

Views

### Implement axis parameter in a custom function

I'm writing a rather trivial function to perform an integration applying trapezoidal rule in log space.
I'd like to add the axis parameter to achieve a functionality similar to the numpy.trapz function, but I'm a bit confused regarding how to correctly implement it.
The non-broadcastable function l...

0

votes

1

answer

14

Views

### Numpy Column Stack with Strings?

I am using numpy.column_stack and having problems
Input = input('Input: ')
Words = ['First','Second','Third','Fourth','Fifth','Sixth','Seventh','Eigth','Ninth']
Numbers = [0.5,1,1.25,1.5,2,3,5,10,15]
Stack = np.column_stack((Words, Numbers))
What I am hoping to achieve is:
Input: Second
Output: 1
In...

0

votes

1

answer

16

Views

### Find largest row in a matrix with numpy (row with highest length)

I have a massive array with rows and columns. Some rows are larger than others. I need to get the max length row, that is, the row that has the highest length. I wrote a simple function for this, but I wanted it to be as fas as possible, like numpy fast. Currently, it looks like this:
Example array:...

1

votes

2

answer

4.2k

Views

### NumPy in IDLE (Python 3.3.2) on Mac OSx 10.8

I know this may be a repeat question, but all the answers I've found on here are WAY above my head. I'm very new at Python, but I would like to use NumPy in IDLE (python 3.3.2). IDLE doesn't seem to access the numpy that comes w/ OSX 10.8 (which is python 2.7). I can't find a Numpy for Python 3 inst...

1

votes

1

answer

3.8k

Views

### Numpy array: concatenate arrays and integers

In my Python program I concatenate several integers and an array. It would be intuitive if this would work:
x,y,z = 1,2,np.array([3,3,3])
np.concatenate((x,y,z))
However, instead all ints have to be converted to np.arrays:
x,y,z = 1,2,np.array([3,3,3])
np.concatenate((np.array([x]),np.array([y]),z))...

1

votes

5

answer

40

Views

### How do I get a boolean array from an array A with multiple conditions in python?

A = np.arange(0,20,1)
A

0

votes

4

answer

56

Views

### Python: Assign Labels to values in an array

I have an array which represents some time series data:
array([[[-0.59776013],
[-0.59776013],
[-0.59776013],
[-0.31863936],
[-0.31863936],
[-0.31863936],
[-0.31863936],
[-0.31863936],
[-0.31863936],
[ 0.31863936],
[ 0.31863936],
[ 0.31863936],
[-0.31863936],
[-0.31863936],
[-0.31863936],
[-0.3186393...

7

votes

1

answer

52

Views

### Why is fancy assignment slower than fancy lookup?

I'm currently trying to get a better understanding of memory related performance issues. I read somewhere that memory locality is more important for reading than for writing, because in the former case the CPU has to actually wait for the data whereas in the latter case it can just ship them out and...

1

votes

1

answer

593

Views

### Numpy or Scipy way to do polynomial fitting in 2 dimensions

I have some data that looks like this
What is the typical way to do a polynomial map of z based on x and y? I have used numpy.polyfit in the past to do similar things in 2 dimensions, so I suppose I could just iterate through all the points and then fit those answers with another 1d polyfit. Howev...

1

votes

1

answer

490

Views

### numpy array split/partition efficiency

This appears to be a simple issue, but I've been struggling trying to efficiently split a 2D array:
start_time = time.time()
M = np.ones((400,400))
for i in range(10000):
e = np.array_split(M, 20)
print time.time() - start_time
However, this process takes ~6 seconds comparing to ~0.5 seconds when im...

1

votes

3

answer

847

Views

### Broadcasting a list in Pandas

I have a dataframe (a) , from which I want to subtract a list (b), column-wise:
import numpy as np
import pandas as pd
In:a=pd.DataFrame(np.arange(0,20).reshape(5,4))
print(a)
Out: 0 1 2 3
0 0 1 2 3
1 4 5 6 7
2 8 9 10 11
3 12 13 14 15
4 16 17 18 19
In: b=[1,2,3,...

1

votes

1

answer

208

Views

### CBLAS segmenation fault with large array

this is my third post and attempt to solve this problem, which first
showed up using numpy.dot(A, A.T) where A is large, 150,000 x 265 elements.
With numpy, I got back an array with many missing values, that were just zeros.
I've tried to call BLAS thru CBLAS. I'm getting a segmentation fault error...

1

votes

1

answer

37

Views

### how to get values from array with a specific rule in numpy

For example, I have a array with value [1,2,4,3,6,7,33,2]. I want to get all the values which are bigger than 6. As I know the numpy.take can only get values with indices.
Which function should I use?

1

votes

2

answer

1.4k

Views

### Pandas: Sum of first N non-missing values per row

I'd like to efficiently sum the first N non-missing values of a pandas DataFrame.
For example, if I had dataframe like this:
"df"
sid 1900 1899 332 855 1285 1413 1063 1768 2320 1117
bid
309 -0.02 -0.03 -0.03...

1

votes

1

answer

4.9k

Views

### cvtcolor in Cv2 - no attribute

I'm writing a program to collect a series of images from a USB microscope and generate an average as a way of denoising. I'm having to convert the image into a numpy array, sum the pixels then divide before converting back to an image.
Somewhere along the line the channels get mixed so my output is...

1

votes

2

answer

209

Views

### NumPy map calculation depending on the indices

I have a NumPy array of shape (Height, Width). Now I want to calculate the value of all the elements present in this array. The value is a function of the position [x,y] of the element.
Suppose I want to assign all the elements the value as (x**2+y**2)/2. I want to do this without using a for loop....

1

votes

1

answer

487

Views

### Fitting a probability distribution to the data and finding cumulative distribution function for it

I want to fit an asymmetric probability distribution to my data and I thought an exponentially modified Gaussian distribution can be a good representative for my data. I
m=array([ 16.25, 16.75, 17.25, 17.75, 18.25, 18.75, 19.25, 19.75,
20.25, 20.75, 21.25, 21.75, 22.25, 22.75, 23.25,...

0

votes

1

answer

23

Views

### problem with condition statement despite using right operator [duplicate]

This question already has an answer here:
Logical operators for boolean indexing in Pandas
3 answers
I wrote this script to create a specific variable that takes different values according to the number of reports. Count of Report is an integer column.
no_audit = df_bei_index['Count of Report'] ==...

0

votes

1

answer

22

Views

### A way to map one array onto another in numpy?

I have a 2-d array and a 1-d array, shown below. What I'd like to do is to fill the blank spaces in the 2-d array with the product of the 2-d and 1-d array - probably simplest to demonstrate below:
all_holdings = np.array([[1, 0, 0, 2, 0],
[2, 0, 0, 1, 0]]).astype('float64')
sub_holdings = np.array(...

0

votes

0

answer

28

Views

### Compute distance matrix for sequences of unequal lengths

I have a multiple sequences (with unequal lengths) of the form:
series = [ [1, 2, 5, 6, 9, 1],
[2, 6, 1, 4, 5],
[3, 5, 6, 7, 1, 4, 8, 7, 9],
[4, 3, 2, 1],
...
[2, 9, 7, 6, 5]
]
shape = (38, )
The length of the sequences is around 1000-2000. There are 38 sequences in total. I need to perform hierarc...

1

votes

2

answer

2.9k

Views

### Is there a faster version of numpy.random.shuffle?

I'm using numpy.random.shuffle in order to compute a statistic on randomized columns of a 2D array. The Python code is as follows:
import numpy as np
def timeline_sample(series, num):
random = series.copy()
for i in range(num):
np.random.shuffle(random.T)
yield random
The speed I get is something li...

0

votes

3

answer

24

Views

### Numpy array: how to row-wise check if first X amount of values are valid?

Problem Description
Consider the following two example arrays:
arr = np.array([
[5.0, 2.0, 1.0, np.nan, np.nan],
[9.0, np.nan, np.nan, np.nan, 2.0],
[4.0, 7.0, 4.0, np.nan, np.nan],
[8.0, np.nan, np.nan, np.nan, np.nan],
[np.nan, np.nan, np.nan, np.nan, np.nan],
[np.nan, np.nan, np.nan, np.nan, 6.0]...

0

votes

2

answer

16

Views

### Correctly converting a NumPy array to a PyTorch tensor running on the gpu

I have initialized my weights using
torch.zeros(1, self.batch_size, self.n_neurons, dtype=torch.float32, device=device)
where device is "cuda:o". I have also created a model that has a simple RNN
basic_rnn = nn.RNN(self.n_inputs, self.n_neurons)
out, hidden = basic_rnn(X, hidden)
X is my input that...

1

votes

1

answer

64

Views

### Perform math opeartion with two Numpy array - limit the number

I have 2 Numpy arrays which I need to perform some basic math operations on them.
But also I can't have the result of this operation to be greater than 255, due to the type (uint8) of the final numpy array (named magnitude). Any Idea? Except of traversing through the array...
# Notice that the data...

1

votes

1

answer

3.4k

Views

### Python SciPy IndexError: the length of bounds is not compatible with that of x0

I'm new to optimizations using Python and scipy. I'm getting the error
IndexError: SLSQP Error: the length of bounds is not compatible with that of x0.
when trying to pass the bounds parameter into scipy.optimize.minimize
x0 = np.array([[2,2,2,2,2,2,2,2,2,2,2],[2,2,2,2,2,2,2,2,2,2,2]])
bounds = ( [(...

1

votes

1

answer

1.2k

Views

### Python- Importing a dat file into an array using numpy

Hey guys im trying to import text file into an array using numpy but it looks like when it imported them with the rows as cols and vice versa. Am I formatting the array wrong or is that what happnened?
I added a picture below:

2

votes

1

answer

40

Views

### how to convert list of arrays of same legth

if I convert a list of arrays (the internal arrays have all the same length) in an array of arrays, python converts it as a matrix with all the elements joined together. I want it to always remain an array of arrays. Here's what happens:
list_1 = [[np.array([1,2,3]),np.array([4,5,6])]]
>>> np.array(...

0

votes

1

answer

11

Views

### Python - Manual - libary installation Failure on Windows with Spacy, Thinc and msgpack-numpy python 3.7

All, I am pursuing a path of manual installation of python libraries, one that unfortunately, I cannot deviate from and it has become challenging because some of the libraries are just not easily found from pypi.org. This is a Windows 10 set up using Anaconda for python 3.7
My goal is to install...

1

votes

2

answer

6.2k

Views

### How to initialize 2D numpy array

Note:
I found the answer and answered my own question, but I have to wait 2 days to accept my answer.
How do I initialize a numpy array of size 800 by 800 with other values besides zero? :
array = numpy.zeros((800, 800))
I am looking for a solution like this, where I could pass in a list (or lists)...

1

votes

3

answer

6.3k

Views

### ValueError: index must be monotonic increasing or decreasing

ser3 = Series(['USA','Mexico','Canada'],index = ['0','5','10'])
here ranger = range(15)
I get an error while using Forward fill in iPython
ser3.reindex(ranger,method = 'ffill')
/Users/varun/anaconda/lib/python2.7/site-packages/pandas/core/index.pyc in _searchsorted_monotonic(self, label, side)
2395...

1

votes

1

answer

215

Views

### Matplotlib x-axis overcrowded label rendering

I'm trying to simply plot a numpy array but I'm having troubles, especially with getting the number of ticks and labels right on the x-axis.
I'm dealing with an array named percents that looks like this:
percents:
-74 0.000041
-61 0.000041
-60 0.000041
-59 0.000041
-57 0.000041
...
71...

1

votes

2

answer

5.9k

Views

### Pandas series to numpy array conversion error

I have a pandas series with foll. value_counts output():
NaN 2741
197 1891
127 188
194 42
195 24
122 21
When I perform describe() on this series, I get:
df[col_name].describe()
count 2738.000000
mean 172.182250
std 47.387496
min 0.000000
25% 171...

1

votes

2

answer

5.3k

Views

### xgboost watchlist parameter: DMatrix object is not iterable

I am trying to train a xgboost model on a dataset X-train, X_test.
Code:
xgb_params = {
"objective": "multi:softmax",
"eta": 0.3,
"num_class": 62,
"max_depth": 10,
"nthread": 4,
"eval_metric": "merror",
"print.every.n": 1
#"silent": 1
}
num_rounds = 2
mask = np.random.choice([False, True], len(X_tr...

1

votes

2

answer

1.7k

Views

### Add a new column to a Pandas DataFrame by using values in another column to lookup values in a dictionary

How do I add a column to a Pandas DataFrame, by multiplying an existing column by a factor from an external dictionary looked up using values from a second column in the same DataFrame as keys?
I have a pd.DataFrame dataframe df roughly of the form
code blah... year nominal
0 T.rrr bl...

21

votes

6

answer

1.6k

Views

### Why does Python copy numpy arrays where the length of the dimensions are the same?

I have a problem with referencing to a numpy array.
I have an array of the form
import numpy as np
a = [np.array([0.0, 0.2, 0.4, 0.6, 0.8]),
np.array([0.0, 0.2, 0.4, 0.6, 0.8]),
np.array([0.0, 0.2, 0.4, 0.6, 0.8])]
and if I now create a new variable
b = np.array(a)
and do
b[0] += 1
print(a)
then a...

0

votes

2

answer

29

Views

### Adding values to a new array

I have an existing python array instantiated with zeros. How do I iterate through and change the values?
I can't iterate through and change elements of a Python array?
num_list = [1,2,3,3,4,5,]
mu = np.mean(num_list)
sigma = np.std(num_list)
std_array = np.zeros(len(num_list))
for i in std_array:
t...

1

votes

1

answer

1.9k

Views

### Set a column in numpy array to zero

I want to set a column in numpy array to zero at different times, in other words, I have numpy array M with size 5000x500. When I enter shape command the result is (5000,500), I think 5000 are rows and 500 are columns
shape(M)
(5000,500)
But the problem when I want to access one column like first c...

1

votes

1

answer

2.3k

Views

### python ValueError: column index exceeds matrix dimensions

I am running one simple python program with my files. This program works fine on one of my machines with the file, fileA.bed. But this program doesn't work in another machine with the same file. I installed same python version, 2.7.6, same required modules, scipy ('0.15.1'), numpy('1.8.2'), iced ('0...

1

votes

1

answer

24

Views

### TypeError: bad operand type for unary ~: 'float' while groupby and apply a function

I get a TypeError while using groupby and a function to remove outliers:
def is_outlier(s):
lower_limit = s.median() - (s.std() * 3)
upper_limit = s.median() + (s.std() * 3)
return ~s.between(lower_limit, upper_limit)
df1 = df[~df.groupby('objectName')['price'].apply(is_outlier)]
print(df1)
I have f...

0

votes

0

answer

4

Views

### Reading vector information as a multidimensional array from VTU files using 'vtkXMLUnstructuredGridReader' in python

I'm trying to read a vector field information from a VTU file in python using vtkXMLUnstructuredGridReader. The vector field to be read is an array of N*3 dimension, where N is the number of cells and 3 the number of components of the vector.
The VTU file looks like this (without the XML data),
1.04...