Questions tagged [numpy]

24173 questions
1

votes
1

answer
18

Views

Implement axis parameter in a custom function

I'm writing a rather trivial function to perform an integration applying trapezoidal rule in log space. I'd like to add the axis parameter to achieve a functionality similar to the numpy.trapz function, but I'm a bit confused regarding how to correctly implement it. The non-broadcastable function l...
Luca Amerio
0

votes
1

answer
14

Views

Numpy Column Stack with Strings?

I am using numpy.column_stack and having problems Input = input('Input: ') Words = ['First','Second','Third','Fourth','Fifth','Sixth','Seventh','Eigth','Ninth'] Numbers = [0.5,1,1.25,1.5,2,3,5,10,15] Stack = np.column_stack((Words, Numbers)) What I am hoping to achieve is: Input: Second Output: 1 In...
B14d3r11
0

votes
1

answer
16

Views

Find largest row in a matrix with numpy (row with highest length)

I have a massive array with rows and columns. Some rows are larger than others. I need to get the max length row, that is, the row that has the highest length. I wrote a simple function for this, but I wanted it to be as fas as possible, like numpy fast. Currently, it looks like this: Example array:...
Mauricio
1

votes
2

answer
4.2k

Views

NumPy in IDLE (Python 3.3.2) on Mac OSx 10.8

I know this may be a repeat question, but all the answers I've found on here are WAY above my head. I'm very new at Python, but I would like to use NumPy in IDLE (python 3.3.2). IDLE doesn't seem to access the numpy that comes w/ OSX 10.8 (which is python 2.7). I can't find a Numpy for Python 3 inst...
Kim Delaney
1

votes
1

answer
3.8k

Views

Numpy array: concatenate arrays and integers

In my Python program I concatenate several integers and an array. It would be intuitive if this would work: x,y,z = 1,2,np.array([3,3,3]) np.concatenate((x,y,z)) However, instead all ints have to be converted to np.arrays: x,y,z = 1,2,np.array([3,3,3]) np.concatenate((np.array([x]),np.array([y]),z))...
Forzaa
0

votes
4

answer
56

Views

Python: Assign Labels to values in an array

I have an array which represents some time series data: array([[[-0.59776013], [-0.59776013], [-0.59776013], [-0.31863936], [-0.31863936], [-0.31863936], [-0.31863936], [-0.31863936], [-0.31863936], [ 0.31863936], [ 0.31863936], [ 0.31863936], [-0.31863936], [-0.31863936], [-0.31863936], [-0.3186393...
Murray
7

votes
1

answer
52

Views

Why is fancy assignment slower than fancy lookup?

I'm currently trying to get a better understanding of memory related performance issues. I read somewhere that memory locality is more important for reading than for writing, because in the former case the CPU has to actually wait for the data whereas in the latter case it can just ship them out and...
Paul Panzer
1

votes
1

answer
593

Views

Numpy or Scipy way to do polynomial fitting in 2 dimensions

I have some data that looks like this What is the typical way to do a polynomial map of z based on x and y? I have used numpy.polyfit in the past to do similar things in 2 dimensions, so I suppose I could just iterate through all the points and then fit those answers with another 1d polyfit. Howev...
Matt
1

votes
1

answer
490

Views

numpy array split/partition efficiency

This appears to be a simple issue, but I've been struggling trying to efficiently split a 2D array: start_time = time.time() M = np.ones((400,400)) for i in range(10000): e = np.array_split(M, 20) print time.time() - start_time However, this process takes ~6 seconds comparing to ~0.5 seconds when im...
neither-nor
1

votes
3

answer
847

Views

Broadcasting a list in Pandas

I have a dataframe (a) , from which I want to subtract a list (b), column-wise: import numpy as np import pandas as pd In:a=pd.DataFrame(np.arange(0,20).reshape(5,4)) print(a) Out: 0 1 2 3 0 0 1 2 3 1 4 5 6 7 2 8 9 10 11 3 12 13 14 15 4 16 17 18 19 In: b=[1,2,3,...
Chris
1

votes
1

answer
208

Views

CBLAS segmenation fault with large array

this is my third post and attempt to solve this problem, which first showed up using numpy.dot(A, A.T) where A is large, 150,000 x 265 elements. With numpy, I got back an array with many missing values, that were just zeros. I've tried to call BLAS thru CBLAS. I'm getting a segmentation fault error...
wbg
1

votes
1

answer
37

Views

how to get values from array with a specific rule in numpy

For example, I have a array with value [1,2,4,3,6,7,33,2]. I want to get all the values which are bigger than 6. As I know the numpy.take can only get values with indices. Which function should I use?
Samuel
1

votes
2

answer
1.4k

Views

Pandas: Sum of first N non-missing values per row

I'd like to efficiently sum the first N non-missing values of a pandas DataFrame. For example, if I had dataframe like this: "df" sid 1900 1899 332 855 1285 1413 1063 1768 2320 1117 bid 309 -0.02 -0.03 -0.03...
ssquaxe
1

votes
1

answer
4.9k

Views

cvtcolor in Cv2 - no attribute

I'm writing a program to collect a series of images from a USB microscope and generate an average as a way of denoising. I'm having to convert the image into a numpy array, sum the pixels then divide before converting back to an image. Somewhere along the line the channels get mixed so my output is...
Microscopist
1

votes
2

answer
209

Views

NumPy map calculation depending on the indices

I have a NumPy array of shape (Height, Width). Now I want to calculate the value of all the elements present in this array. The value is a function of the position [x,y] of the element. Suppose I want to assign all the elements the value as (x**2+y**2)/2. I want to do this without using a for loop....
chaithu
1

votes
1

answer
487

Views

Fitting a probability distribution to the data and finding cumulative distribution function for it

I want to fit an asymmetric probability distribution to my data and I thought an exponentially modified Gaussian distribution can be a good representative for my data. I m=array([ 16.25, 16.75, 17.25, 17.75, 18.25, 18.75, 19.25, 19.75, 20.25, 20.75, 21.25, 21.75, 22.25, 22.75, 23.25,...
Dalek
0

votes
1

answer
23

Views

problem with condition statement despite using right operator [duplicate]

This question already has an answer here: Logical operators for boolean indexing in Pandas 3 answers I wrote this script to create a specific variable that takes different values according to the number of reports. Count of Report is an integer column. no_audit = df_bei_index['Count of Report'] ==...
Filippo Sebastio
0

votes
1

answer
22

Views

A way to map one array onto another in numpy?

I have a 2-d array and a 1-d array, shown below. What I'd like to do is to fill the blank spaces in the 2-d array with the product of the 2-d and 1-d array - probably simplest to demonstrate below: all_holdings = np.array([[1, 0, 0, 2, 0], [2, 0, 0, 1, 0]]).astype('float64') sub_holdings = np.array(...
Chrisper
0

votes
0

answer
28

Views

Compute distance matrix for sequences of unequal lengths

I have a multiple sequences (with unequal lengths) of the form: series = [ [1, 2, 5, 6, 9, 1], [2, 6, 1, 4, 5], [3, 5, 6, 7, 1, 4, 8, 7, 9], [4, 3, 2, 1], ... [2, 9, 7, 6, 5] ] shape = (38, ) The length of the sequences is around 1000-2000. There are 38 sequences in total. I need to perform hierarc...
araina
1

votes
2

answer
2.9k

Views

Is there a faster version of numpy.random.shuffle?

I'm using numpy.random.shuffle in order to compute a statistic on randomized columns of a 2D array. The Python code is as follows: import numpy as np def timeline_sample(series, num): random = series.copy() for i in range(num): np.random.shuffle(random.T) yield random The speed I get is something li...
Midnighter
0

votes
3

answer
24

Views

Numpy array: how to row-wise check if first X amount of values are valid?

Problem Description Consider the following two example arrays: arr = np.array([ [5.0, 2.0, 1.0, np.nan, np.nan], [9.0, np.nan, np.nan, np.nan, 2.0], [4.0, 7.0, 4.0, np.nan, np.nan], [8.0, np.nan, np.nan, np.nan, np.nan], [np.nan, np.nan, np.nan, np.nan, np.nan], [np.nan, np.nan, np.nan, np.nan, 6.0]...
Xukrao
0

votes
2

answer
16

Views

Correctly converting a NumPy array to a PyTorch tensor running on the gpu

I have initialized my weights using torch.zeros(1, self.batch_size, self.n_neurons, dtype=torch.float32, device=device) where device is "cuda:o". I have also created a model that has a simple RNN basic_rnn = nn.RNN(self.n_inputs, self.n_neurons) out, hidden = basic_rnn(X, hidden) X is my input that...
dearn44
1

votes
1

answer
64

Views

Perform math opeartion with two Numpy array - limit the number

I have 2 Numpy arrays which I need to perform some basic math operations on them. But also I can't have the result of this operation to be greater than 255, due to the type (uint8) of the final numpy array (named magnitude). Any Idea? Except of traversing through the array... # Notice that the data...
Dor
1

votes
1

answer
3.4k

Views

Python SciPy IndexError: the length of bounds is not compatible with that of x0

I'm new to optimizations using Python and scipy. I'm getting the error IndexError: SLSQP Error: the length of bounds is not compatible with that of x0. when trying to pass the bounds parameter into scipy.optimize.minimize x0 = np.array([[2,2,2,2,2,2,2,2,2,2,2],[2,2,2,2,2,2,2,2,2,2,2]]) bounds = ( [(...
Nyxynyx
1

votes
1

answer
1.2k

Views

Python- Importing a dat file into an array using numpy

Hey guys im trying to import text file into an array using numpy but it looks like when it imported them with the rows as cols and vice versa. Am I formatting the array wrong or is that what happnened? I added a picture below:
Joseph hooper
2

votes
1

answer
40

Views

how to convert list of arrays of same legth

if I convert a list of arrays (the internal arrays have all the same length) in an array of arrays, python converts it as a matrix with all the elements joined together. I want it to always remain an array of arrays. Here's what happens: list_1 = [[np.array([1,2,3]),np.array([4,5,6])]] >>> np.array(...
BossShell
0

votes
1

answer
11

Views

Python - Manual - libary installation Failure on Windows with Spacy, Thinc and msgpack-numpy python 3.7

All, I am pursuing a path of manual installation of python libraries, one that unfortunately, I cannot deviate from and it has become challenging because some of the libraries are just not easily found from pypi.org. This is a Windows 10 set up using Anaconda for python 3.7 My goal is to install...
John Vaughan
1

votes
2

answer
6.2k

Views

How to initialize 2D numpy array

Note: I found the answer and answered my own question, but I have to wait 2 days to accept my answer. How do I initialize a numpy array of size 800 by 800 with other values besides zero? : array = numpy.zeros((800, 800)) I am looking for a solution like this, where I could pass in a list (or lists)...
Rock Lee
1

votes
3

answer
6.3k

Views

ValueError: index must be monotonic increasing or decreasing

ser3 = Series(['USA','Mexico','Canada'],index = ['0','5','10']) here ranger = range(15) I get an error while using Forward fill in iPython ser3.reindex(ranger,method = 'ffill') /Users/varun/anaconda/lib/python2.7/site-packages/pandas/core/index.pyc in _searchsorted_monotonic(self, label, side) 2395...
Varun
1

votes
1

answer
215

Views

Matplotlib x-axis overcrowded label rendering

I'm trying to simply plot a numpy array but I'm having troubles, especially with getting the number of ticks and labels right on the x-axis. I'm dealing with an array named percents that looks like this: percents: -74 0.000041 -61 0.000041 -60 0.000041 -59 0.000041 -57 0.000041 ... 71...
SpicyClubSauce
1

votes
2

answer
5.9k

Views

Pandas series to numpy array conversion error

I have a pandas series with foll. value_counts output(): NaN 2741 197 1891 127 188 194 42 195 24 122 21 When I perform describe() on this series, I get: df[col_name].describe() count 2738.000000 mean 172.182250 std 47.387496 min 0.000000 25% 171...
user308827
1

votes
2

answer
5.3k

Views

xgboost watchlist parameter: DMatrix object is not iterable

I am trying to train a xgboost model on a dataset X-train, X_test. Code: xgb_params = { "objective": "multi:softmax", "eta": 0.3, "num_class": 62, "max_depth": 10, "nthread": 4, "eval_metric": "merror", "print.every.n": 1 #"silent": 1 } num_rounds = 2 mask = np.random.choice([False, True], len(X_tr...
sinhayash
1

votes
2

answer
1.7k

Views

Add a new column to a Pandas DataFrame by using values in another column to lookup values in a dictionary

How do I add a column to a Pandas DataFrame, by multiplying an existing column by a factor from an external dictionary looked up using values from a second column in the same DataFrame as keys? I have a pd.DataFrame dataframe df roughly of the form code blah... year nominal 0 T.rrr bl...
curlew77
21

votes
6

answer
1.6k

Views

Why does Python copy numpy arrays where the length of the dimensions are the same?

I have a problem with referencing to a numpy array. I have an array of the form import numpy as np a = [np.array([0.0, 0.2, 0.4, 0.6, 0.8]), np.array([0.0, 0.2, 0.4, 0.6, 0.8]), np.array([0.0, 0.2, 0.4, 0.6, 0.8])] and if I now create a new variable b = np.array(a) and do b[0] += 1 print(a) then a...
sholli
0

votes
2

answer
29

Views

Adding values to a new array

I have an existing python array instantiated with zeros. How do I iterate through and change the values? I can't iterate through and change elements of a Python array? num_list = [1,2,3,3,4,5,] mu = np.mean(num_list) sigma = np.std(num_list) std_array = np.zeros(len(num_list)) for i in std_array: t...
Leigh
1

votes
1

answer
1.9k

Views

Set a column in numpy array to zero

I want to set a column in numpy array to zero at different times, in other words, I have numpy array M with size 5000x500. When I enter shape command the result is (5000,500), I think 5000 are rows and 500 are columns shape(M) (5000,500) But the problem when I want to access one column like first c...
S.AMEEN
1

votes
1

answer
2.3k

Views

python ValueError: column index exceeds matrix dimensions

I am running one simple python program with my files. This program works fine on one of my machines with the file, fileA.bed. But this program doesn't work in another machine with the same file. I installed same python version, 2.7.6, same required modules, scipy ('0.15.1'), numpy('1.8.2'), iced ('0...
sony
1

votes
1

answer
24

Views

TypeError: bad operand type for unary ~: 'float' while groupby and apply a function

I get a TypeError while using groupby and a function to remove outliers: def is_outlier(s): lower_limit = s.median() - (s.std() * 3) upper_limit = s.median() + (s.std() * 3) return ~s.between(lower_limit, upper_limit) df1 = df[~df.groupby('objectName')['price'].apply(is_outlier)] print(df1) I have f...
ahbon
0

votes
0

answer
4

Views

Reading vector information as a multidimensional array from VTU files using 'vtkXMLUnstructuredGridReader' in python

I'm trying to read a vector field information from a VTU file in python using vtkXMLUnstructuredGridReader. The vector field to be read is an array of N*3 dimension, where N is the number of cells and 3 the number of components of the vector. The VTU file looks like this (without the XML data), 1.04...
user297271

View additional questions