# Questions tagged [scipy]

5582 questions

1

votes

1

answer

1.9k

Views

### Multiclass linear SVM in python that return probability

How can I implement a linear SVM for multi-class which returns the proabability matrix for the test samples.
Train samples: mxn
Train labels: mxc
Test labels : mxc, where column has the probability of each class.
The function in sklearn which does "one-vs-the-rest" LinearSVC doesn't return probablit...

1

votes

1

answer

740

Views

### Transition line in heat map - python

I have problem that I can't seem to work around. I have a grid of values that I have interpolated using scipys griddata. The values have been visualized as a heat map with values in [0,1]. Now I would like to plot a transition line for values 1/2.
Is this possible? My first idea was to extract the c...

1

votes

1

answer

5.5k

Views

### Pass Pandas DataFrame to Scipy.optimize.curve_fit

I'd like to know the best way to use Scipy to fit Pandas DataFrame columns. If I have a data table (Pandas DataFrame) with columns (A, B, C, D and Z_real) where Z depends on A, B, C and D, I want to fit a function of each DataFrame row (Series) which makes a prediction for Z (Z_pred).
The signature...

1

votes

1

answer

2.3k

Views

### python ValueError: column index exceeds matrix dimensions

I am running one simple python program with my files. This program works fine on one of my machines with the file, fileA.bed. But this program doesn't work in another machine with the same file. I installed same python version, 2.7.6, same required modules, scipy ('0.15.1'), numpy('1.8.2'), iced ('0...

1

votes

1

answer

438

Views

### Calculating probability distribution from time series data in python

I have a question about probability distribution function I have a time series data and I want to calculate the probability distribution of data in different time windows.
I have developed the following code but i could not find the value of probability distribution for this function.
a = pd.DataF...

1

votes

1

answer

1.3k

Views

### Python 2.7: Area opening and closing binary image in Python not so accurate

I am using Python 2.7 and I used following Python and Matlab function for removing noises and fill holes in this image
.
1. Code to remove noise and fill holes using Python and Opencv
img = cv2.imread("binar.png",0)
kernel = np.ones((5,5),np.uint8)
open = cv2.morphologyEx(img, cv2.MORPH_OPEN, kernel...

13

votes

1

answer

166

Views

### Finding alpha and beta of beta-binomial distribution with scipy.optimize and loglikelihood

A distribution is beta-binomial if p, the probability of success, in a binomial distribution has a beta distribution with shape parameters α > 0 and β > 0. The shape parameters define the probability of success.
I want to find the values for α and β that best describe my data from the perspectiv...

0

votes

0

answer

6

Views

### How to count non-zeroes values using binned_statistic

I need to efficiently process very large 1D arrays extracting some statistics per bin and I have found very useful the function binned_statistic from scipy.stats as it includes a 'statistic' argument that works quite efficiently.
I would like to perform a 'count' function but without considering zer...

0

votes

0

answer

13

Views

### invalid version of numpy or scipy or sickitLearn

On a ubuntu 16.04 image in a Docker Container -
I try to install :
FROM ubuntu:16.04
MAINTAINER Amazon AI
RUN apt-get -y update && apt-get install -y --no-install-recommends \
wget \
python3.5 \
nginx \
libgcc-5-dev \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
# Here we get all python packag...

1

votes

2

answer

175

Views

### Faster way to build a matrix of expected frequencies from a sparse matrix of counts

I have a Compressed Sparse Row matrix containing counts. I want to build a matrix containing the expected frequencies for these counts. Here's the code I'm currently using:
from scipy.sparse import coo_matrix
#m is a csr_matrix
col_total = m.sum(axis=0)
row_total = m.sum(axis=1)
n = int(col_total.su...

1

votes

1

answer

1k

Views

### Faster way of calculating a distance matrix with numpy?

I am calculating a matrix with numpy/scipy like this:
cost = np.empty([chroma1.data.shape[1], chroma2.data.shape[1]])
for x, cx in enumerate(chroma1.transpose()):
for y, cy in enumerate(chroma2.transpose()):
cost[x, y] = sp.distance.euclidean(cx, cy)
This takes quite an amount of time. Is there any...

0

votes

0

answer

5

Views

### Do I have to type the formula code in scipy optimize if the cells in excel already have formula inside?

I'm currently trying to solve an optimization problem using python scipy optimize. The data I need is in an excel file and the formulas of the variables are already in the cells. I already linked the excel file to python using openpyxl.
My question is that do I have to type in the formula once agai...

1

votes

1

answer

664

Views

### Scipy curve_fit multiple series of data

I'm trying to have a curve fit that takes into account multiple series of y based on same values of x and same (exponential) law. The y values among the series vary a little since they're experimental but are still close (at same x).
I tried to build two arrays: one with the x and one with the two d...

1

votes

1

answer

3.4k

Views

### How to load and convert .mat file into numpy 2D array?

I have a data in mat file (observations and features) and i want to load it into numpy 2D array. I dont want to convert it into csv first and then load csv into numpy.

1

votes

2

answer

1.9k

Views

### How do I implement the Probability density function of a Gaussian Distribution

I need to implement a class in Python, that represents a Univariate (for now) Normal Distribution. What I have in mind is as follows
class Norm():
def __init__(self, mu=0, sigma_sq=1):
self.mu = mu
self.sigma_sq = sigma_sq
# some initialization if necessary
def sample(self):
# generate a sample, whe...

1

votes

2

answer

618

Views

### Solve nonlinear equation in python

I am trying to find the fundamental TE mode of the dielectric waveguide. The way I try to solve it is to compute two function and try to find their intersection on graph. However, I am having trouble get the intersect point on the plot.
My code:
def LHS(w):
theta = 2*np.pi*1.455*10*10**(-6)*np.cos(...

1

votes

2

answer

363

Views

### How do I perform a gaussian curve fit in the presence of a linear background with scipy?

I have the given data set:
Of which I would like to fit a Gaussian curve at the point where the red arrow is directed towards. I have attempted to do so by restricting the data points to a range of channels close to the peak, using scipy.optimize.curve_fit and a gaussian function to obtain the fit a...

1

votes

3

answer

1.2k

Views

### imgaug: load and save images

I am using Python+Tensorflow for CNN training on a high-performance computing cluster. I am training a convolutional neural network, but have a relatively small dataset. So I am implementing techniques to augment it. Now this is the first time i am working on a core computer vision problem so am rel...

1

votes

2

answer

307

Views

### Scipy Euclidean distance between two points

The below code for calculating the Euclidean distance between two points returns [[9.]] :
from scipy import spatial
d1 = [[11 , 3]]
d2 = [[2 , 3]]
print(spatial.distance.cdist(d1 , d2 , "euclidean"))
But the Euclidean distance between these two points is 3?
Has the Euclidean distance been impleme...

1

votes

1

answer

67

Views

### problem with constraints in minimize from scipy.optimize

I have searched about this a bit, and couldn't solve this problem. I am working with the minimize function from scipy.optimize, and keep getting the error:
'str' object not callable.
My code is complicated, so I looked up an easy example online to make sure I was inputting everything correctly, and...

1

votes

2

answer

84

Views

### Calculate mean across a row of an array using a row of a masking array

In converting this simple line of Octave code to Python, I’m wondering if there's a quicker/cleaner way:
Given two matrices, the Octave command averages a row of matrix Y using a Boolean matrix R to tag the values that are of interest:
load (‘mydata.mat’)
row1_mean = mean( Y(1, R(1, :) ) )
Nic...

1

votes

1

answer

46

Views

### Result from function call is not an array of floats

Following my previous two posts (post1, post 2), I have now reached the point where I use scipy to find a curve fit. However, the code I have produces an error.
A sample of the .csv file I'm working with is located in post1. I tried to copy and substitute examples from the Internet, but it doesn't s...

1

votes

1

answer

29

Views

### Get the relative extrema from 1D numpy array

I'm writing code that includes the algorithm to find local maximum/minimum values in array. But I failed to find the proper function.
At first, I used argrelextrema in scipy.signal.
b = [6, 1, 3, 5, 5, 3, 1, 2, 2, 3, 2, 1, 1, 9, 10, 10, 9, 8, 7, 7, 13, 10]
scipy.signal.argrelextrema(np.array(b), np....

1

votes

1

answer

53

Views

### Scipy minimize error: 'numpy.float64' object is not callable

I have defined a Class to minimize a standard function called rosen, in order to minimize rosen, the scipy.minimize function needs to call rosen repeatedly to minimize it.
from scipy.optimize import minimize
import numpy as np
class LocalMultivariateOptimization:
def __init__(self, initial_guess_par...

1

votes

1

answer

40

Views

### What is the difference between qgeom in R and scipy.stats.geom.ppf in Python?

My understanding is both ways should give the quantile corresponding to lower tail probability. However, I get different results.
e.g:- qgeom(0.99,0.5) gives 6 in R, wheres geom.ppf(0.99,0.5) gives 7 in Python.

0

votes

0

answer

26

Views

### Python Optimization: How to speed up matrix inverse operation?

My code contains a for loop with large number of iterations. Within the loop I need to so matrix multiplication and matrix inverse (normally a matrix of size 12 x 12). My loop needs to run 120,000 times and I am currently getting a speed of 14s, which is relatively very high compared to MATLAB (1s)...

2

votes

1

answer

14

Views

### Interpolate a plot on certain intervals only

Interpolation overshoots the data at zeros
Attempting a vertical plot (x vs. y) with a smooth curvature of the data along the y axis. Tried SciPy.interpolate's splev and splrep on the entire graph (y[0]-y[-1]). On this interval, the x data is positive, oscillating, and drops to zero several times. T...

1

votes

1

answer

47

Views

### How to get a non-smoothing 2D spline interpolation with scipy

I want a 2D cubic spline fit to some irregullary spaced data - i.e. a function that exactly fits the data at the given points - but can also return values in between.
All I can find (for irregural spaced data) is scipy.interpolate.SmoothBivariateSpline. I can't figure out how to turn 'smoothing' of...

1

votes

2

answer

74

Views

### Obtain indices of a array where 4 or more than 4 adjacent elements are 1

I want to obtain indices of array (data) where 4 or more than 4 adjacent elements are 1:
import numpy as np
data = np.array([[0, 0 , 0 , 0 , 0 , 0 , 1 , 0],
[0, 1 , 1 , 0 , 0 , 1 , 1 , 0],
[1, 1 , 0 , 1 , 1 , 0 , 0 , 0],
[0, 0 , 0 , 1 , 1 , 0 , 0 , 0],
[0, 0 , 0 , 1 , 1 , 0 , 0 , 1],
[1, 1 , 0 ,...

1

votes

1

answer

13

Views

### Getting strange results from signal.lsim when t[0] != 0

I'm running a simulation of LTI state-space model and I need to run it at different times with varying input. In other words, simulate from t0 = 0 to t1=1second, make changes to the inputs of the system based on the result at t1, then continue from t1= 1 second to t2= 2 seconds.
I've tried running w...

1

votes

3

answer

1.8k

Views

### Scipy: fill a histogram reading from a DB, event by event, in a loop

You sometimes don't want to fill a histogram after creating a huge list. You want to read a DB and fill the histogram event by event. Eg:
collection = db["my_collection"]
for event in collection.find():
histogram.fill(event['a_number'])
So, if I have 10Bn entries in the collection, I can fill any hi...

1

votes

2

answer

200

Views

### How can I solve an equation like 'x^3/3x == 4' using Scipy? [closed]

or another Open Source Python Library: Numpy, Matplotlib ...

1

votes

1

answer

7.3k

Views

### Plotting specific range of values within an array Pyplot

I want to plot a certain range of values within 2 arrays using matplotlib,
seconds = 200
dt = .001
x = array of values
time = arange(0, seconds, dt)
Goal is to plot everything after the first 25 seconds. I Know I could create 2 new arrays using a for loop, but I'm certain there's some simple built...

1

votes

2

answer

2.5k

Views

### defining function for scipy.optimize.curve_fit

As I'm lazy I don't want to define function for optimizing this way:
f = lambda x, a, b, ...: a * x + c + ...
but I want to do such thing:
f = lambda x, p: p[0] * x + p[1] + ...
where p is array of initial parameters. The problem is - I don't know how to implement it. I tried
popt = optimize.curve_...

1

votes

1

answer

2k

Views

### Python SciPy chisquare test returns different p value from Excel and LibreOffice

After reading a recent blog post about an application of the Poisson distribution, I tried reproducing its findings using Python's 'scipy.stats' module, as well as Excel/LibreOffice 'POISSON' and 'CHITEST' functions.
For the expected values shown in the article, I simply used:
import scipy.stats
for...

1

votes

2

answer

1.6k

Views

### Python - Interpolating between lines of data

I have data on a 2d grid characterized by points (x,Y,Z). The X and Y values indicate each point's position and Z is "height" or "intensity" at each point.
My issue is that my data coordinates along the X axis are extremely closely spaced (~1000 points), while my Y coordinates are spread out (~50...

1

votes

1

answer

994

Views

### py2exe with scipy.sparse.sparsetools.csr

I get the following error message using python v2.7.3 and scipy v0.11.0 with py2exe v0.6.9:
ImportError: No module named _csr
my setup.py:
from distutils.core import setup
import py2exe
setup(console=['demo.py'],
options = {
"py2exe": {"includes":["scipy.sparse", "scipy.sparse.sparsetools.csr",
"sci...

1

votes

1

answer

564

Views

### Scipy: Sparse Matrix giving incorrect values

Below is my code for generating my sparse matrix:
import numpy as np
import scipy
def sparsemaker(X, Y, Z):
'X, Y, and Z are 2D arrays of the same size'
x_, row = np.unique(X, return_inverse=True)
y_, col = np.unique(Y, return_inverse=True)
return scipy.sparse.csr_matrix( (Z.flat,(row,col)), shape=(...

1

votes

1

answer

144

Views

### Order a NXM Numpy Array according to cumulative sums of each sub-array

I have 5 numpy arrays within a single nxm array, with each array containing a set of random values. What I need is to print them according to their decreasing cumulative sums. For instance, my nXm array looks like:
v = [[1,2,3,4], [2,3,4,5], [11,21,3,4], [4,33,21,1], [2,4,6,5]]
and what I need it to...

1

votes

3

answer

434

Views

### Indices of resampled array in scipy

I have two 1D-array, of the same length, containing a time series and value series, for example
t = linspace(0, 5, 5) # [0, 1.25, 2.5, 3.75, 5]
x = array(range(10, 25)) # [10, 11, 12, 13, 14]
I have to resample the x array using different sampling points in time, (with same start and end point, but...