# Questions tagged [scipy]

7420 questions

1

votes

1

answer

691

Views

### How to obtain the chi squared value as an output of scipy.optimize.curve_fit?

Is it possible to obtain the value of the chi squared as a direct output of scipy.optimize.curve_fit()?
Usually, it is easy to compute it after the fit by squaring the difference between the model and the data, weighting by the uncertainties and summing all up. However, it is not as direct when the...

0

votes

0

answer

5

Views

### LOGNORM.INV function of excel in python scipy

I need to implement the below excel function in python and need to create 10k such calculations. How can I implement this logic in scipy ? Can I use lognormal api of scipy to get lognormal inverse ?
B10 - Probability event will happen
C10 - Lower bound
D10 - Upper bound
=IF(RAND()

1

votes

1

answer

54

Views

### Is fsolve good to any system of equations?

I don't have a lot of experience with Python but I decided to give it a try in solving the following system of equations:
x = A * exp (x+y)
y = 4 * exp (x+y)
I want to solve this system and plot x and y as a function of A.
I saw some a similar question and give fsolve a try:
`from scipy.optimize i...

1

votes

2

answer

77

Views

### Resample pandas dataframe and interpolate missing values for timeseries data

I need to resample timeseries data and interpolate missing values in 15 min intervals over the course of an hour. Each ID should have four rows of data per hour.
In:
ID Time Value
1 1/1/2019 12:17 3
1 1/1/2019 12:44 2
2 1/1/2019 12:02 5
2 1/1/2019 12:28 7
Out:...

1

votes

3

answer

36

Views

### assigning attribute to a method

I have a class for a dynamical system, with methods for the ODE function and also the event function to terminate integration. I want to solve this with scipy.integrate.solve_ivp. According to the docs, this is done by assigning an attribute to your event function, as myEvent.terminal = True. Howeve...

5

votes

2

answer

200

Views

### Filter out troughs based on distance between peaks

I have the following dataframe:
date Values
3/1/2018
3/3/2018 0
3/5/2018 -0.011630952
3/8/2018 0.024635792
3/10/2018
3/10/2018 0.013662755
3/13/2018 2.563770771
3/15/2018 0.026081264
3/17/2018
3/25/2018 4.890818119
3/26/2018
3/28/2018 0.994944572
3/30/2018 0.0985...

7

votes

2

answer

260

Views

### Minimize quadratic function subject to linear equality constraints with SciPy

I have a reasonably simple constrained optimization problem but get different answers depending on how I do it. Let's get the import and a pretty print function out of the way first:
import numpy as np
from scipy.optimize import minimize, LinearConstraint, NonlinearConstraint, SR1
def print_res( re...

1

votes

1

answer

2.3k

Views

### Python: Intersection over Union

I have the following question. I try to calculate the Intersection over Union, which is the overlap of two components divided by the unioin of two components. Lets assume component1 is a matrix with ones where the first object is and component2 is a matrix with ones where the second object is. The...

1

votes

2

answer

150

Views

### Python numpy array negative indexing

I'm a bit confused about the indexing of numpy. Assume the following example:
>>> import numpy as np
>>> x = np.arange(10)
>>> x.shape = (2,5)
>>> x
array([[0, 1, 2, 3, 4],
[5, 6, 7, 8, 9]])
>>> x[0:-1]
array([[0, 1, 2, 3, 4]])
>>> x[1:-1]
array([], shape=(0, 5), dtype=int64)
>>> x[1:]
array([[5, 6,...

1

votes

3

answer

55

Views

### What is the shortest Python code to get a maximum from the two discrete functions?

For given two discrete functions like this:
y1 = lambda(x): x**2
y2 = lambda(x): (x-1.5)*10
x1 = np.linspace(0,12,50)
x2 = np.linspace(1,10,12)
plot(x1, y1(x1), '.-')
plot(x2, y2(x2), '.-')
what is the shortest Python code to get the maximum of them?

1

votes

1

answer

32

Views

### Smoothing a fitted function

Good afternoon everybody,
I have been trying to smoothen out a fitted function I have as an output of an exponential fitting process. The lines of code I am using look like:
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
x = (0.005, 0.025, 0.05, 0.075, 0.1)
y...

1

votes

0

answer

91

Views

### stop scipy fsolve or hybrd immediately when the tolerance is reached

I am dealing with a time-consuming optimization problem where each function evaluation takes about 20 minutes, and I have to perform the optimization for several parameters. Therefore, I would like to terminate the scipy fsolve or hybrd immediately when the maximum of the function return values ar...

1

votes

0

answer

92

Views

### SciPy: Interpolation/Smoothing without edge bleeding

I have a raster that is being stored in a numpy array that holds an aerial photo of an area that may be any shape. There is a good amount of noise in the data within the area that needs to be smoothed out. The edge of the image (where no data is) is marked by 0s that extend to the edge of the raster...

1

votes

0

answer

118

Views

### Have `solve_ivp` return partial result when interrupted

I use scipy's solve_ivp to solve a differential equation. sometimes the integration takes too long and I want to abort it, but return the partial result that was obtained so far.
If I set a time limit in advance I can hack the events argument to check if too much time has elapsed and then stop the...

1

votes

1

answer

70

Views

### Scipy: pickle an ode solver instance

I am running long simulations which involve integration of an ode. Due to the long running times, I decided to generate fallbacks which allow to continue the simulations from a checkpoint. For the ode integration, I decided to pickle the scipy ode solver instance. This approach works for the reentra...

1

votes

0

answer

126

Views

### Python - Hierarchical clustering with starting weights

I'm trying to perform hierarchical/agglomerative clustering using Ward's method on 'pre-clustered' clusters. Since these inputs may not have the same cardinality, the updated distances using scipy's linkage function will be skewed.
To be clear, I have n input clusters, a compressed distance matrix...

1

votes

1

answer

201

Views

### How to get columns from big sparse csc matrix

I have a sparse matrix X
and I have an array
columns_to_use
It consist of 10000 id of columns of matrix X. I want to use only these columns and drop another columns. I try to use such code:
X_new = X[:, columns_to_use]
And it works good with small X (10 000 rows), but with 100 000 rows or more I g...

1

votes

0

answer

61

Views

### Python threads and scipy.interpolate.interp2d

I have this class that creates random 2d maps. The following code does nothing except creating the map since I've stripped it down to isolate the bug:
import numpy as np
from scipy.interpolate import interp2d
class CRelevanceMap:
def __init__(self):
self.relmap = self._create_map(4, 6)
def _create_m...

1

votes

2

answer

94

Views

### plannar fit with odrpack

I am trying to use scipy.odr to get a best fit plane for some x, y, z points.
I define the plane equation implicitly as ax + by + cz + d = 0 and I perform a least squares (with scipy.linalg.lstsq) to provide the odr with an initial estimation.
The components of the beta vector (where beta = [a, b, c...

1

votes

0

answer

141

Views

### Limit the output of ODEINT Python

I'm trying to define 'hard' limits to the return values of Scipy's odeint function but am unsure if function allows for such. I've modified this example from Scipy Cookbook so that the coupled two bodied system could collide.
Image Link
Graph Collision
Imaginary Stop
Specifically, the spring consta...

1

votes

1

answer

1.1k

Views

### Changing WAV format from 32-bit floating point to 16 bit PCM

I'm trying to convert my 32-bit floating point sampled WAVE file to 16-bit integer WAVE file.
I want to generate 16-bit integer WAV file from ultrasonic sine wave which range from -1.414 to 1.414. In SciPy.io.wavfile, the output format is fixed according to type array which I generated. But I chang...

1

votes

0

answer

138

Views

### Different results using leastsq and least_square (with lm) with Scipy 1.0.0?

I wonder if it is expected that Scipy's 1.0.0 leastsq and least_square (with lm) optimizers produce different results. The following code demonstrates that problem for me. (It's fitting Steinhart-Hart parameters.)
#!/usr/bin/env python3
import numpy as np
from scipy import optimize
#r = np.array([10...

1

votes

1

answer

347

Views

### Compute system response to time-varying input by odeint (or solve_ivp)

I’m trying to solve a simple ODE to visualise the temporal response, which works well for constant input conditions using the new solve_ivp integration API in SciPy. For example:
def dN1_dt_simple(t, N1):
return -100 * N1
sol = solve_ivp(fun=dN1_dt_simple, t_span=[0, 100e-3], y0=[N0,])
However, I...

1

votes

1

answer

293

Views

### scipy basinhopping with callback: stop the algorithm when a condition is met

my objective is, while using a basinhopping algorithm in order to find a global minimum, to stop the alogorithm whenever a condition is met after a minimum is found. by looking at the docs here (https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.optimize.basinhopping.html), with speci...

1

votes

0

answer

42

Views

### Numpy: Walk along vector within 2d matrix

Consider a function A(x(z, y), y(z)). I want to start at arbitrary (x, y), and walk along z. I have
dy/dz = 1
dx/dz = z y(1-x)/(y(y+z)). For epsilon z, this gives y(1-x)/(yy)
Now, take the following discretization:
import numpy as np
x = np.linspace(0.01, 0.99, 100)
y = np.linspace(0.5, 2, 50)
X, Y...

1

votes

0

answer

212

Views

### Count the numbers of neighbors in an array with 3D coordinates

I have the following problem. I have an an array of 3D coordinates like this:
arr = np.array([[21.000,48.000,28.000],[27.000,48.000,31.000],[21.000, 47.000,27.000],[22.000, 21.000, 97.000],[22.000, 20.000, 97.000],[22.000, 20.000, 95.000]])
This is only a small extract of the coordinates, because th...

1

votes

0

answer

56

Views

### Calculate nolinear model of vibration control

I am studding automatic and have a matlab/SIMULINK model of nonlinear vibration. I am trying to resolve the same problem using python instead.
Here we have a schema. The black frame is a part that I have already. I also have a Model tłumnika MR (eng. Magnetorheological Damper). Just I don't know ho...

1

votes

0

answer

70

Views

### Scipy.stats binned_statistic_dd for 3D closest-to-centroid per bin

Action
Select closest to centroid for each bin in binned 3D point set.
Attempts
Using binned_statistic_dd from scipy.stats I tried to take the binned mean of this 3D pointset. However, it will only compute the mean for one of the dimensions at a time.
import numpy as np
from scipy.stats import binne...

1

votes

0

answer

128

Views

### scipy.signal.minimumphase deviates from desired magnitude

I have a problem with the scipy.signal.minimumphase not providing the same magnitude response as i input, it deviates quite a lot.
Long story short. I have a material, where the absorption of said material is measured in octave bands(6 discrete values). I need an impulse response that fit those valu...

1

votes

1

answer

316

Views

### ImportError: scipy/spatial/ckdtree.so

I'm using Anaconda2 v5.0.1 on LinuxMint 17.3 Rosa. When I tried to import scipy.stats I got this error:
ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/dolf/anaconda2/lib/python2.7/site-packages/scipy/sparse/_sparsetools.so
Then I tried this...

1

votes

1

answer

74

Views

### Minimisation of a multivariable function

import math
import numpy
import scipy
def chisq1a(a,b,NN):
return (NN/(a**2)*b**2*(1+3/(a**2))*(1+(math.tan((math.pi)/2*(a+b*math.log(b/0.45))))**2))
x0 = numpy.array([0,0,0])
from scipy.optimize import minimize
result = scipy.optimize.minimize(chisq1a, x0)
I am trying to minimise a multivariable fu...

1

votes

0

answer

72

Views

### Error in minimization using scipy

I am trying to minimize a function using spicy but the estimate of sigma is way off. Any help would be greatly appreciated.
Here's my code so far:-
import numpy as np
from scipy.optimize import minimize
import matplotlib.pyplot as plt
import matplotlib.mlab as mlab
from pylab import plot, show, gr...

1

votes

0

answer

88

Views

### Reducing Memory requirements for distance/adjacency matrix

Based on a subsample of around 5,000 to 100,000 word-embeddings (GloVe, 300-dimensional) I need to construct an adjancency matrix, i.e. a matrix of 1's and 0's indicating if the euclidean (or cosine) distance between two words is smaller than x.
Currently, I'm using scipy.spatial.distance.pdist:
dis...

1

votes

1

answer

188

Views

### python linprog, lp appears to be unbounded

I'm trying to write a simple LP in Python to solve rock paper scissors, here is my code before:
from scipy.optimize import linprog
obj = [0, 0, 0, -1]
A = [[0, 1, -1, -1], [-1, 0, 1, -1], [1, -1, 0, -1], [1, 1, 1, 0]]
b = [0, 0, 0, 1]
pb = (0.0, 1)
wb = (None, None)
res = linprog(obj, A_ub=A, b_ub=b...

1

votes

0

answer

194

Views

### Using scipy.stats.entropy on gmm.predict_proba() values

Background so I don't throw out an XY problem -- I'm trying to check the goodness of fit of a GMM because I want statistical back-up for why I'm choosing the number of clusters I've chosen to group these samples. I'm checking AIC, BIC, entropy, and root mean squared error. This question is about en...

1

votes

0

answer

57

Views

### Efficient computation of a bilinear norm over sparse vectors in Python

Given two column vectors x, y : scipy.sparse.csc_matrix, where len(x) == len(y) == N and max(x.nnz, y.nnz) == M, and a symmetric N × N matrix A : scipy.sparse.csc_matrix, where for all columns j, A[j].nnz = C, I need to compute x.T * A * y = ∑∑ᵢ,ⱼ x[i] * a[j][i] * y[j] efficiently in at mos...

1

votes

0

answer

488

Views

### How to compute weighted cosine similarity between two vectores in Python

Hi i'm trying to calculate the cosine similarity between two vectors. I'm using scipy to do so.
>>> from scipy import spatial
>>> 1 - spatial.distance.cosine([3, 45, 7, 2], [2, 54, 13, 15])
0.97228425171234989
>>> 1 - spatial.distance.cosine([3, 45, 7, 2], [6, 54, 13, 15])
0.97216484313108098
But i...

1

votes

1

answer

897

Views

### Confidence interval in python

Is there a function or a package in python to get the 95% or 99% confidence interval of the distributions present in scipy.stats.
If not what is easiest alternative for that
Here are the distributions (but my priority is halfgennorm and loglaplace)
st.alpha,st.anglit,st.arcsine,st.beta,st.betaprime...

1

votes

0

answer

60

Views

### Best approach to fill signal gaps

I have a numpy 2dim array that represents a multi channel Bio-Signal. This array has dimension 20 x n_samples where the columns represent : Sample number - 16 channels data - time.
Given to bluetooth connection i have some package drops so i have gaps in signal. The array has to be imported into MNE...

1

votes

0

answer

207

Views

### Loading .mat files

I have the following code for loading .mat files
import scipy.io as sio
data = pd.DataFrame(sio.loadmat('data.mat').values()).iloc[7]
Loading these files takes several minutes to load. I believe the issue is from a convert the list to a pandas dataframe. The .mat files are not particularly large: on...