# Questions tagged [scipy]

7409 questions

1

votes

0

answer

91

Views

### stop scipy fsolve or hybrd immediately when the tolerance is reached

I am dealing with a time-consuming optimization problem where each function evaluation takes about 20 minutes, and I have to perform the optimization for several parameters. Therefore, I would like to terminate the scipy fsolve or hybrd immediately when the maximum of the function return values ar...

1

votes

0

answer

92

Views

### SciPy: Interpolation/Smoothing without edge bleeding

I have a raster that is being stored in a numpy array that holds an aerial photo of an area that may be any shape. There is a good amount of noise in the data within the area that needs to be smoothed out. The edge of the image (where no data is) is marked by 0s that extend to the edge of the raster...

1

votes

0

answer

118

Views

### Have `solve_ivp` return partial result when interrupted

I use scipy's solve_ivp to solve a differential equation. sometimes the integration takes too long and I want to abort it, but return the partial result that was obtained so far.
If I set a time limit in advance I can hack the events argument to check if too much time has elapsed and then stop the...

1

votes

1

answer

70

Views

### Scipy: pickle an ode solver instance

I am running long simulations which involve integration of an ode. Due to the long running times, I decided to generate fallbacks which allow to continue the simulations from a checkpoint. For the ode integration, I decided to pickle the scipy ode solver instance. This approach works for the reentra...

1

votes

0

answer

126

Views

### Python - Hierarchical clustering with starting weights

I'm trying to perform hierarchical/agglomerative clustering using Ward's method on 'pre-clustered' clusters. Since these inputs may not have the same cardinality, the updated distances using scipy's linkage function will be skewed.
To be clear, I have n input clusters, a compressed distance matrix...

1

votes

1

answer

201

Views

### How to get columns from big sparse csc matrix

I have a sparse matrix X
and I have an array
columns_to_use
It consist of 10000 id of columns of matrix X. I want to use only these columns and drop another columns. I try to use such code:
X_new = X[:, columns_to_use]
And it works good with small X (10 000 rows), but with 100 000 rows or more I g...

1

votes

0

answer

61

Views

### Python threads and scipy.interpolate.interp2d

I have this class that creates random 2d maps. The following code does nothing except creating the map since I've stripped it down to isolate the bug:
import numpy as np
from scipy.interpolate import interp2d
class CRelevanceMap:
def __init__(self):
self.relmap = self._create_map(4, 6)
def _create_m...

1

votes

2

answer

94

Views

### plannar fit with odrpack

I am trying to use scipy.odr to get a best fit plane for some x, y, z points.
I define the plane equation implicitly as ax + by + cz + d = 0 and I perform a least squares (with scipy.linalg.lstsq) to provide the odr with an initial estimation.
The components of the beta vector (where beta = [a, b, c...

1

votes

0

answer

141

Views

### Limit the output of ODEINT Python

I'm trying to define 'hard' limits to the return values of Scipy's odeint function but am unsure if function allows for such. I've modified this example from Scipy Cookbook so that the coupled two bodied system could collide.
Image Link
Graph Collision
Imaginary Stop
Specifically, the spring consta...

1

votes

1

answer

1.1k

Views

### Changing WAV format from 32-bit floating point to 16 bit PCM

I'm trying to convert my 32-bit floating point sampled WAVE file to 16-bit integer WAVE file.
I want to generate 16-bit integer WAV file from ultrasonic sine wave which range from -1.414 to 1.414. In SciPy.io.wavfile, the output format is fixed according to type array which I generated. But I chang...

1

votes

0

answer

138

Views

### Different results using leastsq and least_square (with lm) with Scipy 1.0.0?

I wonder if it is expected that Scipy's 1.0.0 leastsq and least_square (with lm) optimizers produce different results. The following code demonstrates that problem for me. (It's fitting Steinhart-Hart parameters.)
#!/usr/bin/env python3
import numpy as np
from scipy import optimize
#r = np.array([10...

1

votes

1

answer

347

Views

### Compute system response to time-varying input by odeint (or solve_ivp)

I’m trying to solve a simple ODE to visualise the temporal response, which works well for constant input conditions using the new solve_ivp integration API in SciPy. For example:
def dN1_dt_simple(t, N1):
return -100 * N1
sol = solve_ivp(fun=dN1_dt_simple, t_span=[0, 100e-3], y0=[N0,])
However, I...

1

votes

1

answer

293

Views

### scipy basinhopping with callback: stop the algorithm when a condition is met

my objective is, while using a basinhopping algorithm in order to find a global minimum, to stop the alogorithm whenever a condition is met after a minimum is found. by looking at the docs here (https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.optimize.basinhopping.html), with speci...

1

votes

0

answer

42

Views

### Numpy: Walk along vector within 2d matrix

Consider a function A(x(z, y), y(z)). I want to start at arbitrary (x, y), and walk along z. I have
dy/dz = 1
dx/dz = z y(1-x)/(y(y+z)). For epsilon z, this gives y(1-x)/(yy)
Now, take the following discretization:
import numpy as np
x = np.linspace(0.01, 0.99, 100)
y = np.linspace(0.5, 2, 50)
X, Y...

1

votes

0

answer

212

Views

### Count the numbers of neighbors in an array with 3D coordinates

I have the following problem. I have an an array of 3D coordinates like this:
arr = np.array([[21.000,48.000,28.000],[27.000,48.000,31.000],[21.000, 47.000,27.000],[22.000, 21.000, 97.000],[22.000, 20.000, 97.000],[22.000, 20.000, 95.000]])
This is only a small extract of the coordinates, because th...

1

votes

0

answer

56

Views

### Calculate nolinear model of vibration control

I am studding automatic and have a matlab/SIMULINK model of nonlinear vibration. I am trying to resolve the same problem using python instead.
Here we have a schema. The black frame is a part that I have already. I also have a Model tłumnika MR (eng. Magnetorheological Damper). Just I don't know ho...

1

votes

0

answer

70

Views

### Scipy.stats binned_statistic_dd for 3D closest-to-centroid per bin

Action
Select closest to centroid for each bin in binned 3D point set.
Attempts
Using binned_statistic_dd from scipy.stats I tried to take the binned mean of this 3D pointset. However, it will only compute the mean for one of the dimensions at a time.
import numpy as np
from scipy.stats import binne...

1

votes

0

answer

128

Views

### scipy.signal.minimumphase deviates from desired magnitude

I have a problem with the scipy.signal.minimumphase not providing the same magnitude response as i input, it deviates quite a lot.
Long story short. I have a material, where the absorption of said material is measured in octave bands(6 discrete values). I need an impulse response that fit those valu...

1

votes

1

answer

316

Views

### ImportError: scipy/spatial/ckdtree.so

I'm using Anaconda2 v5.0.1 on LinuxMint 17.3 Rosa. When I tried to import scipy.stats I got this error:
ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/dolf/anaconda2/lib/python2.7/site-packages/scipy/sparse/_sparsetools.so
Then I tried this...

1

votes

1

answer

74

Views

### Minimisation of a multivariable function

import math
import numpy
import scipy
def chisq1a(a,b,NN):
return (NN/(a**2)*b**2*(1+3/(a**2))*(1+(math.tan((math.pi)/2*(a+b*math.log(b/0.45))))**2))
x0 = numpy.array([0,0,0])
from scipy.optimize import minimize
result = scipy.optimize.minimize(chisq1a, x0)
I am trying to minimise a multivariable fu...

1

votes

0

answer

72

Views

### Error in minimization using scipy

I am trying to minimize a function using spicy but the estimate of sigma is way off. Any help would be greatly appreciated.
Here's my code so far:-
import numpy as np
from scipy.optimize import minimize
import matplotlib.pyplot as plt
import matplotlib.mlab as mlab
from pylab import plot, show, gr...

1

votes

0

answer

88

Views

### Reducing Memory requirements for distance/adjacency matrix

Based on a subsample of around 5,000 to 100,000 word-embeddings (GloVe, 300-dimensional) I need to construct an adjancency matrix, i.e. a matrix of 1's and 0's indicating if the euclidean (or cosine) distance between two words is smaller than x.
Currently, I'm using scipy.spatial.distance.pdist:
dis...

1

votes

1

answer

188

Views

### python linprog, lp appears to be unbounded

I'm trying to write a simple LP in Python to solve rock paper scissors, here is my code before:
from scipy.optimize import linprog
obj = [0, 0, 0, -1]
A = [[0, 1, -1, -1], [-1, 0, 1, -1], [1, -1, 0, -1], [1, 1, 1, 0]]
b = [0, 0, 0, 1]
pb = (0.0, 1)
wb = (None, None)
res = linprog(obj, A_ub=A, b_ub=b...

1

votes

0

answer

194

Views

### Using scipy.stats.entropy on gmm.predict_proba() values

Background so I don't throw out an XY problem -- I'm trying to check the goodness of fit of a GMM because I want statistical back-up for why I'm choosing the number of clusters I've chosen to group these samples. I'm checking AIC, BIC, entropy, and root mean squared error. This question is about en...

1

votes

0

answer

57

Views

### Efficient computation of a bilinear norm over sparse vectors in Python

Given two column vectors x, y : scipy.sparse.csc_matrix, where len(x) == len(y) == N and max(x.nnz, y.nnz) == M, and a symmetric N × N matrix A : scipy.sparse.csc_matrix, where for all columns j, A[j].nnz = C, I need to compute x.T * A * y = ∑∑ᵢ,ⱼ x[i] * a[j][i] * y[j] efficiently in at mos...

1

votes

0

answer

488

Views

### How to compute weighted cosine similarity between two vectores in Python

Hi i'm trying to calculate the cosine similarity between two vectors. I'm using scipy to do so.
>>> from scipy import spatial
>>> 1 - spatial.distance.cosine([3, 45, 7, 2], [2, 54, 13, 15])
0.97228425171234989
>>> 1 - spatial.distance.cosine([3, 45, 7, 2], [6, 54, 13, 15])
0.97216484313108098
But i...

1

votes

1

answer

897

Views

### Confidence interval in python

Is there a function or a package in python to get the 95% or 99% confidence interval of the distributions present in scipy.stats.
If not what is easiest alternative for that
Here are the distributions (but my priority is halfgennorm and loglaplace)
st.alpha,st.anglit,st.arcsine,st.beta,st.betaprime...

1

votes

0

answer

60

Views

### Best approach to fill signal gaps

I have a numpy 2dim array that represents a multi channel Bio-Signal. This array has dimension 20 x n_samples where the columns represent : Sample number - 16 channels data - time.
Given to bluetooth connection i have some package drops so i have gaps in signal. The array has to be imported into MNE...

1

votes

0

answer

207

Views

### Loading .mat files

I have the following code for loading .mat files
import scipy.io as sio
data = pd.DataFrame(sio.loadmat('data.mat').values()).iloc[7]
Loading these files takes several minutes to load. I believe the issue is from a convert the list to a pandas dataframe. The .mat files are not particularly large: on...

1

votes

1

answer

30

Views

### regarding sampling a two dimensional array and let the sampling array has three-dimension shape

There are two numerical arrays, train_wiggle and train_spike. Both of which have shape like (1000,101). Physically, they respectively represent 1000 points where each point has 101 features.
I would like to generate two corresponding random samples, wiggle_sample and spike_sample from train_wiggle a...

1

votes

0

answer

167

Views

### Replicating Spectogram of Audacity?

I'm trying to plot the spectrograms of audio samples. While I plot it using my code given below, it figures out to be weirder. However, I imported them into audacity which came out to be so nice. Suggest me the changes I need to make in order to replicate the same in python? I would like to know tha...

1

votes

0

answer

64

Views

### Combining threading and numpy vectorize

do you know a neat routine which has the functionality of
def func_(a,b,c):
'do some stuff'
return (a+b*c)
func = np.vectorize(func_)
a_ = np.linspace(0,1,300)
b_ = np.linspace(-3,3,200)
c_ = np.linspace(10,20,55)
a,b,c = np.meshgrid(a_,b_,c_,indexing='ij')
X = func(a,b,c)
but uses multiple cores, i...

1

votes

1

answer

189

Views

### Python: Get all coordinates between two 3D points or draw a 3D Line

I have the following problem. I have 3D point coordinates and I want to conenct them in an array or 'draw a line' like you can do it in 2D with skimage(http://scikit-image.org/docs/0.13.x/api/skimage.draw.html#skimage.draw.line). The optimal case would be if i could directly draw a cylinder with a r...

1

votes

0

answer

123

Views

### Can Sklearn Imputer distinguish real zero values from zeros in place for missing values?

I am using scipy sparse matrices, which fill missing values in with zeros. I want to impute these missing values with the mean of the respective columns, and planned to do this with the Imputer in Sklearn. My question, does the Imputer distinguish between real 0 values in the data from missing value...

1

votes

0

answer

28

Views

### Adding values of small arrays to positions in a greater array

I have the following problem. I have an array with shape (21,41,91). Now I need to fill given line coordinates in the box array with ones. Now I want to add skimage balls to that coodintates in the box array, so that the surrounding ball coordinates get also ones in the boxarray ( adding the balls t...

1

votes

1

answer

178

Views

### Python numpy.convolve integral limits

I have a convolution integral that has a similar form described in this post: (link)
Instead of integrating from 0 to t, I need to integrate from minus infinity to t. I was not able to do this, however, using numpy.convolve, as it always return results from minus infinity to plus infinity.
Using sci...

1

votes

0

answer

64

Views

### How can I create a sparse matrix representation of a dataset in python?

Assume that I have the following dataset:
CustomerID ItemID Quantity
236 512 2
236 372 3
245 512 5
245 129 2
245 326 3
251 372 5
251 629 3
I want to represent this dataset as a sparse matrix, where the rows are the CustomerIDs, th...

1

votes

0

answer

69

Views

### Scipy Exponential Decay Curve

I am trying to fit my data points to exponential decay curve. So, my code is:
import numpy as np
from scipy.optimize import curve_fit
import matplotlib.pyplot as plt
x = np.array([54338, 54371, 54547])
y = np.array([6.37468,6.36120,6.34980])
# prepare some data
x1 = np.array([54324,54332,54496, 546...

1

votes

1

answer

233

Views

### Vectorized random walk in Python with boundaries

I'm trying to simulate a 2-D random walk in python with boundaries (the particle/object will can't cross the boundaries and must return back). However my version is not vectorised and is very slow. How do I implement it without(or minimising) the use of loops.
Here is my approach
def bound_walk():
O...

1

votes

0

answer

282

Views

### Scipy.optimize.curve_fit gets stuck

I read in a sine wave from an oscilloscope. There are over 30*10^6 points divided into several thousand segments (5000). I need to fit these data, segment for segment. The problem is that the program gets stuck after fitting some segments. The error I get is
RuntimeError: Optimal parameters not fou...