Nonlinear least squares and curve fitting with scipy.optimize.least_squares and curve_fit
Informational article in the Scientific Computing with SciPy topical map — Optimization, Root Finding, and Curve Fitting content group. 12 copy-paste AI prompts for ChatGPT, Claude & Gemini covering SEO outline, body writing, meta tags, internal links, and Twitter/X & LinkedIn posts.
Nonlinear least squares and curve fitting with scipy.optimize.least_squares and curve_fit can fit parametric models to data using least-squares minimization: curve_fit is a convenience wrapper that returns optimal parameters and an estimated covariance matrix (pcov), while least_squares is a general solver supporting 'trf', 'dogbox', and 'lm' methods and explicit bounds. The Levenberg–Marquardt algorithm (lm) implemented via MINPACK is suited to unconstrained problems, whereas trust-region-reflective ('trf') handles bounds; the reduced chi-square used to scale covariance is sum(residuals**2)/(M−N) where M is data points and N is parameters. Parameter standard errors come from sqrt(diag(pcov)) scaled by s^2.
Mechanically, both interfaces minimize a sum-of-squares objective built from residuals r_i(theta) and rely on the Jacobian J = ∂r/∂theta to determine search direction and uncertainty; providing an analytic Jacobian often yields faster convergence and more accurate parameter covariance than finite differences. scipy curve_fit wraps an older Levenberg–Marquardt-based entry point and computes pcov from J^T J, while scipy.optimize.least_squares exposes trust-region-reflective and dogbox algorithms and accepts loss functions and bounds. For nonlinear regression with scipy the Jacobian can be returned from model functions, automatic differentiation tools like JAX can be used for parameter estimation python workflows, and robust loss functions reduce bias from outliers. Sparsity patterns and Jacobian-provided sparsity dramatically reduce memory and CPU for large problems.
A common and consequential misconception is treating curve_fit and least_squares as drop-in equivalents: curve_fit (which historically calls MINPACK’s Levenberg–Marquardt) returns popt and pcov but does not support bounds or robust loss functions, while scipy.optimize.least_squares offers bounds, 'trf' and 'dogbox' methods and robust loss choices. Another frequent error is omitting an analytic Jacobian for stiff models or high-dimensional parameter vectors—finite differences can be slow and inaccurate. Interpreting pcov incorrectly also leads to misleading uncertainties: when absolute_sigma=False (the default), the covariance returned must be interpreted relative to the reduced chi-square s^2 = sum(residuals**2)/(M−N); standard errors require multiplying sqrt(diag(pcov)) by sqrt(s^2). For example, fitting a nonlinear ODE model with 1,000 observations and 10 parameters makes finite-difference gradient noise dominate and can bias convergence and covariance estimates.
Practically, select scipy.optimize.least_squares when parameter bounds, explicit robust loss functions, or large-scale control of iterations and Jacobian sparsity are required; use scipy curve_fit for quick, unconstrained fits when analytic uncertainty estimates and legacy MINPACK behavior are acceptable. Always supply an analytic Jacobian when available, validate pcov scaling with the reduced chi-square, and prefer trust-region or dogbox methods for bounded problems. When uncertainties are critical, profile likelihood or bootstrap Monte Carlo provide alternatives to pcov-based Gaussian intervals. The following article provides a step-by-step framework covering analytic Jacobian implementation, diagnostic plots, performance tuning, and confidence-interval estimation, and numerical stability checks.
- Work through prompts in order — each builds on the last.
- Click any prompt card to expand it, then click Copy Prompt.
- Paste into Claude, ChatGPT, or any AI chat. No editing needed.
- For prompts marked "paste prior output", paste the AI response from the previous step first.
scipy curve fit example
Nonlinear least squares and curve fitting with scipy.optimize.least_squares and curve_fit
authoritative, practical, evidence-based
Optimization, Root Finding, and Curve Fitting
Python developers, data scientists, and researchers with intermediate numerical methods knowledge who need practical, reproducible guidance implementing nonlinear least squares and curve fitting in SciPy for research or production
A hands-on, end-to-end comparison of scipy.optimize.least_squares and curve_fit with reproducible code, diagnostics, performance tuning, production best practices, and a decision checklist for choosing the right solver — combining practical engineering tips and numerical theory in one resource.
- scipy curve_fit
- scipy.optimize.least_squares
- nonlinear regression with scipy
- parameter estimation python
- Jacobian
- Levenberg-Marquardt
- trust-region-reflective
- robust loss functions
- parameter covariance
- Treating curve_fit and least_squares as interchangeable without acknowledging that curve_fit wraps least_squares but returns covariance differently — leads to incorrect uncertainty claims.
- Not supplying an analytic Jacobian when available — relying on numerical differentiation and losing performance and accuracy for stiff or large problems.
- Misinterpreting the covariance matrix returned by curve_fit (not dividing by residual variance or misreading diagonal scaling) and reporting misleading parameter errors.
- Failing to scale parameters and data, which causes solver convergence problems or non-meaningful tolerance behaviors.
- Using Levenberg–Marquardt (method='lm') with bounds or constrained problems (lm ignores bounds) and confusing silent failures.
- Ignoring robust loss functions for real experimental data and using standard least squares that are highly sensitive to outliers.
- Assuming default solvers and tolerances are production-ready — not tuning xtol/ftol/gtol or max_nfev for reliability in pipelines.
- Always provide an analytic Jacobian when possible; if not available, compute and cache Jacobian-vector products or use autograd/JAX for scalable automatic derivatives to drastically speed up least_squares.
- For bounded problems prefer least_squares with method='trf' or 'dogbox'; use 'lm' only for unbounded problems — include a short unit test that fails when bounds are present to avoid silent misuse.
- Estimate parameter uncertainties robustly: compute the covariance from the Jacobian and residual variance, but validate with a bootstrap or MCMC (e.g., emcee) for non-linear or non-Gaussian posteriors before publishing confidence intervals.
- Scale parameters and residuals: rescale variables so typical parameter magnitudes are O(1) to improve condition numbers and convergence; include a small helper function to auto-scale inputs.
- Benchmark solvers on representative synthetic workloads: measure function evaluations, wall time, and final residual; report those in a short table and use numba or JIT compilation for inner-loop speedups when fitting many datasets.
- Use robust loss (soft_l1, huber) during exploratory fitting to reduce the influence of outliers, then switch to a refined fit with least squares for final parameter estimates if appropriate.
- When deploying fits in production, add convergence checks, parameter bounds validation, and fallback strategies (e.g., try different initial guesses or solver methods) and log numeric diagnostics for reproducibility.
- Document SciPy version and machine environment in a small 'reproducibility' snippet; numerical behavior changes across SciPy releases — pin or test against the target SciPy version.