Samples
3
Function evaluations
Bayesian Optimization
Finding the maximum of an unknown function by balancing promising predictions with uncertainty.
Bayesian optimization searches for the maximum of an unknown, expensive-to-evaluate function. Instead of testing every possible input, it builds a probabilistic model of the function and decides where to sample next. This demo uses one-dimensional test functions so the whole learning process stays visible: observations, the Gaussian process posterior, and the next proposed sample.
Best x
—
Best sampled input
Best f(x)
—
Highest observed value
Next sample
—
Proposed input
Posterior
Gaussian Process Posterior
Acquisition
Where Should We Sample Next?
Background
A short account of the loop this page runs. The calculations are the NumPy engine behind the API. The article develops the same ideas with the experiment’s actual numbers.
1. Begin with an unknown function
Each evaluation is treated as expensive, so the search does not test every input. It starts from three fixed locations, \(x = 0.15\), \(0.50\), and \(0.85\), and only learns \(f(x)\) at inputs it chooses to sample. Reset returns to those same three points.
2. Fit a Gaussian process
A zero-mean Gaussian process predicts a likely value \(\mu(x)\) and a standard deviation \(\sigma(x)\). The radial basis function kernel ties nearby inputs together:
\[ k(x,x') = \sigma_f^2 \exp\!\left(-\frac{(x-x')^2}{2\ell^2}\right) \]The length scale \(\ell\) is the slider above. Signal variance is fixed at \(\sigma_f^2 = 1\). For observations \((X, y)\),
\[ K = k(X,X) + \sigma_n^2 I, \]
\[ \mu_* = K_*^\top K^{-1} y, \]
\[ \Sigma_* = K_{**} - K_*^\top K^{-1} K_*. \]
The shaded band is \(\mu \pm 2\sigma\). The factor \(K^{-1}\) is never formed: NumPy factors \(K\) with a Cholesky decomposition and the predictions come from triangular solves. The diagonal noise \(\sigma_n^2 = 10^{-6}\) keeps that factor stable. It is not a noise model the optimizer is trying to learn.
3. Choose the next point
Sampling only where \(\mu(x)\) is largest exploits what the model already believes. Uncertain regions can still hide a better value, so the acquisition function spends some of the budget on exploration.
Upper confidence bound, with exploration weight \(\kappa\):
\[ \mathrm{UCB}(x) = \mu(x) + \kappa\,\sigma(x) \]Expected improvement compares the posterior with the best value observed so far, \(f_{\text{best}}\), using a margin \(\xi\):
\[ I = \mu(x) - f_{\text{best}} - \xi, \]
\[ \mathrm{EI}(x) = I\,\Phi(Z) + \sigma(x)\,\phi(Z), \quad Z = I/\sigma(x). \]
\(\Phi\) and \(\phi\) are the standard normal distribution and density. The scores are computed on a grid of 200 inputs in \([0, 1]\), and the largest eligible score is the next sample. Points that have already been observed are skipped.
4. Observe and update
The objective is evaluated at that one input. The new pair is appended, the posterior is recomputed, and the acquisition curve is drawn again. Run 5 Steps repeats this loop, refitting before every choice. It does not take five points from the original curve.
The optional true function is drawn only for comparison. The Gaussian process never sees it except at the inputs it has actually sampled. Bayesian optimization is useful when those samples are the costly part: a model of the unknown function, plus a rule for where the next sample is worth buying, can reach a high value with far fewer evaluations than a uniform grid.