Optimization: Finding a Minimum in One Dimension
Claude prompt
This page was generated by Claude Code from the following prompt (with additional prompting):
Ok, let's make a demo. First part will be bisection, newton, and golden section search on a 1-D problem. Show both the function and its derivative so that it is clear that bisection and newton are finding a root of the derivative. [...]
Split the 1d search and gradient descent into two demos, but both linked to the gradient descent lecture in the lecture table. Make it easier to read f and f' on the plots and use different colors for f and f'.
The function from Chapra's chapter on one-dimensional optimization, negated so that we are minimizing: \[ f(x) = \frac{x^2}{10} - 2\sin x, \qquad f'(x) = \frac{x}{5} - 2\cos x, \qquad f''(x) = \frac{1}{5} + 2\sin x. \] It has a minimum near \(x \approx 1.43\) and a maximum near \(x \approx 5.27\). At either one, \(f'(x) = 0\). So one way to find the minimum is to hand \(f'\) to a root finder from earlier in the course. Bisection and Newton-Raphson below do exactly that: they only ever look at the bottom plot. Golden-section search is different: it only evaluates \(f\) itself, never the derivative.
Bisection and Newton converge to a point where \(f' = 0\), but nothing in them cares whether that point is a maximum or a minimum. Drag the Newton start to the right and it lands on the maximum instead. The sign of \(f''\) tells the two apart. Golden-section search keeps the bracket around the smaller of its two interior values, so it only ever closes in on a minimum (negate \(f\) to find a maximum).