Let’s say we have the following unrestricted quadratic problem to solve:
We further assume that is a symmetric positive definite matrix.
Let’s say that we have an iterative algorithm to find the solution of problem . The algorithm works by selecting a “good” direction at iteration and then walk some amount in that direction.
Moreover, let’s say that we know how to compute in order to minimize the problem: