In a previous post we have seen the conditions in which extremum values are guaranteed to exist. Nonetheless, there are cases in which the extremum values are there but the Weierstrass-Extremum value theorem didn’t tell us that much. The function has a global minimum on the interval , for example. Further, the function
has a global maximum and a global minimum on the open interval .
Let’s say we have the following unrestricted quadratic problem to solve:
We further assume that is a symmetric positive definite matrix.
Let’s say that we have an iterative algorithm to find the solution of problem . The algorithm works by selecting a “good” direction at iteration and then walk some amount in that direction.
Moreover, let’s say that we know how to compute in order to minimize the problem:
Consider the following optimization problem:
Let’s rewrite the objective function such that is a solution of the original problem if and only if it is also a solution for the modified objective function. Denote the new objective function: