Alejandro Garcés Ruiz - Mathematical Programming for Power Systems Operation

Let us consider a mathematical optimization problem represented as ( Equation 2.22).

(2.22)

where f : n→ is the objective function, x are decision variables, Ω is the feasible set, and β are constant parameters of the problem.

A point x~ is a local optimum of the problem, if there exists an open set N(x~), named neighborhood, that contains x~ such that f(x)≥f(x~),∀x∈N(x~). If N=Ω then, the optimum is global. Figure 2.4 shows the concept for two functions in R with their respective neighborhoods N.

Figure 2.4 Example of local and global optima: a) function with two local minima and their respective neighborhoods, b) function with a unique global minimum (the neighborhood is the entire domain of the function).

There are two local minima in the first case, whereas there is a unique global minimum in the second case. This concept is more than a fancy theoretical notion; what good is a local optimum if there are even better solutions in another region of the feasible set? In practice, we require global or close-to-global optimum solutions.

On the other hand, several points may be optimal, as shown in Figure 2.5. In that case, all the points in the interval x 1≤ x ≤ x 2are global optima. Thus, the question is not only if the optimal point is global but also if it is unique. Both globality and uniqueness are geometrical questions with practical implications, especially in competitive markets. Convex optimization allows naturally answering these questions as explained in Chapter 3

Figure 2.5 Example of a function with several optimal points.

It is well-known, from basic mathematics, that the optimum of a continuous differentiable function is attached when its derivative is zero. This fact can be formalized in view of the concepts presented in previous sections. Consider a function f : → with a local minimum in x~. A neighborhood is defined as N={x∈R:x=x~±t,|t|

(2.23)

where t can be positive or negative. If t > 0, then ( Equation 2.23) can be divided by t without modifying the direction of the inequality, to then take the limit when t→0+ t → 0+ as presented below:

(2.24)

The same calculation can be made if t < 0, just in that case, the direction of the inequality changes as follows:

(2.25)

Notice that this limit is the definition of derivative; hence, f′(x~)≥0 and f′(x~)≤0 These two conditions hold simultaneously when f′(x~)=0. Consequently, the optimum of a differentiable function is the point where the derivative vanishes. This condition is local in the neighborhood N.

This idea can be easily extended to multivariable functions as follows: consider a function f:Rn→R (continuous and differentiable) and a neighborhood given by N={x∈Rn:x=x~+Δx} Now, define a function g(t)=f(x~+tΔx) If x~ is a local minimum of f , then

(2.26)

In terms of the new function g , ( Equation 2.26) leads to the following condition:

(2.27)

This condition implies that 0 is a local optimum of g ; moreover,

(2.28)

Notice that g is a function of one variable, then optimal condition g ′ = 0 is met, regardless the direction of Δ x . Therefore, the optimum of a multivariate function is given when the gradient is zero ∇f(x~)=0). This condition permits to find local optimal points, as presented in the next section. Two questions are still open: in what conditions are the optimum global? And, when is the solution unique? We will answer these relevant questions in the next chapter. For now, let us see how to find the optimum using the gradient.

The gradient method is, perhaps, the most simple and well-known algorithm for solving optimization problems. Cauchy invented the basic method in the 19th century, but the computed advent leads to different applications that encompass power systems operation and machine learning. Let us consider the following unconstrained optimization problem:

(2.29)

where the objective function f : n→ is differentiable. The gradient ∇ f ( x ) represents the direction of greatest increase of f . Thus, minimizing f implies to move in the direction opposite to the gradient. Therefore, we use the following iteration:

(2.30)

The gradient method consists in applying this iteration until the gradient is small enough, i.e., until ‖∇ f ( x )‖ ≥ ϵ. It is easier to understand the algorithm by considering concrete problems and their implementation in Python, as given in the next examples.