Convex Optimization Problems

() Convex Optimization Problems
(FOC)
Q.E.D.
July 24, 2009 10 / 11
The rst order condition for (CP). Slater's Condition: There exists an x such that gj () < 0 for all j = 1, . . . m. x
()
Convex Optimization Problems
July 24, 2009
6 / 11
More Homework Problems: (1) Prove: If f and g are both concave functions, the function h = min{f, g} is also concave. (2) Prove: A function f is concave if and only if, for all x and y, and for all α ≤ 0, f (αx + (1 α)y) ≤ αf (x) + (1 α)f (y). (3) If f is concave on Rn , then for any x1 , . . . , xm ∈ Rn , and real numbers αj ≥ 0 with
()
Convex Optimization Problems
July 24, 2009
8 / 11
2. Convex Optimization Problems
We call the following optimization problem a concave maximization problem: maxx∈Rn s.t. f (x) (CP)
()
Convex Optimization Problems
July 24, 2009
2 / 11
Remark: A function f is concave if and only if f is convex. Hence, we will state most results for concave functions only. They all have counterparts for convex functions. Homework: Prove that when both f and g are concave, f + g is also concave. Homework: Prove that if f is concave and increasing on R1 and g is concave, then the composite function f g is also concave. Homework: Prove that a function f on Rn is concave if and only if the set below its graph in Rn+1 , i.e., (x, y) ∈ Rn+1 | x ∈ Rn , y ∈ R, y ≤ f (x) is a convex set.
() Convex Optimization Problems
Q.E.D.
July 24, 2009 5 / 11
Theorem 3:
(1) If f dened on Rn is dierentiable, then f is concave if and only if for any x, y ∈ Rn , f (y) ≤ f (x) + f (x)(y x).
Theorem 2: A function f dened on a convex set C Rn is concave if and
only if its restriction on any line segment in C is a concave function on [0, 1]. Proof: Suppose the restriction of f on any line segment within C is a concave function. Then, for any x, y ∈ C, let g(α) = f (αx + (1 α)y) for α ∈ (0, 1). f (αx + (1 α)y) = g(α) = g(α1 + (1 α)0) ≥ αg(1) + (1 α)g(0) = αf (x) + (1 α)f (y). Hence, f is concave on C. Reversing the argument proves the other half of the theorem.
≤ f (x ) + λ1 (g1 (x) g1 (x )) + . . . + λm (gm (x) gm (x )) (Theorem 3) = f (x ) + λ1 g1 (x) + . . . + λm gm (x) ≤ f (x ) Hence, x is optimal for (CP).
(FOC)
j = 1, . . . , m,
then x is optimal for (CP). Proof: For any feasible x for (CP), f (x) ≤ f (x ) +
f (x )(x x )
(Theorem 3) (FOC)
= f (x ) + [λ1 g1 (x ) + . . . + λm gm (x )] (x x ) = f (x ) + λ1 g1 (x )(x x ) + . . . + λm gm (x )(x x )
(2) If f on Rn is twice dierentiable, then f is concave if and only if its Hessian matrix
2f xi xj
is non-positive denite.
பைடு நூலகம்
The proof of Theorem 2 follows from Theorem 1 and 2 and the chain rule of dierentiation.
g1 (x) ≤ 0 . . . gm (x) ≤ 0.
in which f is concave, and gj are convex. For any concave maximization problem (CP), the rst order condition is also sucient for optimal solutions.
Theorem 1:
(1) If f dened on R1 is dierentiable, then f is concave if and only if its rst order derivative f (t) is weakly decreasing. (2) If f dened on R1 is twice dierentiable, then f is concave if and only if its second order derivative f (t) is non-positive. (3) If f dened on R1 is dierentiable, then f is concave if and only if for any x, y ∈ Rn , f (y) ≤ f (x) + f (x)(y x). Proof: (Homework)
()
Convex Optimization Problems
July 24, 2009
4 / 11
Theorem 1 can be extended to functions with many variables using the following trick. For any convex set C and any pair x, y ∈ C, the line segment between x and y is parameterized by the interval [0, 1] R1 .
Lecture Note 6. Convex Optimization Problems
July 24, 2009
()
Convex Optimization Problems
July 24, 2009
1 / 11
1. Concave (Convex) Functions
Denition: A function f on a convex set C in Rn is concave if for any x, y ∈ C and any α ∈ (0, 1) f (αx + (1 α)y) ≥ αf (x) + (1 α)f (y). It is called strictly concave if the above inequality is always strict. Denition: A function f on a convex set C in Rn is convex if for any x, y ∈ C and any α ∈ (0, 1) f (αx + (1 α)y) ≤ αf (x) + (1 α)f (y). It is called strictly convex if the above inequality is always strict.
j=m j=1
αj = 1,
j=m j=m
f(
j=1
αj xj ) ≥