凸优化和单调变分不等式的收缩算法-南京大学数学系

each iteration.
2 ADMM based customized PPA
The k -th iteration of the proposed Alternating Direction Method of Multipliers in this section is also from a pair of (y k , λk ) to a new pair of (y k+1 , λk+1 ). In the prediction step, we generate a w ˜k
vs
Classical ADMM:
The iteration number of Ye-Yuan’s ADMM is less the one of the classical ADMM.
∗ However, in Ye-Yuan’s ADMM, we need to calculate the step size length αk in
v=
y λ
and
V ∗ = {(y ∗ , λ∗ ) | (x∗ , y ∗ , λ∗ ) ∈ Ω∗ }.
XIII - 4
Applied ADMM to the structure VI
(y k , λk ) ⇒ (y k+1 , λk+1 )
First, for given (y
k
, λk ) , x ˜k is the solution of the following problem
The task of solving the problem (1.1) is to find an (x∗ , y ∗ , λ∗ )
where
Ω = X × Y × ℜm .
XIII - 3
By denoting
T , w = u= , F ( w ) = y − B λ y λ Ax + By − b x
where
∗ αk = γαk ,
γ ∈ (0, 2)
(1.5a)
k k 2 k k ˜ k )T B (y k − y ∥ v − v ˜ ∥ + ( λ − λ ˜ ) ∗ H αk = ∥v k − v ˜k ∥2 H
(1.5b)
and
k ∥v k − v ˜k ∥2 ˜k )∥2 + H = β ∥B (y − y
Note that the mapping F is monotone. We use Ω∗ to denote the solution set of the variational inequality (1.3). For convenience we use the notations


XIII - 1
凸优化和单调变分不等式的收缩算法
第十三讲: 定制 PPA 算法 意义下的乘子交替方百度文库法
Alternating direction method in sense of customized PPA
南京大学数学系 何炳生 hebma@nju.edu.cn
The context of this lecture is based on the manuscript [2]
1 k ˜k 2 ∥λ − λ ∥ . β
The convergence of the classical alternating direction method and Ye-Yuan’s ADMM are demonstrated in Lecture 11.
XIII - 6
Ye-Yuan’s ADMM
k T k
{ y ˜k = Argmin
θ2 (y ) − (λ ) (Ax ˜ + By − b) y∈Y β k 2 ˜ + By − b∥ + 2 ∥Ax ˜ k = λk − β (Ax λ ˜k + B y ˜k − b).
}
(1.4b)
(1.4c)
The sub-problems (1.4a) and (1.4b) are separately solved.
and



x


−A λ
T

θ(u) = θ1 (x) + θ2 (y ),
the first order optimal condition (1.2) can be written in a compact form such as
w∗ ∈ Ω, θ(u)−θ(u∗ )+(w−w∗ )TF (w∗ ) ≥ 0, ∀ w ∈ Ω. (1.3)
XIII - 2
1 Structured constrained convex optimization
We consider the following structured constrained convex optimization problem
min {θ1 (x) + θ2 (y ) | Ax + By = b, x ∈ X , y ∈ Y} (1.1)
{ θ1 (x) − (λ ) (Ax + By − b) x∈X β k 2 + 2 ∥Ax + By − b∥
k T k
}
(1.4a)
x ˜k = Argmin
k Use λ and the obtained x ˜k ,
y ˜k is the solution of the following problem
XIII - 5
Classical Alternating Direction Method of Multipliers:
v k+1 = v ˜k .
Ye-Yuan’s Alternating Direction Method of Multipliers:
v k+1 = v k − αk (v k − v ˜k ),
where θ1 (x)
: ℜn1 → ℜ, θ2 (y ) : ℜn2 → ℜ are convex functions (but not necessary smooth), A ∈ ℜm×n1 , B ∈ ℜm×n2 and b ∈ ℜm , X ⊂ ℜn1 , Y ⊂ ℜn2 are given closed convex sets. ∗ ∗ T T ∗ θ ( x ) − θ ( x ) + ( x − x ) ( − A λ ) ≥ 0, 1 1 θ2 (y ) − θ2 (y ∗ ) + (y − y ∗ )T (−B T λ∗ ) ≥ 0, ∀ (x, y, λ) ∈ Ω, (1.2) (λ − λ∗ )T (Ax∗ + By ∗ − b) ≥ 0, ∈ Ω, such that