马尔科夫链蒙特卡洛方法
合集下载
相关主题
- 1、下载文档前请自行甄别文档内容的完整性，平台不提供额外的编辑、内容补充、找答案等附加服务。
- 2、"仅部分预览"的文档,不可在线预览部分如存在完整性等问题,可反馈申请退款(可完整预览的文档不适用该条件!)。
- 3、如文档侵犯您的权益，请联系客服反馈,我们会尽快为您处理(人工客服工作时间：9:00-18:30)。
4
, 1 ‡”MC”, Monte Carlo, KL«3Ä
Previous Next First Last Back Forward
CarloÈ©•{éÈ©?1OŽ. MCM•{ nØ•â3ueã 4•½n: Theorem 1. {Xt , t ≥ 0}•˜ØŒ ,š±Ï G ˜m•Ω ê¼ó,
ùp
πt L«ê¼ó3tž•
>S©Ù. l
_•ê¼ó˜„Ø2´àg
.
XJ t → ∞žÙÂñ ²-©Ù, ½öπ0 = π, K _•ê¼óÒäk
Previous Next First Last Back Forward
²8
5, dž
∗ Pij (t) =
Pji π(j) ∗ = Pij . π(i) ê ¼ ó(reversible markov chain). Œ
Previous Next First Last Back Forward 7
1.1.3
lÑ b G
Markov Chain
˜m
{Xt , t ≥ 0}•˜àgê¼ó, =£VÇ•Pij = P (Xt+1 = j|Xt = i),
Hale Waihona Puke Baidu
²-©Ù•π. Klƒ‡ ••wdê¼ók P (Xt = j|Xt+1 , Xt+2 , . . .) = P (Xt = j|Xt+1 ),
Ù¥x1 , · · · , xm •l©Ùfθ|x (θ|x)¥Ä ±ŠâŒêÆ• ´3˜ {Ò´•d8 Ù¥Ä
.
x1 , · · · , xm Õáž, KŒ
þnª•uáž, g Âñ E[g(θ|x). ¯ ´š~(J , MCMC•
¯K¥, l©Ùfθ|x (θ|x)¥Ä )
, Ù1˜‡”MC”, Markov Chain, ÒL«l8I© e, A^Monte
Kz~ê⌱¤•— Kz~ꙕž, dÈ©
3
Ý. 3 “d©Û¥, π(·)•˜
Previous Next First Last Back Forward
•Œ±OŽ. (©f©1¥ 3 éõ¯Ke,
Kz~ê-ž
). ù´š~Ð
˜‡5Ÿ, Ï•
Kz~êéJOŽ. vkw«L«, ꊕ{•éJO ˜‡éÐ OŽ•{.
‘ÅCþX, …é?¿¼êf ,
n
Eπ |f (X)|•3ž, …n → ∞ž,
f (Xt ) → Eπ f (X),
t=1
a.s.
Previous Next First Last Back Forward
5
¯ e¡·‚OŽH{þŠfn Kf t •
2 τn
•
. Pf t = f (Xt ), γk = cov(f t , f t+k ),
UXe•ª½Â˜‡ê¼ó: 1. lž•t j. 2. ±VÇmin{1,
pj pi
G
i=£
e‡ž•
G
, d=£ØQ)¤˜‡ÿÀ
G
} Ée˜ž• G •Xt+1 = j, ÄKXt+1 = i. ê¼ó =£VÇ:
·‚OŽ˜ed«•ªe (a). i = j: Pij = =
P (Xt+1 = j, T A|Xt = i) = P (Xt+1 = j|Xt = i)P (T A) Qij min{1, pj } pi
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
Previous Next First Last Back Forward
1
Chapter 1 Markov Chain Monte Carlo Methods
∗ ePij = Pij , dž¡dê¼ó•Œ _ Œ
_ê¼ó
Œ_5²~ L«•([—²ï•§, detailed balance equations) π(i)Pij = π(j)Pji .
l XJ˜‡8I©Ùπ÷vd[—²ï•§, KN´ yÙÒ´²-©Ù: π(i)Pij =
j j
π(j)Pji π(j)Pji
”TA” - the event ”transition is accepted” (b). i = j: Pii = P (Xt+1 = i, T A|Xt = i) + P (Xt+1 = i, T A|Xt = i)
Previous Next First Last Back Forward 10
= Œ±y²
σ2 + 2
k=1
n−k n−k γk = σ 2 [1 + 2 ρk ]. n n k=1
∞
2 τn → τ 2 = σ 2 [1 + 2 k=1
ρk ]
¯ . ÏdH{þŠfn
• • σ2 n−k ¯ [1 + 2 ρk ]. V arπ (fn ) = n n k=1
Previous Next First Last Back Forward 6
²-©Ù, @ol8I©Ùf (t)¥ ‰Åó¥ )Ù ´». ·
A«ïáù
•{: MetropolisŽ{, MetropolisóATäk¯„·Ü(rapid
HastingsŽ{, ±9GibbsÄ
•{. ˜‡Ð
mixing)5Ÿ—l?¿ ˜Ñu鯈 ²-©Ù.
1.1.1
Integration problems in Bayesian inference
.
¼ê. ÏdŒ±ég(θ)?1íä. 'Xg(θ) = θž, O.
KEg(θ|x) = E[θ|x] Œ±Š•θ éda¯K·‚•Ä˜„/ª:
Eg(Y ) =
g(t)π(t)dt . π(t)dt Ï"½Â:
ùpπ(·)•˜‡—ݽöq,. eπ•˜—Ý, KdÏ"=•~„ Eg(Y ) = g(t)fY (t)dt. eπ•˜q,, KI‡˜‡ —Ý. π(·)
Lecture 8: Markov Chain Monte Carlo Methods (˜) ˜ (ê ‰ Å „A k Û •{ ) ê
Ü•² Monday 26th October, 2009
Contents
1 Markov Chain Monte Carlo Methods 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . 1.1.1 Integration problems in Bayesian inference . 1.1.2 Markov Chain Monte Carlo Integration . . 1.1.3 Markov Chain . . . . . . . . . . . . . . . . 1.2 The Metropolis-Hastings Algorithm . . . . . . . . . 1.2.1 Metropolis-Hastings Sampler . . . . . . . . 1.2.2 The Metropolis Sampler . . . . . . . . . . . 1.2.3 Random Walk Metropolis . . . . . . . . . . 1.2.4 The Independence Sampler . . . . . . . . . 1.3 Single-component Metropolis Hastings Algorithms 1.4 Application: Logistic regression . . . . . . . . . . . 1 1 2 4 8 13 14 22 23 33 37 39
∗ =‡••5w, E•ê¼ó(reversed markov chain, _•ê¼ó). PPij (t)•
‡••wê¼ólt + 1ž•=£
∗ Pij (t)
tž• =£VÇ, K P (Xt+1 = i|Xt = j)P (Xt = j) P (Xt+1 = i
= =
P (Xt = j|Xt+1 = i) = Pji πt (j) πt+1 (i)
1.1 Introduction
MCMC(Markov Chain Monte Carlo) •{ ˜„nصeŒ±ëw Metropolis et al. (1953)±9 Hastings (1970), ±9Ù¦ˆ«0 !·‚0 ù«•{ Ä gŽÚA^. MCMC ;Í.
5¿3c¡0 Monte Carlo•{ OÈ© g(t)dt,
2
l #:
ŠâBayes½n, Œ±ÏL
x = (x1 , · · · , xn ) fx|θ (x)π(θ) fx|θ (x)π(θ)dθ
&Eéθ
©Ù?1•
fθ|x (θ|x) == K3 ©Ùe, g(θ) Ï"• Eg(θ|x) = dÈ©•Š• x
.
g(θ)fθ|x (θ|x)dθ =
g(θ)fx|θ (x)π(θ)dθ fx|θ (x)π(θ)dθ
n−1
Theorem 3. e˜‡ó´˜—
AÛH{(=£VDZ„Ýλt , 0 < λ < 1 , Kk n→∞
ñ) , ±9 f ƒéu²-©Ùπ´²•ŒÈ
¯ √ fn − Eπ f (x) n → N (0, 1), τ ù‡½né E²-©Ùπ ù ½n`² : • ØŒ +š±Ï+ • LLN: ê¼ó ~ˆ=⇒ •˜ ²-©Ù. ëê
,˜•¡, 3éõäN¯K¥dÈ©Š Ž, AO´3p‘ž,
MCMC•{édaÈ©Jø
1.1.2
Markov Chain Monte Carlo Integration
g(θ)fθ|x (θ|x)dθ g= ¯ 1 m Monte Carlo O•
m
È© E[g(θ|x)] =
þŠ
g(xi ),
i=1
˜&«mJø nØ•â.
¢Š¼ê H{þŠ±VÇ1Âñ
4•©Ùe þŠ. ©Ù. •ž
• CLT: H{þŠŠÜ· Cz•©ÙÂñ IO Ïd, ·‚Œ±ïᘇ±π•²-©Ù m , Tê¼óÒ¬ˆ ²-G
ê¼ó, K3$1dóv ŠÒƒ
, džê¼ó
ul©Ùπ¥Ä
. l Œ±ŠâH{½néÈ©?1 O. MCMC•{ `:3u: 1. ·^u2•…(J ¯K; 2. ¯K‘ê O\Ï~جü$ÙÂñ„ݽö¦Ù•E,.
j
⇐⇒ ⇐⇒ ~ {pj , j ∈ Ω,
π(i)
j
Pij =
π(i) =
j j
π(j)Pji
²-©Ù
½Â
pj = 1} •·‚a,
8I©Ù, ±9Q•?˜Ø
Previous Next First Last Back Forward
9
Œ
G
˜m•Ω ê¼ó =£VÇÝ , …÷vé¡5^‡ Qij = Qji , i, j ∈ Ω
π•²-©Ù, π0 ´Ð©©Ù, K πt → π, ùpπt L«ê¼ó3tž• t → ∞. dê¼ó$1¿©
>S©Ù. d½n`²
•žm (t = n, néŒ), K3ž•n, Xn
©Ù•π, …ÚЩ©ÙÃ'.
Theorem 2. (Markov chain Law of Large Numbers)eX1 , X2 , · · · •˜H { ²-©Ù•π ©Ù•π K 1 ¯ fn = n ê¼óŠ(Ù¥z‡Xt Œ±´õ‘ ), KXn •©ÙÂñ
•σ 2 = γ0 , kÚƒ'Xêρk = γk /σ 2 . l = 1 ¯ nV arπ (fn ) = nV ar( n 1 n
n n
f t) =
t=1
1 V arπ ( f t) n t=1
n
=
V arπ (f t ) + 2
t=1 n−1
1 n
n−t
cov(f t , f t+k )
k=1 n−1
Nõ¯KÑ´MCMC•{ A^. lBayesian *:5w,
Bayesianíä¥ .¥
*ÿCþÚëêÑ´‘ÅCþ. Ïd,
x = (x1 , · · · , xn )Úë
êθéÜ©ÙŒ±L«• fx,θ (x, θ) = fx|θ (x1 , · · · , xn )π(θ).
Previous Next First Last Back Forward
A
´rdÈ©L«¤˜‡é,‡VÇ—Ýf (t)e ¤l8IVÇ—Ýf (t)¥ ) ‘Å .
Ï". l
È©
O¯K=z
Previous Next First Last Back Forward
1
3MCMC•{¥, Äkïá˜‡ê ±$1dê )‘Å ‚ò0
‰Åó, ¦
f (t)•Ù²-©Ù. KŒ
‰Åó¿©•žm†– Âñ , Ò´lˆ ²-G ê ‰Åó ê
, 1 ‡”MC”, Monte Carlo, KL«3Ä
Previous Next First Last Back Forward
CarloÈ©•{éÈ©?1OŽ. MCM•{ nØ•â3ueã 4•½n: Theorem 1. {Xt , t ≥ 0}•˜ØŒ ,š±Ï G ˜m•Ω ê¼ó,
ùp
πt L«ê¼ó3tž•
>S©Ù. l
_•ê¼ó˜„Ø2´àg
.
XJ t → ∞žÙÂñ ²-©Ù, ½öπ0 = π, K _•ê¼óÒäk
Previous Next First Last Back Forward
²8
5, dž
∗ Pij (t) =
Pji π(j) ∗ = Pij . π(i) ê ¼ ó(reversible markov chain). Œ
Previous Next First Last Back Forward 7
1.1.3
lÑ b G
Markov Chain
˜m
{Xt , t ≥ 0}•˜àgê¼ó, =£VÇ•Pij = P (Xt+1 = j|Xt = i),
Hale Waihona Puke Baidu
²-©Ù•π. Klƒ‡ ••wdê¼ók P (Xt = j|Xt+1 , Xt+2 , . . .) = P (Xt = j|Xt+1 ),
Ù¥x1 , · · · , xm •l©Ùfθ|x (θ|x)¥Ä ±ŠâŒêÆ• ´3˜ {Ò´•d8 Ù¥Ä
.
x1 , · · · , xm Õáž, KŒ
þnª•uáž, g Âñ E[g(θ|x). ¯ ´š~(J , MCMC•
¯K¥, l©Ùfθ|x (θ|x)¥Ä )
, Ù1˜‡”MC”, Markov Chain, ÒL«l8I© e, A^Monte
Kz~ê⌱¤•— Kz~ꙕž, dÈ©
3
Ý. 3 “d©Û¥, π(·)•˜
Previous Next First Last Back Forward
•Œ±OŽ. (©f©1¥ 3 éõ¯Ke,
Kz~ê-ž
). ù´š~Ð
˜‡5Ÿ, Ï•
Kz~êéJOŽ. vkw«L«, ꊕ{•éJO ˜‡éÐ OŽ•{.
‘ÅCþX, …é?¿¼êf ,
n
Eπ |f (X)|•3ž, …n → ∞ž,
f (Xt ) → Eπ f (X),
t=1
a.s.
Previous Next First Last Back Forward
5
¯ e¡·‚OŽH{þŠfn Kf t •
2 τn
•
. Pf t = f (Xt ), γk = cov(f t , f t+k ),
UXe•ª½Â˜‡ê¼ó: 1. lž•t j. 2. ±VÇmin{1,
pj pi
G
i=£
e‡ž•
G
, d=£ØQ)¤˜‡ÿÀ
G
} Ée˜ž• G •Xt+1 = j, ÄKXt+1 = i. ê¼ó =£VÇ:
·‚OŽ˜ed«•ªe (a). i = j: Pij = =
P (Xt+1 = j, T A|Xt = i) = P (Xt+1 = j|Xt = i)P (T A) Qij min{1, pj } pi
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
Previous Next First Last Back Forward
1
Chapter 1 Markov Chain Monte Carlo Methods
∗ ePij = Pij , dž¡dê¼ó•Œ _ Œ
_ê¼ó
Œ_5²~ L«•([—²ï•§, detailed balance equations) π(i)Pij = π(j)Pji .
l XJ˜‡8I©Ùπ÷vd[—²ï•§, KN´ yÙÒ´²-©Ù: π(i)Pij =
j j
π(j)Pji π(j)Pji
”TA” - the event ”transition is accepted” (b). i = j: Pii = P (Xt+1 = i, T A|Xt = i) + P (Xt+1 = i, T A|Xt = i)
Previous Next First Last Back Forward 10
= Œ±y²
σ2 + 2
k=1
n−k n−k γk = σ 2 [1 + 2 ρk ]. n n k=1
∞
2 τn → τ 2 = σ 2 [1 + 2 k=1
ρk ]
¯ . ÏdH{þŠfn
• • σ2 n−k ¯ [1 + 2 ρk ]. V arπ (fn ) = n n k=1
Previous Next First Last Back Forward 6
²-©Ù, @ol8I©Ùf (t)¥ ‰Åó¥ )Ù ´». ·
A«ïáù
•{: MetropolisŽ{, MetropolisóATäk¯„·Ü(rapid
HastingsŽ{, ±9GibbsÄ
•{. ˜‡Ð
mixing)5Ÿ—l?¿ ˜Ñu鯈 ²-©Ù.
1.1.1
Integration problems in Bayesian inference
.
¼ê. ÏdŒ±ég(θ)?1íä. 'Xg(θ) = θž, O.
KEg(θ|x) = E[θ|x] Œ±Š•θ éda¯K·‚•Ä˜„/ª:
Eg(Y ) =
g(t)π(t)dt . π(t)dt Ï"½Â:
ùpπ(·)•˜‡—ݽöq,. eπ•˜—Ý, KdÏ"=•~„ Eg(Y ) = g(t)fY (t)dt. eπ•˜q,, KI‡˜‡ —Ý. π(·)
Lecture 8: Markov Chain Monte Carlo Methods (˜) ˜ (ê ‰ Å „A k Û •{ ) ê
Ü•² Monday 26th October, 2009
Contents
1 Markov Chain Monte Carlo Methods 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . 1.1.1 Integration problems in Bayesian inference . 1.1.2 Markov Chain Monte Carlo Integration . . 1.1.3 Markov Chain . . . . . . . . . . . . . . . . 1.2 The Metropolis-Hastings Algorithm . . . . . . . . . 1.2.1 Metropolis-Hastings Sampler . . . . . . . . 1.2.2 The Metropolis Sampler . . . . . . . . . . . 1.2.3 Random Walk Metropolis . . . . . . . . . . 1.2.4 The Independence Sampler . . . . . . . . . 1.3 Single-component Metropolis Hastings Algorithms 1.4 Application: Logistic regression . . . . . . . . . . . 1 1 2 4 8 13 14 22 23 33 37 39
∗ =‡••5w, E•ê¼ó(reversed markov chain, _•ê¼ó). PPij (t)•
‡••wê¼ólt + 1ž•=£
∗ Pij (t)
tž• =£VÇ, K P (Xt+1 = i|Xt = j)P (Xt = j) P (Xt+1 = i
= =
P (Xt = j|Xt+1 = i) = Pji πt (j) πt+1 (i)
1.1 Introduction
MCMC(Markov Chain Monte Carlo) •{ ˜„nصeŒ±ëw Metropolis et al. (1953)±9 Hastings (1970), ±9Ù¦ˆ«0 !·‚0 ù«•{ Ä gŽÚA^. MCMC ;Í.
5¿3c¡0 Monte Carlo•{ OÈ© g(t)dt,
2
l #:
ŠâBayes½n, Œ±ÏL
x = (x1 , · · · , xn ) fx|θ (x)π(θ) fx|θ (x)π(θ)dθ
&Eéθ
©Ù?1•
fθ|x (θ|x) == K3 ©Ùe, g(θ) Ï"• Eg(θ|x) = dÈ©•Š• x
.
g(θ)fθ|x (θ|x)dθ =
g(θ)fx|θ (x)π(θ)dθ fx|θ (x)π(θ)dθ
n−1
Theorem 3. e˜‡ó´˜—
AÛH{(=£VDZ„Ýλt , 0 < λ < 1 , Kk n→∞
ñ) , ±9 f ƒéu²-©Ùπ´²•ŒÈ
¯ √ fn − Eπ f (x) n → N (0, 1), τ ù‡½né E²-©Ùπ ù ½n`² : • ØŒ +š±Ï+ • LLN: ê¼ó ~ˆ=⇒ •˜ ²-©Ù. ëê
,˜•¡, 3éõäN¯K¥dÈ©Š Ž, AO´3p‘ž,
MCMC•{édaÈ©Jø
1.1.2
Markov Chain Monte Carlo Integration
g(θ)fθ|x (θ|x)dθ g= ¯ 1 m Monte Carlo O•
m
È© E[g(θ|x)] =
þŠ
g(xi ),
i=1
˜&«mJø nØ•â.
¢Š¼ê H{þŠ±VÇ1Âñ
4•©Ùe þŠ. ©Ù. •ž
• CLT: H{þŠŠÜ· Cz•©ÙÂñ IO Ïd, ·‚Œ±ïᘇ±π•²-©Ù m , Tê¼óÒ¬ˆ ²-G
ê¼ó, K3$1dóv ŠÒƒ
, džê¼ó
ul©Ùπ¥Ä
. l Œ±ŠâH{½néÈ©?1 O. MCMC•{ `:3u: 1. ·^u2•…(J ¯K; 2. ¯K‘ê O\Ï~جü$ÙÂñ„ݽö¦Ù•E,.
j
⇐⇒ ⇐⇒ ~ {pj , j ∈ Ω,
π(i)
j
Pij =
π(i) =
j j
π(j)Pji
²-©Ù
½Â
pj = 1} •·‚a,
8I©Ù, ±9Q•?˜Ø
Previous Next First Last Back Forward
9
Œ
G
˜m•Ω ê¼ó =£VÇÝ , …÷vé¡5^‡ Qij = Qji , i, j ∈ Ω
π•²-©Ù, π0 ´Ð©©Ù, K πt → π, ùpπt L«ê¼ó3tž• t → ∞. dê¼ó$1¿©
>S©Ù. d½n`²
•žm (t = n, néŒ), K3ž•n, Xn
©Ù•π, …ÚЩ©ÙÃ'.
Theorem 2. (Markov chain Law of Large Numbers)eX1 , X2 , · · · •˜H { ²-©Ù•π ©Ù•π K 1 ¯ fn = n ê¼óŠ(Ù¥z‡Xt Œ±´õ‘ ), KXn •©ÙÂñ
•σ 2 = γ0 , kÚƒ'Xêρk = γk /σ 2 . l = 1 ¯ nV arπ (fn ) = nV ar( n 1 n
n n
f t) =
t=1
1 V arπ ( f t) n t=1
n
=
V arπ (f t ) + 2
t=1 n−1
1 n
n−t
cov(f t , f t+k )
k=1 n−1
Nõ¯KÑ´MCMC•{ A^. lBayesian *:5w,
Bayesianíä¥ .¥
*ÿCþÚëêÑ´‘ÅCþ. Ïd,
x = (x1 , · · · , xn )Úë
êθéÜ©ÙŒ±L«• fx,θ (x, θ) = fx|θ (x1 , · · · , xn )π(θ).
Previous Next First Last Back Forward
A
´rdÈ©L«¤˜‡é,‡VÇ—Ýf (t)e ¤l8IVÇ—Ýf (t)¥ ) ‘Å .
Ï". l
È©
O¯K=z
Previous Next First Last Back Forward
1
3MCMC•{¥, Äkïá˜‡ê ±$1dê )‘Å ‚ò0
‰Åó, ¦
f (t)•Ù²-©Ù. KŒ
‰Åó¿©•žm†– Âñ , Ò´lˆ ²-G ê ‰Åó ê