Introduction to Sequential Monte Carlo Methods

1) 1 j y1:n 1 )
f ( xn j xn 1 ) g ( yn j xn ) p ( yn j y1:n 1 )
1 ) p ( xn 1 j y1:n 1 ) dxn 1:n .
=
Z
g ( yn j xn ) f ( xn j xn
We will also simply write p ( x1:n j y1:n ) ∝ p ( x1:n
Introduction to Sequential Monte Carlo Methods
Arnaud Doucet
NCSU, October 2008
Arnaud Doucet ()
Introduction to SMC
NCSU, October 2008
1 / 36
Preliminary Remarks
Arnaud Doucet ()
1
+ W k , Wk
i.i.d.
N (0, Σ) ,
1 0 T 1 , T 2 /2 T
ACV 0 ΣCV 0
0 ACV 0 ΣCV
1)
, ACV = , ΣCV =
T 3 /3 T 2 /2
1 , Σ) .
= N (xk ; Axk
Introduction to SMC
1 ) dxk 1 ) dxk 1 ) p ( xk 1 j y1:k 1 ) dxk 1
We have “broken" an high dimensional integral into the product of lower dimensional integrals.
Arnaud Doucet () Introduction to SMC NCSU, October 2008 12 / 36
Z
p ( y1:n j x1:n ) p (x1:n ) dx1:n .
n
We also have the following decomposition p (y1:n ) = p (y1 ) ∏ p ( yk j y1:k
k =2 1)
where p ( yk j y1:k
1)
= = =
Z
Z
p ( yk , xk j y1:k
Sequential Monte Carlo (SMC) are a set of methods allowing us to approximate virtually any sequence of probability distributions. SMC are very popular in physics where they are used to compute eigenvalues of positive operators, the solution of PDEs/integral equations or simulate polymers. We focus here on Applications of SMC to Hidden Markov Models (HMM) for pedagogical reasons... In the HMM framework, SMC are also widely known as Particle Filtering/Smoothing methods.
Arnaud Doucet ()
p ( y1:n j x1:n ) p (x1:n ) dx1:n .
NCSU, October 2008 9 / 36
Introduction to SMC
Sequential Inference in HMM
In particular, we will focus here on the sequential estimation of p ( x1:n j y1:n ) and p (y1:n ); that is at each time n we want update our knowledge of the hidden process in light of yn . There is a simple recursion relating p ( x1:n 1 j y1:n 1 ) to p ( x1:n j y1:n ) given by p ( x1:n j y1:n ) = p ( x1:n where p ( yn j y1:n
Introduction to SMC
NCSU, October 2008
4 / 36
Figure: Graphical model representation of HMM
Arnaud Doucet ()
Introduction to SMC
NCSU, October 2008
5 / 36
Tracking Example
n
We do not observe fXk gk 1 ; the process is hidden. We only have access to another related process fYk gk 1 .
1
g ( j xk ) .
k =1
∏ g ( yk j xk ) .
Arnaud Doucet ()
k =2
Arnaud Doucet () Introduction to SMC
k =2 n
1)
NCSU, October 2008
3 / 36
Observation Model
We assume that, conditional on fXk gk 1 , the observations fYk gk are independent and marginally distributed according to Yk j (Xk = xk ) Formally this means that p ( y1:n j x1:n ) =
k =2 n n 1) ,
Likelihood: p ( y1:n j x1:n ) = Using Bayes’rule, we obtain p ( x1:n j y1:n ) =
Z
k =1
∏ g ( yk j xk )
p ( y1:n j x1:n ) p (x1:n )Fra bibliotekp (y1:n )
where the marginal likelihood is given by p (y1:n ) =
We have the following standard model Xk = φXk so that f ( xk j xk We observe Yk = β exp (Xk /2) Wk , Wk so that g ( yk j xk ) = N yk ; 0, β2 exp (xk ) .
1
µ( )
= xk
1)
f ( j xk
1) .
We introduce the notation xi :j = (xi , xi +1 , ..., xj ) for i by de…nition p (x1:n ) = p (x1 ) ∏ p ( xk j x1:k
n 1)
j. We have
= µ (x1 ) ∏ f ( xk j xk
Arnaud Doucet ()
Introduction to SMC
NCSU, October 2008
2 / 36
Markov Models
We model the stochastic processes of interest as a discrete-time Markov process fXk gk 1 . fXk gk 1 is characterized by its initial density X1 and its transition density Xk j (Xk
Z
g ( yk j xk ) p ( xk j y1:k g ( yk j xk ) f ( xk j xk
Closed-form Inference in HMM
We have closed-form solutions for
Finite state-space HMM; i.e. E = fe1 , ..., ep g as all integrals are becoming …nite sums Linear Gaussian models; all the posterior distributions are Gaussian; e.g. the celebrated Kalman …lter. A whole reverse engineering literature exists for closed-form solutions in alternative cases...
Arnaud Doucet ()
1 j y1:n 1 ) f
( xn j xn
1) g
( yn j xn ) .
10 / 36
Introduction to SMC
NCSU, October 2008
In many papers/books in the literature, you will …nd the following two-step prediction-updating recursion for the marginals so-called …ltering distributions p ( xn j y1:n ) which is a direct consequence. Prediction Step p ( xn j y1:n
1 1 i.i.d.
N (0, Σe )