连续时间马尔科夫链

=
lim
h→0+
1
−
pi,i(h) . h
,
limh→0+
1−pi,i(h) h
.
4.66
对任何
i ̸= j,
qi,j
=
limt→0+
pi,j (t) t
存在有限.
.
, 0 < ε < 1/3,
δ > 0,
t≤δ ,
pi,i(t) > 1 − ε, pj,j(t) > 1 − ε.
(4.31)
0≤h≤t≤δ ,
≤ qi −
qi,r .
r∈EK \{i}
81
(4.35)
草稿 不要打印
EK → E
lim ± pi,j(t +
h→0+
h) − pi,j(t) h
≤
∑ ± qi,rpr,j(t).
r∈E
,
lim
h→0+
pi,j(t + h) − pi,j(t) h
≤
∑ qi,rpr,j (t)
r∈E
≤
lim
h→0+
Q
{Xt : t ≥ 0}
.
∑
i ∈ E, qi = j̸=i qi,j < ∞,
Q
.
E ,Q
.
qi,i = −qi, ,
4.68 设 Q 是保守的, 则 ∑
p′i,j (t) = qi,rpr,j (t),
r∈E
上式称为 Kolmogorov 向后方程.
∀i, j ∈ E, t ≥ 0.
. E
h → 0,
t→0
0,
i=j i ̸= j.
76
(4.29)
草稿 不要打印
:
不
.
,
不
.
t ≥ 0,
pi(t) = P {Xt = i}, i ∈ E.
pi(0) = P {X0 = i}, i ∈ E.
{Xt : t ≥ 0}
.
,
∑ pj(t) = pi(0)pi,j(t),
i∈E
j ∈ E, t ≥ 0.
,
|pi,j(t − h) − pi,j(t)| ≤ |1 − pi,i(h)|.
(4.29)
.
4.65
对任何
i ∈ E,
极限
qi
:= limh→0+
1−pi,i(h) h
存在（可以为
∞）并且对任
何 t > 0, (1 − pi,i(t))/t ≤ qi.
.
q˜i = ∞, qi q˜i < ∞.
q˜i
=
lim
h→0+
1
−
pi,i(h) h
qi = ∞,
.
t > 0,
1
−
pi,i(t) t
≤
q˜i.
0 < ε < 1/t.
h > 0,
1
−
pi,i(h) h
≤
q˜i
+
ε ,
2
( ε) h q˜i + 2 < 1.
0≤s≤h ,
εt
1 − pi,i(s) ≤
. 2
( ε) pi,i(h) ≥ 1 − h q˜i + 2 > 0
s > 0, pi,i(s) > 0.
, nh > t ,
pi,j(nh) ≥ pi,i(nh − t)pi,j(t) > 0.
, {Xnh : n ≥ 0}
1.
83
草稿 不要打印
4.73 对任何 h > 0 以及 i, j ∈ E, limn→∞ pi,j(nh) 存在.
εt
pi,i(s) ≥ 1 −
. 2
78
(4.30)
草稿 不要打印
t = nh + s,
n
, 0 ≤ s < h. C-K ,
∑ pi,i(t + s) = pi,r(t)pr,i(s) ≥ pi,i(t)pi,i(s).
r∈E
,
pi,i(t)
≥
(pi,i(h))npi,i(s)
(
( ε ))n
(
−
pi,j (t)
−
∑
pi,r (t)qr,j
= 0.
r∈E
, pi,j(t) .
Kolmogorov
.
82
草稿 不要打印
pj(t + h) − pj(t) h
=
∑
pi(0)
∑
pi,r
(t)
pr,j
(h)
− h
pr,j
(0)
i∈E
r∈E
=
∑
pr
(t)
pr,j
(h)
− h
pr,j
(0)
,
r∈E
(4.36) .
草稿 不要打印
4.7
,
.
4.7.1
4.62 设随机过程 {Xt : t ≥ 0} 的状态空间 E 是至多可数集, 若对任何整数 n ≥ 1, 参数 0 ≤ t0 < t1 < · · · < tn < tn+1 以及状态 i0, i1, · · ·, in+1 ∈ E, 有
P {Xtn+1 = in+1|Xt0 = i0, · · · , Xtn = in} = P {Xtn+1 = in+1|Xtn = in}, 则称 {Xt : t ≥ 0} 为连续时间马尔可夫链.
)
ε
≥ n (1 − ε) − 1 − ε pi,j(h)pj,j(t − mh)
≥ n(1 − 3ε)pi,j(h).
, (4.32) .
(4.32)
h,
limh→0+ nh = t,
pi,j(h) ≤ 1 · pi,j(t) .
h
1 − 3ε nh
,
lim pi,j(h) ≤ 1 · pi,j(t) .
方程.
4.70 有限状态齐次马尔可夫链的转移概率函数满足 Kolmogorov 向前和向后
Kolmogorov
P ′(t) = P (t)Q (
), P ′(t) = QP (t) (
).
P (s + t) = P (s)P (t)
4.68
,
P ′(s + t) = P ′(t)P (s) = P ′(s)P (t),
(4.31) ,
m∑−1
ε
pi,i;j(mh) ≥ pi,i(mh) −
pi,j;j (lh)
≥
(1
−
ε)
−
1
−
, ε
l=1
0 ≤ m ≤ n.
,
∑n
pi,j(t) ≥
pi,j;j(mh)pj,j(t − mh)
m=1
∑n
≥
pi,i;j((m − 1)h)pi,j(h)pj,j(t − mh)
m=(1
i
j,
i → j.
i → j, j → i,
ij ,
i ↔ j. E
,
不.
4.72 {Xt : t ≥ 0} 与它的 h-骨架的状态具有相同的可达性.
.
i, j ∈ E,
t>0
pi,j(t) > 0,
n,
pi,j(nh) > 0.
,
lims→0 pi,i(s) = 1
pi,i(s) ≥ (pi,i(s/n))n
r∈E
∀i, j ∈ E, t ≥ 0.
上式称为 Kolmogorov 向前方程. 此外, 下述 Fokker-Planck 方程成立:
∑ p′j(t) = pr(t)qr,j.
r∈E
(4.36)
. ,
h > 0, C-K ,
pi,j (t
+
h) h
−
pi,j (t)
=
∑ pi,r(t) pr,j(h)
pi,j(t + h) − h
pi,j (t)
, pi,j(t) ,
. 0<h<t ,
,
Kolmogorov
t − h t, .
4.69 设 qj < ∞ 且 limh→0+ pr,j(h)/h = qr,j 关于 r ∈ E \ {j} 一致成立, 则
∑ p′i,j (t) = pi,r(t)qr,j ,
, {Xt : t ≥ 0}
,
pi,j(t) = P{Xs+t = j|Xs = i} = P {Xs+t − Xs = j − i}

e−λt
(λt)j−i (j−i)!
,
t > 0, j ≥ i,
=

0, δi,j ,
j < i, t=0
s
.
, {Xt : t ≥ 0}
.
4.7.2 Kolmogorov
s, t ≥ 0.
4.69
,
P ′(s + t) = P (t)P ′(s) = P (s)P ′(t), s, t ≥ 0.
4.7.3
,
.
4.71 给定常数 h > 0, 称 {Xnh : n ≥ 0} 为 {Xt : t ≥ 0} 的 h-骨架.
{Xt : t ≥ 0}
i, j,
t>0
pi,j(t) > 0,
.
4.64 对任何 i, j ∈ E, pi,j(t) 在 [0, ∞) 上一致连续. 77
草稿 不要打印
.
h > 0. C-K ,
∑
pi,j(t + h) − pi,j(t) =
pi,r(h)pr,j(t) − pi,j(t)
r∈E
∑
= pi,i(h)pi,j(t) − pi,j(t) + pi,r(h)pr,j(t)
− h
pr,j(0) .
r∈E
pi,j (t
+
h) h
−
pi,j (t)
−
∑
pi,r (t)qr,j
∑ ≤ pi,r(t)
pr,j (h)
− h
pr,j (0)
−
qr,j
.
r∈E
r∈E
,
M,
h
,
pr,j (h)
− h
pr,j (0)
−
qr,j
≤ M,
∀r ∈ E.
,
lim
h→0+
pi,j (t
+
h) h
.
,
.
4.63 强度为 λ 的 Poisson 过程是齐次马尔可夫链.
. {Xt : t ≥ 0}
λ Poisson .
i0 ≤ i1 ≤ · · · ≤ in+1 ∈ E,
0 ≤ t0 ≤ t1 < · · · < tn+1
P {Xtn+1 = in+1|Xt0 = i0, · · · , Xtn = in} = P {Xtn+1 − Xtn = in+1 − in|Xt0 − X0 = i0, · · · , Xtn − Xtn−1 = in − in−1} = P {Xtn+1 − Xtn = in+1 − in} = P {Xtn+1 = in+1|Xtn = in}.
r̸=i
≥ (pi,i(h) − 1)pi,j(t)
≥ pi,i(h) − 1.
∑
∑
pi,j(t + h) − pi,j(t) ≤ pi,r(h)pr,j(t) ≤ pi,r(h) = 1 − pi,i(h)
r̸=i
r̸=i
t−h>0
|pi,j(t + h) − pi,j(t)| ≤ |1 − pi,i(h)|. t−h t
m=1
∑n ≥ (1 − ε) pi,j;j(mh)
m=1
79
草稿 不要打印
,
∑n
ε
pi,j;j (mh)
≤
1
−
. ε
m=1
m∑−1
m∑−1
pi,i(mh) = pi,i;j(mh) + pi,j;j(lh)pj,i((m − l)h) ≤ pi,i;j(mh) + pi,j;j(lh),
l=1
l=1
(t)
r∈E\EK
≤ ∑ pi,r(h) = 1 − pi,i(h) − ∑ pi,r(h) .
h
h
h
r∈E\EK
r∈EK \{i}


lim ± pi,j(t + h) − pi,j(t) −
h→0+
h
∑
qi,rpr,j (t)
r∈EK
≤
qi
−
∑ lim h→∑ 0+ r∈EK \{i}
pi,r (h) h
(4.28)
(4.28)
,
P {Xt+s = j|Xs = i}, s, t ≥ 0, i, j ∈ E.
s
i,
t
j
,
pi,j(s, s + t) = P {Xs+t = j|Xs = i}.
pi,j(s, s + t) s
,
{Xt : t ≥ 0}
.
,
.
{Xt : t ≥ 0}
P (t) = [pi,j(t)]i,j∈E,
pi,j (h)
≤
pi,j (t) n
·
1
1 .
− 3ε
(4.32)
n = ⌊t/h⌋. pi,k;j(mh)
i,
vh, 1 ≤ v ≤ m − 1 不
j,
mh
k
. 0≤h≤t≤δ ,
∑ ε > 1 − pi,i(t) = pi,k(t)
k̸=i
≥ pi,j(t)
∑n
≥
pi,j;j(mh)pj,j(t − mh)
h > 0, C-K ,
pi,j(t +
h) − pi,j(t) h
=
∑
pi,r
(h)
− h
pi,r
(0)
pr,j
(t).
r∈E
EK ,
i ∈ EK ,
pi,j(t + h) − pi,j(t) − h
∑
pi,r
(h)
− h
pi,r
(0)
pr,j
(t)
r∈EK
≤
∑
pi,r
(h)
− h
pi,r
(0)
pr,j
h→0+ h
1 − 3ε t
t → 0+,
lim pi,j(h) ≤ 1 lim pi,j(t) .
h→0+ h
1 − 3ε h→0+ t
ε
,,
lim pi,j(h) ≤ lim pi,j(t) .
h→0+ h
h→0+ t
,
qi,j
=
limh→0+
pi,j (t) t
. (4.34)
qi,j < ∞.
∑ 4.67 j∈E\{i} qi,j ≤ qi.
80
(4.33) (4.34)
草稿 不要打印
. t→0
1 − pi,i(t) = ∑ pi,j(t) ,
t
t
j̸=i
.
, i ̸= j , qi,j = p′i,j(0);
i = j , qi = −p′i,i(0).
Q = [qi,j ]i,j∈E .
,
:
(i). pi,j(t) ≥ 0, i, j ∈ E, t ≥ 0; ∑
(ii). j∈Z pi,j(t) = 1, i ∈ E;
(iii). C-K
: P (s + t) = P (s)P (t),
∑ pi,j(s + t) = pi,k(s)pk,j(t).
k∈E
{Xt : t ≥ 0}
{
lim pi,j(t) = δi,j = 1,
) εt
≥
1−h (
q˜i (
+
· 2
ε ))
1− (
2) εt
≥ 1 − nh q˜i + 2 · 1 − 2
εt ( ε )
≥ 1 − 2 − t q˜i + 2
ቤተ መጻሕፍቲ ባይዱ
= 1 − t(q˜i + ε).
ε→0
(4.30) .
1
−
pi,i(t) t
≤
q˜i.
+
ε.
lim
t→0+
1
−
pi,i(t) t
≤
q˜i