关于微分熵的一类强偏差定理

J.Sys.Sci.&Math.Scis.26(6)(2006,12),693–700(300160)(300161)LaplaceLaplaceMR(2000)60F151{X n,n≥1}(Ω,F,P)g n(x1,x2,...,x n),x k≥0,1≤k≤n,n=1,2,. (1)g n>0.f n(ω)=−1nln g n(X1,X2,···,X n),(2)ωX k X k(ω),f n(ω){X k,1≤k≤n}Q F(X1,X2,···,X n)Qq n(x1,x2,···,x n)>0,x k≥0,1≤k≤n.L n(ω)=ln gn(X1,X2,···,X n)q n(X1,X2,···,X n);(3)L(ω)=lim supn→∞1nL n(ω)=−lim infn→∞1nlnqn(X1,X2,···,X n)g n(X1,X2,···,X n);(4)D(g n||q n)=EL n=E ln gn(X1,X2,···,X n)q n(X1,X2,···,X n).(5)2004-04-14,2005-06-28.69426L n(ω),L(ω),D(g n||q n)q n(x1,x2,···,x n)(n≥1){X k,1≤k≤n}g nq n[1].Shannon[2][2−7].Shannon-McMillan().Shannon-McMillan[8]Laplacelim sup n→∞1nlnqn(X1,X2,···,X n)g n(X1,X2,···,X n)≤0a.s.(6)Z n=q n(X1,X2,···,X n)g n(X1,X2,···,X n)Z n a.s.Z∞(0≤Z∞<∞).(6)(6)L(ω)≥0a.s.(7) {X n,n≥1}(1)f k(x k)X kπn(x1,x2,···,x n)=nk=1f k(x k)(8)φn(ω)φ(ω)πnφn(ω)=ln gn(X1,X2,···,X n)πn(X1,X2,···,X n);(9)φ(ω)=lim supn→∞1nφn(ω)=−lim infn→∞1nlnπn(X1,X2,···,X n)g n(X1,X2,···,X n).(10)(7)φ(ω)≥0a.s.(11)φ(ω)πn(x1,x2,···,x n){X n}g n(x1,x2,···,x n)φ(ω)−ln f k(x k)Laplacefk(s)=E e −s(−ln f k(X k))=∞e−s(−ln f k(x k))f k(x k)d x k.(12)6695 21{X n,n≥1}φ(ω) f k(s)D={ω:φ(ω)<+∞}. f k(s) (−2s0,2s0)lim sup n→∞1nnk=1fk(−2s0)=M(s0)<+∞.(13)lim inf n→∞1nnk=1[−ln f k(X k)−h(X k)]≥α(φ(ω)),a.s. D.(14)lim sup n→∞1nnk=1[−ln f k(X k)−h(X k)]≤β(φ(ω)),a.s. D.(15)h(X k)=E[−ln f k(X k)]=− ∞f k(x k)ln f k(x k)d x k,X kα(x)=sup{g(s,x),0<s<s0},x≥0,(16)β(x)=inf{−g(−s,x),−s0<s<0},x≥0,(17)g(s,x)=−2e−2s(2s0−s)2M(s0)−xs,x≥0,0<s<s0.(18)0=α(0)≥α(x)≥−2e−2√x1+√xs02−√x1+√x2M(s0)−√x(1+√x)s0.(19)(16),(17)α(x)=−β(x).f(s,x k)=e−s(−ln f k(x k))f k(x k)fk(s),(20)∞f(s,x k)d x k=1.(21)f n(s;x1,···,x n)=nk=1f(s,x k)=nk=1e−s(−ln f k(x k))f k(x k)fk(s).(22)(21)f n(s;x1,···,x n)nt n(s,ω)=f n(s;X1,X2,···,X n)g n(X1,X2,···,X n).(23)A(s)∈F,P(A(s))=1lim sup n→∞1nln t n(s,ω)≤0,ω∈A(s).(24)69626(22)–(24),lim sup n →∞1n−n k =1s (−ln f k (X k ))−n k =1ln f k (s )−φn (ω) ≤0,ω∈A (s ).(25)s =0,lim infn →∞1nφn (ω)≥0,ω∈A (0).(26)φ(ω)≥0,ω∈A (0).(27)0<s <s 0(25)lim sup n →∞(a n −b n )≤d ⇒lim sup n →∞(a n −c n )≤lim sup n →∞(b n −c n )+d.(28)1−1x≤ln x ≤x −1(x >0),−s lim inf n →∞1nnk =1[−ln f k (X k )−h (X k )]≤lim sup n →∞1n n k =1[ln f k (s )+s h (X k )]+φ(ω)≤lim sup n →∞1n nk =1[f k (s )−1+s h (X k )]+φ(ω)=lim supn →∞1nnk =1E [e −s (−ln f k (X k ))−1(−s )(−ln f k (X k ))]+φ(ω),ω∈A (s ).(29)0≤e x −1−x ≤12x 2e |x |,0≤e −s (−ln f k (x k ))−1−(−s )(−ln f k (x k ))≤12(s ln f k (x k ))2e |s ln f k (x k )|.(30)(29),(30),−s lim inf n →∞1nnk =1[−ln f k (X k )−h (X k )]≤12s 2lim supn →∞1nn k =1E (ln f k (X k ))2e s (−ln f k (X k ))+φ(ω),ω∈A (s ).(31)g (x )=t x x 2(0<t <1)x =−2ln t[0,+∞)g (−2ln t )=4e −2(ln t )2,sup (e −s )−ln f k (x k )[ln f k (x k )]2,k ≥1 ≤4e −2s2,s >0.(32)sup (e s )−ln f k (x k )[ln f k (x k )]2,k ≥1 ≤4e −2s2,s <0.(33)6697(31),(32)lim inf n →∞1nnk =1[−ln f k (X k )−h (X k )]≥−s 2lim sup n →∞1n n k =1∞0(ln f k (x k ))2e (−2s 0+s )(−ln f k (x k ))e 2s 0(−ln f k (x k ))f k (x k )d x k −φ(ω)s ≥−2e −2s (2s 0−s )2M (s 0)−φ(ω)s=g (s,φ(ω)),0<s <s 0,ω∈A (s )DA (0).(34)Q ∗(0,s 0)A ∗=s ∈Q ∗A (s ),P (A ∗)=1.(34),lim inf n →∞1nnk =1[−ln f k (X k )−h (X k )]≥g (s,φ(ω)),ω∈A ∗ D A (0),∀s ∈Q ∗.(35)x =∞g (s,x )ss >0g (s,∞)=−∞.(16),(27)ω∈A ∗ DA (0),s n (ω)∈Q ∗,n =1,2,···lim n →∞g (s n (ω),φ(ω))=α(φ(ω)).(36)(35),(36),lim inf n →∞1nnk =1[−ln f k (X k )−h (X k )]≥α(φ(ω)),ω∈A ∗DA (0).(37)P (A ∗A (0))=1,(37)(14).−s 0<s <0(25),(28),(33),−s lim sup n →∞1n nk =1[−ln f k (X k )−h (X k )]≤lim sup n →∞1n n k =1[ln f k (s )+s h (X k )]+φ(ω)≤12s 2lim sup n →∞1n nk =1E [(ln f k (x k ))2e s (ln f k (x k ))]+φ(ω)≤s 22lim sup n →∞1nn k =1 ∞0(ln f k (x k ))2e (−2s 0−s )(−ln f k (x k ))e 2s 0(−ln f k (x k ))f k (x k )d x k +φ(ω)≤2e −2s 2(2s 0+s )2M (s 0)+φ(ω),ω∈A (s ).(38)−s ,lim sup n →∞1nnk =1[−ln f k (X k )−h (X k )]≤−g (−s,φ(ω)),ω∈A (s )DA (0).(39)69826Q ∗(−s 0,0)A ∗=s ∈Q ∗A (s ),P (A ∗)=1.(39),lim sup n →∞1nnk =1[−ln f k (X k )−h (X k )]≤−g (−s,φ(ω)),ω∈A ∗ D A (0),∀s ∈Q ∗.(40)g (s,x )ω∈A ∗DA (0),τn (ω)∈Q ∗,n =1,2,···,lim n →∞−g (−τn (ω),φ(ω))=β(φ(ω)).(41)(40),(41),lim sup n →∞1nnk =1[−ln f k (X k )−h (X k )]≤β(φ(ω)),ω∈A ∗DA (0).(42)P (A ∗A (0))=1,(42)(15).(19).(16)(18)x >00≥α(x )≥gs 0√x1+√x,x =−2e −2√x 1+√xs 0 2−√x 1+√x2M (s 0)−√x (1+√x )s 0,(43)α(0)=lim s →0+−2e −2s(2s 0−s )2M (s 0)=0,(44)(19)12f n (ω)(2)1lim inf n →∞ f n (ω)−1nh (X 1,X 2,···,X n ) ≥α(φ(ω))−φ(ω)+H ∗,a .s .D ;(45)lim sup n →∞f n (ω)−1n h (X 1,X 2,···,X n ) ≤β(φ(ω))+H ∗,a .s .D,(46)h (X 1,X 2,···,X n )=E [−ln g n (X 1,X 2,···,X n )](X 1,X 2,···,X n )H ∗=lim inf n →∞1nn k =1h (X k )−h (X 1,X 2,···,X n )=lim inf n →∞1n D (g n ||πn );(47)H ∗=lim supn →∞1n nk =1h (X k )−h (X 1,X 2,···,X n ) =lim sup n →∞1n D (g n ||πn ).(48)(2),(11),(37)(47),lim inf n →∞ f n (ω)−1nh (X 1,X 2,···,X n )6699≥lim inf n →∞1n ln nk =1f k (X k )g n (X 1,X 2,···,X n )+lim inf n →∞1n nk =1[−ln f k (X k )−h (X k )]+lim inf n →∞1nn k =1h (X k )−h (X 1,X 2,···,X n )≥−φ(ω)+α(φ(ω))+H ∗,ω∈A ∗DA (1).(49)(49)(45).(2),(39),(42)(46),lim sup n →∞f n (ω)−1n h (X 1,X 2,···,X n )≤lim sup n →∞1n ln n k =1f k (X k )g (X 1,X 2,···,X n ) +lim supn →∞1n nk =1[−ln f k (X k )−h (X k )]+lim sup n →∞1n n k =1h (X k )−h (X 1,X 2,···,X n ) ≤β(φ(ω))+H ∗,ω∈A ∗DA (1).(50)(50)(46).21{X n ,n ≥1}12Z ∞(ω)>0,a.s.,12{X n ,n ≥1}g n (x 1,x 2,···,x n )≤πn (x 1,x 2,···,x n ),∀(x 1,x 2,···,x n )∈R n ,n ∈N ,Z n (ω)≥1→Z ∞(ω)≥1→φ(ω)≡0.122{X n ,n ≥1}Z ∞(ω)=00<φ(ω)<+∞.12{ξn ,n ≥1}(Ω,F ,P )ξn f (x )= e −x,x >0,0,x ≤0,,ξn {X n ,n ≥1}X n =(ξ1+ξ2+···+ξn )n,{X n ,n ≥1}1231{X n ,n ≥1}s 0>0(13)lim n →∞ f n (ω)−1nh (X 1,X 2,···,X n ) =0,a .s .(51)H ∗=H ∗=0,φ(ω)≡0,D =Ω.α(0)=β(0)=0,(51)(45)(46)3Shannon-McMillan22lim inf n →∞f n (ω)−1nh (X 1,X 2,···,X n ) ≥α(φ(ω))−φ(ω)+h ∗,a .s .D ;(52)70026lim sup n→∞f n(ω)−1nh(X1,X2,···,X n)≤β(φ(ω))+h∗,a.s.D,(53)h∗=lim infn→∞[h(X n)−h(X n|X1,X2,···,X n−1)];(54) h∗=lim supn→∞[h(X n)−h(X n|X1,X2,···,X n−1)].(55)[1]CesaroH∗=lim infn→∞1nnk=2[h(X k)−h(X k|X1,X2,···,X k−1)]≥lim infn→∞[h(X n)−h(X n|X1,X2,···,X n−1)]=h∗.(56) (52)(45)(56)(53).[1]Cover T M and Thomas J A.Elements of Information Theory.Wiley,New York,1991.[2]Shannon C.A mathematical theory of communication.Bell System Tech.,1948,27:327–423,623–656.[3]McMillan B.The basic theorems of information theory.Ann.Math.Statist.,1953,24:196–216.[4]Chung K L.The ergodic theorem of information theory.Ann.Math.Statist.,1961,32:612–614.[5]Breiman L.The individual ergodic theorem of information theory.Ann.Math.Statist.,1975,28:809–811.[6]Shannon-McMillan1994,14(3):337–345.[7]Liu Wen and Yang Weiguo.An extension of Shannon-McMillan Theorem and some limit propertiesfor nonhomogeneous Markov chains.Stochastic Process.Appl.,1996,61:129–145.[8]Liu Wen.Relative entropy densities and a class of limit theorems of the sequence of m-valuedrandom variables.Ann.Probab.,1990,18(2):829–839.A CLASS OF STRONG DEVIATION THEOREMS FORDIFFERENTIAL ENTROPYZhao Jing(School of Science,Tianjin Polytechnic University,Tianjin300160)Wei Jie(Department of Basic Sciences,Military Traffic Institute,Tianjin300161) Abstract In this paper a class of strong deviation theorems of continuous random vari-ables on the sample entropy is established by using the notion of sample relative entropy rate and relative entropy with respect to reference product distributions.In the proof,an approach of applying the Laplace transform to the investigation of the strong deviation theorems on differential entropy is proposed.Key words Differential entropy,strong deviation theorem,relative entropy rate,Laplace transform.。