信息论第二次作业

3.5 AEP. Let ,,21X X be independent identically distributed random variables
drawn
according
to
the
probability mass
function {}m x x p ,2,1),(∈. Thus ∏==n
i i n x p x x x p 1
21)(),,,( . We know that
)(),,,(log 1
21X H X X X p n n →- in probability. Let ∏==n
i i n x q x x x q 1
21)(),,,( , where q is another probability mass function on {
}m ,2,1. (a) Evaluate ),,,(log 1
lim 21n X X X q n
-, where ,,21X X are i.i.d. ~ )(x p .
8.1 Preprocessing the output. One is given a communication channel with transition probabilities )|(x y p and channel capacity );(max )(Y X I C x p =. A helpful statistician preprocesses the output by forming )(_
Y g Y =. He claims that this will strictly improve the capacity. (a) Show that he is wrong.
(b) Under what condition does he not strictly decrease the capacity?
8.3 An addition noise channel. Find the channel capacity of the following discrete memoryless channel:
Where {}{}2
1Pr 0Pr ====a Z Z . The alphabet for x is {}1,0=X . Assume that Z is independent of X . Observe that the channel capacity depends on the value of a .
8.5 Channel capacity. Consider the discrete memoryless channel
)11 (mod Z X Y +=, where ⎪⎪⎭⎫
⎝⎛=1/3 1/3, 1/3,3 2, ,1Z and {}10,,1,0 ∈X . Assume that
Z is independent of X .
(a) Find the capacity.
(b) What is the maximizing )(*x p ?
8.12 Time-varying channels. Consider a time-varying discrete memoryless channel. Let n Y Y Y ,,21 be conditionally independent given
n X X X ,,21 , with conditional distribution given by ∏==n
i i i i x y p x y p 1)|()|(.
Let ),,(21n X X X X =, ),,(21n Y Y Y Y =. Find );(max )(Y X I x p .
9.1 Differential entropy. Evaluate the differential entropy ()h ln X f f =-⎰ for the following: (a) (),0x f x e x λλ-=≥. (b) ()1
2
x f x e λλ-=
(c) The sum of X1 and X2, where they are independent normal random variables with means 1μ and variance 2t t=σ，1,2.
10.2 A channel with two independent looks at Y . Let 1Y and 2Y be conditionally independent and conditionally identically distributed given
X .
(a) Show );();(2),;(21121Y Y I Y X I Y Y X I -=. (b) Conclude that the capacity of the channel
X
(Y1,Y2)
is less than twice the capacity of the channel
X
Y1
10.3 The two-look Gaussian channel. Consider the ordinary Shannon Gaussian channel with two correlated looks at X , i.e., ),(21Y Y Y =, where
2
211Z X Y Z X Y +=+= with a power constraint P on X , and ),0(~),(221K N Z Z ,
where
⎥⎦
⎤⎢⎣⎡=N N N N K ρρ. Find the capacity C for (a) 1=ρ (b) 0=ρ (c) 1-=ρ
10.4 Parallel channels and waterfilling. Consider a pair of parallel Gaussian
channels,
i.e.,
⎪⎪⎭
⎫
⎝⎛+⎪⎪⎭⎫ ⎝⎛=⎪⎪⎭⎫ ⎝⎛212121Z Z X X Y Y , where
⎪⎪⎭⎫ ⎝⎛⎥⎥⎦⎤⎢
⎢⎣⎡⎪⎪⎭⎫ ⎝⎛222121 00 ,0~σσN Z Z ,And there is a power constraint P X X E 2)(2221≤+.
Assume that 2221σσ>. At what power does the channel stop behaving like a single channel with noise variance 22σ, and begin behaving like a pair of channels?。