第五章：无失真编码1(英文)

Single message（signal）source －－continuous variable source
For the continuous variable source
U a, b P pu
pu 为连续变量u的概率密度
Note：
u U R1 0,
Steady source
很多实际信源输出的消息往往是由一系列符号序列所组成的。可以把这种信源 输出的消息看做时间上或空间上离散的一系列随机变量，即为随机矢量。这时， 信源的输出可用N维随机矢量X=( X１，X２…XN)来描述，其中N可为有限正整 数或可数的无限值。这N维随机矢量X有时也称为随机序列。 一般来说，信源输出的随机序列的统计特性比较复杂，分析起来也比较困难。 为了便于分析，我们假设信源输出的是平稳的随机序列，也就是序列的统计性 质与时间的推移无关。很多实际信源也满足这个假设。 若信源输出的随机序列X= (Ｘ１，Ｘ２，…，ＸＮ )中，每个随机变量 Xi (i=1,2,…，N)都是取值离散的离散型随机变量，即每个随机变量Xi的可能取值是 有限的或可数的。而且随机矢量 X的各维概率分布都与时间起点无关，也就是 在任意两个不同时刻随机矢量 X的各维概率分布都相同。这样的信源称为离散 平稳信源。如中文自然语言文字，离散化平面灰度图像都是这种离散型平稳信 源。
Chapter 5 lossless source coding
Source info.

Content which the Info. Theory researches on the source:

Source modeling: use suitable random process to describe signals

source info.


Characteristics and classifications of information source Examples of practical info. source
Characteristics and classifications of information source

2）Main characteristics of information source

Characteristics and classifications of information source

Description and classifications of info. source

Practical source
Characteristics and classifications of information source

Single message（signal）source

ห้องสมุดไป่ตู้

It is the simplest also is the most basic source, is the composition actual letter source basic unit. It may use the dual foreword [U,P(u)] composed of the scope U of the source random variable and the correspondent probability distribution P (u) to express. Once the source is assigned, its corresponding probability space has been assigned; Otherwise, if the probability space is assigned, which indicates the corresponding source has been assigned. Therefore, the probability space can attribute the discrete source statistical property, sometimes we call this probability space the source space
U u1 0, u1 1 P p ,p 0 1
当p 0 p1

1 2
0 ,1 1 1 , 2 2
Single message（signal）source －－continuous variable source

Single message (signal) source:

Discrete source Continuous variable source

Steady source Source with/without memory Markov source Random waveform source

concerned：info. which the signals carry

Calculation of info. efficiency carried by source output signals

Entropy rate, redundancy
Source coding

Valid expression of the source output
Single message（signal）source －－discrete source

For the discrete source
U U u1 U ui U un P p p p 1 i n
例：对于二进制数据、数字信源： U={0,1} ， 则有

For some letter sources, although the output is the single signal (code) message, but the number of the possibly appear messages is the non- limiting value which cannot be counted, namely the output message mark collection A value is continual, or the value is the real number (-∞， ∞). For example, the voice signal, the thermal noise signal continual value data at some time, the obtained continual data in the external guidance system such as the related voltage, the temperature, the pressure etc.. These data values are continual, but also is stochastic. We may use the uni-dimensional continuous random variable X to describe these messages. This kind of source is called continuous source. Its mathematical model is the continuous probability space:
Single message（signal）source －－discrete source


The message number which possibly output by these sources is limited or may be counted, moreover each time only output one message. Therefore, we can use a discrete random variable X to describe this message which the source outputs. The sample space of random variable X is mark collection A; But probability distribution of X is the first-examine probability of various message appears, the probability space of the source surely is a perfect set. In the actual situation, many such sources exist. For example, throwing the coin, the correspondence writing, the computer code, the telegraphic indicator, Arab digital code and so on. These sources output a single signal (or a code), their mark collection value is limited or may be counted. We may use the uni-dimensional discrete random variable X to describe these sources outputs. Its mathematical model is the discrete probability space:

Source without memory


In some simple discrete steady source case, the signal is statistically independent. Namely in the random vector X=(X１X２…XＮ）, each random variable Xi (i=1,2,…N) is statistically independent. So the union probability distribution of the N dimension random vector satisfies P(X)=P １ （Ｘ １ ） P ２ （Ｘ ２ ） …P Ｎ （ＸＮ） We call the source X described by source space ［X， P(x) ］ the discrete non-memory source. The signals send by this kind of source are statistically independent.

Statistical characteristics of information source

1）what is info. source?

Info. Source is the origin of information, in practical communication, the common source are: voice, word, picture, data…. In the Info. Theory, info. source is the origin of message （ signal ） , message （ signal ） sequence and continuous message. In mathematics, the source is the origin to produce random variable U，random sequence U and random process U(t,ω). The basic characteristic of source is statistical uncertain. It can be described by the probability statistical characteristics.