5. LDPC

Random Linear Codes!
It can also be shown that random linear codes achieve capacity. Unlike their non-linear brethren, linear codes can be encoded in polynomial time, rather than exponential time.
Gallager’s exam源自leR=1/4Contributions of Gallager
Gallager proved some distance-properties of these codes and described a probability-based decoding algorithm with promising empirical performance. However, it appears that GL codes have been generally forgotten, the assumption perhaps being that concatenated codes were superior for practical purposes (R.G. Gallager, personal communication).
An example
r check nodes (r: number of the check bits in a codeword) n massage nodes(n:length of a codeword)
The codewords are those vectors (c1,…… ,cn) such that for all check nodes the sum of the neighboring positions among the message nodes is zero. LDPC usually is a nonsystematic code. But as all known that it is equivalent to its systematic form because the two
the LDPC code defined by the graph is the set of vectors (c1, c2, ) such that T 0 . Hc The matrix H is called a parity check matrix for the code. However, not every binary linear code has a representation by a sparse bipartite graph. If it does, then the code is called a lowdensity parity-check (LDPC) code. The sparsity of the graph structure is key property that allows for the algorithmic efficiency of LDPC
Mackay said in his paper published in 1996.
Galois field:a finite field with q elements
How to create sparse random parity check matrices
(MacKay, 1997) Construction 1A: An M x N matrix (M rows, N columns) is created at random with weight per column t (e.g. t = 3), and weight per row as uniform as possible, and overlap between any two columns no greater than 1. (The weight of a column is the number of nonzero elements; the overlap between two columns is their inner product). Construction 2A: Up to M/2 of the columns are designated weight 2 columns, and these are constructed such that there is zero overlap between any pair of columns. The remaining columns are made at random with weight 3, with the weight per row as uniform as possible, and overlap between any two columns of the entire matrix no greater than 1.
Sparsity of the graph (Decoding of LDPC uses BP algorithm) One very important aspect of belief propagation is its running time. Since the algorithm traverses the edges in the graph, so the graph is sparse the number of edges traversed is small. Moreover, if the algorithm runs for a constant number of times, then each edge is traversed a constant number of times,
Mackay’s Contributions
During MacKay’s work on MN codes he realized that it is possible to create ‘good’ codes from very sparse random matrices,
and to decode them using approximate probabilistic algorithms.
“We eventually reinvented Gallager’s decoding algorithm and GL codes. In this Letter we report the empirical performance of these codes on Gaussian channels. We have proved theoretical properties of GL codes (essentially, that the channel coding theorem holds for them) elsewhere. GL codes can also be defined over GF(q). We are currently implementing this generalization.”
Constructions 1B and 2B A small number of columns are deleted from a matrix produced by constructions 1A and 2A, respectively, so that the bipartite graph corresponding to the matrix has no short cycles of length less than some length l. The above constructions do not ensure that all the rows of the matrix are linearly independent, so the M x N matrix created is the parity matrix of a linear code with rate at least R = K/N, where K = N-M. Results are reported here on the assumption that the rate is R. The generator matrix of the code can be created by Gaussian elimination.
nK k
Good Random Linear Codes Gallager’s LDPC
LDPC codes are linear codes obtained from sparse bipartite graphs to construct and decode a reduced codebook size ‘Good code’ , with good performance and with reasonable code rate.
Shannon’s Contributions
Shannon proved the existence of codes of rates arbitrarily close to capacity for which the probability of error of the maximum likelihood decoder goes to zero as the block length of the code goes to infinity. (In fact, Shannon proved that the decoding error of the maximum likelihood decoder goes to zero exponentially fast with the block length, but we will not discuss it here.)