SymmetricRVLC

Codes Using BacktrackingHsien-Wen Tseng and Chin-Chen Chang Department of Computer Science and Information Engineering National Chung Cheng University, Chiayi 621, Taiwan, R.O.C.E-mail: {hwtseng,ccc}@.tw Correspondence address:Chin-Chen ChangProfessorDepartment of Computer Science and Information Engineering National Chung Cheng UniversityChiayi 621, Taiwan, R.O.C.E-mail: ccc@.twTEL: 886-5-2720411 ext. 33100FAX: 886-5-2720859Codes Using BacktrackingHsien-Wen Tseng and Chin-Chen ChangDepartment of Computer Science and Information Engineering,National Chung Cheng University, Chaiyi, Taiwan 621, R.O.C.E-mail: {hwtseng,ccc}@.twAbstractMany coding standards, such as JPEG, H.261, H.263, MPEG-1, MPEG-2, use variable length codes (VLC) as their entropy coding strategy. However, VLC have a big drawback when transmitting over a noisy channel. This drawback is an error propagation problem. For this reason, reversible variable length codes (RVLC) have been used to enhance the error resilient capabilities of VLC. This paper presents a new algorithm using backtracking that can construct symmetrical RVLC. Depth first node generation is applied to this algorithm and bounding function is used to replace nodes with their symmetrical children. The experimental results show that our algorithm can generate better codes than those of previous methods. In addition, our proposed algorithm provides a shorter maximum code length. The shorter maximum code length can usually achieve more efficient decoding.Keywords: reversible variable length codes (RLVC), Huffman Code, H.263, MPEG-4, JPEG-2000, error resilience1. IntroductionTraditionally, variable length codes (VLC) have been used as entropy coding in many image coding standards (JPEG [1]) and video coding standards (H.261 [2], H.263 [3], MPEG-1 [4], MPEG-2 [5]). An example of VLC is the Huffman Code, which is well known to give the optimal code with minimum redundancy. But in recent years, more and more new standards such as JPEG-2000 [6], H.263+ [7], H.263++ [8], MPEG-4 [9] have adopted reversible variable length codes (RVLC), because VLC have the problem of error propagation. Even one single bit error will cause many following codewords to be misinterpreted. This is a big problem in an error-prone environment. In order to enhance the error resilient capabilities of VLC, Fraenkel and Klein [10] presented the bidirectional Huffman coding in 1990, Takishima et al.[11] proposed the RVLC for avoiding continuous errors in 1995. RVLC are not only a prefix code but also a suffix code. A code is called a prefix code namely if no codeword is a prefix of any other codewords. Conversely, a code is called a suffix code namely if no codeword is a suffix of any other codewords. Therefore, RVLC can be decoded both in the forward and backward directions so as to provide error resilient transmission over a noisy channel.RVLC are very useful because they provide the capability of error resiliency and because they can be decoded in two directions. Except for the adoption in manystandards as we mentioned above, RVLC can also be applied to speed up searching of encoded data. For example, we can begin by searching the encoded data in the forward and backward directions at the same time. This can significantly reduce the search time and this kind of search is impossible when using VLC.There are two types of RVLC, one is symmetrical and the other is asymmetrical. Symmetrical RVLC share the same code table when decoding both in the forward and backward directions, because the code is symmetrical. But two types of code tables are necessary for asymmetrical RVLC. For this reason, symmetrical RVLC is simpler than asymmetrical RVLC; meanwhile, the memory requirement of symmetrical RVLC is less than that of asymmetrical RVLC. However, asymmetrical RVLC always provides better efficiency than symmetrical RVLC because a more flexible code selection is allowed.In this paper, we will concentrate on symmetrical RVLC and devise a new method of constructing symmetrical RVLC. The remainder of this paper is organized as follows. Section 2 introduces the related works. Section 3 explains the details of our proposed algorithm. Experimental results are presented in Section 4. The conclusions are drawn in Section 5.2. Previous WorksIn 1995, Takishima et al. [11] proposed a coding scheme to generate RVLC. It starts from a non-reversible VLC, such as Huffman Code, and converts the code by a top-down scheme into a symmetrical RVLC. Recently, Tsai and Wu [12, 13] proposed a more efficient symmetrical RVLC construction algorithm that is based on Takishima et al.’s algorithm. This algorithm also starts from a list of given Huffman Code, but uses a new codeword selection mechanism to select candidate codewords in each level. We first explain some terms in relation to their construction algorithms, and then briefly summarize their algorithms.The number of symmetrical codewords on a full binary tree of level L is given as follows:m o(L) = 2└(L+1)/2┘,where └x┘is the largest integer less than or equal to x. Let p(L) denote the total number of symmetrical codewords at level L unavailable due to the violation of the prefix condition. The calculation of p(L) can be found in [11]. Hence, the number of available symmetrical codewords, m(L), at level L is calculated as follows:m(L) = m o(L) - p(L).If we ignore the codeword selection mechanism in both Takishima et al.’s algorithm and Tsai and Wu’s algorithms, then their algorithms can be summarized asfollows:------------------------------------------------------------------------------------------------------- (i) Initialize the bit length vector of the target symmetrical RVLCn rev(i) = n(i),where n(i) is the bit length vector of a list of given Huffman Code.(ii) Count m(i), which is the total number of available symmetrical codewords of a full binary tree at level i.If n rev(i) > m(i){n rev(i+1) = n rev(i+1) + n rev(i) - m(i),n rev(i) = m(i),}Assign the first n rev(i) candidate codewords as target codewords.(iii) Repeat Step (ii) until the number of target codewords is equal to the number of codewords.------------------------------------------------------------------------------------------------------- The bit length vector is denoted as (n(1), n(2), ……, n(L max)), where n(i) is the number of codewords having length i and L max is the maximum code length. We observe a common phenomenon in their algorithm, which is∑=1 irev(i)n<= ∑=1in(i).It means that the total number of symmetrical RVLC of length less than or equal to i never exceeds the total number of Huffman Code of length less than or equal to i. As a result, the bit length vector of Huffman Code dominates the bit length vector of symmetrical RVLC. The Huffman Code is well known to give the optimal code with minimum redundancy. But this is not always true when applying it to the RVLC domain. For example, in Table 1, the average code length of symmetrical RVLC (C14) is shorter than C12 and C13, where C11 is a list of Huffman Code, C12 is a symmetrical RVLC generated by Takishima et al.’s algorithm, and C13 is a symmetrical RVLC generated by Tsai and Wu’s algorithm.Table 1. Huffman Code and RVLCsSymbol Probability C11 C12 C13 C14A 0.33 00 00 00 1B 0.30 01 11 11 00C 0.18 11 010 010 010D 0.10 100 101 101 0110E 0.09 101 0110 0110 01110Average code length 2.19 2.46 2.46 2.32Table 1, reveals that the symmetrical RVLC generated from a given Huffman Code may not be good enough. This is because more constraints are bundled with symmetrical RVLC, such as the prefix condition, the suffix condition and symmetry. Meanwhile, the number of available symmetrical codewords at each level is limited tothe bit length vector of a given list of Huffman Code. These result in the waste of symmetrical codewords at some level. For example, in Table 1, the symmetrical codewords with length 1 (‘0’ and ‘1’) are not available in Takishima et al.’s algorithm and Tsai and Wu’s algorithm. But they produce a shorter average code length when using them in the symmetrical RVLC C14.Besides, the affix codes (a code having both prefix and suffix properties) have been extensively studied in Fraenkel and Klein [10]. They mentioned that searching for affix codes should not be restricted to the set of Huffman codes. Because there are optimum codes which cannot be obtained via the Huffman algorithm. For example, given the source weight distribution (7, 3, 3, 3, 3, 1, 1, 1, 1), the only bit length vector minimizing the average code length is (2, 3, 3, 3, 3, 4, 4, 4, 4), but none of the Huffman Codes with this bit length vector has the affix property.Hence we will introduce a novel symmetrical RVLC construction algorithm that is not based on a given Huffman Code. The backtracking technique is employed in our proposed algorithm. For certain sources, the number of prefix codes can be large enough to make an exhaustive search for an affix code forbidden, the backtracking method has as its virtue the ability to yield the good result with far fewer searches. Its basic idea is to build up the solution vector one by one and to use the bounding function to test whether the vector being formed has any chance of success. In manycases, a backtracking algorithm generally has unfavorable performance. This is not always the case, because a good method and data structure makes the algorithm efficiently. Our proposed algorithm is obviously better than Tsai and Wu’s algorithm in respect of the processing time to construct the symmetrical RVLC. Further, the average code length is also improved. This will be shown in our experimental results.3. Proposed AlgorithmOur algorithm employs the technique of backtracking. Many problems which deal with searching for a set of solutions can be solved using the backtracking formulation. We assume that symbol a i occurs in the data with probability Pi. The desired solution can be expressed as an n-tuple (C1, C2, …, C n), where C i is the codeword of symbol a i and n is the number of symbols. Our algorithm calls for finding one vector that minimizes a bounding function B(C1, C2, …, C n). The bounding function is defined as,B(C1, C2, …, C n) = ∑n PiLi1where Pi is the occurrence probability of symbol a i and Li is the length of codeword C i .That is to say, the bounding function is the calculation of average code length.Before explaining the details of the proposed algorithm, we define some terms employed in our algorithm.Symmetrical ChildrenIn a full binary tree, the symmetrical children of a node X are defined by all of the first symmetrical codewords on paths from node X to leaf nodes. For example, in Figure 1, the black nodes are symmetrical codewords, the symmetrical children of A(‘0’) are C(‘00’), F(‘010’), and J(‘0110’), and the symmetrical child of D(‘11’) isH(‘111’).Fig. 1 A sample treeReplacementGiven a source probability distribution P = (P1, P2, …, P n) in decreasing order. Our method starts with P and generates the symmetrical RVLC level by level. Suppose we have selected m available symmetrical codewords from Level 1 to Level l and they are represented by a target list tlist = (c1, c2, …, c m) in increasing order of, where code length. Then the average code length of these codes is tavg = ∑m PiLi1Li is the length of codeword c i. The replacement test is performed on each component in tlist one by one. Assume that codeword c i has k symmetrical children (c i1, c i2, …, c ik). If we replace the codeword c i with its symmetrical children, we will obtain a new list (c1, c2, …, c i1, c i2, …, c ik, …, c m). The new list is then rearranged in increasing order of code length and we will obtain a list nlist= (c'1, c'2, …, c'm+k-1). The probability distribution P is remapped to nlist. The average code length of the first m codes innlist is navg =∑mi 'PiL 1, where L 'i is the length of codeword c 'i . Now we make acomparison, if tavg > navg then the replacement is carried out and the target list tlist is replaced with the new list nlist . In a nutshell, we use the bounding function to replace nodes with their symmetrical children, and the bounding function is the calculation of average code length.In summary, our proposed symmetrical RVLC algorithm is specified as follows: ------------------------------------------------------------------------------------------------------- Step 1. Assign the codeword ‘1’ to the target list. Step 2. For each symmetrical codeword at level i (i > 1)Assign the available symmetrical codewords to the target list based on the increasing order of code length.Step 3. If any code in the target list satisfies the condition of replacement , replace thiscode with its symmetrical children .Step 4. Repeat Step 2 and Step 3 until the size of the target list is greater than or equalto the number of codewords.------------------------------------------------------------------------------------------------------- We are going to illustrate this by an example. Five symbols with probabilities (0.26, 0.24, 0.18, 0.17, 0.15) are given. Table 2 shows the process of generatingsymmetrical RVLC, level by level. There is no replacement carried out from Level 1 to Level 4, because they cannot minimize the bounding function B. In Level 5, the only available symmetrical codeword is ‘01110’, so we assign codeword ‘01110’ to the target list. Now the target list = (1, 00, 010, 0110, 01110), and the bounding function B is 0.26 * 1 + 0.24 * 2 + 0.18 * 3 + 0.17 * 4 + 0.15 * 5 = 2.71. Then we check the replacement of the target list. If we replace the code ‘1’ with its symmetrical children ‘11’ and ‘101’, the bounding function B will gain 0.26 * 2 + 0.24 * 2 + 0.18 * 3 + 0.17 * 3 + 0.15 * 4 = 2.65. It satisfies the condition of replacement, so the replacement takes place. Finally, there are no other codewords in the target list satisfying the condition of replacement, and the size of the target list is greater than the number of codewords. Therefore, our algorithm ends. This results in the codes ‘00’, ‘11’, ‘010’, ‘101’, ‘0110’, and the average code length is 2.65.Table 2. Sample of proposed algorithmLevelTarget List(Before replacement)Target List(After replacement)AverageCode Length1 (1) (1) 0.262 (1, 00) (1, 00) 0.743 (1, 00, 010) (1, 00, 010) 1.284 (1, 00, 010, 0110) (1, 00, 010, 0110) 1.965 (1, 00, 010, 0110, 01110) (00, 11, 010, 101, 0110, 01110) 2.654. Experimental ResultsOur experiments are done by C++ Builder run on a notebook computer with Mobil Pentium III 600MHz CPU and 128MB RAM. The proposed algorithm has been tested on the English alphabet and file set from Canterbury Corpus (/). The Canterbury Corpus file set was developed specifically for testing new compression algorithms. The files were selected based on their ability to provide representative performance results. Meanwhile, as for the compression efficiency, the average code length is employed. It is defined as,Average Code Length = ∑n PiLi1where n is the number of symbols, Pi is the occurrence probability of symbol a i and Li is code length of symbol a i. The average code length is the number of bits needed, on average, to represent a symbol.From Table 3, it can be seen that, to the average code length, our proposed algorithm outperforms Tsai and Wu’s algorithm. It is superior to Tsai and Wu’s algorithm by 0.14264718. This is the number of bits saved, on average, to represent a symbol. It seems small, but it becomes significant when the input is large enough.Table 3. Comparison of symmetrical RVLC constructed from Tsai & Wu’s algorithmand our proposed algorithmTable 4 shows the results tested on the Canterbury Corpus file set. It can be seen that our proposed algorithm provides smaller average code length than Tsai and Wu’s algorithm. Moreover, the time needed to generate symmetrical RVLC is provided in Table 4. For the sake of equitableness, we assume that a source probabilitydistribution is given. The algorithm proposed by Tsai and Wu should construct a Huffman code first, and then begins from the Huffman code. In contrast to their work, our proposed algorithm doesn’t start the construction with a given Huffman code. We begin from a given source probability distribution, and construct the symmetrical RVLC one by one. Table 4 shows the time needed to generate the codes by our scheme is shorter than that of Tsai and Wu’s algorithm.Furthermore, our proposed algorithm, like Tsai and Wu’s algorithm, can avoid the codeword variation problem compared with Takishima et al.’s algorithm [11]. The codeword variation problem resulting from the Huffman Code is not unique when given the same source. For example, we can convert a Huffman Code to another with the same average code length by exchanging a node or a leaf at the same level. This conversion is called equivalent transform [11]. A different Huffman Code will generate a different symmetrical RVLC in Takishima et al.’s algorithm and the codeword variation problem occurs. Our proposed algorithm generates unique symmetrical RVLC when given the same source probability distribution, hence the codeword variation problem never occurs.Table 4. Comparison of the average code lengthIn Table 5, it is shown that for the most part our proposed algorithm always provides a shorter maximum code length than Tsai and Wu’s algorithm. Traditionally, a Huffman Code has a memory requirement of O(2h) and a computational complexity of O(h), where h is the height of the Huffman tree; it is also the maximum code length [14]. For this reason, the shorter maximum code length usually can achieve more efficient decoding.Table 5. Comparison of the maximum code length and the bit length vectorFile Codeword Mode Max. Code LengthBit Length VectorHuffman Code15 (0,0,1,6,8,7,11,8,5,11,5,1,0,3,2) Tsai and Wu’s Sym. RVLC 15 (0,0,1,4,5,5,9,8,14,11,5,1,0,3,2) asyoulik.txt (68 codewords) Our Sym. RVLC 12 (0,0,3,2,5,5,7,6,10,9,16,5)Huffman Code16 (0,1,0,4,9,8,6,5,4,13,12,3,0,1,4,4) Tsai and Wu’s Sym. RVLC 16 (0,1,0,3,5,5,7,8,8,12,13,3,0,1,4,4) alice29.txt (74 codewords) Our Sym. RVLC 12 (0,0,3,2,5,5,7,6,10,9,16,11) Huffman Code12 (0,0,1,6,9,6,8,9,6,8,11,10) Tsai and Wu’s Sym. RVLC 12 (0,0,1,4,5,5,9,8,13,8,11,10) xargs.1 (74 codewords) Our Sym. RVLC 12 (0,0,3,2,5,5,7,6,10,9,16,11) Huffman Code12 (0,1,0,4,8,8,8,8,5,15,9,10) Tsai and Wu’s Sym. RVLC 12 (0,1,0,3,5,5,7,8,11,12,14,10) grammar.lsp (76 codewords) Our Sym. RVLC 12 (0,1,1,2,5,5,6,7,9,10,15,15)Huffman Code19 (0,0,2,7,3,9,4,4,7,14,5,2,6,1,1,7,4,3,2) Tsai and Wu’s Sym. RVLC 19 (0,0,2,4,4,4,8,6,8,10,9,2,6,1,1,7,4,3,2) plrabn12.txt (81 codewords) Our Sym. RVLC 14 (0,0,4,2,4,4,6,4,8,6,12,8,18,5)Huffman Code16 (0,0,1,8,4,9,5,7,12,13,10,6,2,3,0,4) Tsai and Wu’s Sym. RVLC 16 (0,0,1,4,5,5,9,8,14,13,10,6,2,3,0,4) lcet10.txt (84 codewords) Our Sym. RVLC 13 (0,0,3,3,4,4,7,5,10,8,15,11,14) Huffman Code14 (0,0,0,6,12,8,5,11,13,13,6,5,5,2) Tsai and Wu’s Sym. RVLC 14 (0,0,0,4,6,6,10,10,16,16,6,5,5,2) cp.html (86 codewords) Our Sym. RVLC 12 (0,0,2,2,6,6,8,8,12,12,20,10) Huffman Code13 (0,1,0,3,7,11,7,17,16,17,3,4,4) Tsai and Wu’s Sym. RVLC 13 (0,1,0,3,5,5,7,8,11,12,18,16,4) fields.c (90 codewords) Our Sym. RVLC 13 (0,0,2,2,6,6,8,8,12,12,20,14)Huffman Code17 (1,0,0,1,2,14,11,9,9,7,10,9,13,20,16,11,26) Tsai and Wu’s Sym. RVLC 17 (1,0,0,1,2,4,6,5,9,10,17,17,14,20,16,11,26) ptt5(159 codewords) Our Sym. RVLC 16 (1,0,0,2,3,3,5,5,8,8,13,12,22,23,41,13) Huffman Code14 (0,1,0,0,3,20,16,24,25,45,30,56,29,6) Tsai and Wu’s Sym. RVLC 15 (0,1,0,0,3,6,11,10,16,17,31,31,56,54,19) sum(255 codewords) Our Sym. RVLC 15 (0,1,0,1,5,5,9,10,15,14,26,24,44,43,58) Huffman Code12 (1,0,1,3,1,0,0,1,1,74,146,28)Tsai and Wu’s Sym. RVLC 17 (1,0,1,2,2,0,0,1,1,6,12,12,24,23,46,45,80) kennedy.xls (256 codewords)Our Sym. RVLC17(1,0,1,0,3,1,1,4,8,8,17,16,31,29,59,56,21)5. ConclusionsIn this paper, an efficient symmetrical RVLC construction algorithm has been proposed. This algorithm employs the technique of backtracking. Depth first node generation is applied to this algorithm. Bounding function is used to replace nodes with their symmetrical children. The major contribution of the proposed algorithm is that it provides smaller average code length than does Tsai and Wu’s algorithm. Meanwhile, our proposed algorithm has no codeword variation problem.According to our experimental results, it is seen that our proposed algorithm can generate efficient symmetrical RVLC. Moreover, for the most part, our symmetrical RVLC provides a shorter maximum code length than Tsai and Wu’s symmetrical RVLC. The shorter maximum code length can usually achieve more efficient decoding. Furthermore, improvement of average code length may be possible. The design of more efficient or shorter RVLC will be the topic of our future research.REFERENCES1. Pennebaker, W. and Mitchell, J. (1993) JPEG Still Image Data CompressionStandard. New York: Van Nostrand Reinhold.2. ITU-T Recommendation H.261 (1993) Video Codec for Audiovisual Services atp × 64 kb/s.3. ITU-T Recommendation H.263 (1995) Video Coding for Low Bit RateCommunication.4. ISO/IEC IS 11 172(MPEG-1) (1991) Coding of Moving Pictures and AssociatedAudio for Digital Storage Media at up to About 1.5 Mbits/s.5. ISO/IEC DIS 13 818(MPEG-2) (1994) Generic Coding of Moving Pictures andAssociated Audio.6. ISO/IEC/JTC1/SC29/WG1 (2000) JPEG 2000 Image Coding System, FCDVersion 1.0.7. Wen, J. and Villasenor, J. (1997) A Class of Reversible Variance Length Codesfor Robust Image and Video Coding. Proc. IEEE Int. Conf. Image Proc., 2, 65-68.8. Wen, J. and Villasenor, J. (1998) Reversible Variable Length Codes for Efficientand Robust Image and Video Coding. Proc. IEEE Data Compression Conference, 471-480.9. ISO/IEC 14 496-2 (1998) Final Draft International Standard: InformationTechnology －Coding of Audio-visual Objects: Visual.10. Fraenkel, A. S. and Klein, S. T. (1990) Bidirectional Huffman Coding. TheComputer Journal, 33 , no. 4.11. Takishima, Y.; Wada, M., and Murakami, H. (1995) Reversible Variable LengthCodes. IEEE Trans. on Commun., 43, 158-162.12. Tsai, C.-W. and Wu, J.-L. (2001) Modified Symmetrical ReversibleVariable-Length Code and Its Theoretical Bounds. IEEE Trans. on Information Theory, 47, 2543-2548.13. Tsai, C.-W. and Wu, J.-L. (2001) On Constructing the Huffman-Code-BasedReversible Variable-Length Codes. IEEE Trans. on Commun., 49, 1506-1509. 14. Huffman, D. A. (1952) A Method for the Construction of Minimum RedundancyCodes. Proceedings of IRE, 40, 1098-1101.20。