Curved wavelet transform for image coding

Curved wavelet transform for image codingDemin Wang, Liang Zhang, André Vincent, and Filippo SperanzaCommunications Research Centre Canada,3701 Carling Avenue, Ottawa, Ontario, Canada K2H 8S2Tel : 1-613-991-5621, Fax : 1-613-990-6488, E-mail: demin.wang@crc.caAbstract—The conventional two-dimensional (2-D) wavelet transform used in existing image coders is usually performed through one-dimensional (1-D) filtering in the vertical and horizontal directions, which cannot efficiently represent edges and lines in images. The curved wavelet transform presented in this paper is carried out by applying 1-D filters along curves, rather than being restricted to vertical and horizontal straight lines. The curves are determined based on image content and are usually parallel to edges and lines in the image to be coded. The pixels along these curves can be well represented by a small number of wavelet coefficients. The curved wavelet transform is used to construct a new image coder. The code-stream syntax of the new coder is the same as that of JPEG2000, except that a new marker segment is added to the tile headers. The new coder therefore retains all the features and functionalities of JPEG2000. Results of image coding and subjective quality assessment show that the new coder performs better than or as well as JPEG2000. The new coder is particularly efficient for images that contain sharp edges and can provide a PSNR gain of up to 1.6 dB for natural images compared with JPEG2000.Index Terms—Image coding, wavelet transform, curved wavelet, image compression, sub-band coding.I. I NTRODUCTIONOver the past decade, wavelet transform (WT) has become a powerful tool for image and video coding [1][2]. A number of wavelet-based image and video coders have been developed, such as those described in [3]-[11]. These coders provide desirable features and functionalities with compression efficiency comparable to that of the most efficient DCT-based image coders. Typically, the two-dimensional (2-D) WT used in these coders is performed through one-dimensional (1-D) filtering along each row and each column of the image to be coded.The main shortcoming of this conventional WT is that it does not provide a compact representation for edges and lines in images. When an image is filtered along rows and columns, the filter often crosses edges and lines of the image. The sequence of pixels across an edge usually has a broad frequency spectrum, from low to high frequencies. The wavelet transform decomposes the energy of the pixel sequence into a large number of frequency bands (called scales). Many wavelet coefficients are therefore required to properly reconstruct the edge, resulting in a representation that is not compact. As a result, the wavelet-based image coders produce “ringing” artifacts around edges, especially at low bit rates.In this paper, we present a curved wavelet transform, referred to as curved WT, for image coding. The curved WT is carried out by applying 1-D filters along curves, rather than being restricted to horizontal and vertical straight lines. The curves are determined based on image content. The basic idea behind the curved WT is that if a curve is parallel to an edge or a line in the image to be coded, the sequence of pixels along the curve consists mainly of low frequency components with no or little high frequency components and can be well represented by a small number of wavelet coefficients. The curved WT can therefore provide a more compact representation for edges and lines than the conventional WT and improve the compression efficiency of wavelet-based image coders.The orientations of edges and lines are taken into account in several image coding methods. In [12], an image is partitioned into rectangular regions and each region is resampled along a pair of axes. A conventional subband decomposition is then applied in the resampled domain. After resampling, the supports of the regions are no long rectangular and may not connect one to another. Therefore, each region has to be decomposed and encoded independently. The independent decomposition of each region reduces the coding efficiency and results in visible artifacts around the region boundaries. In [13], the geometric flows are curved. That study, however, was focused on the development and optimization of a new class of representation bases, called bandelet. Recently, several new transforms that take edge orientation into account have been proposed in order to compactly represent edges. These transformsinclude curvelets [14][15], ridgelets [15][16], contourlets [17], and directional wavelet transforms [18][19]. The underlying theories of these transforms and their application to image coding are still under development. Finally, 1-D wavelet filters along motion trajectories have been recently used in video coding [20][21].We initially proposed the curved WT in [22], where the curved WT was implemented using convolution wavelet filters and overlapped extension, and was used to improve the well-known SPIHT image coder [4]. In this paper, the curved WT is implemented with a lifting structure and is used to construct a new image coder with EBCOT (embedded block coding with optimal truncation). EBCOT has been adopted in JPEG2000 [8].This paper is organized as follows. Following the introduction, Section II is devoted to the description of the curved WT, where we present constraints on the curves, the algorithms used to determine the curves, and the implementation of the curved WT via the lifting structure. In Section III, we describe the new image coder based on the curved WT. Section IV reports experimental results of image coding using the new coders and the performance comparison with JPEG2000. The comparison includes objective, i.e., PSNR, and subjective measures of picture quality obtained through a subjective quality assessment experiment. Finally, conclusions of the paper and perspectives for future study are presented in Section V.II. C URVED W AVELET T RANSFORMThe concept of the curved WT is illustrated in Fig. 1. The first step of the curved WT is to determine , for 0 ≤i < I, as shown in Fig. 1 (a). Low-pass and high-pass wavelet filters are a set of vertical curves xiapplied separately along the curves, and the outputs of these filters are sub-sampled by discarding every other row, resulting in low-pass coefficients L(m, n) and high-pass coefficients H(m, n). The filters can beis then determined for the resulting any pair of 1-D wavelet filters. A set of horizontal curves yicoefficients L(m, n), as shown in Fig. 1 (b). The low-pass and high-pass filters are applied along these curves, and the outputs are sub-sampled by discarding every other column, producing two bands ofcoefficients LL and HL. In a similar way, a third set of curves z i is determined for coefficients H(m , n )and is used to decompose the H coefficients into the LH and the HH coefficients. The process described above achieves the first level transform of the curved WT, producing four bands of coefficients LL, HL, LH, and HH, as shown in Fig. 1 (c). This process is repeatedly applied to the resulting LL bands to obtain the second and higher level transforms.Alternatively, the first set of curves at each transform level can be horizontal curves, while the second and third sets are vertical curves. Different orders of the curve sets (i. e., vertical curves first or horizontal curves first) usually produce different transform results because the curves usually change with the order. Therefore, the order, as well as each set of curves, should be determined based on image content.In order to obtain a dyadic decomposition like that produced by the conventional WT, there must be some constraints on the curves. In the following subsections, we describe these constraints, the algorithms used to determine the curves, and the implementation of curved WT via the lifting structure.(a) (b)(c)Fig.1. Curves and the first level of curved WT. (a) Vertical curves x i , (b) horizontal curves y iand z i for low pass coefficients L and high pass coefficients H, (c) the four bands ofcoefficients resulting from the first level transform.A. Constraints on the CurvesIn order that the curved WT provides a dyadic decomposition, a vertical curve is defined by a continuous single-valued function of the vertical coordinate m. This means that a vertical curve does not contain any horizontal straight-line segment. The pixels along a vertical curve come from a set of successive rows of the image to be transformed, with one and only one pixel per row. A curve may connect with other curves and may pass between two adjacent pixels. Each pixel in the image to be filtered has to be associated with at least one curve. Similarly, a horizontal curve is defined by a continuous single-valued function of the horizontal coordinate n. With these constraints, the output of the filters along a set of vertical (horizontal) curves can be sub-sampled by discarding every other row (column), and a multi-level curved WT produces a dyadic decomposition.To reduce the computational cost for determining the curves and the coding cost for compressing the curves, the following additional constraints are imposed on the curves in our implementation:(i) All curves consist of line segments with a few discrete orientations. For vertical curves, theDOORZHG RULHQWDWLRQV DUH DQG- DQGLV LGHQWLF al with the one of orientation -DQGFig. 2, the line segments of orientations-SDVV EHWZHHQ WZR DGMDFHQW SL[HOV Half-pixel values are required for filtering along these segments.(a) (b)Fig. 2. Allowed orientations of the line segments constituting the curves, (a) for vertical curves,and (b) for horizontal curves. The solid circles denote full-pixels and the empty onesdenote half-pixels.B. Determination of the CurvesIdeally, for a given image and target bit rate, the best curves could be determined through rate-distortion optimization. This would require a large number of calculations because any change affecting a curve also impacts other curves at the same and higher levels of the transform.To reduce the computational complexity, we propose a simple algorithm to determine the curves. This algorithm searches for the curves that minimize the energy of high-pass wavelet coefficients within each block. After the curved WT is applied to an image with the curves determined in this manner, the energy of the image is concentrated in the low-pass coefficients at the highest level, and the high-pass coefficients at all levels contain a small amount of energy. A low energy of high-pass coefficients means that the high-pass coefficients require a small number of bits to code. Since the number of the high-pass coefficients is much larger than that of the low-pass coefficients, reducing the number of bits required by-the high-pass coefficients can effectively reduce the total number of bits required for the image. The proposed algorithm for determining each set of curves consists of the following steps:1) The high-pass wavelet filter is applied to the image along the straight lines of each allowedorientation.2) The filtered image is divided into blocks of K×L pixels.3) In each block, the energy of the resulting high-pass coefficients is calculated for each of theallowed orientations.4) The orientation that results in the lowest energy is chosen, and the line segments of thisorientation within the block are kept.5) The kept line segments are extended to adjacent blocks and connected with the kept line segmentsof the adjacent blocks to form a set of curves.6) The total energy Ecof the high-pass coefficients produced by a set of vertical (or horizontal)curves is compared with the energy EL produced by the conventional WT. If Ec> αELwith0 < α < 1, this set of vertical (or horizontal) curves is replaced by a set of vertical (or horizontal)lines, where α is a predetermined parameter. In this study, α was experimentally set to 0.9.The last step takes into account the compromise between the energy reduction and the coding cost resulting from the curved WT. On the one hand, the curved WT can reduce the energy of high-pass wavelet coefficients and thus the number of bits required for encoding the coefficients. On the other hand, the curves have to be coded and transmitted to the decoder, requiring a coding cost. Therefore, a set of curves is not used in the curved WT if it cannot reduce the energy sufficiently to overcome the associated coding costs.As mentioned above, each level of the curved WT requires two sets of curves (the third set is identical to the second one). If the first set consists of vertical curves, then the second one will consist of horizontal curves, or vice versa. The orientation order should be determined based on image content. In our implementation, the order that produces the smaller energy of high-pass coefficients at the first transform level is chosen for all the levels.The size K×L of the blocks should vary with the level of the transform because the number of LL coefficients becomes smaller at a higher level. In our experiments, the block size for the first set of curves xiis predefined as 32×32 for the first level of transform and 16×16 for the second and higher levels. Theblock sizes for yi are half of those for xi. Specifically, if yiare horizontal (vertical) curves, the blockheights (width) are half of those for xi.Fig. 3 illustrates some of the vertical curves that are determined using the proposed algorithm for the image Barbara 7KH FXUYHV DUH JHQHUDOO\ SDUDOOHO WR WKH HGJHV DQG OLQHV RI RULHQWDWLRQ EHWZHHQLQ WKH LPDJH 1RWH WKDW IRU LOOXVWUDWLRQ SXUSRVHV RQO\ VRPH RI WKH FXUYHV DUH VKRZQ LQ WKLV ILJXUHFig.3. Some of the vertical curves determined for image Barbara with block size of 32×32 pixels.C. Implementation with Lifting StructureIt is well known that wavelet filters can be implemented with a lifting structure. For the 5/3 and 9/7 wavelet filters, each lifting step involves three pixels. When one of the wavelet filters is applied along a column (or row) in an image, a pixel is predicted or updated using the two adjacent pixels on the same column (or row) at each lifting step [8].The lifting structure is suitable for the implementation of the curved WT because at each step it involves only two adjacent pixels, instead of more pixels along a curve. For simplicity, we describe below only the implementation with vertical curves. Suppose that a pixel is on a vertical curve. At each lifting step, this pixel is predicted or updated with its two adjacent pixels that are on the same curve. One of the adjacent pixels is on the row above the pixel and the other on the row below the pixel. For the example shown in Fig. 4 (a), pixels a , b , and c are on the same vertical curve x (m ). Pixel a is predicted or updated by a + λ(b + c ), where λ is the lifting coefficient.(a) (b) (c) Fig. 4. Three situations in the lifting implementation of the curved WT. (a) the normalsituation, (b) requiring half-pixel interpolation, (c) intersection of two or more curves.There are two special situations to handle in the implementation. The first one involves half-pixelinterpolation. As shown in Fig. 2, the line segments of orientations -SDVV EHWZHHQ two adjacent pixels. Half-pixel interpolation is required for these line segments. For the example shown inFig. 4 (b), pixel a is on a line segment (part of a curve) RI RULHQWDWLRQThe code-stream syntax of the new coder is the same as that of JPEG2000 [8], except that a new marker segment is added to the tile headers (the image to be coded may be divided into tiles). This marker segment consists of two bytes indicating the start of the marker segment, two bytes specifying the length of the marker segment, one bit indicating the curve set order, and then the codes of all the curves for the tile. The new coder, therefore, retains all desired features and functionalities of JPEG2000.Clearly, the new encoder requires more calculations than the JPEG2000 encoder because the forward curved WT involves the determination of curves and curve set order, whereas the conventional WT does not. However, the computational cost of the new decoder is about the same as that of the JPEG2000 decoder. Indeed, the inverse curved WT in the new decoder does not need the determination of curves and curve set order. Additionally, the decoding of curves is fast because the amount of data representing curves is very small.Fig. 5. Block diagram of the new image encoder CWT-EBCOT.IV. E XPERIMENTAL R ESULTS AND P ERFORMANCE E VALUATION In this section, we first show the energy compactness of the curved WT, then present image coding results of the new coder and the comparison with JPEG2000 in terms of PSNR. Finally, we present the results of a subjective quality assessment experiment in which a group of viewers assessed the picture quality of the decoded images.A. Energy CompactnessTo demonstrate that the curved WT can provide a more compact representation for edges and lines than the conventional WT, we synthesized a test image of size 480×720, hereafter called Zoneplate. This image, shown in Fig.6 (a), was transformed using both the curved WT and the conventional WT. The results of the first level of transform are shown in Fig. 6 (b) and (c), respectively. For increased visibility, the LH and HH coefficients have been multiplied by a factor of 2. The curved WT preserves 95.6% of the image energy in the LL coefficients and distributes only 4.4% of the energy in the high-pass (LH, HL, and HH) coefficients. By contrast, the conventional WT distributes as much as 28.5% of the energy in the high-pass coefficients, six times more than the curved WT.(a)Fig. 6. Comparison between the curved WT and the conventional WT. (a) a synthetic image called Zoneplate, (b) the curved WT distributes 4.4% of the image energy in the LH, HL, and HH coefficients, (c) the conventional WT distributes 28.5% of the energy in the thesecoefficients.(b)(c)Fig. 6 (continued). Comparison between the curved WT and the conventional WT. (a) a synthetic image called Zoneplate, (b) the curved WT distributes 4.4% of the image energy in the LH, HL, and HH coefficients, (c) the conventional WT distributes 28.5% of theenergy in the these coefficients.B. Comparison in Terms of PSNRSix synthetic and natural grayscale images were used in the experiment for comparing the new image coder and JPEG2000. These images include three well-known images, Lena, Barbara, and Goldhill, of size 512×512; two JPEG2000 test images, Bike and Woman of size 2560×2048; the synthetic image Zoneplate shown in Fig. 6 (a). The images were compressed at bit rates of 0.1, 0.2, 0.3, 0.4, 0.5, 0.7, and 0.9 bits per pixel (bpp) using the new coder and a JPEG2000 coder [23]. The wavelet filters used in these coders were the 9/7 and 5/3 filters [8].Fig. 7 shows the PSNR of the decoded images at various bit rates. The synthetic image Zoneplate, which consists of circles of various widths, contains a large amount of high frequency components. This image is very difficult to compress with existing image coders. In contrast, the new coder was able to follow the contour of the circles and achieved a PSNR of up to 9.69 dB higher than that obtained by JPEG2000. For the images Barbara and Bike, the new coder also significantly outperformed JPEG2000 because these images contain a large amount of edges and lines of various orientations. For example, the PSNR gain of the new coder with the 5/3 filter was 1.67 dB for Barbara at the bit rate of 0.4 bpp and 0.74 dB for Bike at 0.1 bpp. The new coder produced a PSNR slightly higher than JPEG2000 for the image Lena and performed as well as JPEG2000 for Goldhill and Woman. The image Goldhill is quite blurred and does not contain sharp edges and Woman mainly consists of large flat regions (e.g., the image background) and fine textures (e.g., the sweater). Table 1 lists the average PSNR over the tested bit rates for each image as well as the average PSNR gains of the new coder compared with JPEG2000.As expected, a comparison of the two wavelet filters indicated that the 9/7 filter resulted in a higher PSNR than the 5/3 filter. As well, with the new coder, differences in PSNR resulting from the two filters were smaller than those with JPEG2000, as shown in Table 2. This means that the impact of the filters on the new coder is smaller than that on JPEG2000.Fig. 7. PSNR obtained using the new coder and JPEG2000.Fig. 7 (continued). PSNR obtained using the new coder and JPEG2000.Table 1. Average PSNR (dB) and gains of the new coder compared with JPEG2000.Image Filters New coder JPEG2000 Gain5/3 25.32 17.75 7.567Zoneplate 9/7 27.70 20.62 7.0855/3 35.07 34.82 0.241Lena 9/7 35.61 35.57 0.0415/3 30.96 29.52 1.444Barbara 9/7 31.74 30.65 1.0925/3 31.66 31.67 -0.007Goldhill 9/7 32.07 32.08 -0.0065/3 31.85 31.30 0.558Bike 9/7 32.21 31.81 0.3995/3 31.65 31.55 0.103Woman 9/7 32.18 32.21 -0.031Table 2. Differences in the average PSNR (dB) resulting from 9/7 and 5/3filters in each coder.Image New coder JPEG2000Zones plate 2.38 2.87Lena 0.54 0.74Barbara 0.77 1.13Goldhill 0.41 0.41Bike 0.36 0.52Woman 0.53 0.66C. Subjective Picture Quality AssessmentTo further evaluate the new image coder and compare it with JPEG2000, we performed a subjective quality assessment experiment on decoded images of Lena, Barbara, and Goldhill. The decoded images were obtained using the new coder and the JPEG2000 coder with 9/7 filter at bit rates of 0.1, 0.2, 0.3, 0.4, and 0.5 bpp. Eleven viewers, with a mean age of 39.2 years, participated in the experiment. All viewers had normal visual acuity. Three viewers had experience with subjective assessment of image quality. No viewers had experience with the coding technologies and their associated artifact characteristics.Subjective quality was measured using a stimulus comparison method. In each trial, viewers were shown two decoded images of the same source presented side by side on a Sony GDM-F520 CRT monitor. The images were viewed from a distance equal to four times the height of the images on the screen. One of the images was obtained using the new coder at a given bit rate and the other was obtained using JPEG2000 from the same source image at the same bit rate. Thus there were 15 basic, “side by side”, comparisons (i.e., three images at five bite rates). To eliminate any effect due to monitor characteristics, each basic comparison was presented in two different versions. In one version, the image obtained using the new coder was on the left side and the image obtained using JPEG2000 on the right side, while in the other version the position order was reversed. Furthermore, each version was shown twice to reduce the variance of the measured quality. Accordingly, each viewer saw each of the 15 basic comparisons four times for a total of 60 trials. The order of presentation of the basic comparisons and versions was randomized. Different random orders were used for different viewers.Viewers were asked to assess the perceived quality of both the left and right images and rate the difference, if any, using the 10 cm graphic scale illustrated in Fig. 8. For analysis, the responses were recorded as varying from -50, corresponding to extreme left end of the scale, to +50, corresponding to the extreme right end of the scale. In each trial, the images were displayed on the screen until the viewers had provided a rating.Fig. 8. Rating scale used to assess subjective image quality. The numerical values were not printedon the scale and were used only for analysis.As noted previously, the viewers rated each of the basic, “side by side”, comparison 4 times. The ratings of these 4 replications were averaged to obtain the viewer’s rating for that comparison. The viewers’ ratings were then averaged to obtain the mean rating for that comparison. Fig. 9 shows the mean ratings as a function of bit rates for the three source images. In this figure, a positive value indicates that the image obtained using the new image coder was rated as having a higher quality than that obtained using JPEG2000. A negative value indicates the opposite. A zero value means no difference in perceived quality between the images obtained using the two coders.It can be seen from Fig. 9 that the mean ratings for the Barbara and Lena images are different from zero and positive, although the magnitude of the difference decreases with increasing bit rate. These results suggest that the new image coder performs better than JPEG2000 for these images. To corroborate these observations, we evaluated the statistical significance of the observed mean ratings using the t-test. The statistical analysis confirmed that the observed mean ratings were significantly different from zero for the bit rates of 0.1, 0.2, 0.3, and 0.4 bpp (p <0.05 in all cases), but not for the 0.5 bpp case. For the images of Goldhill, it is easy to see that there was no difference between the performances of the two coders at any of the bit rates.Overall, these results are in agreement with those obtained with the PSNR. A couple of minor discrepancies are worth noting, though. First, note that while the perceived difference decreases at higher bit rates, the PSNR does not (see Fig. 7). This indicates that, as it might be expected, the difference in picture quality produced by the two coders at high bit rates becomes perceptually too small to be easily assessed. Secondly, for the image Lena the PSNR resulting from the new coder are only slightly higher than those from JPEG2000. On the contrary, the differences in subjective quality are far more pronounced. These minor discrepancies simply confirm that PSNR does not always provide an accurate estimate of perceived picture quality.Fig. 9. Subjective quality (average of the individual ratings) as a function of bit rates. The error bars represent the 95% confidence intervals.Generally, the new image coder preserves edges and lines better than JPEG2000. This can be observed from Figs. 10 and 11. Fig. 10 shows portions of two decoded Barbara images obtained with the 9/7 filters at 0.2 bpp. In the image produced by the new coder, the lines on the clothes are well reconstructed and the edges of the arm are smooth. However, many lines in the image produced by JPEG2000 are lost or distorted, and there are serious “ringing” artifacts around the edges. Fig. 11 shows portions of two decoded Lena images at 0.2 bpp. The PSNR difference between the two images is only 0.16 dB. However, the edges of the hat brim and the mirror frame in the image produced by the new coder are smooth, whereas those in the image produced by JPEG2000 have significant “ringing” artifacts.(a)(b) (c)Fig. 10. Portions of decoded Barbara images at 0.2 bpp, (a) the original image, (b) the new coder, PSNR=28.67 dB, (c) JPEG2000, PSNR=27.25 dB.(a)(b) (c)Fig. 11. Portions of decoded Lena images at 0.2 bpp, (a) original image, (b) the new coder, PSNR=33.10dB, (c) JPEG2000, PSNR=32.94 dB.V. C ONCLUSIONSIn this paper, we present the curved wavelet transform in which 1-D wavelet filters are applied along curves. A simple algorithm is proposed for determining the curves. The resulting curves are usually parallel to edges and lines in the image to be compressed. Therefore, the curved wavelet transform can exploit the geometrical information on the orientation of edges and lines to provide a compact representation for images. Then, we present a new image coder that is based on the curved wavelet transform and embedded block coding with optimal truncation (EBCOT). The code-stream syntax of the new coder is the same as that of JPEG2000, except that a new marker segment is added. The new coder, therefore, retains all the desired features and functionalities of JPEG2000. Image coding results show that, compared with JPEG2000, the new coder results in higher PSNR for some images and about the same PSNR for the other images. The new coder is particularly efficient for images that contain sharp edges and lines and can provide a PSNR gain of up to 1.6 dB for natural images. The results of a subjective quality assessment experiment confirm that the new coder provides overall higher subjective image quality than JPEG2000, possibly because it preserves edges well.The algorithm proposed in this paper for determining curves is not optimal. We believe that an optimal algorithm can significantly improve the performance of the new image coder. In addition, some constraints imposed on the curves could be relaxed, leading to more general curves. The curved wavelet transform can be combined with any methods that process, reorganize, and compress wavelet coefficients. In [22], the curved wavelet transform was combined with the well-known set partitioning in hierarchical trees (SPIHT) [4]. Combination with other methods remains open for future study.A CKNOWLEDGMENTThe authors would like to thank Dr. Wa James Tam and Ms. Taali Martin for their help in the writing of this paper, and to thank Mr. Phil Blanchfield and Mr. Ronald Renaud for preparing the subjective quality assessment experiment.。