Header for SPIE use Advanced Lifting-Based Motion-Threading (MTh) Technique for the 3D Wave

1. INTRODUCTION
Wavelet transform is an effective method for image and video representation. It can efficiently pack the energy of the image/video and facilitate the latter entropy coding. An additional advantage of wavelet transform is its nice scalability. Studies have been focused on how to efficiently exploit the correlations among frames for wavelet-based video coding. Directly applying the wavelet transform to the predicted residues is usually referred as the MC + 2D wavelet coding [2][3]. Such schemes have been extensively investigated in the past decade. Since the wavelet transform is not good at representing the residual image, it does not show the desired promising efficiency. Another aspect of wavelet video coding is to directly apply wavelet transform in all three directions, i.e., the temporal, horizontal and vertical directions, where the correlations among frames are exploited by the temporal wavelet transform instead of motion compensation. With the inherent scalable property, the 3D wavelet video coding can readily achieve quality, temporal and spatial scalabilities simultaneously, which is a very desirable feature in the video streaming and storage applications [1]. However, due to the existing motion among frames, the pixels located at the same position in different frames may be out of alignment so that the efficiency of the
Header for SPIE use
direct temporal wavelet transform is limited. Some early works have been done to incorporate motion information into the 3D wavelet coding. Taubman and Zakhor used a global displacement model to align different frames before the 3D wavelet transform [4]. Wang et al. [5] proposed to register and warp all frames into a common coordinate system and then apply a shape-adaptive 3D wavelet transform to the warped video. Considering the limitations of the global motion model, Ohm proposed to use the block matching technique similar as that in the traditional video coding schemes and meanwhile specially process the covered/uncovered, connected/unconnected regions [6]. In the above mentioned approaches, either the perfect reconstruction can not be achieved, or the motion accuracy is not sufficient. Even after the motion alignment, the compression efficiency is still not satisfactory. Recently, MPEG is actively investigating and developing the inter-frame wavelet coding technologies. MCEZBC proposed by Chen et al has become prominent because of the excellent performance [8]. In MC-EZBC, each pair of frames is first motion estimated with hierarchical block structure, and then decomposed into a high-band frame and a low-band frame by the motion-aligned lifting-based Haar filter. Further temporal decomposition is performed on each pair of low-band frames with the similar process. MC-EZBC efficiently solves the problems in the fractional-pel motion aligned temporal transform. The performance of MC-EZBC is comparable to that of the JVT standard [9]. However, as a short support wavelet filter, Haar filter may not fully exploit the long-period correlations across video frames. To utilize the long-period correlation along the motion trajectory, Xu et al proposed the motion-threading (MTh) approach [7]. Along with the MB-based estimated motion vectors, pixels are formed into many nonoverlapped motion threads. Multi-layer shape-adaptive wavelet transform is applied to each motion thread. The temporally transformed frames are further decomposed horizontally and vertically within each frame. The entire decomposed coefficients are coded with the embedded entropy coding to organize a scalable bit-stream. Motion-threading technique provides a flexible structure in utilizing the correlation among frames. For simple motion sequences, MTh can achieve a good performance. The limitation of the MTh scheme is the nonoverlapping restriction which intends to ensure the perfect wavelet reconstruction upon motion threads. For some complex motion sequence, many threads which involve many-to-one pixel mapping have to be artificially truncated under this restriction. The wavelet reconstruction error on these thread boundaries considerably decrease the coding efficiency [15]. This paper proposes an advanced motion-threading technique to improve the coding efficiency of the 3D wavelet coding. We first extend the original motion-threading technique with the lifting wavelet structure [10]. Even if many-to-one mapping arises, all the motion threads can still be filtered without being terminated. This can greatly reduce the boundary effects. Furthermore, fractional-pixel motion estimation and alignment, such as half-pixel and quarter-pixel, can be easily applied in the extended scheme with perfect wavelet reconstruction. To reduce the motion cost, this paper also investigates the correlation of motion vectors among adjacent frames. An R-D optimized criterion is designed to select a proper MB mode, considering the motion correlations during motion estimation and motion vector coding. Results show that the proposed advanced MTh scheme improves the coding efficiency up to 1.5~5.5dB generally, and achieves a competitive performance to the state-of-the-art JVT standard [9]. The paper is organized as follows. Section 2 introduces the basic structure of the original motion threading technique. In section 3, an advanced lifting-based motion-threading 3D wavelet coding scheme is proposed. Fractional pixel motion alignment and multi-layer motion threading are discussed in detail. An R-D optimized scheme is designed to consider both the mismatching problem and the motion cost reduction. Experimental results are given in section 4. Finally, we conclude this paper in Section 5.