A Motion Estimation CHip for Block Based MPEG-4 Video Applications

A Motion Estimation Chip for Block BasedMPEG-4 Video ApplicationsMaleeha Abbas, Batool Talha, Shoab Khan and *Adeel AbbasCollege of Electrical and Mechanical Engineering, National University of Sciences and Technology, Pakistan*Department of Electrical and Computer Engineering, Johns Hopkins University{Emails: maleeha@, batool_engineer@}Abstract - This paper presents an efficient VLSI architecture for the MPEG-4 video-coding standard. While its predecessors focused on the storage and transmission of audiovisual sequences, MPEG-4 being the first object-based coding standard has much more to offer. Motion Estimation (ME), the most computationally intensive part of MPEG, requires very high power throughput on account of its high memory utilization. To cope with the bandwidth and processing power requirements, the best approach entails a dedicated implementation for such a demanding algorithm. The first step of the design involves hardware/software partitioning to map most of the motion estimation functions in hardware. In order for the system to meet area and power dissipation constraints, less time critical functions like intra-frame coding, discrete cosine transform, quantization, variable length coding and zigzag scanning are handled as part of the software portion of the program. We have increased the precision significantly by doubling the accuracy of motion compensation from integer-pel to half-pel. Sum of Absolute Differences (SAD) is chosen as the criterion for evaluating the best match of motion vectors. The block matching is achieved with the novel robust search algorithm called Predictive Diamond Search (PDS). The chip is synthesized at 67 MHz with a power dissipation of 452mW.Keywords: Motion estimation, VLSI architectures for MPEG, Bilinear Interpolation, Video/Image coding1. INTRODUCTIONMotion estimation is an inter-frame coding technique which eliminates temporal redundancy [1]. Temporal redundancy is redundant information in the form of similarities between frames due to motion. Inter-frame compression scheme achieves low bandwidth by coding and transmitting the difference between successive picture frames. The smaller the difference, the better the motion is predicted. Furthermore, the direction of motion is evaluated by searching for the best motion vector on the criterion of Sum of Absolute Differences (SAD) [2]. As a first step in SAD calculation, the absolute differences of corresponding elements in macroblocks are estimated. In the second stage they are summed up to give the final SAD result. SAD for a given motion vector (x, y) and a block of N ×N pixels is given by the following equation:|),(),(|),(0y j x i prev j i curr y x SAD Nj Ni ++−=∑∑==Where curr (i, j) is a pixel in current (reference) frame and prev (i+x, j+y) is a pixel in previous frame with an offset of (x, y). Block based motion estimation algorithms range from the most time consuming i.e. Full-Search Block Matching [3] to a variety of hierarchical algorithms like Three Step Search [4], Four Step Search [5], Two Dimensional Logarithmic Search (TDL) [6], Orthogonal Search Algorithm [1], and Predictive Line Search [7].With the growing demand of multimedia communication over wired and wireless links, a lot of current research is focused on developing SoC architectures for video processing so as to achieve low power consumption and cost [8] and [9]. But most of them offer a hardwired solution with capability to perform only a specific search algorithm and range, resulting in less implementation flexibility [10]. There is a need for a more flexible architecture as different multimedia applications have varying quality, power consumption and computational requirements. We are interested in the design of a 32-bit application specific processor, called MEP-II, which is capable of performing a wide range of MPEG-4 motion estimation functions. With MEP architecture, the search window, motion estimation algorithm and pixel block size for SAD can be reconfigured by simple instruction level modifications. The information for search window and block size is placed in the memory and later used as counters in the Address Generation Unit register file to execute the required sequence. To verify our results we implemented Predictive Diamond Search algorithm [11] due to its superiority in computational complexity and output quality. Besides, the bilinear interpolation is executed to increase the motion vector precision to half pel.2. ARCHITECTURE OVERVIEWThe heart of the architecture consists of four execution units operating in parallel, i.e. a Program Control unit (PCU) a Data Processing Unit (DPU), an Address Generation Unit (AGU), and a Memory Management Unit (MMU). They are discussed in the following sections:2.1 Program Control Unit (PCU)Controller organizes the flow of entire program and binds all the rest of modules together. Control unit is implemented as a micro-coded state machine [12]. Shown in Fig. 1, the PCU architecture supports 4-level nested loop control, branching and subroutine control using a 10-b program counter (PC) and 64-b instruction register (IR).At every clock cycle, single instruction is fetched from one of 550 locations in the instruction ROM and is registered in IR. Next, PC is incremented and instruction word is fed to the decoder, which generates the control signals for rest of the architecture. The next address logic selects the input address for instruction ROM from branch address, PC, Subroutine Return Address (SRA) and output from the for loop block. The instruction op-code holds conditional bit and polarity bit, which are input to the next address logic block. Whenever the polarity bit is high, a conditional jump occurs. The conventional Jump instructions are executed by loading PC with the Branch Address from the instruction word. The SRA is a 10-b register, provided to store the PC value when the subroutine instruction is executed. In case of Return instruction the PC moves to the value in SRA.The architecture takes care of nested loops by three last-in first-out (LIFO) stacks for storing the loop start address, loop end address and the loop counter. The decoding of the repeat instruction involves pushing of these values into the respective stacks and PCU starts executing the instructions in loop. Afterwards the PC is continuously compared with the loop end address value and loop counter is decremented each time loop is completed. When PC becomes equal to the loop-end-address and the loop-counter decrements to zero, the value in each stack is popped. Subsequently, the program follows out of the loop.2.2 Data Path and the SAD Computation UnitData Path’s core operations include calculation of SAD and performing interpolation of the previous frame. Data Path architecture is composed of 32×32-b register file with six read ports and four write ports, 4 processing units (PU), 4 barrel shifters and a 6-input Wallace compression tree of three-level depth (shown in Fig. 2). Four 32-b ports permit fast loading or storing of data from / to the memory, while the other six 32-b ports are used for the simultaneous reading of four operands and the writing-back two of the results on the register file. TheData path is able to perform 16×16 SAD Macro-block calculations in 64 clock cycles.Curr_SADEach processing unit is made up of subtract / accumulate unit, 2×1 multiplexer and a not gate. During SAD [12] the PUs are used to calculate absolute difference between two pixel values while for interpolation, they are used to sum the pixel values. Finally the compression tree adds four absolute differences, correction vector and current SAD result in one clock cycle [14]. The Min_SAD and Curr_SAD registers are used to implement the early SAD truncation technique [15]. Two flags (Abort_bit and Equality_bit) are enabled if the current SAD exceeds or equals the previous minimum SAD calculated for the particular block in specified search window. These flags decide the next instruction to be executed by the controller.Motion vector precision to half pel is achieved by coding the difference between current and interpolated reference frame. During interpolation, the sum of two pixels from each processing unit is averaged by one of the shifters. The 16-b averaged sum from two shifters is thenFig. 2: Data Path and SAD Calculation Unitconcatenated and written on the 32-b write port of the register file.2.3 Memory Management Unit (MMU)The MMU is divided into three-memory banks i.e. current, previous and interpolated frame memory for efficient operand fetch. Each MMU memory location is 32-bit wide and can store 2 pixels. The MMU memory decoder is used to select the type of memory for each instruction. In addition, there is an address multiplexer, which takes input from instruction, AGU or from some external source like the software GUI in our case. This is used to fill the current and previous memories. Select signal from PCU decides which address is to be loaded on the MMU bus.The current and previous memory blocks have the same number of ports i.e. one 32-b write port and two 32-b read ports. But they differ in capacity. For a 176×144 frame, current memory block has a size of 13K×32 bits and previous memory block has a size of 19K×32 bits. This difference in size occurs as previous frame has padding on the corners to calculate motion vectors at the edges. Apart from storing the current frame pixels, the Block Type, Block Number and Search Points are also stored in the current memory block. This information is required for performing the search algorithm, in our case predictive diamond search. The interpolated frame is four times the size of previous frame (75K×32 bits). Interpolated block is a 32-b dual ported memory. It can load or store 4 pixels in a cycle. Interpolated memory has special signals from the PCU to control the bits (lower or upper) to be read or written.2.4 Address Generation Unit (AGU)AGU performs effective address calculations necessary for fetching operands from memory. It can generate and modify two 18-b addresses in one clock cycle. AGU uses integer arithmetic to compute addresses in parallel with other processor resources to minimize address-generation overhead.The AGU is divided into an address Arithmetic Logic Unit (ALU) and an address register file. As stated, each MMU memory location stores 2 pixels. The AGU has an addition/subtraction unit, multiplier, shifter unit, OR gate and AND gate. A 5-b integer value (Immediate_data) is used if one of the operands in ALU operation is a constant. The main address calculated is stored in AGU_addr1 (See Fig. 4) and it reads the first two pixels. AGU_addr1 is incremented to generate AGU_addr2, which in turn addresses the third and fourth pixel. With this mechanism, each read or write can access four pixels in a cycle.The address register file consists of 16 18-bit registers, each of which can be controlled independently to act as temporary data registers or as indirect memory pointers. Data from two read ports of the register file is loaded in parallel on the two address buses feeding the MMU. The value in any register can be modified from three sources via the write port. These sources can be data from memory, result calculated from address ALU and constant value from the instruction. Automatic updating is available when using address register indirect addressing mode. The addressing mode modifies the selected address register in a read-modify-write fashion; the addressregister is read, its contents are modified by the associated modulo arithmetic unit, and the register is written with the appropriate output of the arithmetic unit (See Fig. 4).3. INSTRUCTION SET DESIGNMEP-II architecture is a single cycle hardwired control machine. The instruction set consists of a total of 41 RISC like instructions, each 64-b long. The instruction format consists of 6-b op-code, 2-b select signal for the branch multiplexer, a polarity bit, an Abort bit, a Conditional bit, 2-b memory type selection signal and the source and destination registers of AGU. The instruction set operates on AGU register file and consists of both general purpose and application specific instructions.The general-purpose instructions modify the AGU registers and facilitate data transfer. MEP-II features a wide range of arithmetic and logical instructions including addition / subtraction, multiplication, shift etc. The data transfer is made possible by instructions like MEMORY_READ (memory read takes 2 cycles), STORE_PIXELS (store calculated interpolated pixels), STORE_SAD (store SAD), and STORE_MV (store motion vectors). In addition, CALL (subroutine call), RETURN (return from subroutine), REPEAT (loop instruction) and NOP (No Operation) instructions are also included as part of the instruction set.The application specific instructions for motion estimation operate on Data Path and Memory. The specific instructions for SAD include resetting the Curr_SAD register and initializing the Min_SAD register in Data Path with maximum possible SAD value and aborting the SAD subroutine when most optimum SAD is calculated. The SAD subroutine loads the Data Path bus with 8 pixels in parallel, computes their corresponding differences andaccumulates the result to current SAD. This process continues till all the pixels in macro block have been operated or SAD is aborted. For performing interpolation we have interpolation subroutine. Single pixel of previous frame requires four additions (2 cycles) to be mapped to interpolated frame. Instructions Interpolate_Initial and Interpolate_Final are required to complete the process.4. SOFTWARE DEVELOPMENT TOOLKITThe MEP-II software is responsible for encoding and decoding a video file, along with implementing interface to the video source through built in Visual C routines. A specifically designed Graphical User Interface (GUI) (shown in Fig. 5) assists the user to select video source, start/stop video capturing and create/open or play/halt the MPEG video file. In addition, the software initializes the MEP-II memory and calculates SAD. If the current frame has less correlation with the previous frame (high SAD value) the frame is intra coded. Intra-frame coding is a compression technique, which reduces the special redundancy of a picture by performing Two Dimensional Discrete Cosine Transform (2D DCT). Once the DCT coefficients are computed, they are quantized and Huffman coded to generate variable length codes. The final encoded bit stream is produced after the coefficients are arranged by zigzag scanning. A reverse of the preceding process takes place at the decoder, which involves Inverse Discrete Quantization (IDCT) and de-quantization.5. RESULTS AND VERIFICATIONThe BURAQ processor toolkit [16] was used to generate assembler and simulator from the instruction set. At first, simulation and testing for hardware and software was conducted by feeding test vectors to each separately. Then the formal verification was performed by integrating both on the Programmable Logic Interface (PLI). Finally, the Register Transfer Level (RTL) verilog code of the design was synthesized using Xilinx’s "XCV300e-8bg352" from Vertex-E library fabricated with 0.18 micron CMOS technology. Based on the test results, our chip can operate at 67 MHz with a power dissipation of 452 mW from a 1.8 V supply. Details are shown in Table 1. As shown, the core size is very small and the powerFig. 4: Address Generation Unit (AGU) Fig. 5: MPEG-4 Video File “cat.mp4”consumption is low so that it is cost effective and will be a good option for portable devices. The compression results of MEP-II are also noteworthy, for example the “cat.mp4” file (shown in Fig.5) is compressed from 198 K bits to 15.4 K bits.Frequency 67MHzPower Dissipation 452mWSupply Voltage 1.8VGate Count 61,603Area (Core Only) 2.10 mm2ROM Size 35K bitsSRAM Size 3.4M bits6. CONCLUSIONSIn this paper, we presented a 32-bit ASIC architecture called MEP-II designed for MPEG-4 video applications, which supports the complex motion estimation operations. Because the interface of MEP-II is very simple and is controllable at the instruction level, it can be used to perform a variety of search algorithms for variable block sizes. Our approach achieves close to 100% hardware utilization by employing the same data path for performing SAD and Interpolation. Moreover, the chip has specially designed software, which combines flexibility with a reduced turn-around time.REFERENCES[1] J. Kneip, B. Schmale, H. Moller and R. Gmbh, “Applyingand implementing the MPEG-4 multimedia standard,”IEEE Micro, vol. 19, pp. 64-74, Nov. 1999.[2] J. Mitchell, W. Pennebaker, C. Fogg and D. LeGall,“MPEG video compression standard,” Digital multimediastandard series, Chapman and Hall, 1996.[3] M. Chen, L. Chen, and T. Chiueh, “One-dimensional fullsearch motion estimation algorithm for video coding,”IEEE Trans. on Circuits and Systems for VideoTechnology, vol. 4, pp. 504–509, Oct. 1994.[4] R. Li, B. Zeng, and M. L. Liou, "A new three-step searchalgorithm for block motion estimation," IEEE Trans. onCircuits and Systems for Video Technology, vol. 4, pp.438-4, Aug. 1992.[5] L. Po and W. Ma, “ A Novel Four-Step Search Algorithmfor Fast Block Motion Estimation”, IEEE Trans. onCircuits and Systems for Video Technology, vol. 6, no. 3,pp. 313-317, June 1996.[6] S. Zhu and K. Ma, “A new diamond search algorithm forfast block matching motion estimation,” Int. Conf.Information, Communications and Signal Proc, vol. 1,pp. 292-6, 1997.[7] Y. Huang, S. Ma, C. Shen and L. Chen, “Predictive linesearch: An efficient motion estimation algorithm forMPEG-4 encoding systems on multimedia processors,”IEEE Trans. on Circuits and Systems for VideoTechnology, vol. 13, pp. 111-117, Jan. 2003.[8] J. Chen and K. Liu, “Efficient architecture and design ofan embedded video coding engine,” IEEE Trans. onMultimedia, vol. 3, pp. 285-297, Sep. 2001.[9] Z. He, M. L. Liou, “Design of fast motion estimationalgorithm based on hardware consideration,” IEEE Trans.on Circuits and Systems for Video Technology, vol.7, pp.819-823, Oct.1997.[10] C. Vleeschouwer, T. Nilsson, K. Denolf and J. Bormans,“Algorithmic and architectural co-design of a motion-estimation engine for low-power video devices,” IEEETrans. on Circuits and Systems for Video Technology,vol. 12, pp. 1093-1105, Dec. 2002.[11] A. Tourapis, G. Shen, M.L. Liou, O. C. Au, and I.Ahmad, "A New Predictive Diamond Search Algorithmfor Block Based Motion Estimation,” VisualCommunications and Image Processing 2000 (VCIP-2000), vol.3, pp.1365-1373, June, 2000.[12] B. Bomar, "Implementation of MicroprogrammedControl in FPGAs," IEEE Trans. on Industrial.Electronics, vol. 49, pp. 415-422, April 2002.[13] S. Wong, S. Vassiliadis and S. Cotofana, “A sum ofabsolute differences implementation in FPGA hardware,”28th Euromicro Conference, pp. 183-188, Sep. 2002. [14] S. Wong, B. Stougie and S. Cotofana, “Alternatives inFPGA-based FPGA SAD implementations,” 1st IEEEInternational Conference on Field-ProgrammableTechnology, pp. 449-452, Dec. 2002.[15] S. Richmond and H. Packard, D.S. " A Low-PowerMotion Estimation Block for Low Bit-Rate WirelessVideo," International Symposium on Low PowerElectronics and Design, pp. 60-63, Aug. 2001.[16] A. Abbas and S. Khan, “BURAQ: A re-configurableprocessor toolkit for VLIW architectures,” 17th ISCAInternational Conference on Computers and TheirApplications, vol. 1, pp. 156-159, April 2002.Table 1: Chip Specifications。