A Multi-Core System-on-Chip Architecture for Multimedia Signal

HiBRID-SoC:A Multi-Core System-on-Chip Architecture for Multimedia SignalProcessing ApplicationsHans-Joachim Stolberg,Mladen Berekovic,Lars Friebe,S¨o ren Moch,Sebastian Fl¨u gel, Xun Mao,Mark B.Kulaczewski,Heiko Klußmann,and Peter PirschInstitut f¨u r Mikroelektronische Systeme,Universit¨a t HannoverAppelstr.4,30167Hannover,Germanystolberg@ims.uni-hannover.deAbstractThe HiBRID-SoC multi-core system-on-chip targets a wide range of applicationfields with particularly high processing demands,including general signal processing applications,video and audio de-/encoding,and a combi-nation of these tasks.For this purpose,the HiBRID-SoC integrates three fully programmable processors cores and various interfaces onto a single chip,all tied to a64-Bit AMBA AHB bus.The processor cores are individually opti-mized to the particular computational characteristics of dif-ferent applicationfields,complementing each other to de-liver high performance levels with highflexibility at reduced system cost.The HiBRID-SoC is fabricated in a0.18µm 6LM standard-cell technology,occupies about82mm2,and operates at145MHz.1.IntroductionThe tremendous progress in VLSI technology allows the integration of an ever-increasing number of transistors on a single chip.Likewise,continuous improvements in al-gorithm research enable increasingly sophisticated multi-media applications,demanding a steadily rising amount of processing power.Due to the high innovation rate in this field,the development and standardisation process of multi-media applications is characterized by rapid changes in the algorithms and tools used.One example is the MPEG-4 video coding standard[1]of the Moving Picture Experts Group(MPEG),which has been introduced in1999with the Simple Profile,followed by—among others—the Ad-vanced Simple Profile(ASP)in2001,and its successor, MPEG-4part10or Advanced Video Coding(A VC),sched-uled for2003.In total,more than10video profiles have been defined since1999within MPEG-4for various appli-cation targets employing numerous algorithm options[2].While the technological progress generally offers the po-tential to keep pace with the growing processing demands, the suitability of an implementation for advanced multime-dia processing is determined by the architectural concept employed,i.e.,how the transistors are actually spent on the chip.In the era of system-on-chip(SoC),multiple process-ing units can be integrated together with an extensive choice of interface modules on a single chip.In order to meet the demands of multimedia signal processing applications, however,an SoC must provide,in addition to a high level of arithmetic processing power,a sufficient degree offlexibil-ity,integrate a powerful on-chip communication structure, and employ a well-balanced memory system to account for the growing amount of data to be handled when targeting higher-quality applications,e.g.,in the area of video.Existing approaches are either narrowly focused on a specific set of algorithm options,such as dedicated chips for the MPEG-4Simple Profile[3],or consists of a very general processing core[4]without specialization towards particular properties of the targeted algorithm class,poten-tially lacking processing performance for schemes with spe-cial processing demands.An extension of a programmable core with dedicated modules,as,e.g.,in the Trimedia[5], does not help when the functions that have been hard-wired change in a new version of a multimedia standard.The HiBRID-SoC multi-core system-on-chip architec-ture,developed at the University of Hannover,combines high processing power for multimedia schemes with high flexibility due to full software programmability.With three specifically adapted programmable cores on a single chip, various on-chip memory modules,and a64-Bit AMBA AHB system bus,the HiBRID-SoC provides a versatile so-lution for stationary or mobile multimedia applications such as MPEG-4video up to full TV resolution,surveillance ap-plications including object recognition and coding,or on-board data processing for reconnaissance applications.For future developments,the HiBRID-SoC serves as an exten-sible platform where components may easily be exchanged for customization towards particular applicationfields.In the following section,the architecture of the program-mable multi-core SoC is presented in more detail,includ-ing an overview of the three programmable cores and their software development environment.Section3describes the design and test methodology employed.Implementation re-sults of the chip and performance results for typical appli-cations are presented in Section4,and Section5concludes the paper.2.HiBRID-SoC Architecture OverviewThe main components of the HiBRID-SoC,the three programmable cores,are each adapted towards a specific class of algorithms.The combination of the cores in an SoC design methodology has placed particular demands on the architectural concept of the chip as well as on the software development environment.2.1.Multi-Core SoC ArchitectureWith the rapid development cycles in today’s multimedia algorithm research,programmability is a key requirement for a versatile platform designed to follow new generations of multimedia applications and standards.With program-mable cores,several different algorithms can be executed on the same hardware,and the functionality of a specific system can be easily upgraded by a change in software.Multi-core SoCs are attractive candidate architectures for multimedia processing as multimedia schemes in gen-eral can be partitioned in stream oriented,block oriented, and DSP oriented functions,which can all be processed in parallel on different cores.Each core can be adapted to-wards a specific class of algorithms,and individual tasks can be mapped efficiently to the most suitable core.In general,different approaches exists to accelerate exe-cution on programmable processors.In most cases,some kind of parallelization technique is employed on instruc-tion level(e.g.,very long instruction word,VLIW),data level(e.g.,single instruction multiple data,SIMD),or on task level(e.g.,simultaneous multithreading).Another very powerful means to accelerate multimedia processing is to adapt programmable processors to specific algorithms by introducing specialized instructions for frequently occur-ring operations of higher complexity[6].The HiBRID-SoC multi-core architecture,shown in Fig-ure1,comprises three programmable cores that have each been specifically optimized towards a particular class of algorithms by employing different architectural strategies. The HiPAR-DSP is a16-datapath SIMD processor core controlled by a four-issue VLIW and is particularly opti-mized towards high-throughput two-dimensional DSP-style processing,such as FFT-intensive applications orfiltering. The second core,the Stream Processor(SP),consists of a scalar32-Bit RISC architecture that is more optimized to-wards control-dominated tasks such as bitstream processing or global system control with a particular focus on high-level language programmability.The Macroblock Proces-sor(MP),finally,has been designed specifically for the effi-cient processing of data blocks or macroblocks that are typ-ical for many video coding schemes.It has a heterogeneous data path structure consisting of a scalar and a vector unit controlled by a dual-issue VLIW,offersflexible subword parallelism,and contains instruction set extensions for typ-ical video processing computation steps.Figure1.HiBRID-SoC multi-core architecture.A64-bit AMBA AHB system bus[7]connects all cores to off-chip SDRAM memory via a64-Bit SDRAM inter-face,to two versatile32-Bit host interfaces for access,e.g., to a host PC via PCI,and to serialflash memory for stand-alone applications.While the system bus operates at full in-ternal clock frequency,the SDRAM and host interfaces sup-port a programmable internal-to-external clock ratio in or-der to facilitate adaptation to various system environments. The direct exchange of data and control information be-tween the programmable cores without placing a burden on the system bus is supported by three dual-port shared mem-ories.Based on the standardized AMBA system bus,the ar-chitectural concept of the HiBRID-SoC follows a platform approach where individual cores may be exchanged for cus-tomization;another core may be added for additional func-tionality,or a core may be removed if not required for in-creased silicon efficiency.2.2.HiPAR-DSPThe HiPAR-DSP is a highly parallel DSP core with a VLIW-controlled SIMD architecture,which has been previ-ously developed at our institute[8].Figure2shows a block diagram of the HiPAR-DSP.The core consists of16identi-cal16-Bit data paths(DP0–DP15),each of which contains a local registerfile,an ALU,a shift/round unit and a16/36-Bit multiply/accumulate unit.A data path can process two 32-Bit instructions in parallel.Depending on local status flags,a data path can also ignore the current instruction(Au-tonomous SIMD).Each data path has its own local cache memory(LC)for autonomous random access of local data.2P2PFigure2.HiPAR-DSP architecture.A shared on-chip memory,called Matrix Memory,al-lows concurrent accesses of all data paths in matrix-shaped access patterns.This memory concept provides an easy data exchange between the data paths,which is required for manyfilter and image processing algorithms.The HiPAR-DSP is controlled by a global control unit, which has its own local registerfile and ALU.It is used for global control operations like address calculation,loops,or branches.Furthermore,the controller fetches the128-Bit VLIW from the instruction cache and broadcasts the arith-metic instructions to all data paths.An autonomously operating DMA unit serves all cache misses and performs data prefetch transfers to the ma-trix memory.The DMA unit is connected directly to the AMBA AHB bus and the dual-port memories connecting the HiPAR-DSP with the MP and the SP.At the targeted clock frequency of145MHz,the HiPAR-DSP achieves a performance of2.3GMACs,making it a versatile processor for all kind of applications with high nu-merical processing demands.2.3.Stream Processor(SP)The SP has been optimized for high-level programmabil-ity and efficient processing of control-driven applications.It has a Harvard architecture with a32-Bit data path consisting of5pipeline stages and controlled by32-Bit RISC instruc-tions.The SP supports conditional execution,forwarding, interlocks,and provides full interrupt capability.Independent caches are used for instructions and data, each4kByte large with4-way set associativity.They con-vert the64-Bit AMBA bus width to the32-Bit internal width.Both caches share a memory protection unit which can distinguish up to eight regions in the address space.The data cache also performs the memory mapping for the ac-cess of the inter-core dual-port memories.2.4.Macroblock Processor(MP)The MP has a heterogeneous data path structure consist-ing of a scalar and a vector data path,as shown in Fig-ure3.The scalar data path operates on32-Bit data words in a32-entry registerfile and provides control instructions such as jump,branch,and loop.The vector data path is equipped with a64-entry registerfile of64-Bit width.Spe-cial function units(SFU)provide instruction set extensions for common video and multimedia core algorithms.The 64-Bit-wide arithmetic execution units in the vector path, e.g.,MUL/MAC or ALU,incorporate SIMD-style subword parallelism by processing either two32-Bit,four16-Bit,or eight8-Bit data entities in parallel within a64-Bit register operand.The MUL/MAC unit even delivers a128-Bit result by writing back two64-Bit registers,thus preserving the full precision of the computed result.This way four16-Bit mul-tiplications with32-Bit accumulation can be performed in parallel.Originally,SIMD-style processing of data types packed into a single word assumes the same operation to be ap-plied uniformly to all data items.Whenever subword data items are to be treated differently,program execution has to be serialized,resulting in a loss of performance.This lim-itation was removed by the introduction offield-wise con-ditional execution in the MP which allows to sustain the SIMD scheme even for data-dependent processing.With its powerful andflexible support for subword parallelism,the vector data path is particularly suited to process the repet-itive operations of typical macroblock algorithms at high throughput.The MP’s parallel data paths are controlled by a dual-issue64-Bit VLIW.By default,thefirst slot’s instruction is issued to the vector path,and the second one is issued to the scalar path,enforcing parallel execution.However,also two vector or two scalar instructions can be paired within a VLIW.With four read and two write ports on the vec-tor registerfile,even two vector instructions can execute in parallel provided they do not belong to the same instruction group(i.e.,are not executed on the same hardware unit).If parallelization is not possible,the instruction decoder au-Scalar Data Path Vector Data PathFigure3.Macroblock processor data paths. tonomously serializes the execution of instructions.The flexible utilization of the VLIW minimizes the number of void instruction slots and promotes code density.Instructions and data are supplied to the MP via local memories,which are accessible within a single clock cycle. Transfers between external memory and local memories are performed in the background by the AMBA bus controller through programmed DMA as the program execution con-tinues.For the core algorithms,scalar and vector path can operate in concert to exchange blocks of video data between the local data memory and the vector registerfile without stall cycles by performing stores and loads concurrently. 2.5.Software Development EnvironmentThe processor cores provide various special architectural features.Especially the HiPAR and the MP use data paral-lelism via SIMD and splittable ALU,instruction parallelism via VLIW,and special instructions optimized for certain video and image processing algorithms.For each core,opti-mizing assemblers are available that support these features and perform code reordering,instruction scheduling,and VLIW parallelization.Based on the assemblers,C and C++ compilers have been developed for each core.The com-piler for the MP is mainly used for application controlling, whereas the most computation-intensive core algorithms are typically written in assembly language for full exploitation of all data path features.The HiPAR-DSP compiler has been extended to support matrix data types for SIMD and Matrix Memory access programming.The compiler for the SP supports the full set of features provided by the architec-ture.During the architecture definition phase of the proces-sor cores,cycle-accurate simulators have already been de-veloped and utilized for the optimization of architectural features and instruction sets.These simulators continue to be used for application development.While at the current stage the application software for each core is developed and simulated separately,integration of all simulators into a unified SoC simulator is under way.The unified simula-tion and development environment will then allow a precise simulation and profiling of AMBA bus traffic and inter-core data exchange in addition to the cycle-accurate simulation of each core.3.Design and Test Methodology3.1.Design and Verification FlowThe HiBRID-SoC features an IP-core-based design ap-proach.Modularity and reusability have been major as-pects throughout the development process.The design has a single clock domain and is fully synchronous,includ-ing all on-chip memories.All IP blocks,i.e.,all proces-sor cores,I/O interfaces,AMBA arbiter,and BIST logic were described in VHDL.For simulation,Model Technol-ogy’s ModelSim was used.The design was synthesized us-ing Synopsys’Design Compiler.A hierarchical approach without top-level optimization achieved the best results.The HiPAR-DSP core is a modified version of the HiPAR-DSP16,which has been implemented in silicon be-fore[8].It has been adapted to support the required modu-larity,e.g.,by extending the capabilities of the DMA con-troller to access the AMBA bus and the dual-port memo-ries.The MP core is the successor of a previous design[9], augmented also by an AMBA bus interface.All remaining components were new developments.For verification purposes,key software components had already been developed using the available cycle-true sim-ulators.Various applications like,e.g.,MPEG-4video de-coding on the MP and SP core,FFT and motion estima-tion on the HiPAR-DSP,and a multi-tasking operating sys-tem for the SP were simulated with the VHDL implemen-tation,and the results were compared with simulator out-puts,with previous results(for the HiPAR-DSP),or visu-ally inspected(for the MPEG-4decoder).For the verifi-cation of the AMBA AHB bus,behavioral bus master and slave models with an extensive set of properties were em-ployed.Behavioral models for externalflash memory and SDRAM served as communication partners for the corre-sponding I/O interfaces.The host interfaces were imple-mented as C models and simulated using ModelSim’s for-eign language interface(FLI).This way,the HiBRID-SoC has been successfully verified in a complete system environ-ment.In addition,a Quickturn Mercury emulation system was employed for the verification of the SP and MP cores, which enabled to run larger applications and to achieve a higher verification coverage.3.2.Memory BIST and Scan TestThe HiBRID-SoC design contains10memory wrappers with a total number of57memory modules in single-,dual-, and quad-port configurations.Built-In Self Test(BIST)logic for the memories is provided within the wrapper mod-ules.Each RAM wrapper consists of a BIST controller,one or more memory block(s),memory bypass logic,a com-parator and several multiplexers(see Figure4).Figure4.Memory wrapper with BIST.In BIST mode(bist=1)the BIST controller provides memory addresses,write data,memory control signals and reference data for the comparator.Even if several mem-ory modules are grouped together in one wrapper,only one BIST controller is used.Multiplexers in front of the memories select system mode or BIST mode input signals according to the exter-nal bist control signal.Multiplexers behind the memories prevent the propagation of uninitialized memory content in scan mode.Instead,memory bypass logic(xor gates and flipflops)is used to observe the memory input signals and to provide test patterns at the read data output in scan mode. In BIST or system mode(scan=0)the memory module output data are transferred to the read data output bus and the comparator logic.In BIST mode,the comparator asserts a fail signal if wrong data come from the memory.A special wrapper module is used for negative edge clocked RAM blocks.The multiplexers behind the mem-ories are substituted by enableflipflops,which are disabled in scan mode.Additional observationflipflops are used in-stead of the bypass logic to avoid memory shadow effects.Each BIST controller performs a March C–test,modi-fied for word-oriented memories and read disturb fault de-tection(see Table1and also[10],[11]).To test a quad-port(r/r/w/w)registerfile,the BIST con-troller state machine for dual-port(rw/rw)memories is em-ployed and a read and write port pair of the memory is used as a virtual read/write port.In thefirst step,read port A/ write port A and read port B/write port B are paired,and in the second step,read port A/write port B and read port B/write port A are used as a virtual read/write port.All tests are performed with different data background patterns in order to detect intra-word coupling faults[12]. For a16-Bit wide memory,the following patterns are used: X”0000”,X”5555”,X”3333”,X”0F0F”,X”00FF”.The full march test operation sequence is applied for all data patterns.Test time is therefore not optimized,but theTable1.March Test Operation Sequences:for1-Port Memories for2-Port(rw/rw)Memories⇓(w0:n);⇑(r0:r0,w1:r0[a+1]);⇓(w0);⇑(r1:r1,r1:r1,w0:r1[a+1]);⇑(r0,w1);⇓(r0:r0,r0:r0,w1:r0[a-1]);⇑(r1,r1,w0);⇓(r1:r1,w0:r1[a-1]);⇓(r0,r0,w1);⇑(r0:r0,r0[a+1]:w1);⇓(r1,w0);⇑(r1:r1,r1[a+1]:w0);⇑(r0)⇓(r0:r0,r0[a-1]:w1);⇓(r1:r1,r1[a-1]:w0);⇑(r0:r0)memory BIST is designed to run at full internal clock fre-quency;one complete iteration takes less than2ms.A full scan test is implemented for the BIST logic as well as for the rest of the standard-cell logic within the chip. With32scan chains and1920automatically generated test vectors,a fault coverage of more than99%is achieved.4.Implementation ResultsThe HiBRID-SoC has been implemented in a0.18µm 6LM standard-cell CMOS technology and integrates about 14million transistors on chip.The layout of the HiBRID-SoC is shown in Figure5.In total,the HiBRID-SoC oc-cupies about82mm2,with more than half of the area con-sumed by the HiPAR-DSP and its memories.MP and SP core including memories account together for about30% of the area,and the rest is occupied by the dual-port mem-ories and interfaces.The chip operates at a frequency of 145MHz with an estimated power consumption of3.5W.First performance results on selected multimedia appli-cations are already available from existing software imple-mentations,due to the early availability of cycle-accurate simulators and—for the HiPAR-DSP—proven silicon of a previous version.On the MP/SP combination,an MPEG-4ASP decoder for full TV resolution(720×576,25Hz)at bit rates of1.5–3Mbit/s has been implemented.The per-formance requirements of the relevant decoder subtasks on the MP are given in Table2.In total,around120MHz are required on the MP to perform real-time MPEG-4ASP de-coding.This application scenario is well complemented by the HiPAR-DSP,which in addition offers the opportunity to implement sophisticated motion estimation algorithms for high-compression video encoding,segmentation algorithms for object detection and tracking,or complexfiltering oper-ations for custom pre-or postprocessing steps,making the HiBRID-SoC aflexible implementation platform for a wideFigure5.Chip layout of the HiBRID-SoC.Table2.MPEG-4ASP decoder performanceon MP and SP,720×576@25Hz,1.5–3Mbit/s:Motion Compensation100.4Reconstruction 5.3Other 4.0Overall119.7range of multimedia applications.5.ConclusionsThe HiBRID-SoC provides a powerful and versatile system-on-chip solution for various kinds of multimedia signal processing applications.With its three program-mable cores adapted to different classes of algorithms,dif-ferent applications can efficiently be mapped onto this ar-chitecture.The full software programmability of all three cores facilitates to keep pace with the rapid algorithm de-velopments in thisfield.The extensible platform approach of the HiBRID-Soc allows to exchange or add additional IP cores for further customization towards specific system environments.6.AcknowledgmentsThis work was funded by the Fraunhofer Gesellschaft under contract T/F31D/1A232/P1307.The authors would like to thank Micronas GmbH for tape-out assistance. References[1]ISO/IEC JTC1/SC29/WG11N4668,“Overview of theMPEG-4Standard,”Jeju,March2002.[2]S.Bauer,J.Kneip,T.Mlasko,B.Schmale,J.V ollmer,A.Hutter,and M.Berekovi´c,“The MPEG-4multi-media coding standard:Algorithms,architectures and applications,”J.VLSI Signal Processing Syst.,vol.23, pp.7–26,Oct.1999.[3]M.Takahashi,T.Nishikawa,M.Hamada,T.Takayanagi,et al.,“A60MHz240-mW MPEG-4videophone LSI with16-Mb embedded DRAM,”IEEE Journal Solid-State Circuits,vol.35,no.11, pp.1713–1721,Nov.2000.[4]N.Seshan,“High VelociTI Processing,”IEEE SignalProcessing Mag.,pp.86–101,March1998.[5]Philips,TriMedia TM-1300Media Processor DataBook,Sep.2000.[6]M.Berekovi´c,H.-J.Stolberg,M.B.Kulaczewski,P.Pirsch,et al.,“Instruction set extensions for MPEG-4video,”J.VLSI Signal Processing Syst.,vol.23, pp.27–50,Oct.1999.[7]ARM Ltd.(1999,May).AMBA Specification Rev.2.0.[Online].Available:[8]J.P.Wittenburg,W.Hinrichs,H.Lieske,H.Kloos,L.Friebe,P.Pirsch,“HiPAR-DSP—a scalable fam-ily of high performance DSP-cores,”Proc.of the13th Annual IEEE International ASIC/SOC Conference, pp.92–96,Sep.2000.[9]H.-J.Stolberg,M.Berekovi´c,P.Pirsch,H.Runge,etal.,“The M-PIRE MPEG-4codec DSP and its mac-roblock engine,”Proc.2000IEEE Int.Symp.Circuits Syst.,pp.II192–195,May2000.[10]A.J.van de Goor,S.Hamdioui,“Fault models andtests for two-port memories,”Proceedings of the16th IEEE VLSI Test Symposium,pp.401–410,1998. [11]S.Hamdioui,M.Rodgers, A.J.van de Goor, D.Eastwick,“March tests for realistic faults in two-port memories,”Records of the2000IEEE International Workshop on Memory Technology,Design and Test-ing,pp.73–78,2000.[12]A.J.van de Goor,I.B.S.Tlili,“March tests for word-oriented memories,”Proceedings Design,Automation and Test in Europe,pp.501–508,1998.。