Network Processor Architecture for Next Generation Packet Format

Network Processor Architecture for Next Generation Packet FormatAnkush Garg Department of Computer Science University of California.Davis Davis,California95616garg@Prantik Bhattacharyya Department of Computer Science University of California,Davis Davis,California95616 pbhattacharyya@ABSTRACTOver the past few years,network processors have become an important component in packet delivery over the internet with its packet processing rate deter-mining the bottleneck of usage of bandwidth of trans-mission medium.The architectures of network proces-sors have mainly focused on achieving a particular line rate for IPv4packets.As the usage of new format IPv6 increases,changes in architecture models become one of the primary concerns.In this paper,we propose to modify an existing network processor architecture to improve performance by reducing the overload of pre-processing a packet header.We also introduce the con-cept of cache for forwarding address lookup tofind the packet destination address to increase efficiency of the lookup system.1IntroductionOne of the most vital component of any network system is the path taken by a packet as it travels from host to destination machines.The current internet infrastructure is based on best-effort delivery options i.e.the setup is such that it will make its best effort to send the packet to its destination.The system does not work on the model of providing a guaranteed service delivery option i.e.it never promises to the host or to the destination machine that it would be able to deliver the packet with a hundred percent certainty.Such a particular model helps in building the components as there is always less demand on service providers to promise to customers and hence deliver the packet.With this model in focus,we have network processor in the path which helps in data transmission over the infrastructure.We have till now used the term‘data’instead of packet so as to give the overview of internet data delivery structure without referring to packet switching or any other technologies.From now on we will use the term packet as packet switching is the pre-dominant technology and all the components being built satisfy the demand of this technology.The main function of a network processor[1][2]is to accept packets from its incoming port and using the processors built inside it,‘route’these packets appropriately,i.e.forward these packets appropriately through the output ports. Thus,the architecture of the router becomes vital to the packet transmission rate.As we have more and more bandwidth available in the physical path, the performance of the router becomes a bottleneck to the packet forwarding rate[3].A lot of research has been done to build better router architectures[4] so that these routers are able to do justice to the bandwidth available otherwise.One of the primary focus of research has been analyzing the different 1architectures which can exhaustively use the packet formats to take the forwarding decisions.As the packet format changes from IPv4to IPv6, we observe that major change(s)in the header format is taking place.The change in these versions is not merely a four times up-gradation from32bits of IPv4addressing mode to128bits of IPv6addressing mode but a major change in the headerfields.In other words,we are talking of the packet formats and specially the header formats of each packet format that has been designed for the new version.As these transitions take place,one of the key challenges has been upgrading or installing new network processors in the data path so that new formats can be identified and any processing activity can be done on them accordingly.Current technology solutions has mainly focused on integrating IPv6packets with IPv4packets as the restricted deployment of the newer version has not provided companies enough motivation to deploy only IPv6focused network processors.As the upgraded versionfinds widespread popularity,one of the key research issue has thus become designing newer architecture models for network processors so that the various new characteristics of the IPv6can be properly used for higher performance rates.In this paper,we present our research on the pos-sible architectures for routers suitable for next gener-ation packet formats.The paper is organized as fol-lows.In Section2,we make a study of the existing network processors and their architectures.We also talk about how the current processing of IPv4is done in these network processors and how IPv6deployment on the current architectures may perform,thus stating our motivation of these research.In Section3,we give details about our model.In Section4we try to math-ematically analyse the performance enhancement that our model can provide and Section5deals with the performance evaluation of our proposed model.These are followed by conclusion and possible future work in Section6.2BackgroundIn this section we study why network processors are required and the architectures of existing network processors.We observe that due to the advent of optical fibres the data transfer rate across transmission lines has increased to the order of Gbps(Gigabit/second). According to Intel[5],a packet arrives every35ns for a10Gbps connection and every8ns for a40Gbps connection.Assuming a minimum packet size of40 bytes,from Sherwood et.al.[6]we observe that a line card operating at10Gbps can process upto32million packets every second.General Purpose Processors are incapable of handling data at such a high rates which has lead to the use of Network Processors(NP).An excellent starting place to understand what a network processor is given by Shah[1].Extending the knowledge we have gained,we can define a network processor as a processing unit targeted for networking application.In other words,a device with architectural features and/or special circuitry for packet work processors are quite similar to general purpose central processing unit that are used in different types of equipment and products.Below we discuss the architecture of a networkprocessor.Figure1.A General Network ProcessorA network processor has mainly a central process-ing unit where the data is processed.It also has two types of memory units,termed as the SRAM(Static Random Access Memory)and DRAM(Dynamic Random Access Memory).DRAM is generally used to store large forwarding table(s)while SRAM isFigure2.Multi Core Network Processorused to store the small incoming packets.A table look up is done to determine the next forwarding address and other instructions are executed and then the packets come out and are put into the output line. Figure1shows the environment in which a general Network Processor works.The core is equipped with special instructions suited to satisfy the demand of networking applications i.e.the processor is equipped with instructions to speed up packet processing. These instructions are generally optimized to yield better performance with regard to the specialized demands of packet transmissions over any network.Network processors have evolved over the time and most of the processors currently available have multiple cores with each core capable of running multiple threads.Figure2shows a high level diagram for this kind of a processor.We have based our work on Intel IXP2800processor which has multi core architecture capable of running threads and handling more than one packet at a time.A more detailed discussion of this processor is given later. The following paragraphs of the current section talks about the motivation for our project.Thus,we have NPs with special instructions to handle packet(s).Also,it is an interesting fact to note that in packet processing,its mainly the header of a packet which is processed and the data part of the packet is ignored for most of the computation. Thus,it becomes essential that we have NPs with architectures and instruction set suited specially for header formats.Lets take a look at the packet formats of IPv4and IPv6(shown in Figures3and4)to understand what adaptations in NPs are required to make them better suited to handle the new version of packet format.We also take a look on why such a revision in the packet format isrequired.Figure3.IPv4PacketFigure4.IPv6PacketThe IP address in IPv4is32bit long implying that the maximum number of distinct IP address that can be assigned is232.The phenomenal growth of computers and their connectivity to the internet has lead to an IP address exhaustion problem[7]. This inspired the research of a new addressing mode which can provide enough IP addresses to everyone asking for it and yet remain un-exhausted over the years.IPv6has come up as the solution and a large scale deployment of IPv6has already started.An IP address in IPv6is128bit long and thus can be used to represent2128addresses,a number so hugethat a possible exhaustion of available IP addresses is virtually impossible.As the packet header format changes,naturally it becomes important to look into other issues that may be effected due to this change. Correspondingly,Network Processors also need to adapt to these changes in near future.In the new header format of IPv6,the Network Processor needs to read the128bits of the IP ad-dress(es)present in the header and then it determines the next hop address for this packet from its forward-ing table.Also,observed are the facts that in IPv6,the header size isfixed(the size of the header is40bytes) which means a preprocessing is not required to sepa-rate the header and the data from the packet and also the fact that a number of instructions are not required to be carried out compared to the previous packet for-mat.For example,as thefield of checksum has been entirely removed from IPv6,the processor need not carry out the complex computation on the data bits and match them to the CRCfield.The packet size for IPv6is alsofixed implying that an incoming packet need not be broken down into more packets as was the case sometimes with IPv4.Thus,it may think of sep-arating the header and the data from the packet from the input line itself so that faster processing can be done.This provided us with the possibility of doing research on this topic.In the next sections,we de-scribe our proposed model and present our simulation results.3Proposed ModelWe decided to work on the Intel IXP2800 Network Processor as it has been widely used in industry.The Intel Product Brief[8]says that IXP 2800is based on the same model as the earlier IXP 1200version.Basically,the chip has one Xscale Core that runs at700Mhz and16Micro-engines each working at a frequency of1.4Ghz.The microengines are capable of handling multiple threads.The SRAM unit in IXP2800supports a800Mbytes/sec read and 800Mbytes/sec write.The SRAM interface hasaFigure5.Existing Model of Processing in IXP2800peak bandwidth of1.6GBytes/sec per channel using 200MHz SRAMs.In the existing model,as shown in Figure5,the incoming packetfirst comes to the Xscale Core.The core checks the version of the packet(IPv4or IPv6) and after doing the CRC Check and other operations forwards the packet information to a microengine that is free and puts the packet in SRAM.If the input buffer is full then the packet is dropped.The informa-tion provided to the microengine is the location in the SRAM where the packet is stored.The microengine operates on the packet andfinds that on which output line the packet needs to be sent after comparing the source and destination addresses of the packet with the data of the forwarding table stored in the DRAM. Also notable is the fact that the microengine has to get the source and destination address of the packet from the SRAM.This introduces a latency as many cycles are wasted in the process due to the slow speed of SRAM.The packet is then sent on the selected out-put line.3.1Reducing LatencyThe IPv6packet has afixed header size and it also need not do a CRC Check on the IPv6packet.This suggests that the Xscale Core can be replaced by a fast and less complex processing unit.As no extra checks are needed to be done on the packet,the onlyFigure6.Proposed Model of Processing in IXP2800work the core is left with is to direct the packets to the microengine that is free.As shown in Figure6,we propose to replace the core with another microengine operating at1.4Ghz.Now,the header of the packet is forwarded to a microengine and the rest of the data is stored in the SRAM.Along with the header,the mi-croengine also gets the location of the packet from the SRAM.As the microengine does not need to fetch the source and destination address from the SRAM it can start processing the header immediately reducing the idle time.Afterfinding the output line for a packet,the microengine fetches the rest of the packet data from the SRAM in the last step,reassembles it and sends it on the output line.Clearly,the microengine can start working on the packet without wasting any time thus reducing the latency.3.2Introducing CacheAnother important point,as noted by Hu et.al.[9], is that microengines have no caches in them.But an important observation made by Harai et.al.[10]is that40%of the packets follow the same path on the network.This is a very significant observation as the microengine has to read the forwarding table from the DRAM again and again.The DRAM is even slower than the SRAM implying that for each packet many cycles are wasted for the getting the data of the for-warding table.Thus,the lookup time can be reduced drastically by introducing a cache.As40%of the times the microengine needs the same data from the DRAM the hit rate can be taken to be0.4.This means that the cache can be incorporated to reduce the mem-ory access time.The architectural modifications can be summarized as follows:1.Modification1(M1)-Replace the Xscale Corewith a Microengine to exploit the change in the header from IPv4to IPv6.And forward the header to the microengine reducing the SRAM access time.2.Modification2(M2)-Provide the microengineswith a cache to exploit the inherent property of the internet that many of the packets follow a similar path.4Theoretical AnalysisIn this section,we try to compute a theoretical im-provement that might be achieved using the changes that we have proposed.Park et.al.in[11]say that an average program written for IXP2800executes around 654instructions.Out of those105instructions are ex-ecuted on the core and the rest on the microengine. Tan et.al.in[12]note that both the core and the mi-croengine are have6pipeline stages.Again Tan et. al in[13]say that around20%of the instructions im-plemented on microengine are memory accesses with a latency of54microengine clock cycles.Thus,we can compute the time taken by the processor for one packet as follows:time old=(105∗6/700∗106)+((80/100∗549∗6)/1.4∗109)+((20/100∗549∗54)/1.4∗109)=7.017∗10−6sIn the above equation thefirst term is for the core, the second for microengine instructions which are not memory accesses and third for the memory accesses for the microengine.The average packet size over the internet is550bytes and as there are16microenginesin IXP2800,we can compute the throughput to be:throughput old=1/time old∗550∗8∗16bits/s=1.0032∗1010bits/s=10.032GbpsWe can see the original IXP2800was able to achieve a line speed of10Gbps+.Now,we will compute the throughput for the M1(modification1).The only change will occur in thefirst term of the time equation. Thus,the time for the processor with a microengine in place of the core can be computed as follows: time M1=(105∗6/1.4∗109)+((80/100∗549∗6)/1.4∗109)+((20/100∗549∗54)/1.4∗109)=6.567∗10−6sThis gives a throughput ofthroughput M1=1/time M1∗550∗8∗16bits/s=1.0719∗1010bits/s=10.719GbpsThus,from the introduction of M1a performance en-hancement of throughput M1/throughput old=1.07 can be achieved.Now,if we also introduce M2then the memory access term for the time will be modified. As,we noted earlier that around40%of the packets follow a similar path on the network.Thus,for40% of the times there will be a cache hit thereby reducing the latency from54to6for those instructions.The time equation can be modified to:time M1+M2=(105∗6/1.4∗109)+((80/100∗549∗6)/1.4∗109)+((20/100∗549∗54∗0.6)/1.4∗109)+((20/100∗549∗6∗0.4)/1.4∗109)=5.0616∗10−6s The throughput for this machine can be found out to be:throughput M1+M2=1/time M1+M2∗550∗8∗16bits/s=1.3908∗1010bits/s=13.908GbpsThus,the total enhancement that can be achieved by introducing both M1and M2is throughput M1+M2/throughput old=1.39.This means that our model can in theory give an ehance-ment of1.39over the original IXP2800.The following section will illustrate the simulation results that were obtained for the model proposed.5Simulation ResultsWe wrote a discrete event simulator to simulate the architecture.The IXP2800product brief[8]says that the size of of the input buffer is8192bytes and assuming an average packet size of550bytes we get the buffer size to be15.The Core checks the incoming packet and then directs that packet to the microengine which is free.Thus,the core has been modelled as an M/M/1queue with a buffer size of 15.The microengines have also been modelled asM/M/1queues.As soon as the packet is processed, the microengine becomes free and becomes ready for another packet to be sent to it.λOriginal M1M1+M20.660923360.82803812126352129690416312474381.2942287669316717841.4165943113461053214911.6229771919803107763491.82841874253579913020082332007130215771801040λOriginal M1M1+M20.64999697499988549999990.849859874993939499982614851553491844649962831.24528865466535149641111.44170293432695548392631.63851144400985346118281.83579071373210943490052333997234892204099489To simulate M1(modification1)the speed of the core was increased to make it comparable to that of a microengine.Thus,it was able to process the incom-ing packets faster.And to model M2(modification 2)on top of M1,the speed of the microengines was increased to account for lesser latency for the memory accesses.The incoming rate(λ)was changed in steps of0.2(starting from0.6)of the10Gbps packet rate (2272727.3packets/s).Thefirst table shows the num-ber of packets dropped for the original processor and the processors with the modifications proposed.The second table shows the number of packet processed by the processors.Thus,thefirst row of upper table tells that for an incoming packet rate of6Gbps the number of packets dropped by the three processors are609,233and6respectively.We can see from the data that the processor with both M1and M2in it performs better than the other two processors.Forλ=1.4the number of packets dropped for M1+M2are one order of magnitude lesser than for the other two processors.This shows that the results match with the theoretical analysis.Figure7shows the number of packetsdropped for the three processors.And Figure8showsthe num-ber of packets processed by the three processors.The blue line is for the theoretical limit of1.39that was proved in the previous section.It can be seen from the graphs that the number of packets dropped for the processors with both M1and M2are least and the ex-plosion in the number of packets dropped occurs close to the blue line showing that the simulation results areFigure7.Packets Dropped by different ProcessorsFigure8.Packets Processed by different Processorsnot very far off from the analytical results.6Conclusion And Future WorkWith the proposed microengine processor introduced in place of the core processor for pre-processing of packet headers,we observe that an improvement over the packet procesing rate can be achieved.Though the improvement seems small in terms of numerical data,the significant achievement will be in the cost of the processor as a processor with less complexity has been used.We further introduce the concept of caches as many of the packets from a particular source to a destination follow the same route to increase the total throughput of the network processor.A significant improvement in the process-ing rates has been observed due to this concept.One of the interesting future research work can be inspecting various policies for the cache replacement. This would require intensive analysis of how packetsflow between different network processors and setting up an appropriate policy.Also,an important part of our research focus has been using simple processing unit in place of the complex and costly Xscale core. We believe that a smaller and less complex proces-sor can be used in place of our proposed micro-engine model to achieve a better performance ratio as the sim-ple version of IPv6requires a small number of instruc-tions for it pre-processing.We leave this topic as part of future work.REFERENCES[1]Niraj Shah.Understanding network processors.UCB.[2]Patrick Crowley,Marc E.Fiuczynski,Jean-LoupBaer,and Brian N.Bershad.Characterizing pro-cessor architectures for programmable network interfaces.Proceedings of the2000International Conference on Supercomputing,2000.[3]Xiaoning Nie,Lajos Gazsi,Frank Engel,andGerhard Fettweis.A new network processor ar-chitecture for high-speed communications.In Proceedings of the IEEE Workshop on Signal Processing Systems(SIPS’99),1999.[4]Mel Tsai,Chidamber Kulkarni,Christian Sauer,Niraj Shah,and Kurt Keutzer.A benchmark-ing methodology for network processors.1st Network Processor Workshop,8th Int.Symp.on High Performance Computer Architectures (HPCA),2002.[5]Intel.Next generation network processor tech-nologies.Technical report,Network Processor Division,Intel Corporation,2001.[6]Timothy Sherwood,George Varghese,and BradCalder.A pipelined memory architecture for high throughput network processors.In Proceed-ings of30th Annual International Symposium on Computer Architecture(ISCA’03),2003. [7]Expert Research Team on Number Re-sources Utilization.Analysis and recom-mendations on the exhaustion of ipv4address space.Technical report,Japan NetworkInformation Center(JPNIC),2006.[8]Intel.Intel ixp2800network processor.Techni-cal report,2004.[9]Xianghui Hu,Xinan Tang,and Bei Hua.High-performance ipv6forwarding algorithm for multi-core and multithreaded network processor.In PPoPP’06:Proceedings of the eleventh ACM SIGPLAN symposium on Principles and practice of parallel programming,pages168–177,New York,NY,USA,2006.ACM.[10]Hiroaki Harai and Masayuki Murata.High-speed buffer management for40gb/s-based pho-tonic packet switches.IEEE/ACM w., 14(1):191–204,2006.[11]Jaehyung Park,Myoung Hee Jung,SujeongChang,Su il Choi,Min Young Chung,and Byung Jun Ahn.Performance evaluation of the flow-based router using intel ixp2800network processors.Workshop on Information Systems Information Technologies,2006.[12]Zhangxi Tan,Chuang Lin,Hao Yin,and Bo Li.Optimization and benchmark of cryptographic algorithms on network processors.IEEE Micro, 24(5):55–69,2004.[13]Yao Yue,Chuang Lin,and Zhangxi Tan.Npcryptbench:a cryptographic benchmark suite for network processors.SIGARCH Comput.Ar-chit.News,34(1):49–56,2006.[14]Intel.Intel ixp2855network processor.Techni-cal report,Intel.[15]Cheng Sheng,Zhang Xu,Cao Yingxin,andDing Wei.Implementation of10gigabit packet switching using ixp network processors.Interna-tional Conference on Communication Technol-ogy(ICCT’2003),2003.。