Resource Discovery in a dynamical grid based on Re-routing Tables

Resource Discovery in a dynamical grid based on Re-routing TablesKonstantinos I.Karaoglanoglou *,Helen D.KaratzaAristotle University of Thessaloniki,Department of Informatics,University Campus,Thessaloniki 54124,Greecea r t i c l e i n f o Article history:Received 17January 2008Received in revised form 1April 2008Accepted 10April 2008Available online 22April 2008Keywords:Grid Resource Discovery Re-routing Tables Dynamical grid Offline resourcea b s t r a c tThis paper studies the Resource Discovery problem in a dynamical grid based on a grid-router model.This model suggests that the grid can be seen as an environment comprisedby routers and resources,where each router is in charge of its local resources.We addressthe Grid Resource Discovery problem as a problem of discovering the appropriate resourcefor a specific request within that environment.Attempting to solve the Grid ResourceDiscovery problem,several mechanisms have been proposed in the past.One of thosemechanisms is the Routing Tables mechanism,which can guarantee finding the appropri-ate resource for a specific request within a static grid environment,where resources arepermanently online,connected in the grid.This paper investigates the effectiveness of aRouting Tables mechanism,called Re-routing Tables which can guarantee finding theappropriate resource in a dynamical grid environment,where resources could disconnectfrom the grid,and therefore get in an offline state.Due to an offline resource situation,the resource request must be re-routed in order to be satisfied.Ó2008Elsevier B.V.All rights reserved.1.IntroductionThe term ‘‘Grid”was introduced in early 1998with the publication of the book ‘‘The Grid:Blueprint for a new computing infrastructure”.Until then researchers and scientists referred to ‘‘the computing resources transparently available to the user via a networked environment”with the terms metasystem or metacomputer [23–25].The release of this book laid the groundwork of the field and provided definitions regarding that field.Since that time,many technological changes have occurred in computer science resulting to the evolution of the grid technology.Therefore,the need for clear definitions and descriptions about the grid technologies is now more than obvious,mainly because new aspects of the grid need to be included [1].In [2],the authors extracted characteristics and definitions from main grid literature sources in order to provide a clear and complete grid definition.According to the list of characteristics extracted from literature,a grid can be defined as ‘‘a large-scale,geographically distributed,hardware and software infrastructure composed of heterogeneous networked resources owned and shared by multiple administrative organizations which are coordinated to provide transparent,dependable,pervasive and consistent computing support to a wide range of applications.These applications can perform distributed computing [26,28],high throughput computing,on-demand computing [30],data-intensive computing [29],col-laborative computing or multimedia computing”.It is obvious,that the base of grid technology is the concept of resource sharing.The types of resources shared in a grid infrastructure could be desktop systems,clusters,storage devices and large data-sets.The question is what happens when a remote user requests access to a remote resource either to execute a job or to have access in the resource’s data?A 1569-190X/$-see front matter Ó2008Elsevier B.V.All rights reserved.doi:10.1016/j.simpat.2008.04.010*Corresponding author.Tel.:+302310886294.E-mail address:kkaraogl@csd.auth.gr (K.I.Karaoglanoglou).Simulation Modelling Practice and Theory 16(2008)704–720Contents lists available at ScienceDirectSimulation Modelling Practice and Theoryj o u r n a l h o m e p a g e :w w w.e l s e v i e r.c o m /l o c a t e /s i m p a tK.I.Karaoglanoglou,H.D.Karatza/Simulation Modelling Practice and Theory16(2008)704–720705 mechanism provided by the grid infrastructure should be available to discover an appropriate resource for a request.There-fore,one of the main capabilities a grid infrastructure needs to support is a Resource Discovery mechanism[3].Discovering a specific resource in traditional computing environments is relatively easy because the number of shared resources is small and all resources are under central control.In a grid environment,there are certain factors that make the resource discovery problem difficult to solve.These factors are:the huge number of resources,distributed ownership, heterogeneity of resources,resource failure,and resource evolution(upgrades changing a resource’s technical characteris-tics).An efficient Resource Discovery mechanism should take into consideration the above factors.Another important aspect in a grid infrastructure is the dynamicity of such a system.Resources,shared in a grid,exist in two states:online and offline.Not all resources are permanently online in a grid.A resource’s state could change during time from online to offline and backwards.The main reason that this happens is the distributed ownership of grid resources.An owner may establish a policy on a workstation that states that a foreign job can be run on a machine only at certain periods of time.Another reason for a resource to exit the grid,therefore get in an offline state,is due to heavy local load.An owner may establish a policy on a workstation that states that a foreign job can be run on a machine when the local load is under a certain limit.When the local load reaches that limit,a resource must exit the grid and execute local jobs only.Finally,the huge number of resources in a grid increases the probabilities of resource failures.Resource failures are unpredictable and common due to hardware faults,software faults or power outages.Taken into consideration the dynamicity that characterizes a grid system,we propose a new framework for resource dis-covery based on Routing Tables,called Re-routing Tables.Based on a large number of tests,we investigate the effectiveness of the Re-routing Tables mechanism which can guaranteefinding the appropriate resource in a dynamical grid environment, where resources could disconnect from the grid,and therefore get in an offline state.This paper is organized as follows:Section2presents related work on resource discovery in a grid environment.Section3 describes the Resource Discovery mechanism based on Routing Tables.Section4describes our deployed framework of the Re-routing Tables mechanism.Section5presents experiments and testing of the mechanism.2.Related workAttempting to solve the Grid Resource Discovery problem,several mechanisms and frameworks have been proposed dur-ing the past years.In this section,we present some popular approaches for the Grid Resource Discovery problem mentioned in bibliography.One of the popular approaches for the Grid Resource Discovery problem is the so called matchmaking one[4].The match-making framework was designed to solve real problems encountered in the deployment of Condor,a high throughput com-puting system.In bibliography,several other research papers make use of the matchmaking framework trying to add new aspects in the existing mechanism[5,12,15,16].According to this framework,all entities comprising a grid system are di-vided into two categories:requestors,and providers.All entities advertise their characteristics and requirements in classified advertisements.A matchmaking service is responsible offinding a match between the advertisements and informing the rel-evant entities of the match.The matched entities connect and cooperate for the service’s execution.In recent years,significant interest has focused on two systems:grids and peer-to-peer systems.Both systems share the same main idea.They are both resource-sharing environments.Their difference is that they have followed different evolu-tionary paths.grid systems are mainly used in complex scientific applications,while peer-to-peer systems are developed around mainstream services such asfile-sharing.Research papers,concerning thatfield,suggest the use of existing protocols developed for peer-to-peer systems into grid systems[6,7,14,18,21,27].Another notable approach to the Resource Discovery problem is the Semantic Communities one[8,11,13,17,19].Motiva-tion behind the Semantic Communities approach is that grid communities and human communities consist of members that are engaged in sharing and communication.Main target in this approach is to create grid communities based on similar-interests policies allowing community nodes to learn of each other without relying on a central meeting point.In addition to approaches that have been proposed in the past,many research papers suggest combinations of existing approaches.In bibliography,existing approaches,such as the matchmaking one,are combined with Semantic Web concepts. Using ontologies,asymmetric descriptions of resources and requests,and background knowledge combined with matchmak-ing rules attempt to solve the Resource Discovery problem[9,20,22].The Routing Tables mechanism,previously examined in[10],suggests a way of directing resource requests in a grid net-work.Based on a grid-router model,each router maintains Routing Tables in order to direct requests for certain resources in a grid system.In this paper,we extend the framework,creating the Re-routing Tables mechanism in a de-centralized man-ner,focusing on its effectiveness in a dynamical grid,where resources are not permanently online.3.Grid Resource Discovery based on Routing TablesBefore we start examining the Re-routing Tables mechanism,we have to demonstrate the Routing Tables mechanism in a static grid environment using the grid-router model.This model suggests that a grid system can be seen as an environment comprised by routers and resources.Each router is in charge of its local resources.Fig.1demonstrates a grid system com-prised by12routers,where each router controls a specific number of resources,3–5.We assume that there are20differenttypes of resources in the network.Each resource has unique technical characteristics,and therefore can satisfy a specific re-quest.The edges shown in Fig.1indicate a direct connection between the routers.For example,Router 1is directly con-nected to Router 2,Router 6,and Router 11and is in charge of Resource 1,Resource 3,and Resource 16.Assuming that at a point of time a request for a specific resource is created in one of the routers in the grid system,an effi-cient Resource Discovery mechanism should be able to find that specific resource in the network.In the simplest case,the rou-ter checks if one of its local resources meets the request’s requirements.If not,the router should forward the request to its neighbors randomly.This random forwarding could be sufficient in a small network,but would not satisfy a large network’s needs.The Routing Tables mechanism is used in order to forward the requests in the grid network in a non-random way.Each router in the grid network maintains a Routing Table with size equal to the number of different resources in the network.Each data element in that table is the minimum distance measured in hops from that router to all the resources available in the network (Fig.2presents such a table).Note that each edge in Fig.1stands for 1hop including the edges between a router and its localresources.Fig.1.Demonstration of a grid system based on the grid-router’sapproach.Fig.2.An example of a Routing Table available in a network router.706K.I.Karaoglanoglou,H.D.Karatza /Simulation Modelling Practice and Theory 16(2008)704–720K.I.Karaoglanoglou,H.D.Karatza/Simulation Modelling Practice and Theory16(2008)704–720707 Since the Routing Tables maintain information about the minimum distances from each router to all the resources in the network,it is more than obvious that a shortest distance algorithm plays a central role in the Routing Tables mechanism.We have deployed such an algorithm for our simulation needs and we present it later in the paper(Algorithm1).It is also clear that the Routing Tables mechanism requires a heavy computational initialization phase.In this phase,computations are made in order to discover the minimum distances from all routers to all the different resources in the system.Then the grid system is ready to satisfy resource requests based on the information included in the Tables.Fig.3is a closer view of Fig.1.Let’s examine the case when a request for Resource18is created in Router11.First,Router 11checks if the requested resource exists locally.Router11is in charge of Resources10,11,and17.Since the requested resource does not exist locally in Router11,the request has to be forwarded to Router’s11neighbors.Router11connects directly to Router1and Router6.After checking the Routing Tables of both routers(Figs.4and5),it is obvious that the re-quest will be forwarded to Router1based on the minimum distance information included in Router’s1Routing Table.The distance from Router1to Resource18is equal to2hops.On the contrary the distance from Router6to Resource18is3hops. As a result,the request is forwarded to Router1.Then the request is forwarded to Router2,where the requested Resource18 exists locally.All resource requests created in the network are dealt the same way.First,the router that requests a specific resource checks its local resources.If the resource is not found,the request is forwarded in the network based on the information in-cluded in the Routing Tables.It is guaranteed that the request will discover the resource in the minimum distance of hops since all the data elements inside the Routing Tables are provided by a minimum distance algorithm.As mentioned before,the Routing Tables mechanism assumes that the grid system is static,meaning that all resources are permanently online and cannot get in an offline state.An offline resource situation could result to unwanted side effects and system failures.We have deployed and tested a new framework for the Routing Tables mechanism called Re-routing Tables, which can manage effectively offline resource events.The Re-routing Tables mechanism is presented in the following section.Algorithm1A Minimum Distances Algorithm computing the shortest distances from all routers to all resources in the networkdo for all routescurrent router is thefirst list elementdo while the list is not empty or all routers are not markedi=first list elementextract i from listdo for all the arcs in the networkif the(i,j)arc exists and j is not marked thenj is markedinsert j in the listcalculate hop distance for j’s local resourcessave distances in current router’s Routing Tablesend_ifend_doend_doend_do4.Re-routing Tables4.1.Grid Resource Discovery based on Re-routing TablesWe have deployed a framework for Resource Discovery in a Dynamical grid,taken into consideration the fact that a grid system is characterized by resources that can get in an offline state at any time.An effective Grid Resource Discovery mech-anism should be able to overcome the dynamicity of such a system.The proposed mechanism called Re-routing Tables is based on the Routing Tables mechanism analyzed in previous section.The Re-routing Tables mechanism can effectively deal with offline resource events which are bound to happen in a grid system.In Fig.7,two alternative routes are discovered due to an offline resource event in Router2.A request for Resource8is created in Router11.Based on the information in the Routing Tables(shown in Fig.6)the request is forwarded to Router 1and then the request is forwarded to Router2,where Resource8exists locally.The problem is that Resource8in Router 2is currently offline and cannot satisfy the request.Due to that offline event,the request has to discover an alternative Resource8somewhere in the system so that the re-quest is satisfied.Such a resource type already exists in the system twice.Router9and Router5control Resource8locally.Calling the Minimum Distances Algorithm we compute the new minimum distance from Router 2to Resource 8(4hops).The distances of Resource 8from Router 2are 4hops for both Routers 5and 9.The problem is that there are still routers in the network who still think that Resource 8is online in Router 2and they will keep forwarding requests for that specific resource to Router 2.Even though an alternative resource exists in the system,routers do not have the new information in their Rout-ing Tables.Note that request for Resource 8in Router 11is yet to be satisfied.The next step is to update the Routing Tables in order to achieve the correct re-routing of the request in the network.We propose two solutions for the problem of updating the Routing Tables,and therefore creating the Re-routing Tables.4.1.1.Solution 1When the fact that a resource exited the grid is established,we can call the Minimum Distances Algorithm for all routers in the network in order to update the information in the Routing Tables regarding the offline resource.Of course this solution is computationally intensive especially in cases of large networks.Solution 2can reduce effectively the computational cost.4.1.2.Solution 2Examining the grid system in Fig.7,it is obvious that not all routers forward requests for Resource 8to Router 2.Therefore updating the Routing Tables for all routers is unnecessary.A proper solution is to find the routers that forward requests for Resource 8to Router 2and only update their Routing Tables creating their Re-routing Tables.From a total of 12routersinFig.3.Another view of Fig.1.Request for Resource18.Fig.5.Router’s 6RoutingTable.Fig.4.Router’s 1Routing Table.708K.I.Karaoglanoglou,H.D.Karatza /Simulation Modelling Practice and Theory 16(2008)704–720Fig.6only the following forward requests for Resource 8to Router 2:Router 1,Router 3,Router 6,Router 7and Router 11.So we only update the Routing Tables for these routers achieving to reduce the computational cost.All the other routers in the network can satisfy their requests with the alternative resources,of type Resource 8,existing in Router 5and Router 9.4.2.Updating Routing Tables-Creating Re-routing TablesOnce an offline resource event occurs in a router of the network,we have to determine which routers are going to update their Routing Tables as mentioned in previous section.The idea is to start from the router where the offline resource event occurred and visit all nodes in the network to examine which of them do forward requests for the offline resource tothat Fig.7.Two alternative routes are discovered due to an offline Resource 8in Router2.Fig.6.Routing Tables for all routers regarding Resource 8.K.I.Karaoglanoglou,H.D.Karatza /Simulation Modelling Practice and Theory 16(2008)704–720709710K.I.Karaoglanoglou,H.D.Karatza/Simulation Modelling Practice and Theory16(2008)704–720specific router.Counting hops from the offline resource router and checking the Routing Tables of the visited nodes will determine whether or not a router should update its Routing Table.If the calculated hops are equal to the distance in hops included in a router’s Routing Table for that offline resource then this router has to update its Routing Table creating its Re-routing Table.The algorithm below(Algorithm2)is the one we deployed in order to determine the routers that have to up-date their Routing Tables when an offline resource event is established.Algorithm2Updating Routing Tables-Creating Re-routing Tables after an offline resource event temp_router is the router where the offline resource event occurredtemp_router is thefirst list elementdo while the list is not empty or all routers are notmarkedi=first list elementextract i from listdo for all the arcs(i,j)in the networkif the(i,j)arc exists and is not marked thenj is markedcalculate hops so farif hops calculated are equal to the distance in j’s RoutingTables for the offline_resource thenRouter j’s Routing Table has to be updatedinsert j in the listcall Minimum Distances Algorithm for Router jend_ifend_ifend_doend_doBack to the example based on Fig.7,we have already stated that not all routers are going to update their Routing Tables. Executing the update algorithm will result to the list of routers that have to update their Routing Tables.Starting from Router 2,in which the offline resource event occurred and with the knowledge included in the Routing Tables shown in Fig.6we conclude to the following.The distance of Resource8in Router2from Router1is2hops,equal to the distance represented in Router’s1Routing Table.So Router1forwards requests for Resource8to Router2.The distance of Resource8in Router2 from Router3is2hops,equal to the distance represented in Router’s3Routing Table.So Router3forwards requests for Resource8to Router2.The distance of Resource8in Router2from Router4is3hops,not equal to the distance represented in Router’s4Routing Table(2hops).This means that Router4does not forward requests for Resource8to Router2and for-wards them to another router controlling locally a resource of that type.Following the above procedure for all the routers in the network,we conclude that Routers1,3,6,7,and11including Router2must update their Routing Tables in order to satisfy future requests for the offline Resource8.Calling the Minimum Distances Algorithm for these routers has as a result the creation of Re-routing Tables shown in Fig.8.Note that the distancesFig.8.Re-routing Tables for all routers after the update regarding Resource8.K.I.Karaoglanoglou,H.D.Karatza/Simulation Modelling Practice and Theory16(2008)704–720711 shown here are different for Routers1,2,3,6,7,and11from those in Fig.6because of the update.The distance for all the other routers is the same because they were unaffected from the departure of Resource8in Router2.Based on the information included in the Re-routing Tables,the request for Resource8created in Router11can now be satisfied.Router’s11neighbors are Routers1and6.Based on the Re-routing Tables,the request will be forwarded to Router 6since its distance(4hops)from Resource8is smaller than its distance from Router1(5hops).The request will then be forwarded to Router7and later to Router8.Finally the request reaches Router9,where Resource8exists locally.The request for Resource8in Router11is satisfied in5hops.Fig.9demonstrates the new Re-routing of the request due to the offline resource event in Router2.The total distance in hops for the Resource8request in Router11is7hops.The request required2hops to reach Router2 because Router11was unaware of the fact that Resource8has exited the system.The additional5hops came from the re-quest’s re-routing in order to reach Router9and its alternative Resource8.In our example with the creation of the request for Resource8in Router11,the Re-routing Tables mechanism executed the following steps:Based on the Routing Tables information the request was forwarded through Router1–Router2.In Router2the fact that the requested Resource8exited the system was established.Updating the Routing Tables of routers that sent their requests for Resource8to Router2caused the creation of the Re-routing Tables.Based on the Re-routing Tables the request was forwarded from Router11through Routers6,7,8to Router9where an alternative Resource8existed.To sum up,the Re-routing Tables mechanism deals with offline resource events executing the following procedure.First, the request is forwarded to the router that controls locally the requested resource based on the information included in the Routing Tables.If the forwarded request reaches the router that controls the desired resource,and the resource is currently offline,then we enter in the updating the Routing Tables phase.That phase only occurs to the routers that were forwarding requests for the specific offline resource.After the update,the request is forwarded based on the information included in the Re-routing Tables.The total distance in hops is the sum of the initial hops until to reach the router from where the resourcephase.exited and the re-routing hops after the update Array Fig.9.A re-routing event for Resource8due to an offline Resource8in Router2.712K.I.Karaoglanoglou,H.D.Karatza/Simulation Modelling Practice and Theory16(2008)704–7205.Testing:environment and experimentsWe have tested the Re-routing Tables mechanism in a number of experiments taken into consideration that a Resource Discovery mechanism should perform well in both small and large networks.Driven by this fact,simulations started in net-works of202routers and ended in networks of1002routers.We used the Grid Graph generator created by Resende[31]to produce networks for our simulation needs.Grid Graph is a user-friendly generator used to produce networks.It uses two basic parameters(h,w)and a seed value used to produce random numbers.The two basic parameters are the ones that determine the size and topology of the pro-duced network.For example,if the input parameters h,w are values2,5,respectively,the Grid Graph generator produces a 2Â5grid network with12nodes and17arcs.The Grid Graph generator’s result is depicted in an easy to read textfile (Fig.10).The result textfile provides information about the way the nodes connect in the produced network.Grid Graph generator produced the backbone of the networks,meaning the routers.After this we allocated a certain num-ber of resources to each router of the network.In all tests,we assume that we own20different types of resources.Each re-source has unique technical characteristics and therefore can satisfy a specific request.Each router of the network can control locally3–5resources.Basically in a202router network the total number of resources is approximately800.Note that a resource of a specific type could exist locally more than one time in the same router.The request for a specific resource is created randomly in one of the system’s router.During the simulation,requests for random resources are created in random routers.The number of total requests based on which we tested the Re-routing Ta-bles mechanism is1000.Simulations for all the cases of networks ended when1000requests for specific resources were satisfied.Independently of the creation of requests,the offline resource events happen in a random way also.At some point of time a single random resource in a random router gets in an offline state.In order to assess a clear view on the behavior of the Re-routing Tables mechanism,the offline resource events happen rapidly.From a total of1000requests for resources our goal was to have at least100re-routing events.In order to achieve the goal of at least100re-routing events,a random resource gets in the offline state every3time units of the simulation for the small size networks(202routers).For the medium size networks(402and602routers),an offline resource event happens every2time units of simulation.For the large size net-works(802and1002routers),an offline resource event happens more often,every1time unit of simulation.We have used the ability of the Grid Graph generator of producing different topologies for the same size of network.For a network of202routers,the Grid Graph generator can produce networks strongly connected,medium connected and simply connected.For every size of network presented in the paper,the Re-routing Tables mechanism was tested in four different topologies of that exact size.Due to space limitations not all results and tests are presented here.The results presented here are thefinal averages of the four different executions in four different topologies for each size of network.During the simulation,resources get in the offline state as mentioned previously.We used a limitation for the offline re-source events.Not all resources in a router can exit the system,meaning that a router without local resources would have no meaning.All routers start the simulation with a number of3–5resources.Resources during simulation time get in the offline state,but to avoid a router with no local resources we do not let such an event to happen.5.1.Hybrid Resource DiscoveryWe deployed another Resource Discovery mechanism called Hybrid Resource Discovery in order to compare the behavior of the Re-routing Tables mechanism.Hybrid Resource Discovery mechanism works in a semi-random way,based on a ran-localdom-walk approach.When a request for a specific resource is created in a router,the Hybrid mechanism checks in the Array Fig.10.A sample textfile depicting a network produced by the Grid Graph generator.。