Agent Based Cloud Storage System

Agent Based Cloud Storage SystemProf. Dr. ABDEL-FATTAH HEGAZY *, Prof. Dr. AMR BADR**, MOHAMMED KASSAB** Department of Computer ScienceArab Academy for Science Technology and Maritime TransportEgyptE-mail: kassab.mohd@** Department of Computer ScienceCairo UniversityEgyptAbstract: - Cloud computing technology is envisioned as the next generation architecture of IT Enterprise. It is defined as a set of scalable data servers or chunk servers that provide computing and storage services to clients. The cloud storage is a relatively basic and widely applied service which can provide users with stable, massive data storage space. Our research shows that the architecture of current Cloud Computing System moves from central to distributed one; the reason for such movement is to avoid the bottle neck introduces since all data chunks must be indexed by a master index server. In this paper, we propose new cloud storage architecture based on P2P using agents. The system is based on a new architecture with better scalability, fault tolerance and enhanced performance.Keywords: - Cloud Computing, Architecture, Storage, P2P, Agents, Distributed Systems1IntroductionA cloud computing platform dynamically provisions servers as required. Servers in the cloud can be physical and/or virtual machines. Other computing resources such as network devices, storage area networks, firewall and other security devices are included as well. This paper will focus on the storage service provisioned by the cloud.Some typical cloud systems, such as GFS of Google[1], Elastic Cloud of Amazon[2], Blue Cloud of IBM[3], all have a similar central architecture for storage, with a central entity to index or manage the distributed data storage entities. It is effective to simplify the design and maintenance of the system by a central managed architecture, but the central entity definitely becomes a bottleneck due to the frequent visits. Although systems in practice have used some technique as backup recovery to avoid the probably disaster from the central bottle neck, the flaw from such architecture has not been resolved essentially. To overcome such bottleneck resulting from the central master entity used for indexing, another architecture based on P2P which provides a pure distributed data storage environment without any central entity management is introduced.In this paper, we propose a cloud computing architecture based on agents which provisions the benefits provided by the P2P architecture but with better performance. Rest of the paper is organized as follows, in section 2 we will introduce some related work about cloud storage system and P2P storage system. In section 3 of this paper, we describe a typical scenario to explain the architecture of our proposed cloud computing storage environment. In section 4, there is an introduction to the prototype used. In section 5, we illustrate our results. Section 6 is conclusion and proposal for future work.2Related WorksIn this section, we shall introduce some related work regarding cloud computing system and P2P storage architecture.2.1Google File System (Single Master)The first to give prominence to the term could compute (and maybe to coin it) was Google’s CEO Eric Schmidt, in late 2006[4]. Computing platform [5] which was first developed for the most important application of Google search service[6] and now has extended to other applications. Google cloud computing infrastructure has four systems which are independent of and closely linked to each other. They are Google File System for distributed file storage, MapReduce program model for parallel Google applications[6], Chubby for distributed lock mechanism[7] and BigTable for Google large-scale distributeddatabase[8].A GFS cluster consists of a single master and multiple chunk servers and is accessed by multiple clients, as shown in Fig. 1[1]. Files are divided into fixed-size chunks. Chunk servers store chunks on local disks as Linux files and read or write chunk data specified by a chunk handle and byte range. For reliability, each chunk is replicated on multiple chunk servers and hence the replicas must be synchronized in order to maintain consistency. The master maintains all file system metadata. This includes the namespace, access control information, the mapp.ing from files to chunks, and the current locations of chunk. When a client wants to visit some data on a chunk server, it will first send a request to the Master, and the master then replies with the corresponding chunk handle and locations of the replicas. The client then sends a request to one of the replicas and fetches the data wanted.[1]Fig. 1 Architecture of Google File SystemThe above architecture is clearly based on a master indexed storage system. The defect of such central based index architecture is that the GFS master becomes the bottle neck of the system since any request must be originated from the master index before being directed to the target chunk server, leaving such burden on the GFS master.2.2 P2P Storage SystemThe distributed P2P network indexed by DHT can resolve the problems of bottle neck resulting from central index system. Since the management is distributed equality to every peer in the network, there is no bottle neck any more, but with new alternatives, new problems arise. The new problem is how to maintain the consistency of the replicas in case of read/write operations.Some P2P systems for distributed storage have been developed now, such as Oasis[9], OM[10] and Sigma[11]. They keep the replica consistency in different ways and index the data resource by DHT. In the following section, we will discuss a cloud storage system based on P2P.2.2.2 Cloud Based on P2P ArchitectureThe architecture captioned in the next Fig. 2[12] is divided into three roles. The client application is responsible for the data request from the cloud. A gateway represents the entity which can direct the request/response between the client application and the cloud and lead the request to the nearest node on the network. Finally, the chunk server is the entity which is served as the data resource node and P2P node as well.Each chunk server has three function module interfaces. As illustrated in the Fig. below, an index module which is responsible for the global resource index which is assigned by DHT. Rout module, to pass a lookup request by a next hop routing table. Finally, the data or chunk module whichprovides the data resource stored on the local machine.Fig.2 Cloud Based on P2P ArchitectureFollowing Fig. 3[12] shows that in the index module, there is a chain containing the data index information pointers to all the data blocks with the same name ID will be linked in a sub-chain. A pointer contains the address of a data blockand the update version number of that block.[12]Fig. 3 Index Module Details3 Cloud Based on Agents OE-P2P 3.1 ArchitectureFig.4 Cloud Storage Based on Agent OE-P2P3.2 W ork FlowIn this section, we will present the system working flow with a typical scenario. Before a client can perform the work, data blocks and the corresponding replica should be uploaded to the Chunk Servers. Selection of the chunk servers for storage is the same with the traditional P2P cloud computing platform.1. Client sends a request for a data block with logicidentifier to the Gateway.2. Gateway analyze the request, parse the identifier of thedata block in the request, such as logic address, and change it to 128 bits logic ID by DHT algorithm which can be recognized by chunk servers’ agents on the OE-P2P network.3. Gateway will direct the search request data package toany of the sub-clouds based on the logic ID. In case the logic ID is even the search request data package will be directed to the nearest node on the even sub-cloud, otherwise the search request data package will be addressed to the nearest node on the odd sub-cloud. 4. Gateway constructs an OE-P2P search request datapackage including the logic ID, and sends the request to the chunk server’s agents OE-P2P network.5. The OE-P2P search request package routed among thechunk servers following the OE-P2P search protocol such as Chord[13], Can[13], Pastry[13], and Tapestry[13]. The chunk servers agents now act as a routing nodes of OE-P2P and the routing interface will be taken used of.6. The request reaches the server which contains the indexinformation of the logic ID in searching.7. The index includes all the pointers of the data replicawith the same ID. The chunk server agent now acts as an index server and the index function interface will play its role. The chunk server will select the latest pointer by its version number, if there are more than one candidate, the server should select the nearest node by comparing the IP address of the client and the data server, then return the best address to the client. 8. When the client gets the best address, it will then sendits request to the address of the chunk server which contains the data block. Now the chunk server acts as a data provider as the traditional cloud storage platform does.3.3 ReplicationIn our case the cloud provides storage as a service to users SaaS [14], a frequently met problem is write/read for mutual exclusion. In a central managed system as GFS, this can be resolved by using lock mechanism, but in distributedsystem, it will be more complicated. In this section, we willClient GatewayAgentChunk ServerAgentChunk Serverdiscuss the replica consistency control. Here is an example for writing consistency in our P2P cloud storage system: 1.Client finds the chunk server node which contains theindexed information of the target data block.2.Client tell the index node that it will do writeoperation.3.The index node check the chain of replicas state tosee whether there is another writing being processed.The state of the chain is lock or unlock. If the state ofthe pointer with the latest version is unlock, the indexnode will allow that write operation by returning thechunk server address of the latest version (if there aremultiple candidates, the index node should select thenearest node to the client by comparing the IPaddresses), and change the state to lock; If lock, theindex node will queue the requests until the statereverts to unlock or is timed out, the first writerequest in the queue then can be carried out.4.Client gets the address of the newest version,connects that chunk server, write the update data tothe block, then sends message to the index node tonotify that the write operation has finished.5.When the index node receives the finish message, theversion number of the pointer to the just modifiedblock will be incremented by 1. Then a procedure ofconsistency update to all the replicas will be start. 6.After a replica server finishes update, it will send anupdate response messages to the index node and theversion in its pointer in the chain will be increased bythe index node.7.When all the pointers in the chain have updated tothe newest version, the state of the chain will be setunlock. If the period of lock state is overtime, thestate of the chain will reset unlock by force and willbe state here will be timed out. When the chain hasset unlock, any delayed update response messageswill be discarded and the version of thecorresponding pointer in the chain will remainunaffected. This prevents the system from waitingindefinitely if there is something wrong with somereplica servers in case of update. Since the version ofthe data block on the delayed server is old, no clientwill visit that server for the old data later unless itsupdate is confirmed. An agent-scheduling routine[15]is taken in consideration to make sure that all replicasare to be of the same version and avoidinconsistencies. The procedure in carried out on aregular time interval where all replicas are updated tothe same version number.4 Prototype System We developed a prototype simulation system based on all three architectures discussed in this paper. The design is as follows: Operating system used is Windows XP. The simulator was developed using C#.Net. The number of chunk servers, clients, operation types (read/write) and number of operations are entered as parameters to the system.5 ResultsWe implemented all three architectures and we considered a LAN setting, where latencies on each node are taken in consideration in our simulation. Our simulation was fed with varying number of clients in order to test both response time and throughput. Our test environment was composed of 5 servers accommodating 50 files distributed randomly and the numbers of clients were entered as a parameter ranging as 10, 50, 100, 200 & 400 where all clients each accessed 10 files applying both read/write operations.Fig.7: Average response time during read operation In Fig. 7 we demonstrate that with increasing number of clients, OE-P2P shows lowest response time during read mode compared with P2P and Google as OE-P2P was free from the bottle neck from which Google suffered and smaller search space compared to P2P.NEW ASPECTS of APPLIED INFORMATICS, BIOMEDICAL ELECTRONICS & INFORMATICS and COMMUNICATIONS [9]Oasis:M. Rodrig, and A. Lamarca, “DecentralizedWeighted Voting for P2P Data Management,” in Proc.of the 3rd ACM International Workshop on DataEngineering for Wireless and Mobile Access, 2003,pp. 85–92.[10]OM:H. Yu. and A. Vahdat, “Consistent and AutomaticReplica Regeneration,” in Proc. of First Symposium onNetworked Systems Design and Implementation(NSDI '04), 2004.[11]Sigma:S. Lin, Q. Lian, M. Chen, and Z. Zhang, “Apractical distributed mutual exclusion protocol indynamic peer-to- peer systems,” in Proc. of 3rdInternational Workshop on Peer-to-Peer Systems(IPTPS’04), 2004.[12] Ke Xu 1, Meina Song 2, Xiaoqi Zhang 3, JundeSong4,"A Cloud Computing Platform Based on P2P"ITIME '09 IEEE International Symposium on IT inMedicine & Education, 2009. pp. 427-432[13] RafitIzhak-Ratzin, “Improving the BitTorrent ProtocolUsing Different Incentive Techniques”, University ofCalifornia, LA, Doctor of Philosophy in ComputerScience, 2010[14] A white paper produced by the Cloud Computing UseCase Discussion Group, 2009 “Cloud Computing UseCases”, Version 2.0,/group/cloud-computing-use-cases.[15]E. Sarhan, A. Ghalwash, M.Khafagy, “Agent BasedReplication for Scaling Back-End Databases ofDynamic Content Web Sites”, 12th WSEASInternational Conference on COMPUTERS,Heraklion, Greece, 2008. pp. 867-862。