A New Architecture of Data Access Middleware under Grid Environment

A New Architecture of Data Access Middleware under Grid EnvironmentQingyang Wang1, Jingshu Chen1, Xibin Gao1, Wei Zhou1, Baoping Yan11 Computer Network Information Center, Chinese Academy of Sciences,Beijing 100080, China{qywang, gxbin, weizhou, ybp}@, cjs@AbstractData sharing is one of the most important research areas in data grid. Distributed data resource and heterogeneous data schema bring difficulties to data Access and sharing. This article mainly focuses on how to deal with the heterogeneity of data schema and put forward a blueprint to solve the data access and sharing problem of heterogeneous physical data resources in the grid. To solve this problem we propose the SDB resource model which extracts three data layers from the physical data resource in order to facilitate the access and sharing of data resource.1.IntroductionScientific Database (SDB) of Chinese Academy of Sciences is the collection of hundreds of autonomous databases maintained by nearly fifty institutes of different disciplines. These databases are characterized with distribution, multi-discipline and heterogeneity. With the development of technology and scientific research, the traditional data resource management can not meet the demands of up-to-date research work which needs data resource sharing and collaboration. Faced with tremendous amount of data resource stored in large-scale, distributed and heterogeneous databases, we have to figure out a method to access data resource effectively and then conduct comprehensive and deep data sharing in order to satisfy the requirement of scientific research in future.Data schema heterogeneity is one of the key obstacles to implement data integration [1]. The integration of databases with different data schema is just like the communication of people who come from different countries: people speaking different languages are hard to understand each other. We have already developed a grid middleware-DAS (Data Access System). This system not only use grid service to encapsulate database access, but also shield the heterogeneity of data schema and provides a good platform to data integration in future.This article mainly applies to relational database because most of data resources of SDB are stored in relational databases. Section 2 presents the problems of existing technology and OGSA-DAI [2]. Section 3 gives our solution-the architecture of DAS system and proposes our SDB Resource Model. In section 4 we give a detailed application of SDB Resource Model. Section 5 introduces the application of SDB Resource Model in our DAS system. Finally section 6 concludes the article by talking about some remaining problems and research issues need to be addressed in future.2.Analyze of Existing Technology andSystem2.1Analyze of Existing Integration MethodMost of the Data Integration Systems today are characterized by an architecture based on the media schema and a set of base schema [3]; the bases contain the real data while the media schema provides a virtual and reconciled view of the underlying bases. The real world data sources are autonomous and heterogeneous, and some of them are incomplete and ambiguous in metadata which cause problems to integrate them into mediated schema. Data cleaning and metadata enrichment should be taken before translating base schemas into mediated schema [4, 5]. Even this step has finished, there still likely remains some inconsistencies between mediate schema and base schemas. The query writer has knowledge of the attribute distribution and integrity constraints only in the mediated schema [6]. The tuples retrieved from the sources may violate the integrity constraints of the mediated schema.The universal relation [7] aims at achieving complete access path independence by requesting the system in an appropriate language, expecting the system to figure out the desired access path for itself. The concept of universal relation and various approaches to translate queries posed on the universal relation are described in [7]. A “universal relation” is an imaginary relation that represents all of the data in the databases. A query language that lets us refer to the universal relation rather than to actual database scheme can be much simpler than typical relational query languages, because we need to mention only attributes rather than attribute relation pairs.We find that providing users with the user with a universal relation, instead of a mediated schema, addressed the fundamental problem better. Since no integrity constraints are present in the universal relation, the query writer has to only specify the value of the “known attributes:” and ask for the “unknown attributes”. The tuple retrieved from the sources do not have to satisfy any integrity constraints at the mediated schema.2.2 The OGSA-DAI ProjectOGSA-DAI [2, 9] is a middleware product which supports the exposure of data resources, such as relational or XML databases, on to grids. The project was conceived by the UK Database Task Force and was established to provide a uniform way to access data resources in alignment with OGAS [8]. It enables client applications to submit request documents in order to perform a set of tasks on a remote data resource. Though DAI are widely used as a data access middleware, it still cannot satisfy users in the following aspects:z Data users are not easy to find their interested data resources among hundreds of thousands ofdata services through OGSA-DAI, if they do notknow the exact address of their interested dataservices.z A user can get metadata of a physical data resource through grid service, however, suchmetadata sometimes are hard to understand because they may be too complex or the amountof them is huge, or they need further explorationfor their ambiguity. Thus, data users usually needto spend a lot of time to understand the metadataand then make corresponding query activities. Our DAS tries to solve the above problems.3.Overview of Data Access SystemScientists need a convenient and powerful platform which provides a variety of customizable data applications based on SDB (Scientific Database). They can access data from SDB just as from a large-scale database system without knowing how and where data resources are stored. In addition, the data resources are highly distributed, autonomous, and dynamic, so the complexities of data resources should be made transparent to them.3.1 Architecture of DASThe figure 1 is the Architecture of DAS. It is a kind of Service Oriented Architecture (SOA) [12, 13], SOA consists of three primary components: the service provider provides the service, the service requester is the client that requires a service to be performed and the service agency provides registration and discovery of services. Dynamic service registration and discovery is an additional characteristic of SOA: a service requestor uses UDDI to discover registered service providers and does a dynamic binding to a found service provider.SDGFinder is a service search engine we developed. It is just like the role of UDDI of SOA, data users who are authorized can access SDGFinder through Internet Explorer. Every institute of CAS (Chinese Academy of Sciences) who wants to publish their data resources should at first register their data services into the database of SDGFinder and at the same time submit the metadata and additional description of the data to SDGFinder. If a data user want to find his (or her) interested data service, he (or she) can enter the key words of his (or her) interested data in the SDGFinder, then SDGFinder, the search engine, will return a list of descriptions of related data services and their exact addresses through which data users can access the dataservices.Figure 1: Architecture of DAS System3.2 SDB Resource ModelFigure 2: SDB Resource ModelThe bottom layer is the concrete data resource. It may be relational database, xml database or file system. The second layer is the physical data level storing the metadata which is extracted from the concrete data resource. These metadata are stored in xml files with uniform data format. After this step we can get the initially regulated, predetermined data format no matter how different the concrete data resources are. The third layer is virtual data layer. We also call it the virtual database layer. Virtual database indicates a special sub-database extracted fromphysical data layer according to logical relationship of relations in scientific database. The forth layer, universal relation layer, is user’s logical view of virtual database. In fact, the latter three layers are all exposed to data users according to their needs. So users who want to access scientific data resource will have the chance to choose one of the three layers when calling for data service.Figure 3 illustrates the process of translating the metadata of a physical database into a universalrelation schemaFigure 3. Schema Translation ProcessDBA should be involved during the translation process of data schema because he (or she) knows the detailed information of concrete physical database. We have developed the MappingBuilder, a software which helps DBA to do the mapping work between different levels.Local schema: Firstly we extract metadata information from physical database and then save the metadata information into an xml document according to predefined schema. This step can facilitate metadata cleanness and transformation into a more homogenous situation (both syntactically and semantically) in the next step. In addition, a physical database can provide service to different clients who may have different requirements, so we can define multiple virtual databases from a physical database according to different requirements.Global schema: The second step is to preprocess the metadata which is stored in local schema formed in the first step. Many design and construction of former databases are reasonless and irregular. Database construction institutes want the database to meet the new requirement of scientific research but at the same time do not change the database structure which may affect the current upper application system and research work. In order to reduce adverse effect of reasonless database design to the upper application, we should regulate the metadata in this layer. In addition, DBA can enrich metadata information to current metadata. As we know, many scientific database are constructed many years ago and do not have sufficient metadata information. Considering the autonomy of each database, we can supplement metadata information in upper layers such as global schema layer and universal relation layer without changing the semantic information of original database. With complete metadata information, users understand the database better and reduce the difficulty to access the database. After this preprocessing step, the structure of virtual database is more comprehensible and better suit for the generation of universal relation schema later.Universal relation schema: The third step is to translate the relations in global schema into a single relation in universal relation schema according to mapping rules. This universal relation will be exposed to end users as a logical view. A user just selects attributes of interest and filters the attributes by specifying the values for some of the attributes in the universal relation without denote the access path of underlying relations in database. So users can understand and access database easily. The detailed information of this step will be illustrated in Section 4.Physical data layer, virtual data layer and universal relation layer are different in the presentation form of data stored in physical data resources. Data users can access all these three levels according to their roles and needs. If a user is very familiar with the data in a physical data resource and has the corresponding right to access the data resource, he can choose access the physical data layer through data service, just as the OGSA-DAI does. On the other hand, if a user has little knowledge of a physical data resource, he’d better access the virtual data layer or universal relation layer.4. Application of SDB Resource Model4.1 Running ExampleFigure 4 presents a concrete database which contains four tables. The relationship of the tables is showed out in Figure 4. Relations CUSTOMERS, AGENTS, PRODUCTS and ORDERS are used by a wholesale business to keep track of its customers, the products it sells to these customers, and the agents who place orders for products on behalf of thesecustomersFigure 1: Running Database4.2 Constructing Virtual DatabaseWe can supplement metadata information to physical database during the process of translating local schema into global schema. For example, we can add commentary information to the attribute CUSTOMERS.discount, which point out that this attribute specify the discount of products the corresponding customer enjoys. It reflects the credit level of the customer. We can also change the name “discount” into “product_discount” in the global schema which makes the database users easily understand the meaning of this attribute.This step can enhance DBA‘s ability to control relations and attributes in a database. It is known that the data of scientific research has various confidentiality requirement, different users have different access permission, so it is necessary to conduct access control. In this step we can control the access permission of relations and attributes by judging whether a user has the right to access them according to the permission information stored in global schema. In a word, in this translation process DBA can clean and supplant metadata information of physical database and enhance the access control of database.4.3 Constructing Universal RelationA “universal relation” is an imaginary relation that represents all of the data in the database. A query language that lets us refer to the universal relation rather than to the actual database scheme can be much simpler than typical relational query languages, because we need to mention only attributes than attribute relation pairs [9]. We go for the attribute level mapping to create universal relation because of the following two benefits [7]: zFree the application programmer and the end user from the need to specify access paths(the so-called navigation problem)z Eliminate the need for program modification to accommodate changes in the database structure.So we suppose that the query translator rather than the query writer is in a better position to decide the access paths.In Table 1, we can see the difference between the three schemas, the attributes in four relations of global schema has been mapped in one relation in universal schema, and the names of attributes in local schema has changed when mapped in global schema in orderto make these attributes more comprehensible.Table 1: The CAP Database4.4 Query TranslationFigure 5: Query Translation ProcessFigure 5 shows the translating process of sql query. The data service engine will judge which data layer a user poses sql query on, for example, if the query is posed on universal relation layer, the sql query sentence must be translated into a query based on virtual data layer, and then the translated query should be translated into a new query again based on physical data layer, the eventual translated query will be sent to sql executor module to get query result.4. DAS SystemDAS system is developed above the Globus Toolkit 3.2 platform. Now it provides modes to publish physical date resource. The first one is to through grid service interface; the other is to through IE (internet explorer) to access the physical data resource, for example, a user choose the universal relation layer to access CAP database in IE mode, as is showed infigure 6Figure 6: The Demonstration of Universal Relation in CAP DatabaseThe SDB resource mode has been applied into our DAS system and fundamentally meets the users’ requirement. It also provides a relatively good foundation for data integration in future. The configuration tool of this system –Mapping builder,facilitates DBA to configure local schema, global schema and universal relation schema which can be changed according to new research requirement. When the data resource has changed slightly, DBA can merely adjust the mapping relationship between three schemas without affects the running situation of upper application system, realizing the transparence of underlying database.Secondly, now the metadata regulation work team of SDB has already established some standards and regulations of metadata in several specific disciplines, such as chemistry, biology, bioinformatics, astronomy and meteorology. These standards and regulations guide the construction of new databases and at the same time provide a uniform and regular data format facilitating data sharing among multiple databases. So we can consult these standards and regulations to define local schema, global schema and universal schema.Now DAS system has been deployed in 45 institutes of Chinese Academy of Sciences and still needs to be improved after we receive feedbacks from users.5.Conclusions and Future WorkThis article investigates the database schema of scientific databases. We designed a data access middleware under grid environment to solve the problem of data accessing and sharing among scientists. First, we presented an overview of the architecture of our Data Access System and the design of SDB resource model and the purpose of each data layer in SDB resource model. In addition, we concluded with a brief report of application of our DAS system into 45 institutes of Chinese Academy of Sciences.The key point of the current phase of DAS has been to develop a production-quality software framework in which query operations can be performed. The software enables applications to be built without any data access. In consequence, applications built using data service should be more flexible, in terms of what data can be discovered and accessed. The next phase of development, DAS, will focus on:z Improving the reliability, performance, and scalability of DAS.z Extending support to more software platforms.z Composition of higher-level services to establish reusable programming platforms and improvedmanageability.Acknowledgements: The research work is supported by the National High Technology Research and Development Program of China (863 Program) under Grant No. 2002AA104240 and is an application Grid of “High Performance Computer and its Key Software”, also known as “China National Grid”, a grand special project of the 863 Program in the 10th five-year plan; and is also supported by “The Scientific Database and its Application System” No. INF105SDB, an important project of CAS 10th five-year Informatization Program.References[1]M. Lenzerini. Data integration: a theoretical perspective.In Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, pages 233–246, 2002[2] M. Antonioletti, M. Atkinson, R. Baxter, A. Borley, N.P.Chue Hong, B. Collins, N. Hardman., A. Hume, A.Knox, M. Jackson, A. Krause, S. Laws, J. Magowan, N.W. Paton, D. Pearson, T. Sugden, P. Watson and M.Westhead. The design and implementation of Grid database services in OGSA-DAI. Concurrency and Computation: Practice and Experience 17(2): 357-376. [3] M. Lenzerini. Data integration: a theoreticalperspective. In Proceedings of the twenty-first ACMSIGMOD-SIGACT-SIGART symposium on Principles of database systems, pages 233–246, 2002.[4] M. Bouzeghoub and M. Lenzerini. Introduction to thespecial issue on data extraction, cleaning, and reconcilation. Information Systems, 26(8):535–536,2001.[5] H. Galhardas, D. Florescu, D. Shasha, and E. Simon. Anextensible framework for data cleaning. TechnicalReport, 3742, 1999.[6] Shreepadma Venugopalan, Krishna, P. T. Application ofUniversal Relation to Data Integration./~vshree/cs764/Windik.pdf[7] MAIER, D., ULLMAN, J. D., AND VARDI, M. Y. Onthe foundations of the universal relation model. ACMTrans. Database Syst. 9, 2 (June 1984), 283-308.[8] S. Tuecke, K. Czajkowski, I. Foster, J. Frey, S. Graham,C. Kesselman, T. Maquire, T. Sandholm,D. Snelling, P.Vanderpilt, Open Grid Services Infrastructure, Version1.0, June 27, 2003, GFD.15.[9] J. D. Ullman. Principles of Database and Knowledge-based systems Volume 2: The new technologies. 1988.[10] OGSA-DAI.（/）[11] GlobusToolkits.(/toolkit/)[12] T.Andrews et al: Specification of the Business ProcessExecution Language for Web Services Version 1.1; at:/bpel4s/[13]SOA. /software/solutions/soa/。