Efficient retrieval of complex objects Query processing in a hybrid db and ir system

1 Introduction
A decade ago, a clear distinction between database (DB) systems and information retrieval (IR) systems could be based on the kinds of data stored in the systems as well as on the techniques employed to access them. While DB systems were appropriate for large amounts of structured factual data, IR systems were “normally used to handle bibliographic records and textual data”[SM, p. xi]. The document representations were unstructured strings (or semi-structured records). However, most systems based on the string matching approach show only a limited retrieval capability, especially when applied to full-text databases. As a consequence, alternative IR models employ not only different access methods, but also structured document representations, e.g. [CKT], thus approaching an integration of text and fact retrieval, e.g. [FH]. In the database field, a rising need to cope with unstructured data comes along with the development of multimedia information systems, where not all data can be structured directly, e.g. textual passages, images, sounds, etc. Complex documents, as well as multimedia documents, comprise both structured and unstructured data. Therefore, the structured and unstructured data unavoidably coexist in one information system. In this case, the appropriate DB/IR system must possess the capability to store, control and retrieve both types of data. It should provide efficient access methods based on exact
Abstract
Information retrieval systems are currently being developed to represent and manipulate complex multimedia objects. Such objects contain two types of components, i.e. structured components (e.g. of the type integer, real, fixed-length string, etc.), and unstructured components (e.g. text, images, sounds). Relational database systems (RDBS) are often used to store structured data and retrieve it via exact matching, while unstructured data is organized into inverted files, and accessed by Information storage and Retrieval Systems (IRS), utilizing indeterminate matching. The difficulty is how to fill the gap between the RDBs and inverted files, as well as the gap between the RDBMSs and IRSs based on inverted files. We describe an approach to integrating the two types of systems and the two different types of data.
In: G. Knorz; J. Krause; C.Womser-Hacker (eds.): Proc. of the 1st German Natl. Conf. on Information Retrieval (Regensburg, Sept. 19uctured factual data 1, as well as powerful facilities to handle the uncertainty inherent to searching for less structured data. The latter component should be able to perform a ranking of the retrieved items according to a relevance function. Information storage and Retrieval Systems (IRS), conventionally based on inverted files with an index mechanism, are widely used for processing unstructured (or semi-structured) bibliographic records and textual data. “Virtually all the commercially available systems are based on inverted file design” [SM]. Advanced techniques such as probabilistic model, document and term clustering, automatic indexing, etc, are employed in some prototypical IRSs. Such techniques are very useful in handling less structured data like text, but do not aim at handling structured data efficiently. Database management systems (DBMS), especially relational database management systems (RDBMS) are widely used to store, control and retrieve structured data. However, most RDBMSs lack the ability to handle less structured data like text, images, etc. Long fields, textual fields and even binary fields are featured now in some RDBMSs to enhance the ability to handle less structured data. But the retrieval function is still based on exact matching, i.e. the performance in retrieving relevant items is not sufficient. In sum, RDBMSs provide mature techniques for handling structured data 2, while in the area of IRSs probabilistic approaches provide appropriate means for accessing textual parts of complex documents. Since the problem of integrating DB and IR functions in one system is not only an academic topic, but is also considered important for applications, e.g. office systems, considerable efforts towards an integration were made [for a discussion of the related work in the db area cf. AH, HMS]. According to [HMS] five approaches to integrating IRS and RDBMS can be distinguished: The first approach is called the standard RDBMS approach. It employs standard RDBMS directly to manipulate both structured and unstructured data, even though most RDBMSs do not provide the appropriate functionality to represent and manipulate unstructured data [HMS]. The second extended RDBMS approach extends traditional RDBMS with new data types (e.g. text) that are suitable to represent the unstructured data and processing functions on those new types, e.g. Sybase3, ORACLE/TEXT. The third approach is to extend an IRS to include the relational model for the presentation and manipulation of structured data. The fourth is called the external integration approach, such that an IRS and a RDBMS are embedded in a common environment, e.g. by adding a common query interface. Finally, the full integration approach combines the functionalities of both RDBMS and IRS into a hybrid data management system. The standard RDBMS approach does not really feature functions for manipulating unstructured data. The extended RDBMS or IRS approach requires the augmentation of an existing RDBMS or IRS, which in most cases is a standard commercial system. However, the efficiency and the power of the extension is restricted, since it cannot be as efficient and powerful as an independent facility. In general, there will be an imbalance in the ability to store and manipulate different types