A New Data Model for Time Sequences

A New Data Model for Time SequencesDmitry V. Maslov11 Nayanova University, Samara, Russiadmaslov@industrialauto.ruAbstract. In this paper we analyze the requirements for logical modeling of or-dered data (sequences). We examine the shortcomings of existing models andpresent a new one. Our model inherits the advantages of different existing ap-proaches and has some new features. Its main contributions are: a) integrationof data quality information into the model; b) "transform" operator, allowing tochange the calendar of the sequence to any specified calendar; c) extended ag-gregation operators, that can use the ordering information.1 IntroductionData ordered over time (time sequences) can be found in many diverse financial, in-dustrial and scientific applications. That's why a lot of people face the problem of storage and preliminary processing of sequences in DBMS.Today relational data model is the most prevalent and universal. However rela-tional data model is not well suited for modeling of sequences. The most important reason for this is that a relation – the main structural element of relational model – is an unordered set of tuples. Consequently, although sequence data can be stored in relational database, it cannot be manipulated effectively. For example, it is very diffi-cult (nearly impossible) to query 5-day moving average of some stock price using relational algebra. It is also difficult to transform a sequence with granularity of one day to a sequence with granularity of one month, or to make a join of two sequences with different calendars. The problems of modeling sequence data in relational data-base are more carefully described in [8, 14]. In [14] professor Dennis Shasha formu-lates the requirements for financial time sequences model, in [1, 15] the requirements for industrial time sequences model are formulated.Because of mentioned problems a lot of specialized data models that deal with se-quences were proposed during last 15 years. However, as shown in section 3, all of them have shortcomings. One of the tasks of this research is to determine structural and functional requirements for time sequences data model (or, moreover, arbitrary sequences data model). We propose a new data model that inherits the advantages of all existing approaches. The main contributions of our model are:· integration of data quality information into a model: the domain of the function that is modeled by the sequence, the area where the sequence approximates the function and the quality (mistake of approximation);· extended transform operator;· extended aggregation operators.The rest of this paper is organized as follows: in section 2 the requirements for time sequences data model are described; in section 3 the advantages and shortcomings of existing data models are analyzed; section 4 presents a new data model; section 5 contains the conclusions and the direction for future work.2 Requirements for Time Sequences Data Model2.1 Structural RequirementsHigh Abstraction Level. The model should deal with as wide class of objects as possible.Independent (Separated from Data) Representation of Calendar. Calendar is an ordered set of time points at which the values of the sequence are defined. The model of sequences should have the structural element representing calendar with following functions [2]: a) defining time points at which time sequences have values; b) describing sets of time points (e.g., in queries); c) defining constraints; etc.Data Quality Information. Time sequence is a way to describe some external process (function). That's why the model should include the information about this function's domain and the quality of approximation.2.2 Functional RequirementsAlgebra of Sequences. According to [4], algebra of sequences doesn't have to include all statistical and other functions for time series analysis. There are a lot of specialized software packages with such capabilities. The algebra of sequences should include basic retrieving and transformation capabilities: "transform" operator, changing a calendar of a sequences to a new arbitrary calendar; temporal aggregation; moving window aggregation; calculations that use one or more sequences (possibly of different types). The algebra should be closed, i.e. the result of any operator should be a time sequence.Integration with Relational Data. The model should allow the co-existence of relational and sequence data in one query. All relational operators should be extended to deal with new structural elements of the model.3 Related Research WorkThe model, described in [6], is general for temporal objects (time sequences) and spatial objects. But this model doesn't have the structural element representing calen-dar, and doesn't pay attention to the algebra of sequences.The model for time series, proposed in [15], is a sort of valid-time temporal data model and doesn't address the problems of sequence data modeling.The model of Time Series Management System, presented in [3, 4, 5], contains ba-sic structural elements for modeling of time sequences: events, calendars, time series and groups of time series; and basic operations: calendar transformation, filtration, aggregation. However this is not a formal logical model, but the concrete implementa-tion of Time Series Management System. Moreover, the model doesn't have the ability to integrate relational and sequence data.In the model [2, 9, 10] the concepts of time sequence and time sequence collection are defined. The notion of the type of time sequence (i.e. the type of the function it describes) is presented. This model presents a general form for all operators manipu-lating time sequences, and some specific operators. It also defines the algebra of cal-endars. However this model doesn't include the operation of arbitrary calendar trans-formation. Another drawback of this model is that time sequence exists not as individ-ual object, but as a part of time sequence collection which is modeled as a relation.The SRQL query language [7] defines the minimal extension to the set of relational operators to formulate sequence queries over relational data. The advantage of such approach is the simplicity of implementation and the natural integration of relational and sequence data; the drawback is the limited functionality.SEQ model, described in [11, 12, 13], is the most general logical data model for sequences. It is designed for any kind of ordered data. However it doesn't include the operation of arbitrary calendar transformation (with many-to-many relation between source and destination calendars). Temporal aggregation ("g-collapse") doesn't pre-vent the ordering information inside the group (e.g. it is impossible to use aggregation function that selects n-th element in each group).To the best of our knowledge, no time sequence data model includes data quality information.4 A New Data Model for Time Sequences4.1 FundamentalsLet O be a set (finite or infinite), - full ordering relationship defined on this set.p Definition 1. {}j c =C , m j ,1= is a calendar if the following properties hold:·O c j j Î"|;·k j c c k j k j p Þ<"|,.An empty set Æ is also a calendar (empty calendar ).Calendar has dual nature. On one hand it is a succession, on the other hand it is aset. That's why standard set operations are applicable to the calendars.We define a sequence schema as a list of named attributes ,where each attribute A ),...,,(21n A A A A =i has the type T i , i.e. its values belongs to some set T i . The event of schema A is an element . Event contains the values of the sequence's attributes at some time point.n n T T T a a a a ´´´Î=...),...,,(2121Let T i * be the set of all non-empty subsets of T i . We define an approximate event ofschema A as an element , where each a **2*1**2*1*...),...,,(n n T T T a a a a ´´´Î=i * is some set of values of type T i , reflecting an approximate value of attribute A i .We say that a * is an approximation of a (), if *a a »*|,1i i a a n i Î=".Let be a set of all possible events and letbe a set of all possible approximateevents, where v n T T T E ´´´=...21{...**2null n v T T È´´´}{}*1*uncertain v T E È=null and v uncertain are some special values. Then each element of E * is either some approximate value or special value (v uncertain or v null ). The value v null de-notes the fact that the event is not defined at particular time point, v uncertain denotes the fact that the event is unknown at particular time point (and, moreover, it is unknown if the event is defined or not).Let be some function. For each element that belongs to its domainD it defines an event belonging to . Let a be a value of F at some point c ; and let aE O DF ®Í|n T T T ´´´...21* be an approximation of a . Then a * approximates the function F at point c . We assume that v null approximates any function at the points where the function is not defined.Now we are ready to define a time sequence as a collection of approximate events,corresponding to consecutive time points and approximating the functionE O DF ®Í|.Definition 2. A sequence H of schema A , approximating the function , is a 5-tuple , where{}j c C =E O D F ®Í|),,,,(.in M f D M V C · , m j ,1= is a calendar of the sequence;· {}j v V =, m j ,1= is a collection of values of the sequence, and the following properties hold:o ; *|E v j j Î"o if then approximates F at point с|j "uncertain j v v ¹j v j ;· is a set, where sequence H approximates the function F (which meansthat we know the points inside M , where the function F is defined, and usinginterpolation function f O M Íin we can calculate the approximation of F at any ofthese points);· is a set of the points inside M where F is defined (in generalcase we do not know such points outside M );M D D M Ç=·is an interpolation function that defines an ap-proximate event, approximating the function F , at each point**2*1...),,(n in T T T c V C f ´´´®C c D c M ÏÎ,. Let ℋ be the set of all possible sequences.So, a sequence for each time point of the calendar defines an approximate event.Depending on we can define different types of sequences. Let us note the most important of them:in M f D M F ,,,· Discrete – if , i.e. inside M the function F is defined only at the timepoints of calendar of the sequence. In this case the interpolation function is notused;C D M Í· Step-wise constant – if inside M the function F changes its value only at time points of calendar of the sequence;· Linear – if F is a numerical function and f in is a linear function, approximatingF with an error not greater than some e .Example 1. An example of discrete sequence is an information about some goods sales. To each time point of the calendar an information about a sale that took place at this time point corresponds.Example 2. Suppose some measurements periodically received from some sensor. Let d be the relative error of the sensor and e - the maximum absolute error of linear approximation. Then each value of the sequence is an interval (, where v is a measured value. The interpolation function of the sequence produces the intervals . The set M contains the interval of sensor's normal operation (the intervals of sensor's faults or system faults are excluded from M ). Outside M we cannot calculate the value of F using interpolation.),d d ×+×-v v v v ),(d n e d e ×++×--in in v v v Definition 3. Let R be a relational schema, A – sequence schema. Then an extended relation with schema (R.A) is a set of elements, consisting of a tuple of schema R and a sequence of schema A .Extended relation is an object providing the integration of relational and sequencedata in one model. In extended relation each tuple is extended with sequence, which is viewed not as an attribute of special type, but as an individual object with its own algebra. The paper [12] has already proposed a few operators for such objects. One of the future tasks of our research is to define the algebra for extended relations.Such approach to the integration of relational and sequence data has following ad-vantages:· The ability to add and remove sequences using DML-operations (that is im-possible if each sequence is modeled as a relation);· The ability to select several sequences by some criteria (that is impossible if each sequence is modeled as a relation);· The ability to combine relational and sequence data in one query, e.g. join se-quence and relation (which is impossible if the sequences are modeled as at-tributes of abstract data type);· In degenerate case (if sequence schema contains no attributes) extended rela-tion becomes classical relation, which means that proposed data model con-tains relational data model.Example 3. Suppose that some database contains an information about different companies' stocks sales. Such information can be modeled using extended relation. Each tuple of this extended relation contains general info about the company (relational part) and the history of stocks sales (sequence part).4.2 Algebra of SequencesCalendar Transformation Operator. Assume that the sequence , where , , approximates the function and let be some point. Then the value of H at point c is defined as follows:E O DF ®Í|ì=$|if ,c j v c ),,,,(in M f D M V C H ={}O c C j Í=m j ,1=O c Î)ïïîïïíÏÎÏ=cases other in the ),,,(,if ,if ,)(c V C f D c M c v M c v c H inM null uncertainj j Obviously, if , then approximates F at the point c.uncertain v c H ¹)((c H Let , O c C k Í=}'{'l k ,1= be some calendar. The calendar transformation opera-tor is designed to derive from H a sequence that has approximate values of F at the points of calendar C'.Definition 4. transform , where)',,,','(')',(in M def f D M V C H C H ==·, )}'({'k c H V =l k ,1=; ·, where j is such interpolation function transformation that H' approximates F inside M .)',('C H f in j = Operator j depends upon the implementation of the model. The simplest exampleof possible j is an operator that always return "undefined" interpolation function f undef ,which for all points, that don't belong to , returns (approximateevent that for each attribute contains the set of all its possible values).'C ),...,,(**2*1*n T T T a =Proposed calendar transformation operator has the following advantages:· The result of the operator is always a sequence (which, by definition, approxi-mates the function F );· Being a sequence, the result of the operator contains the information about the quality of approximation (the set of possible values of all attributes).Temporal Aggregation Operator. Like previous operator, temporal aggregation operator changes the calendar of the sequence. But, in contrast to "transform" operator, the result of temporal aggregation is a sequence that is result of computations that use the values of function F .Definition 5. Assume that the sequence , where ),,,,(in M f D M V C H =, , approximates the function . LetO' be some ordered set; y - some function. C' is used to denote C , E O D F ®Í|'{)('c C ==y '|O O ®l k ,1=m l £aggregatio , .)',',''in M f D ,,(H n y aggr {}O c C j Í=m j ,1='}O k Í,','(')_def M V C H func ==, where· , }'{'k v V =l k ,1=E , , whereis a calendar that consists of points such that y ;aggr_func | ℋ ® is an aggregation function that for input sequence de-fines an approximate event; )))'(,((_'1k k c H transform func aggr v -=y C c Î)'(1k c -y k c c ')(=*·; },')(,|''{'M c c c c C c M Î=$Î=y ·;},')(,|''{''M M D c c c c C c D Î=$Î=y · f'in is not used, because D'M' Ì C' (H' is discrete). The operator of temporal aggregation divides the input sequence into non-intersecting parts; to each part an aggregation function is applied. The input of aggre-gation function is a sequence (not a set), that is why it can use the ordering informa-tion. For example, aggregation function can select n-th element from each subse-quence, which is impossible with usual aggregation functions.Now let's define another, simplified, form of temporal aggregation operator. As-sume , and let be some calendar. We define the function y as fol-lows: y . Then O O =')('c C O C Î'min 'c or c =p 'C )'(',''c c c C c Î=)_,,()_,',('*func aggr H n aggregatio func aggr C H n aggregatio C defy =.Positional Operators. In the paper [12] a group of "positional" operators was defined: unary and binary positional operators, and moving window aggregation. Now we define positional operators for sequences defined in this model.Let and be the sets of all possible events with corresponding schemas, let G be some function,11211..n n T T T E ´´´=22221..k k T T T E ´´´=n E |k E ®k i T i ,1,2=®(G E g n i | - its coordinate functions. Let be someapproximate event. Then is defined as follows:***2),...,n n E a Î*1,(a a =*a )*a · Gnull null v v G =)(;uncertain uncertain v v =)(;·if then G , where ; )(|*G dom a a a λ"(|{)2*a g x T x a i i =Î=))(),..,(),(()(**2*1*a g a g a g a k =}*),(a a g i »·if then ; )(|*G dom a a a Ï»"null v a G =)(*· in the other cases .uncertain v a G =)(*Obviously, if a and G then .*a »uncertain v a ¹)(*)()(*a G a G »Definition 6. Assume that the sequence , where),,,,(in M f D M V C H =, , approximates the function . , whereD F Í|)',,('in M f C ({v G ',','D M V )}j ')(def H H G =='V =m ,1j ={}O c C j Í=m j ,1=E O ®·, ; ·; }))((|{'uncertain v c H G c M M =-=·; )}(),(|{)'(''G dom a c H a c M D D M M Ï»"-Ç=·, where j - is such interpolation function transformation that H' approximates G(F) inside M'.),('G H f in j =Let , , be the sets of all possible events with corresponding schemas, let esome function, 1121111..n n T T T E ´´´=2222122..n n T T T E ´´´=33231..k k T T T E ´´´=n n E E G ®´21k E | b k i T i n ,1,32=®***2*122),...,,(n n E b b b Î=**E E g n i |1´***2*111),...,,n n E a a Î*b ),(**b a G ** - its coordinate functions. Let, be some approximate events.Thenis defined as follows: *(a a =·if or , then G ; null v a =*null v b =*null v b a =),(*·if and b , or b and , then ; uncertain v a =uncertain v b a =),(***null v ¹uncertain v =null v a ¹*G *· if ", then, where;)(),(|,G dom b a b b a a λ"»),..,,(),,((),(**2**1**g b a g b a g b a =),,(|{),(3**a b a g x T x b a i i »=Î=**)),(**b a G k },**b b a g i »*·if , then G ; )(),(|,G dom b a b b a a Ï»"»"**null v b a =),(*· in the other cases .uncertain v b a G =),(***Obviously, if a , b , and G , then .*a »b »uncertain v b a ¹),(),(),(**b a G b a G »Definition 7. Assume that , where ),,,,(111111in M f D M V C H ={}O c j Í=C , , approximates the function ,, approximates the function . , where11n E O ®Í22n E O ®Í(2C H =,(1H H G 2in M,',',(')2def M V C H ==)},({'21j j v v G V =2|D F )','in M f 'D m j =,1m j ,1=1|D F ),,,,2222f D M V ·, ; ·; }))(),((|{)('2121uncertain v c H c H G c M M M =-Ç=·; )}(),(),(),(|{)'('2121'21G dom b a c H b c H a c M D D D M M M Ï»"»"-ÇÇ=·, where j - is such interpolation function transformation that H' approximates G inside M'. ),,('21G H H f in j =),(21F FDefinition 8. Assume that the sequence , where ),,,,(in M f D M V C H =, , approximates the function Letaggr_window(с) be some function that for each point defines a calendar insideO .O D F ®ÍO c Îdef {}O c C j Í=m j ,1=E |.')_,_,(_H func aggr window aggr H n aggregatio window =)',',',',(''in M f D M V C =, where , where H · , }'{'j v V =m j ,1=, ,where aggr_func | ℋ ® is an aggregation function that for input se-quence defines an approximate event;)))(_,((_'j j c window aggr H transform func aggr v =*E · Æ};¹ÇÎ=M c window aggr C c M )(_|{'·Æ};¹ÇÎ=M M D c window aggr C c D )(_|{''·f'in is not used, because D'M' Ì C' (H' is discrete). 5 Conclusions and Future WorkProposed data model is a general data model for all kinds of ordered data. Althoughwe call it "A Model for Time Sequences", we never used the assumption that the or-dering domain is time.Our data model integrates the data quality information. We have defined operatorsof flexible calendar transformation and temporal aggregation. Proposed algebra ofsequences is closed, which means we can use the output of any operator as the input ofanother operator, thus constructing complicated expressions.We are looking to the following areas for the further research:· Definition of algebra of extended relations and further development of se-quences algebra;· Development of optimization techniques for operators of sequences algebra; · Development of effective storage methods for sequence data.References1. Bonnet, P., Gehrke, J., Seshadri, P.: Towards Sensor Database Systems. Mobile Data Man-agement (2001) 3-142. Chandra, R., Segev, A.: Managing Temporal Financial Data in an Extensible Database.Proceedings of VLDB (1993) 302-3133. Dreyer, W., Kotz, A.D., Schmidt, D.: An Object-Oriented Data Model for a Time SeriesManagement System. Proceedings of SSDBM (1994) 186-1954. Dreyer, W., Kotz, A.D., Schmidt, D.: Research Perspectives for Time Series ManagementSystems. SIGMOD Record, Vol. 23, No. 1 (1994) 10-155. Dreyer, W., Kotz, A.D., Schmidt, D.: Using the CALANDA Time Series Management Sys-tem. Proceedings of ACM SIGMOD International Conference on Management of Data (1995) 4896. Erwig, M., Schneider, M., Gting, R.H.: Temporal and Spatio-Temporal Data Models andTheir Expressive Power. FernUniversitt Hagen, Informatik-Report 225 (1997) /erwig97temporal.html7. Ramakrishnan, R., Donjerkovic, D., Ranganathan, A., Beyer, K.S., Krishnaprasad, M.:SRQL: Sorted Relational Query Language. Proceedings of SSDBM (1998) 84-958. Schmidt, D., Dittrich, K.A., Dreyer, W., Marti, R..: Time Series, a Neglected Issue in Tem-poral Database Research?. Proceedings of the Int. Workshop on Temporal Databases (1995) 214-2329. Segev, A., Shoshani, A.: A Temporal Data Model Based on Time Sequences. In: TanselA.U. et al.: Temporal Databases – Theory, Design and Implementation. The Benja-min/Cummings Publ. Comp. (1993) 248 - 26910. Segev, A., Shoshani, A.: Logical Modeling of Temporal Data. Proceedings of ACMSIGMOD (1987) 454-46611. Seshadri, P.: Management of Sequence Data. Ph.D. Thesis. University of Wisconsin, CS-Dept (1996)12. Seshadri, P.: SEQ: A Model for Sequence Databases (1995)/2302.html13. Seshadri, P., Livny, M., Ramakrishnan, R.: The Design and Implementation of a SequenceDatabase System. Proceedings of VLDB (1996) 99-11014. Shasha, D.: Time Series in Finance: the array database approach./cs/faculty/shasha/papers/jagtalk.html15. Wolski, A., Kuha, J., Luukkanen, T., Pesonen, A.: Design of RapidBase - An Active Meas-urement Database System. IDEAS (2000) 75-82。