System of KDD Tasks and Results within the STULONG Project

System of KDD Tasks and Results within theSTULONG ProjectPetr Dolejˇs´ı2,V´a clav L´ın1,2,Jan Rauch1,2and MichalˇSebek21Department of Information and Knowledge Engineering,University of Economics,Prague,W.Churchill Sq.4,13067Praha3,Czech Republic {xdolp15,xlinv05,rauch,xsebm08}@vse.cz2European Centre for Medical Informatics,Statistics and Epidemiology–Cardio University of Economics,Prague,W.Churchill Sq.4,13067Praha3,Czech RepublicAbstract.KDD results acquired on the Challenge data are shown.It isargumented that to gain an insight into the data,it is important to suit-ably structure the already discovered knowledge.The needs of knowledgereuse are further addressed.We propose structuring our WWW presen-tation of discovered knowledge in accord with hierarchical decompositionof KDD tasks within the project.Formal description of KDD tasks is at-tempted.Possible applications of the formal approach in management ofthe discovered knowledge are sketched.1IntroductionThe Challenge data were taken over from the STULONG–a research project3, currently underway at the European Center for Medical Informatics,Statistics and Epidemiology,Prague.We have been participating in its part concerning developement of techniques for KDD.We have been miningassociation rules with the4ft-Miner[7]procedure.As we have dealt with complex data,the number of various analyses that could have been performed on the data has also grown large.We have identified reuse of the discovered knowledge an important issue. To enhance possibilities of reuse of the discovered knowledge,we have structured our project’s WWW presentation in accord with the hierarchy of KDD tasks. We have also attempted to describe the hierarchy of KDD tasks formally.In section2are briefly described the analysed data and introduced several notions concerningassociation rule miningwith the4ft-Miner procedure.Section 3shows some of the results mined in the data.Particularly,section3.5discusses related problems and stresses the need of knowledge reuse.Section4describes how we had adressed this issue,i.e.how the presentation of obtained results is structured.Formalization of KDD tasks is attempted in section5.Section5.5in particular discusses possible utilization of such a formal approach in management of the discovered knowledge.Section6concludes the paper.3http://euromise.vse.cz/challenge/2Attribute Groups and4ft-Association Rules2.1Attribute GroupsWe have been analysinga data matrix containingresults of observation of244 attributes of entry examination of each patient.About1500male patients par-ticipated in the research.Attributes from entry examinations are divided into several groups:physical,social characteristisc,biochemical,alcohol consumption and other.Attributes from control examinations concern mainly changes in pa-tient’s behavior,social status,drugconsumption,etc.For research purposes,pa-tients(i.e.database objects)were divided into several(non-overlapping)groups such as”patients with cardiological risk”,”normal patients”,or”patients with pathologic symptoms”.2.24ft-Association Rules and Conditional Association RulesWe mine for association rules with a procedure called4ft-Miner[7].To distin-guish the4ft-Miner’s association rules from rules that are mined by other data miningsystems,we call our rules4ft-association rules.In this section,we will introduce only the most basic notions,see section5and references([1],[2],[6]) for more information.4ft-association rules are expressions of the formϕ≈ψ.In4ft-association rule ϕ≈ψ,ϕis called antecedent,andψis caled succedent.ϕandψare symbolic names of database objects’properties.Example of such a symbolic name could bediast1(75...85)∧height(170...180),standingfor the property of having the diastolic blood pressure in the range form 75to85and being from170to180cm high.Syntactically,ϕandψare formulae, see section5.1.We will say that a database object satisfies a formula if it has the property denoted by the formula.Intuitive meaningofϕ≈ψis thatϕandψare in a relation given by the sym-bol≈,which is called4ft-quantifier.There are many4ft-quantifiers,each of them correspondingto some kind of relationship in data.Each4ft-quantifier corre-sponds to some condition concerning4ft-table,see Tab1.Association ruleϕ≈ψis evaluated on the basis of the condition correspondingto the4ft-quantifier≈. In4ft-table,a is the number of objects from the analyzed data matrix satisfying bothϕandψ,b is the number of objects satisfyingϕand not satisfyingψ,etc.ψ¬ψϕa b¬ϕc dTab.1-4ft-table ofϕandψB y suitable conditions concerning4ft table can be expressed various types of dependencies ofϕandψ,i.e.there can be various4ft-quantifiers(see also section 5.2).We will introduce two of them.The quantifier ⇒p,s of founded implication [1]for 0<p ≤1and s >0is defined by the condition a a +b ≥p ∧a ≥s .It means that at least 100p per cent of objects satisfying(in the g iven data)ϕsatisfy also ψand that there are at least s objects satisfyingboth ϕand ψ.The above average quantifier 4⇒+p,s for 0<p and s >0is defined by thecondition a a +b ≥a +c a +b +c +d (1+p )∧a ≥s .It means that amongobjects satisfying ϕis the relative frequency of objects satisfying ψat least 100p per cent higher than the relative frequency of objects satisfying ψamong all the objects in database.Let us give two examples of 4ft-association rules from our application.Rulediast 1(80...85)∧weight (75...80)⇒0.5;20chlst (200...240)is interpreted as:”50%of patients with diastolic blood pressure in (80,85)and weight in (75,80),has also cholesterol in (200,240);there is 20of such patients in total.”.Rulediast 1(75...85)∧height (170...180)⇒+0.60;28chlst (180...200)is interpreted as:”Among patients satisfying diast 1(75...85)∧height (170...180),there are at least 60%more patients satisfying chlst (180...200),than there are patients sat-isfying chlst (180...200)in the whole data matrix;there is 28of such patients in total.”It is important to note that above average quantifier is symetric,i.e.the followinginterpretation is equivalent:”Among patients satisfying chlst (180...200);there are at least 60%more pa-tients satisfying diast 1(75...85)∧height (170...180),than there are patients satisfying diast 1(75...85)∧height (170...180)in the whole data matrix;there is 28of such patients in total.”.There are also conditional association rules of the form ϕ≈ψ/ξ,where ξis a symbolic name of a database objects’property.Intuitive meaningof such a rule is that ϕand ψare in relation ≈when formula ξis satisfied.Evaluation of conditional association rules is as described above,except for the fact that frequencies a ,b ,c and d are calculated only for objects satisfying ξ.3Examples of Discovered KnowledgeWe were examiningassociational and implicational relations between the g roup of biochemical attributes and the group of physical attributes.These examina-tions were done within the following patient groups:the normal group,the risk group,the pathologic group.We have used the founded implication quantifier and the above-average quantifier .In all the tasks (Task 1–4):–antecedents were derived from physical attributes:diastolic blood pressure (diast 1),systolic blood pressure (syst 1),body height and weight (height ,weight ),body mass index (bmi ),skinfold thickness at subscapular (subsc ),4See also http://lispminer.vse.cz/overview/4ft_quantifier.htmland skinfold thickness at triceps(tric).In Task4,also some aditional at-tributes were allowed,see3.4–succedents were derived from biochemical attributes:urine(moc),cholesterol (chlst),and triglicerides(trigl)–conditions were used to restrict the computations on the patient group con-cerned by the particular task.We will not display conditions in our example rules below,for typesettingreasons.We will show results from four KDD tasks.Discussion of the results follows in section3.5.3.1Task1First task concerned founded implications(with p>0.5∧s>20)between biochemical and physical attributes within the risk-patient group.There were6 rules found:–height(175...185)∧syst1(130...140)∧weight(80...90)⇒0.56;20chlst(230...270)–diast1(80...90)∧subsc(17,18)⇒0.52;23chlst(240...280)–syst1(120...130)⇒0.51;20chlst(240...280)–diast1(80...85)∧height(170...180)∧syst1(130...140)⇒0.51;21chlst(22...260)–syst1(135...145)⇒0.50;20chlst(220...260)–diast1(80...85)∧weight(75...80)⇒0.5;20chlst(200...240)3.2Task2Second task concerned founded implications(with p>0.6∧s>10)between biochemical and physical attributes within the pathologic-patient group.There was only1rule found:–diast1(85...95)⇒0.63;10chlst(240...280)3.3Task3Third task concerned above-average(with p>0.5∧s>20)relations between biochemical and physical attributes within the normal-patient group.There were 19rules found.We list only10strongest rules:–diast1(80...85)∧bmi(170...180)⇒+0.87;26chlst(180...200)–diast1(85...95)∧height(175...180)⇒+0.85;20chlst(210...240)–diast1(90...100)∧height(175...185)⇒+0.84;21chlst(210...240)–diast1(80...90)∧height(170...180)⇒+0.79;21chlst(190...200)–diast1(85...95)∧height(175...185)⇒+0.73;25chlst(210...240)–syst1(110...120)∧bmi(24...26)⇒+0.69;20chlst(210...250)–syst1(135...145)⇒+0.62;21chlst(200...220)–diast1(80...85)∧syst1(115...125)⇒+0.62;20chlst(160...200)–syst1(110...120)⇒+0.61;22chlst(230...250)–diast1(75...85)∧height(170...180)⇒+0.60;28chlst(180...200)3.4Task4Fourth task was a specialization of the third task in the sense that also non-physical attributes were allowed in antecedent.It were the following:leissure-time activities(aktpozam),alcohol consumption(alkohol),pain in the legs (boldk),and pain in chest(bolhr).There were23rules generated.We will list only the three most interestingrules.–diast1(80...85)∧height(170...180)∧boldk(1)⇒+0.95;24chlst(180...200)–syst1(110...120)∧bmi(24...26)∧boldk(1)⇒+0.83;20chlst(210...250)–diast1(90...100)∧height(175...185)∧boldk(1)⇒+0.81;20chlst(200...240)3.5DiscussionWhat are the questions we might ask after getting the results given in sections 3.1,3.2,3.3,and3.4?Apparently,blood pressure(systolic or diastolic)plays its role in almost all the discovered rules in all the tasks.Therefore,one might want to examine KDD tasks that would focus on blood pressure more closely.Task2 proved only one rule;one might want to know,whether this rule holds exactly only in the pathologic-patient group,or also in other groups.Very interesting are results from the fourth task.The discovered rules are very strong(p>0.80). Also,it appears that the pain-of-the-legs attribute(boldk)plays an important role in connection with cholesterol levels.Again,examining further tasks featurig the boldk attribute might be of interest.Another feature of the partial results is that there was no rule with moc or trigl in succedent generated.It would be interestingtofind out why.Although the questions rising from our examples might seem trivial to a do-main expert;they outline the problem we have been encounteringall throug h our data miningefforts.With complexity of g iven data,the number of interest-inganalyses that can be performed can rang e from several dozens to couple of hundreds.Apparently,the partial results give users only a very limited insight into the data.To gain a greater insight,one has to either perform new analyses,or check whether relevant analyses were not already performed.To gain such an insight, it is necessary to compare results of different analyses,consider value distribu-tions of attributes,sumarize the acquired results,etc.Since it would be grosslyinefficient to re-run already performed analyses,well structured presentation and good organization of discovered knowledge grows in importance,i.e retrieval and reuse of KDD results becomes an issue.4WWW Presentation of KDD Tasks and ResultsTo reuse,share and retrive discovered knowledge,we have been developing a WWW presentation of KDD results5.To faciliate navigation through pages de-scribingparticular analyses and discovered knowledg e,we have structured the presentation in accord with a hierarchical decomposition of KDD tasks within the project.KDD task is generally a verbal explication of some range of research prob-lems,which are to be examined in the given data.Indeed,KDD tasks differ in their generality.General tasks can be broken-up into more specific tasks,these into yet more specific and so on.As mentioned above,attributes in our data set are divided into several groups:physical attributes,biochemical attributes, etc.In our application,the most general KDD tasks concern relations between attribute groups(”Examine associational relations between physical and social characteristics”).On a more specific level,these KDD tasks are relativized-relations between given attribute groups are examined within separate patient’s groups(”Examine associational relations between physical and social characteristisc within the risk patient group”).Further,these relativized tasks are broken-up into examiningparticular types of relations correspondingto quantifier classes(see section5.2)-associations, imlications,double implications,and equivalences(”Examine implicational rela-tions between physical and social characteristisc within the risk patient group”). The last step is formulation of the most specific KDD tasks,like are those in section3.Thus gradually specifying KDD tasks,we arrive from the most general tasks to most specific ones.Structure of the WWW presentation reflects the hierarchy of KDD tasks.The whole presentation can be thought of as consisting of several tree-like structures.A root of one such structure is shown infigure1.It is a web page containing tri-angular table displaying all possible combinations of attribute groups,i.e.fields of the table correspond to the most general KDD tasks6.Internal nodes of the tree structure correspond to more specific KDD tasks and arcs(i.e.html links) correspond to gradual specialization of the tasks.Leaves of the tree correspond to particular analyses and discovered knowledge.Navigation through the system of web pages is easy and natural,and meets the objective of faciliatingknowledg e reuse.5http://euromise.vse.cz/,please note that the web site is still under developement 6E.g.thefield at the intersection of the row”smoking”and column”social”corre-sponds to the task of examining relations between the attribute group”smoking”and attribute group”social characteristics”.The”OK”sign means that the task has already been solvedFig.1.A table at root of the KDD task hierarchy within the STULONG project. 5Formalization of KDD TasksWe are going to describe the hierarchy of KDD tasks formally,i.e.we are going to define a simple formalism capable of describingKDD tasks,and we are g oingto define the specialization relation on KDD tasks.Since the discovered knowledge is itself formal(4ft-association rules),we will be able to describe formally every part of our WWW presentation.5.1FormulaeWe will define formulae as symbolic names of database object’s properties.A database object will satisfy a particular formula,if it will have the property denoted by the formula(these notions were informally used already in section 2.2).Let A be a database attribute,let rng(A)be the value range of the attribute A. Literal is an expression of the form A(α),whereα⊂rng(A)∧α=∅. Formulae7are defined as usually:–every literal A(α)is a formula–ifΦandΨare formulae,then¬Φ,Φ∧ΨandΦ∨Ψare also formulae–there are no other formulaeUsual conventions concerningparentheses apply.Symbols¬,∧and∨are com-mon propositional connectives of negation,conjunction and disjunction;formulae ¬Φ,Φ∧ΨandΦ∨Ψhave their obvious meanings.We say that literal A(α)is true for object o in the given database,if and only if the value of attribute A of the object o belongs to the setα.Given object o and formulaΨ,we will say that o satisfiesΨ,iffΨis true for o.It can be easily seen that every lit-eral A(α),α={a1,a2,...,a n},can be re-written with disjunction of literals A(a1)∨A(a2)∨...∨A(a n).Literals A(a i),a i∈rng(A),are called one-point literals.Expressionsϕ,ψandξof section2.2are obviously formulae–elementary con-junctions of literals8.7in[5]-derived boolean attributes8However,the4ft-Miner also allows for negated literals in conjunctions5.2Classes of4ft-QuantifiersThere have been many4ft-quantifiers defined.Due to their statistical and logicalproperties,they can be divided into several classes[1],[5],[6].Associational quantifiers”∼”in rulesϕ∼ψcorrespond to the intuitive notion ofassocation.Probabilisticaly:P((ψ∧ϕ)∨(¬ψ∧¬ϕ))>P((¬ψ∧ϕ)∨(ψ∧¬ϕ)).Note:the above average quantifier is associational.Implicational quantifiers”=⇒”in rulesϕ=⇒ψcorrespond to the intuitive no-tion that”almost allϕareψ”.In terms of conditional probability:P(ψ|ϕ)>P(ψ). Note:the quantifier of founded implication is implicational.Double implicational quantifiers”⇐⇒”in rulesϕ⇐⇒ψcorrespond to the no-tion that iffϕ=⇒ψthan alsoψ=⇒ϕ.Equivalence quantifiers”≡”in rulesϕ≡ψcorrespond to the notion that iffϕ=⇒ψthan also¬ϕ=⇒¬ψ.Let A,I,DI and E denote classes of associational,implicational,double implica-tional and equivalence4ft-quantifiers,respectively.Followinginclusion relationshold[2]:DI⊆I,E⊆I,I⊆A.5.3Pseudo-RulesKDD tasks discussed above can be formalized usingexpressions similar to4ft-association rules.We will call them pseudo-rules.Pseudo-rule is an expressionof the formΦ≈ Ψ/Ξ,whereΦ,ΨandΞare formulae(in sense of section5.1).Symbol≈ stands for a class of4ft-quantifiers,i.e.≈ ∈{∼,=⇒,⇐⇒,≡}. Pseudo-ruleΦ≈ Ψ/Ξis interpreted as a KDD task:”Within the set of objectsthat satisfyΞ,examine relations of type≈ between objects satisfyingΨandobjects satisfyingΦ”.Let us stress that although syntactically similar to associ-ation rules,pseudo-rules correspond to tasks only,not to actual relations in data. Pseudo-rules are capable of representingvery specific KDD tasks,as well as quite general KDD tasks.Considerϕ≡ψ/ξ,whereϕ,ψ,andξare elementary con-junctions of literals and/or negated literals.Formulaeϕ,ψ,andξare symbolic names of complex characteristics of database objects and≡is a fairly special type of relation(equivalence).On the other hand,considerΦ∼Ψ/Ξ,whereΦ,Ψare disjunctions of formulae andΞis an empty(hence true)formula.In sucha case,ΦandΨare symbolic names of groups of characteristics,relations to be found are not restricted by any condition(Ξis empty),and∼is the most general type of relation(association).Therefore,we can use pseudo-rules to represent KDD tasks all alongtheir hierarchy.5.4Specialization/Generalization of Pseudo-RulesLet us attempt to formalize the notion of specialization/generalization relation between pseudo-rules.We are lookingfor a partial-order relation on pseudo-rules, which would satisfy the followingintuition:Given two KDD tasks,represented by two pseudo-rulesτ1andτ2,τ1=(Φ1≈ 1Ψ1/Ξ1),τ2=(Φ2≈ 2Ψ2/Ξ2);τ2is a specialization ofτ1iff:–sets of objects satisfyingΦ2,Ψ2andΞ2are subsets of sets of objects satisfying Φ1,Ψ1andΞ1,respectively.–≈ 2is a special case of≈ 1(see section5.2)Before we proceed to definition of the specialization relation,we need a no-tion of associated propositional formulae(see also[6]).LetΦbe a formula.Then associated propositional formulaπ(Φ)ofΦis a proposition of propositional cal-culus constructed by:–constructingformulaΦ by replacingall the literals inΦby appropriate disjunctions of one-point literals(see section5.1)–π(Φ)is the same stringof symbols asΦ ,but particular one-point literals are understood as propositional variablesNow we are ready to define the specialization relation.Letτ1andτ2be pseudo-rules,τ1=(Φ1≈ 1Ψ1/Ξ1),τ2=(Φ2≈ 2Ψ2/Ξ2).Let C1and C2be classes of4ft-quantifiers denoted by≈ 1and≈ 2,respectively9.Let→be the proposi-tional connective of implication.We will say thatτ2is a specialization ofτ1, symbolicallyτ1 τ2iff:–propositions(π(Φ2)→π(Φ1)),(π(Ψ2)→π(Ψ1))and(π(Ξ2)→π(Ξ1))are tautologies of propositional calculus–C2⊆C1It can be seen easily that thus defined relation satisfies the above listed intuitions and is transitive,reflexive and antisymetric,i.e.it is a partial ordering.5.5Possible Applications of the Formal ApproachIt is possible to include formal desriptions of web pages’content within the pages’source code[4];therefore,the formalisation of KDD tasks indicated above can be used for efficient content-based on-line retrieval of KDD tasks and of the discovered knowledge[3].For example we can have queries such as:”Given a pseudo-ruleτ1,find all analyses that solve some taskτ2,such thatτ1 τ2”.In a similar way,we could retrieve whole sections of web presentations of KDD tasks and results (i.e.”Givenτ1,find within the WWW presentation’s tree a sub-tree described by τ2,such thatτ1 τ2”.)The formal approach would also allow for automated support of integration of WWW presentations of KDD research projects(i.e. appendingone tree to another;ag ain the relation is utilized).Also,we can make use of logical properties of4ft-rules themselves.In[1] were4ft-association rules developed as expressions of special logical calculi that valuate in the analysed data matrices.Furthermore,deduction was defined on 4ft-association rules.Therefore,given a4ft-association ruleϕ≈ψ,we can ask whether there was an analysis performed,which proved a set of4ft-assocaition rules O,such that O|=ϕ≈ψ(see also[5]).9for example,if≈ is=⇒,then C is the class of imlicational quantifiers6ConclusionsWe have reckognized the need for transparency and reuse of knowledge discov-ered in the Challenge data in course of the STULONG project.These needs determinate the design of WWW presentation of the discovered knowledge.To further enhance possibilities of knowledege reuse,we had developed formal means of description of the underlyingKDD task hierarchy.The research has been supported by projects LN00B107and ZA471011of the Ministry of Education of the Czech Republic.References1.H´a jek,P.-Havr´a nek T.:Mechanising Hypothesis Formation-Mathematical Foun-dations for a General Theory.Springer-Verlag,1978,396p.Available online at http://www.cs.cas.cz/vvvvedci/hajek/guhabook2.Iv´a nek,J.:On the Correspondence between Classes of Implicational and EquivalenceQuantifiers.In Principles of Data Mining and Knowledge Discovery.Red.Zytkow, J.Rauch,J.Berlin,Springer Verlag1999,pp.116-1243.L´ın,V.,Rauch,J.,Sv´a tek,V.:Content-based Retrieval of Analytic Reports.In:(Schroeder M.,Wagner G.,eds.)International Workshop on Rule Markup Lan-guages for Business Rules on the Semantic Web.Sardinia,Italy,June2002,p.219-224.ssila,O.,Swick,R.R.:Resource Description Framework(RDF)Model and SyntaxSpecification.Recomendation REC-rdf-syntax-19990222,W3C,February19995.Rauch,J.:Logical Calculi for Knowledge Discovery in Databases.In Principles ofData Mining and Knowledge Discovery,Springer-Verlag,1997.6.Rauch,J.:Classes of Four-Fold Table Quantifiers.In Principles of Data Mining andKnowledge Discovery,(J.Zytkow,M.Quafafou,eds.),Springer-Verlag,1998.7.Rauch,J.-ˇSim˚unek M.:Mining for4ft Association Rules by4ft-Miner.in:INAP2001,The Proceeding of the International Conference On Applications of Prolog.Prolog Association of Japan,Tokyo October2001,pp.285-294。