Towards Inductive Support Logic Programming

Towards Inductive Support Logic ProgrammingJ. F. Baldwin and T. P. MartinJim.Baldwin@, Trevor.Martin@Extended AbstractThe related fields of data mining and knowledge discovery in databases (KDD) have grown rapidly, aiming to produce “knowledge” which gives insight to a human user and which can be used to explain existing patterns and predict future values in the database.At a simple level, we can make a distinction between commercial/industrial and academic approaches. Industrial and commercial applications need to demonstrate profitable results from data mining, and generally focus on simple questions against very large databases. The tools tend to be predominantly statistical in nature, and the underlying models are relatively simple. Strong emphasis is placed on visualisation.Academic approaches, on the other hand, generally focus on the extraction of much richer knowledge structures -typically rules - from smaller amounts of data. Rules are an ideal framework for knowledge discovery - logic is easily understood by most people, and provides a powerful computing formalism through logic programming. Additionally, it is relatively easy to incorporate background knowledge in the form of user-supplied logic statements. Of course, whilst logic is ideal for reasoning about formal, closed systems such as mathematics and small computer programs it suffers from an artificial over-precision in relation to the real world. Thus, rather than using a natural form such as “most young high-spending customers …”we are forced to quantify everything precisely“at least 85.2% of customers aged between 20 and 30 who spend over £200 per week …”The over-precision of logic may also be inappropriate at the data level. The relational model assumes that data is complete and correct - an assumption which is often invalid. A relational database imposes a crisp model of the world, i.e. all categories must be precise and information must be known with complete certainty.Support logic programming [Baldwin 1986] and its practical implementation (Fril) [Baldwin, Martin and Pilsworth 1991]avoids these problems by integrating probabilistic and fuzzy uncertainty into logic programming using the mass assignment framework [Baldwin 1992]. This allows uncertainty to be handled in data values, in relations, in rules, and in inference. To answer a query, a standard logic programming proof tree is created with extended unification to deal with uncertain attribute values; a probabilistic calculation is then carried out over the proof tree to determine the support for the conclusion. All proof paths must be examined to determine the overall support.The Fril data browser [Baldwin and Martin 1997] allows us to extract support logic rules from data. The data browser is a propositional learner, i.e. it summarises attribute values for different classes to produce predictive rules which model the relation between values in a tuple.More sophisticated KDD allows relations to be discovered between values in different tuples. This is implemented most effectively in inductive logic programming [Muggleton 1992], which has been successfully applied in diverse areas such as mutagenicity prediction, protein structure modelling, and program synthesis. Inductive logic programming relies heavily on the dual semantics of logic programs. Taking a database of examples E+ and counter-examples E-, with ∀e+∈E+:H∪B=e+¬∃e−∈E−:H∪B=e−background knowledge expressed as a set of clauses B, we attempt to derive H such thatThe generation of logic programs is a substantial search problem, but successful applications show that with the provision of appropriate background knowledge, this is an effective approach. One of the major drawbacks of inductive logic programming is its lack of a mechanism for dealing with uncertainty. To summarise:•commercial data mining systems have shown promising results with relatively simple tools•logic-based approaches give more understandable models and allow background knowledge to be expressed easily •Fril extends the logic programming approach to give more natural and understandable models by use of fuzzy terms •Inductive logic programming can generate richer knowledge by creating meaningful new predicatesCurrent research within the Bristol AI Group aims to combine the best aspects of inductive logic programming with the uncertainty representation of Fril to create a sophisticated and novel approach to knowledge discovery. In order to do this, we need to extend the present informal semantics of Fril and develop a model-theoretic framework so that we can reason about the declarative semantics of Fril programs as well as their procedural semantics.There have been several attempts to develop the declarative semantics of quantitative logic programs. Support logic is a more general approach in that it also permits uncertainty in attribute values; the Fril interpretation of a rule support as an interval-valued conditional probability also requires careful treatment.In conventional logic programming, a Herbrand interpretation is defined as a subset of the Herbrand base, containing precisely those ground atoms which are true under the interpretation. A Herbrand model of a program is a Herbrand interpretation in which every logical consequence of the program is true. A partial ordering on interpretations allows us to define the least Herbrand model and a method by which it may be constructed.For support logic programs we extend the idea of an interpretation by considering a mass assignment over the Herbrand base instead of a single subset of the Herbrand base. The supports have meaning with respect to the entire set of possible worlds (interpretations) - see also [Bacchus et al. 1996]. For example, with the Herbrand baseB = {p(1), r(1), q(1) }the set of interpretations is {p(1) ∧ r(1) ∧ q(1), p(1) ∧ r(1) ∧ ¬q(1), … , ¬p(1) ∧ ¬r(1) ∧ ¬q(1) }A distribution on the power set gives an extended interpretation - for exampleM = { {p(1) ∧ r(1) ∧ q(1), ¬p(1) ∧ r(1) ∧ q(1) }: 0.4{p(1) ∧ ¬r(1) ∧ q(1) , ¬p(1) ∧ ¬r(1) ∧ ¬q(1)}: 0.1{p(1) ∧ r(1) ∧ ¬q(1) }: 0.5 }The mass assignment restriction operation gives us a partial ordering over the mass assignment (MA) interpretations, and a way of deriving the support pair for any ground atom under the MA interpretation. The necessary support for a ground atom A under a MA interpretation I = {K : mk} is the sum of masses assigned to elements which entail A,while the possible support is the sum of masses assigned to elements which do not entail ¬Agiving the supports [0.5, 1] to p(1), [0.9, 0.9] to r(1) and [0.4, 0.5] to q(1) in the example above. A support logic program computes a support pair for any ground atom. Thus we have a way of defining soundness and completeness (respectively,the computed support pair is contained in/contains the support pair derived from the mass assignment interpretation).Hence we can define a mass assignment model of a support logic program. Further research is needed to fully explore this basic idea and its ramifications - for example, taking two extended interpretations I1 and I2 such that I1 is a restriction of I2, the support pair for a ground atom under I1 will be a sub-interval of the support for the atom under I2. Thus it should be possible to construct a mass assignment model in a similar fashion to computing the least fixed point of the Tp operator in standard logic programming.SummaryFril allows us to express uncertainty in logic programs, and can be used to extract programs from real world data. The Fril data browser summarises attribute values and forms predictive rules; this has been successfully applied in many applications as well as on benchmark machine learning databases. The development of mass assignment model-theoretic semantics will allow us to extract relational knowledge from data, in a manner similar to inductive logic programming.ReferencesBacchus, F., Grove, A. J., Halpern, J. Y. and Koller, D. (1996). “From Statistical Knowledge Bases to Degrees of Belief.” Artificial Intelligence 87(1-2): 75-143.Baldwin, J. F. (1986). "Support Logic Programming" in Fuzzy Sets - Theory and Applications , Ed. A. Jones, D. Reidel.133-170.Baldwin, J. F. (1992). "The Management of Fuzzy and Probabilistic Uncertainties for Knowledge Based Systems" in Encyclopedia of AI , Ed. S. A. Shapiro, John Wiley. (2nd ed.) 528-537.Baldwin, J. F. and Martin, T. P. (1997). "Basic Concepts of a Fuzzy Logic Data Browser with Applications" in Software Agents and Soft Computing: Concepts and Applications , Ed. H. S. Nwana and N. Azarmi, Springer (LNAI 1198). 211-241.Baldwin, J. F., Martin, T. P. and Pilsworth, B. W. (1991). "FRIL: A Support Logic Programming System", Proc.Expert Systems and Optimisation in Process Control , 225 - 238.Fayyad, U. (1997). "Knowledge Discovery in Databases: An Overview" in Inductive Logic Programming , Ed. N. Lavrac and S. Dzeroski, Springer (LNAI 1297). 3-16.Muggleton, S. (1992). "Inductive Logic Programming", Academic Press.S l (A ,I )=m K K ∈I K |=A∑S u (A ,I )=m KK ∈I K |=/¬A ∑。