2003b) Mining the implicit ratings for focused collaborative filtering for paper recommenda

Mining Implicit Ratings for Focused Collaborative Filtering for Paper RecommendationsTiffany Ya Tang and Gordon McCalla1Department of Computer Science, University of SaskatchewanSaskatoon, SK S7N 5A9 CANADA{yat751@ask.ca, mccalla@ask.ca}Abstract. In this paper, we describe our on-going work on applying webmining to guide focused collaborative filtering for paper recommendations in aweb-based learning system. In particular, we propose to first apply a dataclustering technique on web usage data to form clusters (groups) of users withsimilar browsing patterns, which can be viewed as filtering based on implicitratings (browsing sequences) according to [21]. Then, collaborative filteringtechniques would be adopted on each cluster, instead of on the whole pool ofusers for recommendations as in other clustering-based collaborative filteringapproaches. By using our two-layered collaborative filtering approach, we willnot only maintain the diversity of users, but also focus on groups of users withsimilar browsing patterns. Therefore, our proposed approach could not onlymake personalized but also ‘grouplized’ recommendations, thus overcomingprevious claims that data clustering will only produce ‘less-personalrecommendations’ [33]. In addition, both explicit and implicit ratings are takeninto consideration, which can reinforce and complement each other to makemore accurate recommendations.1. IntroductionFor a web-based system where many users interact with each other, with the system and with the open Web, it is not trivial to design a system to make use of the trace data to gain insight into users’ web activities and take actions accordingly. In order to develop and maintain such a kind of system, there exist many challenges, among them:1.How to deal with the huge amount of web data (content, usage, or evenstructural information about the web system itself), which can easily be collected?2.What kind of user data, usage data, context data and environment data wouldbe necessary to model users’ web activities and respond appropriately, effectively and efficiently?3.How to deal with the so-called ‘cold-start’ problem [31, 34] when new user(s)who have not left too many ‘traces’ come in?1This work is supported by the Natural Sciences and Engineering Research Council of Canada.2 Tiffany Ya Tang and Gordon McCalla4.Should these systems recommend items based on individual users orcommunities of users or both?5.Even when these systems recommend items based on an observation ofcommunities of users with similar patterns, how would the system take actions in order to maintain the uniqueness of each individual user?Of central importance is how to model users in such environments in order to find out critical contextual information, such as what are their information needs, what kind of information they are looking for etc. This information would, in turn, help the system support users at an appropriate time effectively and efficiently.In our on-going work, we attempt to explore these issues in the context of an adaptive and evolving web-based learning environment, where learners with various background knowledge engage in an advanced web-based course on data mining and web mining. Specifically, the evolving system can adapt itself not only to its users, but also to the open Web in response to the usage of its learning materials. Our system is open in the sense that learning items (referring to papers in our system) related to the course could be added, maintained, or deleted. A more complete description of the system is discussed in [43]. In this paper, we will focus specifically the two main techniques involved in the system. In particular, we propose to apply both web mining and collaborative filtering (CF) techniques to model individual as well as group behaviors, and made recommendations accordingly. Specifically, we will first perform data clustering on usage data in order to group users with similar browsing behaviors, which will allow us to shrink the candidate set. This step can also be viewed as CF based on implicit ratings. We will then operate CF techniques on each individual cluster instead of on the whole set of data. Therefore, when a new user comes to the system, the system will first match her/him against a specific cluster based on the existing browsing patterns; after that, personalized recommendations will be made within the cluster he/she belongs to. By doing this, we can not only maintain the diversity of users in terms of their learning interests, activities etc, but also are able to identify their uniqueness. To the best of our knowledge, there is little research addressing mining on implicit ratings for focused CF to make recommendations, although these implicit ratings might be of great value and thus provide rich data to make recommendations as effectively as explicit ratings can.The organization of this paper is as follows. In the next section, we will first give a brief background description of CF and web mining techniques. Then, we will present the two-layered, focused CF framework for paper recommendations in a web-based learning environment. A comparison to related work is presented in the third section. We conclude this paper by describing some lessons learned so far, and point to some future research.2. Background: Collaborative Filtering and Web Mining2.1 Collaborative Filtering and Recommender SystemsAs the World Wide Web becomes an increasingly popular medium, information overload intensifies: users are overwhelmed by the information pouring out from theMining Implicit Ratings for Focused Collaborative Filtering for Paper Recommendations 3 web, and are usually confused by which information should be consumed. Recommender systems offer a reliable solution to this issue. They have been studied extensively over the last couple of years [2, 4, 21, 23, 32, 36]. There are two basic approaches to providing personalized recommendations: content-based and collaborative filtering [22]. Regardless of approach, at the core of personalization is the task of building a model of the user. Content-based approaches build user models that link the contents of the information a user has browsed to the preferences of the user concerning those artifacts. CF approaches build user models that link the information preferences of a user to those of other users with similar tastes or preferences. Therefore, in CF, the system will try to match a user’s preferences against other similar clusters of users.A purely content-based approach only considers the preferences of a single user, and is concerned with only the significant features describing the content of an item, whereas, a purely CF approach ignores the contents of the item, and only makes recommendations based on comparing the user against clusters of other similar users. Consider, however, that item information (features that would best categorize the item) can be obtained through a content-based approach, and user information (relative distance of the user to other clusters of users, and users’ opinions) can be obtained from CF. By combining these two techniques, perhaps we can have both individual as well as collective experiences with respect to the items being recommended. [1, 2, 19] discuss systems which apply such hybrid approaches to make movie recommendations.2.2 Web Mining: Knowledge Discovery on the WebWeb mining first appeared in 1996 [15]. As its name indicates, the goal is to apply data mining techniques to extract as well as discover information automatically from web documents and services. It can be grouped into three categorizes, i.e. content, structure, and usage mining.2.2.1 Web Content MiningWeb content mining refers to the process of uncovering interesting and potentially useful knowledge from web contents/documents. At present, content mining focuses mainly on text mining techniques including clustering, classification, and categorization of web contents. Most text mining on the content of web pages is key word driven (e.g. [16]).2.2.2 Web Structure MiningWeb structure mining is interested in both the structure within web documents (intra-links) and the structure of the hyperlinks between web sites (inter-links). The most significant research concerning web structure mining is proposed by Kleinberg [24], who introduces the notion of authorities behind the enormous link structures of the web. Authorities are those highly referenced pages on a certain topic. And the numerous pages that point to these authorities are called hubs. They have been4 Tiffany Ya Tang and Gordon McCallautilized for emerging web community identification [18], focused crawling [10], and scientific literature retrieval and recommendations [26].2.2.3 Web usage miningWeb usage mining has been applied for various kinds of decision support, among them: web site modification/evaluation (e.g. [30]), personalization of information filtering (e.g. [12]), learning about consumers or individual users or groups of users (e.g. [17, 35]). Web usage mining, with an aim of designing efficient techniques to predict as well as analyze user behaviors, is a powerful tool for web site designers and owners to gain insights into the usage patterns and make sound decisions accordingly.3. Focused Collaborative Filtering for Paper Recommendations3.1 Problem DomainProblem Statement: Given a collection of papers and a matrix of user/ratings of these papers, recommend a set of papers, so as to meet both the learner’s knowledge level and interests.As pointed out in [42] that making recommendation for learners is different from making recommendation for users in many other domains. In particular, in e-learning, recommending mechanism should consider not only learners’ interest towards the items like most other recommender systems do, but also their knowledge of domain concepts relative to the learning environment. For instance, not to recommend highly technical papers to a first-year-undergraduate student or popular-magazine articles toa senior-graduate student. A recommendation model for our system is presented in[42], which will not be described here.Our proposed system makes recommendations in the context of an advanced course on data mining and web mining. The course contains 14 chapters, covering basic concepts and operations of data mining and web mining and their applications in e-business, intelligent tutoring, bioinformatics, recommender systems, user modeling etc. Hence, each chapter can roughly be grouped at a category level2. The recommendation part initially would consist of approximately 250 papers and two glossaries. We are also building a web crawler to crawl NEC’s CiteSeer (the largest scientific digital library for computer science) to accommodate more up-to-date papers. The system is aimed at graduate students majoring in computer science, bioinformatics, engineering, or business.2According to [9], prediction model works by inferring a user’s preference for a category from his/her accesses of that category. And categories are organized in a hierarchical structure. We believe that it is reasonable to capture learners’ information needs at the category level as described in [9] because it would enough for us to track down their category–level (coarse-grained) interest in order to make recommendations.Mining Implicit Ratings for Focused Collaborative Filtering for Paper Recommendations 5 3.2 The First Layer: Clustering Communities of LearnersAt the first layer of the system, learners’ web activities will be collected and grouped into clusters of learners’ with similar browsing sequences. In our previous work on this issue, we reported an experimental study where informed clustering is performed based on a feature construction technique to cluster learners with similar knowledge levels [39, 40]. Course content is organized as a directed tree where each node represents a chapter, and each directed-edge denotes the prerequisite relationship between two chapters. It is assumed that there is one main topic in each chapter. For instance, chapter 2 of our system discusses some basic operations of data mining. There are three main properties which describe the difficulty of learning that chapter, i.e. W: writing level, T: technical level, and P: presentation level. The writing level of a topic includes page length, number of conceptual definition, verbal usage and writing organization. Technical level includes mathematical/symbolic formulation, algorithms, proof of theorems, etc. Presentation level contains graphical presentation, say, system architecture or flow diagram, examples, case study, etc. These levels are determined subjectively by tutors based on their experiences. Moreover, in order to learn each chapter smoothly, the learner should understand some important part of its prerequisite chapters.Formally, for chapter/topic j, we have Φj, denoting content complexity, whereΦj = { Ch j W, Ch j T, Ch j P, MinKn j }(1) Ch j W, Ch j T, Ch j P denote chapter j’s writing, technical and presentation level respectively; MinKn j denotes the minimum average knowledge of the prerequisite chapter(s) of chapter j in order to understand chapter j.For learner i, we have L i denoting his/her overall learning characteristics, whereL i = {R, T, A, B, Ex} (2) R, denotes the overall reading skill of each learner. T, or TechnicalKnowledge denotes learners’ previous knowledge, which might help them understand any chapter’s technical aspects. A, or AnalyticalSkill represents learners’ analytical abilities. Some learners may possess relevant knowledge to some extent, denoted as their B or BasicKnowledge, which can facilitate their learning speed and overall learning outcomes. Ex specifies a learner's subjective attitude and expectancy towards the knowledge to be acquired. It can also be seen as how much knowledge the learner should acquire before she moves on. In most research on web learning, it is assumed that learners will move on to learn a new topic only after they have understood current one(s) [37]. However in our framework, they are allowed to proceed as long as they have met their expectations for the topic.Since the learner may visit the same chapter more than once, then, we perform a simple computation by counting the total number of visits (Frequency) and the total time spent in reading each chapter (TotalTime). Thus, for each chapter we get the pair data (Frequency(ch j), TotalTime(ch j)).6i V Then, using features of the curriculum knowledge described in equation (1), we apply the weighted-Euclidean-distance-based clustering to infer learners’characteristics and cluster them accordingly. The weight is defined as:w x = [G j (W) + G j (T) + G j (P)]-1 (5)where G j is the arithmetic-composition function specifically for contrasting x ∈{W ,T , P } (see [39] for details).We conducted an experiment with synthesis datasets, and results showed that although the clustering algorithm could infer learners’ learning characteristics relatively accurately, deviations occur especially when learners differ substantially in terms of their educational levels and abilities. In this case, it is believed that model-based clustering would perform better. Currently, we are conducting experiments on a new model-based clustering.3.3 The Second Layer: Focused Collaborative FilteringCollaborative filtering (CF) works by making recommendations or predictions based on a database of explicit accumulated ratings. One of the key steps involved for CF is to form neighbors for a target user. The goal of neighborhood formation can formally be expressed in the following way.Given a target user u , find an ordered list of l neighboring users N ={N 1, N 2, …N l },such that u ∉N and sim (u ,N 1) > sim (u ,N 2) > …> sim (u , N l ), where sim (u , N i ) denotes the similarity of user model u to its neighbor N i .There are two common similarity measurements in the CF literature: Pearson-correlation based and Cosine-based similarity. Since Pearson-correlation neighborhood forming outperforms the Cosine-based approach [6], we apply the former in our study. Specifically, the Pearson correlation between users a and b is given by:Where V i,k is the vote rated by user i on item k ,is the mean vote for user i ,and K is the set of items co-rated by both a and b .In our proposed approach, after we identify clusters of users with similar interests and knowledge level, we form Pearson-correlation similarity based CF to make recommendations for target user(s). Such web mining not only shrinks candidate sets,but also guides CF in a more delicate and focused manner, which is desirable if web based adaptive systems are to deal with the scalability issue. Therefore, we seek to make recommendations based on more focused sub-groups of users, whose ratings might be of more valuable in comparison to the whole pool of users.For example, suppose ten users have been using our system to learn. Based on specific criteria they could be clustered into 2 groups {User1, User2, User3} and {User4, User5, …, User10}. Thus, CF would then focused on each cluster (Figure 1)∑∑∑−−−−=K b k b K a k a K b k b a k a V V V V V V V V b a W 2,2,,,)()())((),(Mining Implicit Ratings for Focused Collaborative Filtering for Paper Recommendations 7Figure 1. An illustration for focused CFIt is obvious that by first applying data clustering, the number of a user’s neighbors might be less; that is to say, similarity between the user and his/her “shrunken”neighbors would be higher. Hence, from this perspective, we say that data clustering can guide CF into a focused area, where the ‘distance’ between the target user and his/her neighbors would be smaller. In addition, it might be possible that by applying our approach, the recommendation accuracy could be increased. Because we believe that users with similar ratings towards specific items(s) are not necessarily ‘close neighbors’, we will first group users with similar learning patterns for CF. In essence, both explicit and implicit ratings are taken into consideration, which can reinforce and complement each other to make more accurate recommendations.In our approach we do not alter the data. Thus, the clustered data would still faithfully capture the original merits of the data. Therefore, although we have not conducted experiments using our approach, we can safely claim that our proposed approach would yield a performance as least as good as those techniques which perform data clustering directly on the whole pool of data [46].4. Related WorkRelated work can be categorized into the following four groups: making recommendations in e-learning systems, applying web-mining in e-learning systems, recommending scientific literature on the web, and modeling users on the web.4.1 Making Recommendations in E-learning SystemsRecommending course contents to serve the needs of users has been studied extensively in the adaptive hypermedia community. According to [7], generally, there are two kinds of adaptation: adaptive navigation (link level) and adaptive presentation (“content level”). Content-level adaptation can be viewed alternatively as content-based recommendations when users’ past reading items/pages are recorded and analyzed. So far, for web based adaptive learning systems (e.g. [46]),“knowledge/model tracing” has been commonly applied, and has been proved to be very effective [13, 14]. Nevertheless, several researchers have realized that in order to best make both course contents and structure adaptable to learners and thus provide8customized learning environments, learners’ web learning activities should be utilized [7, 25]. There are very few studies addressing this issue, except our recent studies [39, 40].As far as we know, there is no research applying CF techniques for e-learning. However, we believe that the success of and relative fertile research on recommender systems can readily be applied for adaptive web-based learning systems, in order to support social collaborations of learners.4.2 Applying Web Mining in E-learning SystemsBy recording students’ browsed documents, JUPITER is capable of recommending relevant kanji words to them, mainly through text mining techniques [29]. Tang et al [38] propose to construct a personalized e-articles reading tree based on predetermined keyword (s), and recommend it to learners based on keyword-driven text mining. Zaiane [48] adopts association rules mining to make recommendations (mainly learning activities) to learners. In these cases, learners’ learning characteristics (including learning activities, skills, learning patterns etc) are ignored, although the authors admit that certain students’ information should be included.4.3 Recommending Scientific Literatures on the WebCompared to conventional recommendation systems which have been explored extensively, scientific literature recommendations have received little attention until recently thanks to some hugely acclaimed digital libraries, especially NEC’s CiteSeer [26]. Basu et al [3] define the paper recommendation problem as: “Given a representation of my interests, find me relevant papers.” They studied this issue in the context of assigning conference paper submissions to reviewing committee members. For reviewers, they do not need to key in their research interests as they usually do; instead, a novel autonomous procedure is incorporated in order to collect reviewer interest information from the web. Bollacker et al [5] refine CiteSeer, NEC’s digital library for scientific literature, through an automatic personalized paper tracking module which retrieves each user’s interests from a well-maintained heterogeneous user profiles. Citeseer adopts the well-known notion of hubs and authorities originally developed by Kleinberg [24]. Woodruff et al [47] discuss an enhanced digital book with a spreading-activation-geared mechanism to make customized recommendations for readers with different type of background and knowledge. McNee et al [28] use CF techniques to recommend papers for researchers, but the focus is mostly on recommending additional references for a target research paper. Common to all the above approaches is the utilization of citation [3, 5, 47], self-citation [5], fused citation [47] techniques, the first two of which have been widely adopted for document retrievals.Mining Implicit Ratings for Focused Collaborative Filtering for Paper Recommendations 9 4.4 Modeling Users on the WebWebb et al [45] identify two approaches to the modeling of individual users and communities of users: machine learning relying on users’ explicit relevance feedback [11, 23] or labeled user activities [27, 45, 23], and web mining relying on implicit observations of users’ web activities [17, 39].Zaiane [48] further argues that although applications in electronic commerce “relate to forming generic models of user communities”, most work has been carried out in modeling individual users. In fact, data clustering does pretty well in handling communities of users, while association rules mining deals well with finding correlation between items liked by clusters of users and individual user with similar interests.5. Discussion and Conclusions5.1 The Social Aspects of Collaborative FilteringCF is a domain whose social aspects can be explored. For example, the network value of users who either directly or indirectly influence each other’s subsequent behaviors. Unfortunately, it is still unclear how one user’s rating(s) might affect the ratings or activities of another no matter what user groups they are in. Currently, some users do not want to bother to rate items they have tried (movies, CDs, news, books etc) unless they perceive some benefit to providing ratings [20]. For potential target users for which recommendations are made, they are not sure about the sources of the recommendations [22]. Some commercial recommendation systems have already been equipped with a bonus point mechanism to encourage more users to give out their ratings towards the items they have consumed. In our proposed system, we will study this issue, in particular, how learners might take advantage of fellow learners’ratings towards those items they have read during the learning process.5.2 Informed Web Mining for E-LearningA knowledge discovery process guided by prior background knowledge is known as “informed knowledge discovery” (or informed KDD for short) [8]. Although the importance of informed KDD has been realized and discussed extensively in the data mining community, it has largely been ignored by researchers who use data mining to support e-learning. We believe that incorporating course contents, and other knowledge of the learning situation (environment data to aid contextual computing for obtaining situated user models) can allow us to inform web mining algorithms for contextual computing of user models.105.3 A Two-Way-Collaboration between Users, Web System and the Open Web As far as we know, current web-based adaptive learning systems have been focusing on the interrelations between users and the system. Hence, the system, if deemed intelligent, must be capable of detecting users’ needs, following their footsteps, and finally adapting to their needs in a manner that in essence is to collaborate with its users. We argue that only implementing this channel of interactive and collaborative relationships is not enough [41], since to do so we ignore the dynamics of the open Web. Indeed, this is an issue, which has already grown too large to ignore. As such, we believe that two kinds of collaborations should be considered here: one is the collaboration between the system and its users; another is the collaboration between the system and the open Web in response to the changing needs of the users. It is obvious that a system which can fulfill especially the second collaboration, is indeed helping its users to keep up-to-date to the dynamics of information on the Web. It might be worthwhile to design such open web-based systems, although it is evident that more rigorous collaborative research should be carried out between researchers from web information retrieval, user modeling, intelligent user interface, computer supported collaborative work etc.References1.Balabanovic, M. and Shoham, Y. Fab: content-based collaborative recommendation. Comm.Of the ACM. 40(3): 66-72. March 1997.2.Basu, C., Hirsh, H. and Cohen, W.W. Recommendation as classification: using social andcontent-based information in recommendation. In AAAI 1998, 714-720.3.Basu, C., Hirsh, H. and Cohen, W.W, Nevill-Manning, C. Technical paper recommendation:a study in combining multiple information sources. JAIR 1(2001): 231-252.4.Billsus, D and Pazzani, M. A hybrid user model for news story classification. UM’99. 99-108.5.Bollacker, K.D., Lawrence, S. and Giles, C.L. A system for automatic personalized trackingof scientific literature on the web. In ACM DL 1999, 105-113. 1999.6.Breese, J.S., Heckerman, D. and Kadie, C. Empirical analysis of predictive algorithms forcollaborative filtering. In UAI 1998.7.Brusilovsky, P. Adaptive hypermedia. UMUAI, 11(1/2):87-110. 2001.8.Buchanan, B. Informed knowledge discovery: using prior knowledge in discovery programs.In an invited talk, ACM SIGKDD 2000. Boston. (abstract) 3.9.Chen, M., LaPaugh, A.S. and Singh, J.P. Predicting category accesses for a user in astructured information space. In ACM SIGIR 2002, Tampere, Finland. 65-72. 2002.10.C hakrabarti, S., van der Berg, M. and Dom, B. Focused crawling: a new approach to topic-specific web resource discovery. In ACM WWW8, 1999.11.C hi, E.H., Pirolli, P., Chen, K. and Pitkow, J. Using information scent to model userinformation needs and actions in the web. ACM CHI 2001. 490-497.12.C ooley, R., Tan, P.T. and Srivastava. WebSIFT: The web site information filtering system.In WebKDD 1999.13.C orbett, A.T. and Anderson, J.R. Knowledge tracing: modeling the acquisition of proceduralknowledge. UMUAI, 4, 253-278. 1995.14.C orbett, A.T. and Anderson, J.R. Locus of feedback control in computer-based tutoring:impact on learning rate, achievement and attitudes. In ACM CHI 2001. 245-252.。