Innovations in Systems and Software Engineering manuscript No. (will be inserted by the edi

Innovations in Systems and Software Engineering manuscript No.(will be inserted by the editor)Naohiro Matsumura·David E.Goldberg·Xavier Llor`aCommunication Gap Management for Fertile Community Received:date/Accepted:dateAbstract In the paper,wefirst present an approach to extract social networks from message boards on the In-ternet.Then we propose communication gaps based on structural features of the social networks as an indicator of understanding the state of communication.After we classify3,000social networks into three types of com-munication,i.e.,interactive communication,distributed communication,and soapbox communication,we suggest communication gap management to identify the types of communication,the roles of individuals,and important ties,all of which can be used for drawing up a plan for realizing fertile community.1IntroductionWith the advent of popular social networking services on the Internet such as Friendster1and Orkut2,social net-works are again coming into the limelight with respect to this new communication platform.A social network shows the relationships between individuals in a group or organization where we can observe their social activities. For example,individuals who share a serious problem might all join a discussion on ways to solve the problem, while in another case only a few knowledgeable individu-als might give information when individuals seek tofind out more about specific topics.Naohiro MatsumuraGraduate School of Economics,Osaka University1-7Machikaneyama,Toyonaka,Osaka,560-0043Japan Tel.:+6-6850-5231Fax:+6-6850-5231E-mail:matumura@econ.osaka-u.ac.jpDavid E.Goldberg and Xavier Llor`aIllinois Genetic Algorithms Laboratory,Department of Gen-eral Engineering,University of Illinois at Urbana-Champaign 414Transportation Building,104S.Mathews Avenue,Ur-bana,IL61801E-mail:{deg,xllora}@1/2/It is known that well-planned communication real-izes human innovation in communication[4].In this pa-per,we define a“fertile community”as the community where individuals communicate according to their aims, and output innovative outputs.However,the complex-ity of communication makes it difficult to realize fertile community.To overcome this difficulty,we make use of the property that explains how the structural features of a social network affect the types of communication and topics individuals are talking munication gap management is proposed here to identify the types of communication,the roles of individuals,and impor-tant ties,all of which can be used for drawing up a plan for realizing fertile communication.In Chance Discovery research,a“chance”is defined as a rare event or situa-tion which provide us opportunities or risks for human decision making in dynamic environment[12].In this context,we can say that the paper provides a method of discovering a chance towards fertile community.The remainder of this paper proceeds as follows.In Section2,we present an approach to extract social net-works from message boards on the Internet.Then we introducefive indices that can be used to measure the structural features of social networks in Section3.We describe three types of communication found from the classification of3,000social networks in Section4.In Section5,we suggest an approach for communication gap management which enables us to identify the types of communication,the roles of individuals,and impor-tant ties.After related works are shown in Section6,our conclusions and directions for future work are given in Section7.2Extracting Social NetworksIn a social network based upon online communication, the distance between individuals does not mean‘geo-graphical distance’because each person lives in a virtual world.Instead,distance can be considered‘psychologi-2Fig.1A message chain offive messages sent by three indi-viduals.cal distance’and this can be measured by the“influence”wielded among the members of the network.Consider the situation where an individual p has a great deal of influence on an individual q.In this case, we can consider three types of relationship.Case1.p is close to q.Case2.q is close to p.Case3.p and q are close to each other.Cases1and2show uni-directional relationships,and Case3shows a bi-directional relationship.Most previ-ous studies of social networks employed undirectional social networks,i.e.,Case3,because of the simplicity of analysis.However,the relationship between individu-als is not symmetric because of their activities and social situations[16].In addition,the difference of the distances between two individuals could be a key to understanding the relationships since it shows the communication gap between them.For this reason,we treat a social network as an asymmetric network where vertices denote individ-uals and directed links denote theflows of influence.In this paper,we do not need to distinguish Case1from Case2(e.g.,the direction of the relationship distance) because our approach to measuring the communication gap,described later,produces the same results regardless of the direction.We measure the influnce by using the IDM(Influ-ence Diffusion Model)algorithm in which the influence between a pair of individuals is measured as the sum of propagating terms among them via messages[7].Here, let a message chain be a series of messages connected by post-reply relationships,and the influence of a message x on a message y(x precedes y)in the same message chain be i x→y.Then,i x→y is defined asi x→y=|w x∩···∩w y|,(1) where w x and w y are the set of terms in x and y,re-spectively,and|w x∩···∩w y|is the number of terms propagating from x to y via other messages.If x and y are not in the same message chain,we define i x→y as 0because the terms in x and y are used in a different context and there is no influence between them.Based on the influence between messages,we next measure the influence of an individual p on an individ-ual q as the total influence of p’s messages on other’s messages through q’s messages replying to p’s messages. Let the set of p’s messages beα,the set of q’s messages replying to any ofαbeβ,and the message chains start-ing from a message z beξz.The influence from p onto q,j p→q,is then defined asj p→q=∑x∈α∑z∈β∑y∈ξzi x→y.(2)Here we see the influence of p on q as q’s contribution toward the spread of p’s messages.The influence of each individual is also measurable using j p→q.Let the influ-ence of p be k p,and all other individuals beγ.Then,k pis defined ask p=∑q∈γj p→q.(3)As an example of measuring the influence,let us usethe simple message chain shown in Figure1where Anne posted Message1,Bobby posted Message2as a replyto Message1,and Cathy posted Message3and Message4as replies to Message2and Message1,respectively.In thefigure,solid arrows show the replies to previous messages,and dotted arrows show theflows of influence. Here,the influence between a pair of individuals is as follows.–The influence of Anne on Bobby is3(i.e.,j Anne→Bobby=3),because two terms(“Environment”and“Conve-nience”)were propagated from Anne to Bobby,andone term(“Environment”)was propagated from Anneto Cathy via Bobby.–The influence of Anne on Cathy is1(i.e.,i Anne→Cathy=1),because one term(“Price”)was propagated fromAnne to Cathy.–The influence of Bobby on Cathy is1(i.e.,i Bobby→Cathy=1),because one term(“Environment”)was propa-gated from Bobby to Cathy.–The influence of Bobby on Anne and of Cathy on Anne is0(i.e.,i Bobby→Anne=0and i Cathy→Anne=0),because no term was propagated to Anne fromeither Bobby or Cathy.Note that we ignore the influence of Anne on Cathy, even though a term“Environment”was propagated from Anne to Cathy via Bobby,because we want to measure direct influence between individuals.Instead,we considerthe indirect influence of Anne on Cathy via Bobby asthe contribution of Bobby,and add it to the influence of Anne on Bobby.By mapping the influence between individuals,we can obtain a social network showing influence as in Fig-ure2where their relationships are shown as directional links and the influence between them.From thefigure,we can understand the influential relationships between individuals,and guess their roles in the communication,3Fig.2A social network showing the influence from Figure1.Fig.3A social network showing the distance from Figure 1.e.g.,as opinion leaders and followers [?],from the influ-ence.For example,Anne would be an opinion leader be-cause she was the source of the most influence on others.Bobby would be a mediator because he was a recipient of influence and transmitted some of it to Cathy.Cathy would be a follower because she only received influence from others.The influence between individuals also shows the dis-tance between them with respect to contextual similarity since the influence indicates the degree of their shared interest represented as terms.The influence and contex-tual distance between individuals are inversely related;i.e.,the greater the influence,the shorter the distance.Here,let us define the length of a link (i.e.,distance)as follows.The distance of a link:The distance from an individ-ual p to an individual q ,d p →q ,is defined as the value inversely proportionate to the influence from p to q ;i.e.,d p →q =1/j p →q .The distance is between 0and 1when the influence is more than 0.However,the distance cannot be measured by the above definition if the influence is 0.In that case,we define the distance n −1(n is the number of individ-uals participating in communication)as the case of the weakest relationships;i.e.,the diameter of a social net-work where all individuals are connected linearly with maximum distance.In this way,an asymmetric socialnetwork with distance is extracted from message chains as shown in Figure 3.3Communication Gaps in Social Networks Based on the forward and backward distances between two individuals in a social network,we can consider three types of relationship between them.–Type 1.Two-sided communication:Two indi-viduals actively exchange their ideas with each other.In this case,they are equivalent in communication,and their closeness can be identified by the short dis-tances between them.–Type 2.One-sided communication:One indi-vidual’s idea is received by an individual,while an-other individual’s idea is not.In this case,the two senders are not equivalent in communication,as can be identified from the distance between them since the forward distances and backward distances will differ from each other.–Type 3.Sparse communication:Two individuals rarely exchange ideas with each other.In this case,they are not involved in communication,and their relationship can be identified from the long distances between them.These three types,in other words,correspond to var-ious “communication gaps”between two individuals.By measuring the communication gaps for all pairs of indi-viduals in a social network,we can understand the state of their communication.For example,a small gap indi-cates active communication,while a large gap suggests inactive communication.The distances can also be measured for two individ-uals who are not directly connected but indirectly con-nected via other individuals because they can exchange their ideas through others.Interestingly,the shortest path between individuals is often not an existing direct path,but an indirect path.In this paper,we consider the dis-tance of the shortest path as the distance between two individuals since it reflects the real channel of their com-munication.Here,let the distance of the shortest path from an individual p to another individual q be d p →q (d p →q =0if p is equal to q )and the set of all individuals in a so-cial network be γ.We then propose three indices,G diff ,G max and G min ,that measure the collective communi-cation gap in the social network from different points of view.–G diff measures the communication gap by accumu-lating the differences in the distances of the shortest paths between all pairs of individuals.G diff is de-fined asG diff =(12∑p ∈γ∑q ∈γ|d p →q −d q →p |)4×(1n C 2(n −1)),(4)where |d p →q −d q →p |is the absolute value of (d p →q −d q →p ),and n C 2(n −1)is a theoretical maximum of the first term used for normalizing G diff .–G max measures the communication gap by accumu-lating the longer distances of the shortest paths be-tween all pairs of individuals as a bottleneck hinder-ing communication.G max is defined asG max =(12∑p ∈γ∑q ∈γmax(d p →q ,d q →p ))×(1n C 2(n −1)),(5)where max(d p →q ,d q →p )returns the maximum value from {d p →q ,d q →p },and n C 2(n −1)is a theoreti-cal maximum of the first term used for normalizing G max .–G min measures the communication gap by accumu-lating the shorter distances of the shortest paths be-tween all pairs of individuals.G min is defined asG min =(12∑p ∈γ∑q ∈γmin(d p →q ,d q →p ))×(1n C 2(n −1)),(6)where min(d p →q ,d q →p )returns the minimum value from {d p →q ,d q →p },and n C 2(n −1)is a theoreti-cal maximum of the first term used for normalizing G min .We can also consider another approach to identifying the state of communication by using only the distances between individuals instead of the differences between distances since the distance itself shows another aspect of communication gaps.The approach is known as “close-ness centrality”where individuals nearby are more like to give/receive information more quickly than others [2].Based on closeness centrality,we propose two more in-dices for measuring the collective communication gap,C diff and C dist ,as follows.–C diff measures the communication gap by accumu-lating the differences in the closeness centralities of individuals.C diff is defined asC diff =(∑p ∈γ|c in p −c outp |)×(1n (n −1)2),(7)where c in p means the inward closeness centrality that shows the sum of distances of the shortest paths from all other individuals to p ,c out pmeans the outward closeness centrality that shows the sum of distances of the shortest paths from p to all other individuals,and n (n −1)2is a theoretical maximum of the first term used for normalizing C diff .–C dist measures the communication gap by accumu-lating the closeness centralities of individuals.C dist is defined asC dist =(∑p ∈γc out p )×(1n (n −1)2),(8)where n (n −1)2is a theoretical maximum of thefirst term used for normalizing C dist .Note that C distdoesn’t change even if we use c in p instead of c outp .4Three Types of CommunicationTo identify the types of communication,we employed the five indices proposed in Section 3from 3,000social networks.The analysis procedure was as follows.–Step 1.We downloaded 3,000message boards from 15categories of Yahoo!Japan Message Boards.To equalize the number of messages for each message board,we selected message boards having more than 300messages and downloaded the first 300messages.Then,we removed stop words (words except for noun and verb words)from all the messages to accurately measure content-derived influence.In this way,we prepared 3,000message boards with each having 300messages.–Step 2.We extracted a social network from each message board using the approach described in Sec-tion 2.To equalize the number of individuals in a social network,we constructed a social network with the 10most influential individuals identified by Equa-tion (3).We thus obtained 3,000social networks,each consisting of 10individuals.–Step 3.We measured G diff ,G max ,G min ,C dist ,and C diff for the 3,000extracted social networks,and then averaged each index for each category.–Step 4.We tried to identify the types of communi-cation based on the five indices.The values of the five indices and the sum,G sum ,for the fifteen categories are shown in Table 1.We expected that the five indices would reveal different aspects of communication,however they were indistinguishable be-cause the Pearson correlation coefficients between them were over 0.9.For this reason,we employed G sum for identifying the types of communication.From G sum in Table 1,we could identify three major clusters,each cor-responding to a type of communication.We named the clusters as follows.–Interactive Communication:This cluster includes seven categories (“Arts”,“Sciences”,“Health &Well-ness”,“Culture &Community”,“Romance &Rela-tionships”,“Hobbies &Crafts”,and “Family &Home”),the indices of which are considerably smaller than those of other categories.The topics in these cate-gories are common and familiar to many individuals5Table1Five communicatin gap indices and the sum,G sum,for15categories measured from3,000message boards in Yahoo!Japan Message Boards.Categories C dist C diff G diff G max G min G sumFamily&Home0.0320.0280.0320.0480.0170.157Health&Wellness0.0650.0650.0730.1000.0290.332Arts0.0680.0700.0810.1080.0290.356Science0.0720.0680.0780.1100.0340.362Cultures&Community0.0850.0610.0710.1200.0510.388Romance&Relationships0.0810.0790.0890.1250.0380.412Hobbies&Crafts0.0950.0920.1060.1460.0430.482Regional0.1610.1200.1420.2300.0930.746Entertainment0.1510.1290.1570.2280.0750.740Government&Politics0.2170.1670.1970.3130.120 1.014Business&Finance0.2410.1600.1840.3310.150 1.066Schools&Education0.2530.1610.1950.3490.158 1.116Recreation&Sports0.2390.2080.2530.3620.116 1.178Computers&Internet0.4470.2210.2720.5790.315 1.834Current Events0.4550.2200.2710.5880.322 1.856who share these interests.As a consequence,indi-viduals are naturally involved in the communication, and actively exchange their ideas with others.–Distributed Communication:This cluster includes six categories(“Regional”,“Entertainment”,“Busi-ness&Finance”,“Schools&Education”,“Govern-ment&Politics”,and“Recreation&Sports”),the indices of which are generally higher than those of the interactive communication categories.As the top-ics in these categories are somewhat specific and dis-putable,experienced or knowledgeable individuals con-tribute most to the communication.–Soapbox Communication:This cluster includes two categories(“Computers&Internet”and“Cur-rent Events”),the indices of which are higher than those of the above two clusters.The topics in these categories are mainly current affairs or topical news, and the communication is one-way from informers to audiences(or lurkers).From the above results,we can say that there are roughly three types of communication in Yahoo!Japan Message Boards.By assuming that these types are gen-eral properties in communication,the state of commu-nication of any social networks could be identified from communication gap indices.In this research,we select to use G max for further analysis because of its discrim-inability,i.e.,G max proved to be the most discriminative index for identifying clusters.5Communication Gap Management Communication gap management is done to identify the types of communication,understand the roles of individ-uals,and explore remedies for communication gaps in social networks.In this section,we present case studies of communication gap management based on communi-cation types and G max explained before.5.1Communication TypesWe prepared two types of message log,log1and log 2,each of which was extracted from a message board on DISCUS3[3].Log1consisted of72messages posted byfive individuals(one Japanese researcher,one Cata-lan researcher,and three Japanese businesspersons)who discussed“cell phones and women”in English.Log2 consisted of102messages posted by six individuals(two Japanese researchers,one Catalan researcher,one Japanese businessperson,one Chinese student,and one Taiwanese student)who discussed“purchase of a house”in English. Note that the real names of individuals have been re-placed withfictional names to protect their privacy.Figures4and5are social networks showing the dis-tance extracted from log1and log2,respectively,using the approach described in Section2.Once we obtained the social networks,G max was measured as0.069from Figure4and0.364from of paring G max with the clustering results in Section4,we can identify the types of communication of log1and log2as“interac-tive communication”and“distributed communication”, respectively.The topic in log1was familiar to the individuals be-cause they use cell phones everyday.On the other hand, the purchase of a house is a big event in life,and many in-dividuals have no experience of making such a purchase. Therefore,the communication in log2was controlled by a few experienced individuals.Thus,the communication types of log1and log2seem to have properly reflected the types of real communication.5.2Roles of IndividualsAs shown by the definition of G max in Equation(5), G max is measured by summing each individual’s commu-3/6Fig.4The social network with distance extracted from log1.G max =0.069.Fig.5The social network with distance extracted from log 2.G max =0.364.nication gaps with respect to other individuals.That is,G max for each individual is easily measurable by trans-lating the definition of G max .Let the communication gap of an individual p be g max .We can then define g max asg max =(12∑q ∈γmax(d p →q ,d q →p ))×(2n (n −1)),(9)where n (n −1)/2is the theoretical maximum of the firstterm used for normalizing g max .As the G max of Figure 5is high,let us measure each individual’s g max to reveal the source of the communi-cation gap.As shown in Table 2,Fabian has the highest g max .From this,we can understand that there are com-munication gaps around Fabian.The roles of individuals are also identified from their relationships with others.If removing a link raises G max ,the link is considered to help reduce the communication gap and is therefore important.That is,we can measure the importance of each link by comparing the G max of the original social network with that link to G max of a social network without the link.Let the importance of a link be l max .We then define l max as l max =G ′max −G max ,(10)where G ′max is the G max measured from a social network without the link.From the top three l max values in Fig-ure 5listed in Table 3,we can see that a link from Erika to Anna is the most important.Table 2g max of each individual in Figure 5.AnaBill Claudette Danny Erika Fabian g max0.0380.0390.0380.0400.0420.167Table 3Three most important links in Figure 5.Linkl max G ′max G max Erika →Ana 0.5740.9380.364Danny →Ana 0.2700.6340.364Ana →Erika0.2260.5900.364During face-to-face interviews with participants in Figure 5with showing Figure 6,we found that they con-sidered Fabian creative,but strong-willed to the degree that nobody could counter his ideas.As a result,the communication around him did not go well.They also agreed with the results regarding the important links in Table 3because these were the links actively used to exchange ideas during the period under study.5.3Remedies for Communication GapsOnce we can obtain information about who are the causes of communication gaps and which links are most im-portant for communication,we can prepare a remedy to improve communication.For example,the following three approaches could be the candidates of the reme-dies,apart from the feasibility.–Persuade inactive individuals to contribute to the communication,or persuade strong individuals to lis-ten to others’ideas.While this approach is straight-forward,the effectiveness of such persuasion depends on many factors which are beyond the scope of this paper.–Take inactive or strong individuals away from the communication.This approach is easy and has an immediate effect,but it is not constructive since we would lose the potential contributions of the excluded individuals.For example,Fabian in Figure 5should not be taken away because he supplied creative ideas although there were large gaps around him.Instead,we should consider how to get him into the commu-nication.–Add another individual who can communicate with inactive or strong individuals to bridge communica-tion gaps.To find such individuals,the link impor-tance can be used.For example,if we add Mr.X who has a link of distance 1.0to Anne and from Fabian to the social network in Figure 5,G max would dynami-cally drop from 0.364to 0.157.Needless to say,there approaches above should be planned carefully before putting into practice.Having an inverview with individuals for surveying the effect of7a remedy would further enhance the possibility of the success of communicatin gap management.Each remedy mentioned above corresponds to,in other words,a“sce-nario”for realizing fertile community.As the scenario is analytically drawn up from observed communication, it enable us to realize prompt and smooth revolution of communitation.6Related WorksSocial networks have been studied for decades.One of the origins of social network research is Moreno’s work of Sociometry[10].Bavelas analyzedn communication net-work in a small group to identify communication pat-terns[1].Milgram discovered what we now call‘small-world phenomena’which show that everyone is connected to each other through a short chain in their social net-works[9].Rogers classified each person intofive types according to their stage regarding the adoption of new ideas[13].Granovetter insisted that weak ties spanning local relationship boundaries contribute to the diffusion of information[5].Freeman proposed centrality measures to identify the importance of individuals and ties in a social network[2].Scott used social networks to identify gaps in informationflow within an organization tofind ways to get work done more effectively[14].Krackhardt studied the importance of informal networks in organi-zations and revealed the effect of these networks on the accomplishment of tasks[6].Social networks in online communication have been studied as well.Ohsawa et al.classified communities into six types according to the structural features of the word co-occurrence structure of communications[11].Tyler et al.analyzed e-mail logs within an organization to iden-tify communities of practice–informal collaborative net-works–and leaders within the communities[15].Mat-sumura et al.revealed the effect of anonymity and ASCII art on communication through message boards[8].7ConclusionHuman beings are social creatures,and we could not survive without cooperating with others.It also suggests that understanding how relationships are created and functioned is essential to make our lives happier and richer.The range of our social networks is rapidly ex-panding than before as the Internet accelerates our com-munication via E-mail,Chat,Video conference system etc.Reflecting such a situation,managing and utilizing social networks will be an increasingly important part of our lives.We hope this study will contribute to the realization of a better way of life through human rela-tionships over the Internet as well as real world.Acknowledgements The research was supported by Chance Discovery Consortium.We are extremely grateful to Prof. Yukio Ohsawa,Hiroshi Tamura,Yuichi Washida,and all the members of IlliGAL for their insightful discussions.We also greatly appreciate the two anonymous reviewers for their con-structive comments.References1. A.Bavelas,Communication patterns in task-orientedgroups,Journal of Acoustical Society of America,22,725-730,1950.2.L.C.Freeman,Centrality in Social Networks,Social Net-works,1,215–39,1978.3. D.E.Goldberg,M.Welge and X.Llor`a,DISCUS:Dis-tributed Innovation and Scalable Collaboration In Uncer-taion Settings,IlliGAL Report No.2003017,2003.4. D.E.Goldberg,The Design of Innovation,Kluwer Aca-demic Publishing,2002.5.M.Granovetter,The Strength of Weak Ties,AmericanJournal of Sociology,78,1360–1380,1973.6. D.Krackhardt and J.R.Hanson,Informal Networks:TheCompany behind the Chart,Harvard Business Review, July-August1993,104–111,1993.7.N.Matsumura,Topic Diffusion in a Community,YukioOhsawa and Peter McBurney(Eds.),Chance Discovery, Springer Verlag,84–97,2003.8.N.Matsumura,A.Miura,Y.Shibanai,Y.Ohsawa andT.Nishida The Dynamism of2channel,Journal of AI& Society,Springer,19,84–92,2005.gram,The Small World Problem,Psychol.Today,1,61–67,1967.10.J.L.Moreno,Who Shall Survive?Beacon,NY:BeaconHouse,1934.11.Y.Ohsawa,H.Soma,Y.Matsuo,N.Matsumura andui,Featuring Web Communities Based on Word Co-occurrence Structure of Communications,Proc.of WWW2002,736–742,2002.12.Y.Ohsawa and P.McBurney(Eds.),Chance Discovery,Springer-Verlag,2003.13. E.M.Rogers,Diffusion of Innovations(Fourth Edition),New York,Free Press,1962.14.W.R.Scott,Organizations:Rational,Natural,and OpenSystems,Prentice-Hall,Inc.,Englewood Cliffs,New Jer-sey,1992.15.J.R.Tyler, D.M.Wilkinson and B. A.Huber-man,Email as spectroscopy:Automated Discovery of Community Structure within Organizations.(Preprint /arXiv:cond-mat/0303264),2003. 16.P.Wallace,The Psychology of the Internet,CambridgeUniversity Press,1999.Naohiro Matsumura is an assistant professor of the Graduate School of Economics,Osaka University.He re-ceived the BE and the ME in Engineering Science from Osaka University in1998and2000,and the PhD in Engi-neering in2003from the University of Tokyo.His main research interests are Fieldmining and Chance Discov-ery.He is the founder of Fieldmining–a researchfield。