An Efficient Spectral Algorithm for Network Community Discovery

An Efficient Spectral Algorithm for Network Community Discovery and Its Applications to Biological and Social NetworksJianhua Ruan and Weixiong ZhangDepartment of Computer Science and EngineeringWashington University in St LoiusOne Brookings Dr,St Louis,MO63130{jruan,zhang}@AbstractAutomatic discovery of community structures in complex networks is a fundamental task in many disciplines,includ-ing social science,engineering,and biology.Recently,a quantitative measure called modularity(Q)has been pro-posed to effectively assess the quality of community struc-tures.Several community discovery algorithms have since been developed based on the optimization of Q.However, this optimization problem is NP-hard,and the existing al-gorithms have a low accuracy or are computationally ex-pensive.In this paper,we present an efficient spectral algo-rithm for modularity optimization.When tested on a large number of synthetic or real-world networks,and compared to the existing algorithms,our method is efficient and and has a high accuracy.In addition,we have successfully ap-plied our algorithm to detect interesting and meaningful community structures from real-world networks in different domains,including biology,medicine and social science. Due to space limitation,results of these applications are presented in a complete version of the paper available on our website(/˜jruan/).1IntroductionThe study of complex networks has become a fast grow-ing subject in many disciplines,including physics,biol-ogy,and social science.At least part of the reason can be attributed to the discovery that real-world networks from totally different sources can share surprisingly high simi-larities in their topological properties,such as the power-law degree distributions and high clustering coefficients. (See[1,14]for reviews.)One of the key properties in complex networks that have attracted a great deal of interest recently is the so-called community structures,i.e.relatively densely connected sub-networks[15].Community structures have been found in social and biological networks,as well as technological networks such as the Internet and power grid.Automati-cally discovering such structures is fundamentally impor-tant for understanding the relationships between network structures and functions,and has many practical applica-tions.For example,identifying communities from a collab-oration network may reveal scientific activities as well as evolution and development of research areas[12],while de-tecting hidden communities on the World Wide Web may help prevent crime and terrorism[2].To design effective community discovery algorithms, Newman and Girvan[16]proposed a quantitative measure, called modularity(Q),to assess the quality of community structures,and formulated community discovery as a opti-mization problem.Since optimizing Q is a NP-hard prob-lem,several heuristic methods have been developed,as sur-veyed in[4].The fastest algorithm available uses a greedy strategy and suffers from poor quality[3].A more accu-rate method is based on simulated annealing,which requires a prohibitively long running time on large networks[11]. Several spectral algorithms have been developed,which have relatively good performance,but still inefficient for large networks[21,15].In this paper,we propose a spectral algorithm that is ef-fective infinding high quality communities as well as effi-cient on large networks.The algorithm adopts a recursive strategy to partition networks while optimizing Q.Unlike the existing algorithms,our method is a hybrid of direct k-way partitioning and recursive2-way partitioning strate-gies[21,15].We evaluate our algorithm on a large number of synthetic and real-world networks.The results show that the algorithm is more efficient and more accurate than a re-cursive2-way partitioning pared to a direct k-way partitioning method,our algorithm is much more ef-ficient,while having a comparable accuracy.The paper is organized as follows.In Section2,we intro-duce some basic concepts,notations,and previous works.Seventh IEEE International Conference on Data MiningIn Section3,we describe our algorithm and its complexity, and discuss several related methods.We present experimen-tal results in Section4,and conclude in Section5.2Preliminaries2.1Spectral graph partitioningLet G=(V,E)be a network of n vertices in V and m edges in E.Let A=(A ij)be the adjacency matrix of G.A graph partitioning problem is tofind two or more vertex subsets of nearly equal sizes,while minimizing the num-ber of edges cut by the partitioning[8].Known to be NP-hard,the problem exists in many real applications,such as circuit design and load balancing in distributed computing. Many heuristic methods have been developed for the prob-lem,among which spectral methods have received much at-tention and are the most popular.Spectral graph partitioning is in fact a family of methods. These methods depend on the eigenvectors of the Laplacian matrix or its relatives of a graph.Depending on the way they partition a graph,spectral methods can be classified into two classes.Thefirst class uses the leading eigenvec-tor of a graph Laplacian to bi-partition the graph.The sec-ond class of approaches computes a k-way partitioning of a graph using multiple eigenvectors.We briefly review some representatives of these two classes of algorithms below. Let D be the diagonal degree matrix of A,i.e.D ii=j A ij.L=D−A is called the Laplacian matrix of G.Letλ1≤λ2≤···≤λn be the eigenvalues andµ1,µ2,···,µn be the corresponding eigenvectors for the generalized eigen-value problem Lµ=λDµ.It can be shown thatλ1=0, andµ1=1,a vector with all ones.Given the above notation,a representative of bi-partitioning,the SM algorithm[19],works as follows.(1) Computeµ2,the second smallest generalized eigenvector of L.(2)Conduct a linear search onµ2tofind a partition of the graph to minimize a normalized cut criterion[19].It has been shown that when certain constraints are satisfied,the SM algorithm can reach the optima of normalized cuts[19]. Tofind more than two clusters,the SM algorithm can be applied recursively.The most popular algorithm in the second class,the NJW algorithm[17],finds a k-way partition of a network directly as follows,where k is given by the user.(1)Compute the k smallest generalized eigenvectors of L and stack them in columns to form a matrix Y=[µ1,µ2,···,µk].(2)Nor-malize each row of Y to have unit length.(3)Treat each row as a point in R k,and apply standard k-means algorithm (or any other geometric clustering algorithm)to group them into k clusters.2.2Modularity and community structures Given a partition of a network,Γk,which divides its vertices into k communities,the modularity is defined asQ(Γk)=ki=1(e ii/c−(a i/c)2),where e ii is the number of edges with both vertices within community i,a i is the number of edges with one or both vertices in community i, and c is the total number of edges[16].Therefore,the Q function measures the fraction of edges falling within com-munities,subtracted by what one would expect if the edges were randomly placed.A larger Q value means stronger community structures.If a partition gives no more within-community edges than would be expected by chance,the modularity Q≤0.For a trivial partitioning with a single community,Q=0.It has been observed that most real-world networks have Q>0.3[16].The Q function provides a good quality measure to com-pare different community structures.Several algorithms have been developed to search for community structures by looking for the division of a network that optimizes Q(see[4]for a survey).White and Smyth proposed a spectral algorithm(WS),which is effective on small net-works[21].They show that,when the number of commu-nities k is given,the optimization of Q is equivalent to an eigen decomposition problem,if relaxing the discrete mem-bership constraint[21].Therefore,they directly applied a k-way spectral graph partitioning algorithm for this pur-pose.To automatically determine the number of communi-ties,the spectral algorithm is executed multiple times,with k ranging from the user defined minimum K min to maxi-mum K max number of communities.The k that gives the highest Q value is deemed the most appropriate number of communities.A slightly modified version of the WS algo-rithm is as follows.(1)For each k,K min≤k≤K max, apply NJW tofind a k-way partition,denoted asΓk.(2) k∗=arg max k Q(Γk)is the number of communities,and Γ∗=Γk∗is the best community structure.While the WS algorithm is effective infinding good community structures,it scales poorly to large networks, because it needs to execute k-means up to K max times. Without any prior knowledge of a network,one may over-estimate K max in order to reach the optimal Q.For sparse networks,K max can be linear in the number of vertices in the worst case,making it impractical to iterate over all pos-sible k’s for large networks.3The Kcut algorithmIn order to develop a method that scales well to large networks while retaining effectiveness infinding good com-munities,we may take the strategy used in the SM algo-rithm,i.e.,to recursively divide a network into smaller ones. However,two issues remain.First,when should the algo-rithm halt,or in other words,how do we decide whethera (sub)network should be partitioned?Since our goal is to find a partition with a high modularity,we can test whether the Q value increases after the partition.If no partition can improve the modularity,the (sub)network should not be di-vided.Second,it has been empirically observed that if there are multiple communities,using multiple eigenvectors to di-rectly compute a k -way partition is better than recursive bi-partitioning methods [17].Here,we propose an algorithm that is a unique combination of recursive partitioning and direct k -way methods,which will achieve the efficiency of a recursive approach,while also having the same accuracy as a direct k -way method.We follow a greedy strategy to recursively partition a net-work to optimize Q .Unlike the existing algorithms that al-ways seek a bi-partition,we adopt a direct k -way partition strategy as in the WS algorithm.Briefly,we compute the best k -way partition with k =2,3,···,l using the NJW algorithm,and select the k that gives the highest Q value.Then for each subnetwork the algorithm is recursively ap-plied.To reduce the computation cost,we restrict l to small integers.As we will shown in experiments,the algorithm with l as small as 3or 4can significantly improve mod-ularity over the standard bi-partitioning strategy.Further-more,the computation cost is also reduced with a slightly increased value of l compared to bi-partitioning.Given a network G and a small integer l that is the max-imal number of partitions to be considered for each subnet-work,our algorithm Kcut executes the following steps.1.Initialize Γto be a single cluster with all vertices,and set Q =0.2.For each cluster P in Γ,(a)Let g be the subnetwork of G containing the ver-tices in P .(b)For each integer k from 2to l ,i.Apply NJW to find a k -way partitioning of g ,denoted by Γg k ,pute new Q value of the network as Qk =Q (Γ Γgk \P ).(c)Find the k that gives the best Q value,i.e.,k ∗=arg max k Q k .(d)If Q k ∗>Q ,accept the partition by replacing P with Γgk ∗,i.e.,Γ=Γ Γg k ∗\P ,and set Q =Q k ∗.(e)Advance to the next cluster in Γ,if there is any.The inner loop,step 2(b),is similar to the first step of the WS algorithm,except that in 2(b)(ii)we compute the mod-ularity of the whole network G ,which is different from the modularity Q (Γg k ).On the other hand,we do not need to iterate over all communities in the network to re-compute Q .From the definition of Q in Section 2.2,the contributionof each community towards Q is independent of the other communities.Therefore,after g is partitioned,Q can be ef-ficiently updated with the communities that have just been created in g .At step 2(c),we decide the best way to parti-tion g that can improve Q the most.This step turns out to be crucial in identifying globally good community structures with high Q values.At step 2(d),we test if partitioning g can contribute positively towards Q ,and the partition is ac-cepted only if Q increases.When the algorithm terminates,no communities can be further created to improve Q ,thus Γcontains the best community structure.3.1Computational complexityWe first review the computational complexity of the WS algorithm,since the inner loop of Kcut is simply the WS algorithm,except that the computation of Q is slightly dif-ferent.The WS algorithm contains two major components:computing eigenvectors and executing k -means to partition the network.Note that although WS calls NJW multiple times,the eigen problem needs to be solved only once to obtain all K max eigenvectors.To compute eigenvectors,we used the eigs function in MATLAB,which has a time complexity in O (mKh +nK 2h +K 3h ),where m and n are respectively the numbers of edges and vertices of the graph,K =K max is the number of eigenvectors to be computed,and h is the number of iterations for eigs to con-verge [21].Since K <n ,the running time of eigs can be simplified to O (mKh +nK 2h ).Second,we adopted a fast k -means algorithm [6]in our implementation,which takes approximately O (nKe )time,where e is the number of iter-ations for k -means to converge.Since k -means is called K times,the total running time is O (mKh +nK 2h +nK 2e ),where the first two terms are for eigs and the last term is for k -means.Assuming e and h constants,the overall time complexity of WS is O (mK +nK 2),which can be close to O (n 3),since the maximal number of communities for a sparse network may be linear in n .The running time of Kcut depends on the depth of the re-cursive calls.In the worst case,the partitions can be highly imbalanced,and the depth of the recursion is merely the number of partitions produced,K .A more practical es-timate,however,is the average depth,which is close to log l K ,where l is the maximal number of partitions con-sidered by NJW .Therefore,the running time taken by eigs can be estimated to be O ((mlh +nl 2h )log l K ),which can be further simplified to O (mlh log l K ),since l is small and therefore in general m >nl .Similarly,the average-case running time taken by k -means is O (nl 2e log l K ),and the total complexity is given by O ((mlh +nl 2e )log l K ).Our experimental results show that for large networks and small values of l ,the time taken by eigs dominates,giving an overall time complexity in O (mlh log l K )=O (mh ln K lstant,also given that l is small and K=O(n),the total complexity is O(m log n),which is much smaller than the O(n3)running time of the WS algorithm.An important ob-servation from the analysis is that the total running time of Kcut is not a monotonically increasing function of l.Ana-lytically,the minimum value of l/ln l is achieved at l=3. Empirically,we observed that Kcut is most efficient with l=3to5(see Section4.2).The memory complexity of both algorithms is O(m), linear to the number of edges.3.2Related methodsBesides our algorithm and the WS method,several other algorithms have also been developed for identify-ing communities by modularity optimization.Newman proposed an algorithm that is based on recursive spectral bi-partitioning[15].The algorithm computes the leading eigenvector of a so-called modularity matrix,and divides the vertices into two groups according to the signs of the elements in the eigenvector.The algorithm runs recursively on each subnetwork,until no improvement to Q is pared to our method,this algorithm is faster for small networks,since no k-means is performed.On the other hand,the modularity matrix is very dense,with al-most no zero entries.Therefore,the algorithm takes O(n2) memory even for sparse networks,in contrast to O(m)for our method.Furthermore,the algorithm takes O(n2log n) running time,therefore,it does not scale well to large net-works.Importantly,we will show that by combining k-way partitioning with a recursive method,Kcut usually achieves higher modularity than the Newman method.There are also several methods that are not spectral-based.The edge betweenness algorithm[9]and the ex-tremal optimization algorithm[5]are known to be very slow,with O(n3)and O(n2log2n)running time,respec-tively.Another greedy approach,the C NM algorithm[3], has approximately the same time complexity(O(m log2n)) as our method,but the communities returned often have poor quality[15].4EvaluationWe now evaluate our algorithm on a variety of networks and compare it with three existing algorithms that were mentioned in Section3.2:the WS algorithm,the CNM algo-rithm,and the Newman’s algorithm(NM).In what follows, the results of our algorithm are denoted by K-2,K-3,···,for l=2,3,···.Note that Newman suggested in[15]a refining step to improve Q after the initial partitioning.To make a fair comparison,this refining step was omitted in our study, since in theory the same strategy can be applied to any other algorithm as well.Figure1.Results on computer-generated networks.Q relative=Q discovered−Q true.4.1Computer-generated networksTo evaluate Kcut,wefirst tested it on computer-generated networks with artificially embedded community structures.Each network had256vertices forming8com-munities of equal sizes.Edges were randomly placed with probability p in between vertices within the same commu-nity and with probability p out between vertices in different communities.We varied p in from0.8to0.3,representing networks with dense to sparse communities.For each p in, we varied p out from0to p in50.For each pair of(p in,p out),we generated100networks and clustered them with WS(K min=2,K max=15),Kcut (l=2,3,4and5),and NM algorithms.To measure the ac-curacy of the results,we computed the Jaccard Index[20], which is roughly the percentage of within-community edges that were predicted correctly.The Jaccard Index between the true community structure(Γ)and predicted community structure(Γ )is defined asJ(Γ,Γ )=|S(Γ)∩S(Γ )|Table1.Q values for real-world networks.Qn m K∗K maxKarate0.4200.3900.4200.4200.4200.3930.3830.420[21] Football0.6020.5240.6000.5960.5900.4930.577Jazz0.4390.4440.4440.4390.4390.3940.4390.445[5] PPI0.3620.3320.3440.3480.3640.3410.337Internet0.6040.5940.6000.6010.6010.5240.620 Physicists-0.7340.7380.7390.743-0.6590.723[15] K max:maximal number of communities for WS.K∗:number of communities returned by WS.The last column are the best Q values achieved by existing methods in the literature,and references to the methods.Table2.Total CPU time(seconds). NetworkKarate1.10.70.60.7 1.10.30.04 Jazz8k40263123580.8 Internet-6k3k2k2k-283 *A significant difference between CNM and the other algorithms here is that CNM was implemented in C,while all the other algo-rithms compared here were implemented in MATLAB m-files. Fig.1(a)shows the Jaccard Index as a function of p out for p in=0.5.Results for other values of p in or using other types of accuracy measurement are similar(data not shown).The WS algorithm,which explicitly searches over all k’s,has the best accuracy.On the other hand,Kcut with large l values can better approximate WS than with small l values.Moreover,as shown in Fig.1(b),the Q values achieved by the algorithms match their accuracies:WS has the highest modularity,followed by K-5,K-4,...,and the Newman algorithm at last.A third measure,the number of times an algorithm predicted k correctly,also shows that WS>K-5>···>K-2>NM(data not shown).The CNM algorithm has an accuracy similar to K-2for smaller p out, but its accuracy drops significantly when p out increases.4.2Real-world networksWe further tested our method on several real-world net-works.These include an acquaintance network in a Karate club[22],the opponent network of American NCAA Di-vision I college football teams in the year2000[9],a co-performing network of Jazz Bands[10],a protein-protein interaction network of E.coli[18],the Autonomous Sys-tems topology of the Internet[7],and a collaboration net-work of physicists[13].As shown in Table1,the WS algo-rithm usually returns community structures with the highest Q value.Although Kcut with l=2often performed poorly, Kcut with l≥3can usually achieve Q values as good as that by WS,whereas with a much reduced running time. Moreover,for the three networks(Karate,Jazz,Physicists) that have been analyzed by others,Kcut canfind modularity values that are comparable to or better than the best known ones.The NM algorithm(without the refining step)and the CNM algorithm usually have much worse accuracy com-paring to WS and Kcut.The WS and Newman algorithms failed tofinish on the physicist network,due to their exces-sive running time or memory usage.In addition,the communities returned by Kcut are often very close to the known communities if they are available. For example,for the Karate club network,Kcut precisely predicted the actual separation of the club caused by a dis-pute among its members[9].For the football network,Kcut correctly revealed the official NCAA conference structure of the football teams[9],except for a few teams that do not belong to any conference.Because of space limit,we omit the detailed results here.4.3Running timeTable2shows the running time of the four algorithms on the six real-world networks.Table3shows the time spent on eigs and k-means by WS,Kcut and M is based on a different rationale and does not have these two com-ponents.As shown in Table2,although WS is efficient for small networks of up to a few hundred of vertices,it is very inefficient on large networks.The Kcut algorithm,on the other hand,can handle networks of several thousand of ver-tices in less than half minute.It appears in Table2that CNM is the most efficient,especially for small networks.At least part of the reason is that CNM was implemented in the C language,while the other three algorithms were all imple-mented in MATLAB M-files.M-files are interpreted at run time,and therefore have higher overhead.Also observe that Kcut is often faster with l=3,4,5 than with l=2.Based on the analysis in Section3.1,the time Kcut spent in eigs is approximately linear to l/ln l, which reaches its minimum at l=3.In contrast,the time Kcut spent on k-means is proportional to l2/ln l,which is monotonically increasing for l≥2.The experimental re-sults in Table3partially support the theoretical analysis. For large networks,the total running time of Kcut is dom-inated by eigs.Therefore,Kcut can take advantage of a slightly increased l to reduce its running time.When l be-Table3.CPU time(seconds)for program components. Network K-2K-4NM0.080.160.10.110.080.220.10.810.210.230.230.70.110.290.310.230.250.5147857204118122892195119--24511091473170。