Clustering Signed Networks with the Geometric Mean of Laplacians PedroMercado1,FrancescoTudisco2andMatthiasHein1 1SaarlandUniversity,Saarbrücken,Germany 2UniversityofPadua,Padua,Italy 7 1 0 2 Abstract n a Signednetworksallowtomodelpositiveandnegativerelationships. Weanalyze J existing extensions of spectral clustering to signed networks. It turns out that 3 existingapproachesdonotrecoverthegroundtruthclusteringinseveralsituations whereeitherthepositiveorthenegativenetworkstructurescontainnonoise. Our ] analysisshowsthattheseproblemsariseasexistingapproachestakesomeformof L arithmeticmeanoftheLaplaciansofthepositiveandnegativepart. Asasolution M weproposetousethegeometricmeanoftheLaplaciansofpositiveandnegative partandshowthatitoutperformstheexistingapproaches. Whilethegeometric . t meanofmatricesiscomputationallyexpensive,weshowthateigenvectorsofthe a t geometricmeancanbecomputedefficiently,leadingtoanumericalschemefor s sparsematriceswhichisofindependentinterest. [ 1 v 1 Introduction 7 5 7 Asignedgraphisagraphwithpositiveandnegativeedgeweights. Typicallypositiveedgesmodel 0 attractiverelationshipsbetweenobjectssuchassimilarityorfriendshipandnegativeedgesmodel 0 repellingrelationshipssuchasdissimilarityorenmity. Theconceptofbalancedsignednetworks . canbetracedbackto[10,3]. Later,in[5],asignedgraphisdefinedask-balancedifthereexists 1 apartitionintok groupswhereonlypositiveedgesarewithinthegroupsandnegativeedgesare 0 betweenthegroups. Severalapproachestofindcommunitiesinsignedgraphshavebeenproposed 7 1 (see [23] for an overview). In this paper we focus on extensions of spectral clustering to signed : graphs. Spectralclusteringisawellestablishedmethodforunsignedgraphswhich,basedonthe v firsteigenvectorsofthegraphLaplacian,embedsnodesofthegraphsinRk andthenusesk-means i X tofindthepartition. In[16]theideaistransferredtosignedgraphs. Theydefinethesignedratio andnormalizedcutfunctionsandshowthatthespectrumofsuitablesignedgraphLaplaciansyielda r a relaxationofthoseobjectives. In[4]otherobjectivefunctionsforsignedgraphsareintroduced. They showthatarelaxationoftheirobjectivesisequivalenttoweightedkernelk-meansbychoosingan appropriatekernel. Whiletheyhaveascalablemethodforclustering,theyreportthattheycannot findanyclusterstructureinrealworldsignednetworks. We show that the existing extensions of the graph Laplacian to signed graphs used for spectral clusteringhaveseveredeficiencies. Ouranalysisofthestochasticblockmodelforsignedgraphs showsthat,evenfortheperfectlybalancedcase,recoveryoftheground-truthclustersisnotguaranteed. Thereasonisthattheeigenvectorsencodingtheclusterstructuredonotnecessarilycorrespondto thesmallesteigenvalues,thusleadingtoanoisyembeddingofthedatapointsandinturnfailure of k-means to recover the cluster structure. The implicit mathematical reason is that all existing extensionsofthegraphLaplacianarebasedonsomeformofarithmeticmeanofoperatorsofthe positiveandnegativegraphs. Inthispaperwesuggestasasolutiontousethegeometricmeanof the Laplacians of positive and negative part. In particular, we show that in the stochastic block modelthegeometricmeanLaplacianallowsinexpectationtorecovertheground-truthclustersin 29thConferenceonNeuralInformationProcessingSystems(NIPS2016),Barcelona,Spain. anyreasonableclusteringsetting. Amainchallengeforourapproachisthatthegeometricmean Laplacianiscomputationallyexpensiveanddoesnotscaletolargesparsenetworks. Thusamain contributionofthispaperisshowingthatthefirstfeweigenvectorsofthegeometricmeancanstillbe computedefficiently. OuralgorithmisbasedontheinversepowermethodandtheextendedKrylov subspacetechniqueintroducedby[8]andallowstocomputeeigenvectorsofthegeometricmean A#BoftwomatricesA,BwithoutevercomputingA#Bitself. InSection2wediscussexistingworkonLaplaciansonsignedgraphs. InSection3wediscussthe geometricmeanoftwomatricesandintroducethegeometricmeanLaplacianwhichisthebasisofour spectralclusteringmethodforsignedgraphs. InSection4weanalyzeourandexistingapproachesfor thestochasticblockmodel. InSection5weintroduceourefficientalgorithmtocomputeeigenvectors of the geometric mean of two matrices, and finally in Section 6 we discuss performance of our approachonrealworldgraphs. . 2 Signedgraphclustering Networksencodingpositiveandnegativerelationsamongthenodescanberepresentedbyweighted signedgraphs. Considertwosymmetricnon-negativeweightmatricesW+ andW−,avertexset V = {v ,...,v },andletG+ = (V,W+)andG− = (V,W−)betheinducedgraphs. Asigned 1 n graphisthepairG± = (G+,G−)whereG+ andG− encodepositiveandthenegativerelations, respectively. Theconceptofcommunityinsignednetworksistypicallyrelatedtothetheoryofsocialbalance. Thistheory,aspresentedin[10,3],isbasedontheanalysisofaffectiveties,wherepositivetiesarea sourceofbalancewhereasnegativetiesareconsideredasasourceofimbalanceinsocialgroups. Definition1([5],k-balance). Asignedgraphisk-balancedifthesetofverticescanbepartitioned intoksetssuchthatwithinthesubsetsthereareonlypositiveedges,andbetweenthemonlynegative. Thepresenceofk-balanceinG±impliesthepresenceofkgroupsofnodesbeingbothassortative inG+ anddissassortativeinG−. Howeverthissituationisfairlyrareinrealworldnetworksand expectingcommunitiesinsignednetworkstobeaperfectlybalancedsetofnodesisunrealistic. InthenextsectionwewillshowthatLaplaciansinspiredbyDefinition1arebasedonsomeformof arithmeticmeanofLaplacians. AsanalternativeweproposethegeometricmeanofLaplaciansand showthatitisabletorecovercommunitieswheneitherG+isassortative,orG−isdisassortative,or both. ResultsofthispaperwillmakeclearthattheuseofthegeometricmeanofLaplaciansallowsto recognizecommunitieswherepreviousapproachesfail. 2.1 LaplaciansonUnsignedGraphs Spectralclusteringofundirected,unsignedgraphsusingtheLaplacianmatrixisawellestablished technique(see[19]foranoverview). GivenanunsignedgraphG=(V,W),theLaplacianandits normalizedversionaredefinedas L=D−W L =D−1/2LD−1/2 (1) sym whereD = (cid:80)n w isthediagonalmatrixofthedegreesofG. BothLaplaciansarepositive ii j=1 ij semidefinite,andthemultiplicitykoftheeigenvalue0isequaltothenumberofconnectedcompo- nentsinthegraph. Further,theLaplacianissuitableinassortativecases[19],i.e.fortheidentification ofclustersundertheassumptionthattheamountofedgesinsideclustershastobelargerthanthe amountofedgesbetweenthem. Fordisassortativecases,i.e.fortheidentificationofclusterswheretheamountofedgeshastobe largerbetweenclustersthaninsideclusters,thesignlessLaplacianisabetterchoice[18]. Giventhe unsignedgraphG=(V,W),thesignlessLaplaciananditsnormalizedversionaredefinedas Q=D+W, Q =D−1/2QD−1/2 (2) sym BothLaplaciansarepositivesemi-definite,andthesmallesteigenvalueiszeroifandonlyifthegraph hasabipartitecomponent[6]. 2 2.2 LaplaciansonSignedGraphs RecentlyanumberofLaplacianoperatorsforsignednetworkshavebeenintroduced. Considerthe signedgraphG± =(G+,G−). LetD+ =(cid:80)n w+ bethediagonalmatrixofthedegreesofG+ ii j=1 ij andD¯ =(cid:80)n w++w− theoneoftheoveralldegreesinG±. ii j=1 ij ij ThefollowingLaplaciansforsignednetworkshavebeenconsideredsofar LBR =D+−W++W−, LBN =D¯−1LBR, (balanceratio/normalizedLaplacian) (3) LSR =D¯ −W++W−, LSN =D¯−1/2LSRD¯−1/2, (signedratio/normalizedLaplacian) andspectralclusteringalgorithmshavebeenproposedforG±,basedontheseLaplacians[16,4]. LetL+ andQ− betheLaplacianand thesignlessLaplacianmatricesofthegraphs G+ andG−, respectively. WenotethatthematrixL blendstheinformationsfromG+andG−into(twice)the SR arithmeticmeanofL+andQ−,namelythefollowingidentityholds L =L++Q−. (4) SR Thus, asanalternativetothenormalizationdefiningL fromL , itisnaturaltoconsiderthe SN SR arithmetic mean of the normalized Laplacians L = L+ + Q− . In the next section we AM sym sym introducethegeometricmeanofL+ andQ− andproposeanewclusteringalgorithmforsigned sym sym graphsbasedonthatmatrix. Theanalysisandexperimentsofnextsectionswillshowthatblending theinformationfromthepositiveandnegativegraphstroughthegeometricmeanovercomesthe deficienciesshowedbythearithmeticmeanbasedoperators. 3 GeometricmeanofLaplacians Wedefineherethegeometricmeanofmatricesandintroducethegeometricmeanofnormalized Laplaciansforclusteringsignednetworks. LetA1/2betheuniquepositivedefinitesolutionofthe matrixequationX2 =A,whereAispositivedefinite. Definition2. LetA,Bbepositivedefinitematrices. ThegeometricmeanofAandBisthepositive definitematrixA#BdefinedbyA#B =A1/2(A−1/2BA−1/2)1/2A1/2. OnecanprovethatA#B = B#A(see[1]fordetails). Further, thereareseveralusefulwaysto representthegeometricmeanofpositivedefinitematrices(seef.i.[1,12]) A#B =A(A−1B)1/2 =(BA−1)1/2A=B(B−1A)1/2 =(AB−1)1/2B (5) Thenextresultrevealsfurtherconsistencywiththescalarcase,infactweobservethatifAandBhave someeigenvectorsincommon,thenA+BandA#Bhavethoseeigenvectors,witheigenvaluesgiven bythearithmeticandgeometricmeanofthecorrespondingeigenvaluesofAandB,respectively. Theorem1. LetubeaneigenvectorofAandBwitheigenvaluesλandµ,respectively. Then,uis √ aneigenvectorofA+BandA#Bwitheigenvalueλ+µand λµ,respectively. Proof. Using the identities Au = λu and Bu = µu we have (A + B)u = (λ + µ)u. For the geometric mean, observe that for any positive definite matrix M, if Mx = λ(M)x, then M1/2x=λ(M)1/2x. Inparticularwehave A−1/2BA−1/2u=λ−1/2A−1/2Bu=λ−1/2µA−1/2u=(µ/λ)u (cid:112) thus(A−1/2BA−1/2)1/2u= µ/λu. Asaconsequence (cid:112) (A#B)u=A1/2(A−1/2BA−1/2)1/2A1/2u=λ1/2A1/2(A−1/2BA−1/2)1/2u=( λµ)u whichconcludestheproof. 3.1 Geometricmeanforsignednetworksclustering ConsiderthesignednetworkG± =(G+,G−). WedefinethenormalizedgeometricmeanLaplacian ofG±as L =L+ #Q− (6) GM sym sym 3 We propose Algorithm 1 for clustering signed networks, based on the spectrum of L . By GM definition2, thematrixgeometricmeanA#B requiresAandB tobepositivedefinite. Asboth theLaplacianandthesignlessLaplacianarepositvesemi-definte,inwhatfollowsweshallassume that the matrices L+ and Q− in (6) are modified by a small diagonal shift, ensuring positive sym sym definiteness. That is, in practice, we consider L+ + ε I and Q− + ε I being ε and ε sym 1 sym 2 1 2 smallpositivenumbers. Forthesakeofbrevity,wedonotexplicitlywritetheshiftingmatrices. Input: SymmetricweightmatricesW+,W− ∈Rn×n,numberkofclusterstoconstruct. Output: ClustersC ,...,C . 1 k 1 Computethekeigenvectorsu1,...,uk correspondingtotheksmallesteigenvaluesofLGM. 2 LetU =(u1,...,uk). 3 ClustertherowsofU withk-meansintoclustersC1,...,Ck. Algorithm1:SpectralclusteringwithL onsignednetworks GM ThemainbottleneckofAlgorithm1isthecomputationoftheeigenvectorsinstep1. InSection5we proposeascalableKrylov-basedmethodtohandlethisproblem. Letusbrieflydiscussthemotivatingintuitionbehindtheproposedclusteringstrategy. Algorithm1, aswellasstate-of-the-artclusteringalgorithmsbasedonthematricesin(3),relyontheksmallest eigenvalues of the considered operator and their corresponding eigenvectors. Thus the relative ordering of the eigenvalues plays a crucial role. Assume the eigenvalues to be enumerated in ascendingorder. Theorem1statesthatthefunctions(A,B) (cid:55)→ A+B and(A,B)(cid:55)→A#B map eigenvalues of A and B having the same corresponding eigenvectors, into the arithmetic mean (cid:112) λ (A)+λ (B)andgeometricmean λ (A)λ (B), respectively, whereλ (·)istheith smallest i j i j i eigenvalueofthecorrespondingmatrix. Notethattheindicesiandj arenotthesameingeneral, astheeigenvectorssharedbyAandBmaybeassociatedtoeigenvalueshavingdifferentpositions intherelativeorderingofAandB. Thisintuitivelysuggests thatsmalleigenvaluesofA+Bare related to small eigenvalues of both A and B, whereas those of A#B are associated with small eigenvaluesofeitherAorB,orboth. Thereforetherelativeorderingofthesmalleigenvaluesof L is influenced by the presence of assortative clusters in G+ (related to small eigenvalues of GM L+ )orbydisassortativeclustersinG−(relatedtosmalleigenvaluesinQ− ),whereastheordering sym sym ofthesmalleigenvaluesofthearithmeticmeantakesintoaccountonlythepresenceofboththose situations. Inthenextsection,fornetworksfollowingthestochasticblockmodel,weanalyzeinexpectation the spectrum of the normalized geometric mean Laplacian as well as the one of the normalized Laplacianspreviouslyintroduced. Inthiscasetheexpectedspectrumcanbecomputedexplicitlyand weobservethatinexpectationtheorderinginducedbyblendingtheinformationsofG+ andG− troughthegeometricmeanallowstorecoverthegroundtruthclustersperfectly,whereastheuseof thearithmeticmeanintroducesabiaswhichreverberatesintoasignificantlyhigherclusteringerror. 4 Stochasticblockmodelonsignedgraphs InthissectionwepresentananalysisofdifferentsignedgraphLaplaciansbasedontheStochastic BlockModel(SBM).TheSBMisawidespreadbenchmarkgenerativemodelfornetworksshowinga clustering,community,orgroupbehaviour[22]. Givenaprescribedsetofgroupsofnodes,theSBM definesthepresenceofanedgeasarandomvariablewithprobabilitybeingdependentonwhich groupsitjoins. Toourknowledgethisisthefirstanalysisofspectralclusteringonsignedgraphs withthestochasticblockmodel. LetC ,...,C begroundtruthclusters,allhavingthesamesize|C|. 1 k Weletp+ (p−)betheprobabilitythatthereexistsapositive(negative)edgebetweennodesinthe in in samecluster,andletp+ (p− )denotetheprobabilityofapositive(negative)edgebetweennodesin out out differentclusters. Calligraphiclettersdenotematricesinexpectation. InparticularW+ andW− denotetheweight matricesinexpectation. WehaveW+ = p+ andW− = p− ifv ,v belongtothesamecluster, i,j in i,j in i j whereasW+ =p+ andW− =p− ifv ,v belongtodifferentclusters. Sortingnodesaccording i,j out i,j out i j tothegroundtruthclusteringshowsthatW+andW−haverankk. 4 (E+) p+out

Clustering Signed Networks with the Geometric Mean of Laplacians

Similar Clustering Signed Networks with the Geometric Mean of Laplacians