Generative Network Automata: A Generalized Framework for Modeling Adaptive Network Dynamics Using Graph Rewritings

0.7 MB·English

by  Hiroki Sayama

Generative Network Automata: A Generalized
Framework for Modeling Adaptive Network
Dynamics Using Graph Rewritings
HirokiSayamaandCraigLaramee
9
0
0
2

n
a
J
Abstract A variety of modeling frameworks have been proposed and utilized in
2
complex systems studies, including dynamical systems models that describe state

] transitionsona systemoffixedtopology,andself-organizingnetworkmodelsthat
O
describe topologicaltransformationsof a network with little attention paid to dy-
A namical state changes. Earlier network models typically assumed that topological
. transformations are caused by exogenous factors, such as preferential attachment
n
ofnewnodesandstochasticortargetedremovalofexistingnodes.However,many
i
l real-worldcomplexsystemsexhibitbothofthesetwodynamicssimultaneously,and
n
[ theyevolvelargelyautonomouslybasedonthesystem’sownstatesandtopologies.

Here we show that, by usingthe conceptof graphrewriting,both state transitions
1
and autonomoustopology transformationsof complex systems can be seamlessly
v
6 integratedandrepresentedinaunifiedcomputationalframework.Wecallthisnovel
1 modelingframework“GenerativeNetwork Automata(GNA)”. In this chapter,we
2 introducebasicconceptsofGNA,itsworkingdefinition,itsgeneralitytorepresent
0
otherdynamicalsystemsmodels,andsomeofourlatestresultsofextensivecompu-
.
1 tationalexperimentsthatexhaustivelysweptoverpossiblerewritingrulesofsimple
0 binary-stateGNA.TheresultsrevealedseveraldistincttypesoftheGNAdynamics.
9
0
:
v
i 1 Introduction
X
r
a Avarietyofmodelingframeworkshavebeenproposedandutilizedforresearchon
thedynamicsofcomplexsystems[1,2,3].Amajorclassofmodelingframeworks
HirokiSayamaandCraigLaramee
CollectiveDynamicsofComplexSystemsResearchGroup/DepartmentofBioengineering,Bing-
hamtonUniversity,StateUniversityofNewYork,P.O.Box6000,Binghamton,NY13902-6000,
USA,e-mail:[email protected],[email protected]
HirokiSayamaisalsoanAffiliateoftheNewEnglandComplexSystemsInstitute,24Mt.Auburn
St.,Cambridge,MA02138,USA,e-mail:[email protected]
1
2 HirokiSayamaandCraigLaramee
isthatofdynamicalsystemsmodels,includingordinaryorpartialdifferentialequa-
tionsanditerativemaps[4],artificialneuralnetworks[5,6],randomBooleannet-
works[7,8,9],andcellularautomata[10,11].Whiletheyarecapableofproducing
strikingly complex and even biological-like behaviors [12, 13, 14, 15, 16], these
toolsgenerallyassumeanetworkmadeofafixednumberofcomponentsorganized
in a fixed topology.Theirdynamicsare consideredastrajectoriesofsystem states
inaconfinedphasespacewithtime-invariantdimensions.
The recent surge of network theory in statistical physics has demonstrated yet
anothergraph-theoreticapproachtocomplexsystemsmodeling[17,18,19].Itad-
dressestheself-organizationofnetworkstructurevialocaltopologicaltransforma-
tionssuchasrandomorpreferentialaddition,modificationandremovalofcompo-
nentsandtheirinteractions(i.e.,nodesandlinks).Amongthemostactivelyinvesti-
gatedissuesinthisfieldishowstatisticalpropertiesoftheentirenetworktopology
will be affected by additions (growth or augmentation) and removals (failures or
attacks) of nodes and links, and in particular, how networks can be more robust
against the latter [20, 21, 22, 23, 24]. Those additions and removals are typically
assumed as perturbationscomingfrom externalsources, not incorporatedinto the
dynamicsofthenetworkitself.Theyarealsolimitedinthatnotmuchattentionhas
beenpaidtodynamicalstatechangesonthenetwork.Researchersrecentlystarted
investigatingdynamicalstatechangesoncomplexnetworks[25,26,27,28,29,30].
Theyarestilllargelyfocusingonfixednetworktopologiesortopologiesvariedby
exogenousperturbations.
When looking into real-world complex networks, however, one can find many
instancesofnetworkswhosestatesandtopologies“coevolve”,i.e.,theykeepchang-
ingoverthesametimescalesduetothesystem’sowndynamics(Table1).Inthese
networks, state transitions of each component and topological transformations of
networksaredeeplycoupledwitheachother.Understandinganddescribingtheco-
evolutionofstatesandtopologiesofnetworksisnowrecognizedasoneofthemost
importantproblemstoaddress[21,31].Severaltheoreticalmodelsofcoevolution-
arynetworkshavebeenproposedandstudiedmostrecently[32,33,34,35,36],yet
each of these studies used different model formulations for different phenomena,
withlimitedimplicationsgivenforhowthesecoevolutionarynetworkmodelscould
belinkedtootherexistingcomplexsystemsmodels.
Hereweaimtoaddresstheabove-mentionedlackoflinkagesbetweencoevolu-
tionarynetworkmodelsandotherexistingcomplexsystemsmodelsbydeveloping
amorecomprehensiveformulation.Specifically,weshowthat,byusingtheconcept
ofgraphrewriting,bothstatetransitionsandautonomoustopologytransformations
ofcomplexsystemscanbeseamlesslyintegratedandrepresentedinaunifiedcom-
putationalframework.Wecallthisnovelmodelingframework“GenerativeNetwork
Automata(GNA)”[37].Thenameindicatestheintegrationofknowledgeaccumu-
latedindynamicalsystemstheory,networktheory,andgraphgrammartheory.
In the following sections, we will introduce basic concept of graph rewriting,
a working definition of GNA, its generality to represent other dynamical systems
models,andsomeofourlatestresultsofextensivecomputationalexperimentsthat
GenerativeNetworkAutomata 3
Table1 Real-worldexamplesofcomplexnetworkswhosestatesandtopologieschangeoverthe
sametimescalesduetothenetwork’sowndynamics.
Network Nodes Links Exampleof Exampleof Exampleof
nodestates nodeaddition topological
orremoval changes
Organism Cells Celladhesions, Gene/protein Celldivision, Cellmigration
intercellular activities celldeath
communica-
tions
Ecological Species Ecological Population, Speciation, Changesin
community relationships intraspecific invasion, ecological
(predation, diversities extinction relationships
symbiosis, viaadaptation
etc.)
Epidemio- Individuals Physical Pathologic Death, Reductionof
logical contacts states quarantine physical
network contacts
Socialnetwork Individuals Social Sociocultural Entrytoor Establishment
relationships, states,political withdrawal or
conversations, opinions, from renouncement
collaborations wealth community ofrelationships
exhaustivelyswept overpossible rewriting rules of simple binary-state GNA. The
resultsrevealedseveraldistincttypesoftheGNAdynamics.
2 About GraphRewriting
The key characteristic of GNA is that it should have mechanisms for transforma-
tionsoflocalnetworktopologiesaswellastransitionsoflocalstates. Topological
transformationsmaybemodeledasarewritingprocessoflocalnetworkconfigura-
tions.Wewillthereforeadoptmethodsandtechniquesdevelopedingraphgrammar
theory[38]toconstructgeneralformulationsofGNA.
Graphgrammars,studiedsincelate 1960’sin theoreticalcomputerscience[39,
40, 41, 42], are an extensionof formalgenerativegrammarsin computationallin-
guistics to discuss similar rule-based generativeprocesses of graphs,or networks.
They recursively define a set of “valid” graph topologies that can be generated
throughrepetitiveapplicationsofagivensetofnodeand/orlinkreplacementrules.
Acomputationalimplementationofsuchprocessesiscalledagraphrewritingsys-
tem, often used to simulate particular generative processes of network topology.
Heretheword“generative”meansthatthereplacementsaretriggeredbylocaltopo-
logicalfeaturesofthenetworkitself,andnotbyexternalsourcesofperturbationas
typicallyassumedinmodernnetworktheory.
Aclassic,andprobablymostwidelyknown,exampleofgraphrewritingsystems
is the Lindenmayersystem, or L-system [43]. It is a simple rewriting system that
4 HirokiSayamaandCraigLaramee
canproduceself-similarrecursivestructuresinasequentialstring(inthissense,the
L-system remains within the range of classic formalgrammars).What makes this
system outstandingis thatit comeswith an interpretationthat convertsa resultant
stringintoatree-liketopologicalstructure,whichmayappearjustlikeanaturaltree
ifparametersareappropriatelychosen.Thisexampleshowsthecapabilityofgraph
rewritingsystemsto describetheemergenceofnaturalcomplexstructuresusinga
setofsmalllocalrules.
Although their relevance to biology was initially recognized [39, 44], appli-
cations of graph grammars have so far remained within computer science, such
as pattern recognition, compiler design, and data type and process specification
[38,40,41,42],andtheirusehasbeennotsocommonevenwithincomputerscience
duetounintuitive,complicatedformulationandlackofsoftwaretoolsformodeling
[45].Moreover,mostapplicationswereprimarilyfocusedoncontext-freerewriting
rules,andtheyrarelyconsidereddynamicalstatetransitionsonnetworks.Recently,
context-dependentgraphgrammarshavebeenappliedtodescribereactionrulesin
artificial life/artificial chemistry, includingmodels of self-replication[46, 47, 48],
self-assembly[49], morphogenesis[50, 51] anddynamicstate changes[51] ofar-
tifacts. However, none of them integrated graph grammars into complex systems
modelinginaflexible,generalizablewaysoastobereadilyapplicabletonetworks
studiedinotherdomains.
To the best of our knowledge, our GNA framework is among the first to sys-
tematicallyintegrategraphrewritingsintherepresentationandcomputationofthe
dynamics of complex networks that involve both state transition and autonomous
topologicaltransformation.Ourlong-termgoalistodevelopacomprehensivethe-
oryofGNAandasetofanalytical/computationaltoolsthatcanbebroadlyapplied
tothemodelingofvariouscomplexsystems.
3 Definition ofGNA
A workingdefinitionofGNA isa networkmadeofdynamicalnodesanddirected
linksbetweenthem.Undirectedlinkscanalsoberepresentedbyapairofdirected
linkssymmetricallyplacedbetweennodes.Eachnodetakesoneofthe (finitelyor
infinitely many) possible states defined by a node state set S. The links describe
referential relationships between the nodes, specifying how the nodes affect each
otherinstatetransitionandtopologicaltransformation.Eachlinkmayalsotakeone
ofthepossiblestatesinalinkstatesetS′.AconfigurationofGNAataspecifictime
t isacombinationofstatesandtopologiesofthenetwork,whichisformallygiven
bythefollowing:
• V: A finite set of nodes of the network at time t. While usually assumed as
t
time-invariant in conventional dynamical systems theory, this set can dynami-
callychangeintheGNAframeworkduetoadditionsandremovalsofnodes.
• C :V →S:Amapfromthenodesettothenodestate setS.Thisdescribesthe
t t
globalstate assignmentonthenetworkattimet.Iflocalstates arescalar num-
GenerativeNetworkAutomata 5
bers,thiscanberepresentedasasimplevectorwithitssizepotentiallyvarying
overtime.
• L :V →{V ×S′}∗:Amapfromthenodesettoalistofdestinationsofoutgoing
t t t
linksandthestatesoftheselinks,whereS′isalinkstateset.Thisrepresentsthe
globaltopologyofthenetworkattimet,whichisalso potentiallyvaryingover
time.
States and topologies of GNA are updated through repetitive graph rewriting
events,eachofwhichconsistsofthefollowingthreesteps:
1. ExtractionofpartoftheGNA(subGNA)thatwillbesubjecttochange.
2. ProductionofanewsubGNAthatwillreplacethesubGNAselectedabove.
3. EmbeddingofthenewsubGNAintotherestofthewholeGNA.
ThetemporaldynamicsofGNAcanthereforebeformallydefinedbythefollowing
triplethE,R,Ii:
• E:AnextractionmechanismthatdetermineswhichpartoftheGNAisselected
fortheupdating.ItisdefinedasafunctionthattakesthewholeGNAconfigura-
tionandreturnsaspecificsubGNAinittobereplaced.Itmaybedeterministic
orstochastic.
• R: A replacementmechanism that producesa new subGNA from the subGNA
selected by E and also specifies the correspondence of nodes between the old
andnewsubGNAs.ItisdefinedasafunctionthattakesasubGNAconfiguration
andreturnsapairofanewsubGNAconfigurationandamappingbetweennodes
in the old subGNA and nodes in the new subGNA. It may be deterministic or
stochastic.
• I:AninitialconfigurationofGNA.
ThereareacoupleofothercommonlyusedproceduresneededtosimulateGNA
dynamics,such as the removalof the selected subGNA fromthe whole GNA and
the re-connectionof “bridge”links (i.e., links that were between the old subGNA
and the rest of the GNA) when embedding the new subGNA. Because the work-
ingsoftheseproceduresarefairlyobvious,weomitdetailedexplanationsforthem.
TheaboveE,R,I aresufficienttouniquelydefinespecificGNAmodels.Theentire
pictureofarewritingeventisillustratedinFig.1,whichvisuallyshowshowthese
mechanismsworktogether.
Thisrewritingprocess,ingeneral,maynotbeappliedsynchronouslytoallnodes
or subGNAs in a network, because simultaneous modifications of local network
topologiesatmorethanoneplacesmaycauseconflictingresultsthatareinconsis-
tent with each other. This limitation will not apply though when there is no pos-
sibility of topologicalconflicts, e.g., when the rewriting rules are all context-free,
orwhenGNAisusedtosimulateconventionaldynamicalnetworksthatinvolveno
topologicalchanges.
We notethatitisauniquefeatureofGNAthatthemechanismofsubgraphex-
traction is explicitly described in the formalism as an algorithm E, not implicitly
assumedoutsidethegrammaticalruleslikewhatothergraphrewritingsystemstyp-
ically adopt (e.g. [51]). Such algorithmic specification allows more flexibility in
6 HirokiSayamaandCraigLaramee
Fig.1 GNArewritingprocess.(a)TheextractionmechanismEselectspartoftheGNA.(b)The
replacementmechanismRproducesanewsubGNAasareplacementoftheoldsubGNAandalso
specifiesthecorrespondenceofnodesbetweenoldandnewsubGNAs(dashedline).Thisprocess
mayinvolvebothstatetransitionofnodesandtransformation oflocaltopologies. The“bridge”
linksthatusedtoexistbetweentheoldsubGNAandtherestoftheGNAremainunconnectedand
open.(c)ThenewsubGNAproducedbyRisembeddedintotherestoftheGNAaccordingtothe
nodecorrespondencealsospecifiedbyR.Inthisparticularexample,thetopgraynodeintheold
subGNAhasnocorrespondingnodeinthenewsubGNA,sothebridgelinksthatwereconnected
tothatnodewillberemoved.(d)Theupdatedconfigurationafterthisrewritingevent.
representing diverse network evolution and less computational complexity in im-
plementingtheirsimulations,significantlybroadeningtheareasofapplication.For
example, the preferential attachment mechanism widely used in modern network
theory to construct scale-free networks is hard to describe with pure graph gram-
marsbutcanbeeasilywritteninalgorithmicforminGNA,asdemonstratedinthe
nextsection.
Whilethedefinitiongivenaboveisoneofthesimplestpossibleformulationsof
GNA, it already has considerable complexity compared to conventional dynami-
cal systems models. The possibility of temporalchanges ofV and L particularly
t t
GenerativeNetworkAutomata 7
makesitdifficultto investigateits dynamicalpropertiesanalytically.However,the
updating process of GNA is algorithmically described and hence their dynamics
canbeexperimentedthroughcomputersimulationrelativelyeasily.Wehavedevel-
opeda packagein Wolfram Research Mathematicaforsmall-scale simulationand
visualizationofGNAwithnodestates1.Theresultspresentedinthischapterwere
obtainedusingthispackage.
4 Generality ofGNA
TheGNAframeworkishighlygeneralandflexiblesothatmanyexistingdynamical
networkmodelscanberepresentedandsimulatedwithinthisframework.
Forexample,ifRalwaysconserveslocalnetworktopologiesandmodifiesstates
ofnodesonly,thentheresultingGNAisaconventionaldynamicalnetworkmodel,
including cellular automata, artificial neural networks, and random Boolean net-
works(Fig.2(a),(b)).AstraightforwardapplicationofGNAtypicallycomeswith
asynchronousupdatingschemes,asintroducedintheprevioussection.Sinceasyn-
chronousautomatanetworkscanemulateanysynchronousautomatanetworks[52],
the GNA framework covers the whole class of dynamics that can be produced
by conventionaldynamicalnetwork models. Moreover,as mentionedearlier, syn-
chronousupdatingschemescouldalso be implementedin GNA forthis particular
classofmodelsbecausetheyinvolvenotopologicaltransformation.
Ontheotherhand,manynetworkgrowthmodelsdevelopedinmodernnetwork
theorycanalsoberepresentedasGNAifappropriateassumptionsareimplemented
inthesubGNAextractionmechanismEandifthereplacementmechanismRcauses
nochangeinlocalstatesofnodes(Fig.2(c)).
5 Computational Exploration ofPossibleDynamicsofSimple
Binary-State GNA
In this section we reportour latest results of extensivecomputationalexperiments
that exhaustivelyswept over possible rewriting rules of simple binary-state GNA.
Theresultsshownherewereobtainedwithmuchlessrestrictedrulesetsthanthose
assumedinourpreviouswork[37].
1TheMathematicapackageisstillunderactivedevelopmentbutmaybeavailableuponrequest.
8 HirokiSayamaandCraigLaramee
Fig. 2 Various dynamical network models simulated using GNA. These examples were repre-
sentedinthesameformatofhE,R,Ii(seetext)andsimulatedusingthesamesimulatorpackage
implementedinMathematica.(a)Simulationofasynchronous2-Dbinarycellularautomatawith
vonNeumannneighborhoods andlocalmajorityrules.Spacesize:100×100.(b)Simulationof
anasynchronousrandomBooleannetworkwithN=30andK=2.Timeflowsfromlefttoright.
NodesofrandomBooleannetworksarenon-homogeneous,i.e.,theyobeydifferentstate-transition
rules.Hereeachnode’sownstate-transitionruleisembeddedaspartofitsstate,andthereplace-
mentmechanismRreferstothatinformationwhencalculatingthenextstateofanode.(c)Simu-
lationofanetworkgrowthmodelwiththeBaraba´si-Albertpreferentialattachmentscheme.Time
flowsfromlefttoright.Eachnewnodeisattachedtothenetworkwithonelink.Theextraction
mechanismEisimplementedsothatitdeterminestheplaceofattachmentpreferentiallybasedon
thenodedegrees,whichcausestheformationofascale-freenetworkinthelongrun.
GenerativeNetworkAutomata 9
5.1 Assumptions
There are infinitely many possible mechanisms for E and R because there are no
theoreticalupperboundsin termsof the size of the old subGNA selected by E (it
could be infinitely largeas the GNA grows) and the new subGNA producedby R
(itcouldbearbitrarilylargebythedesignofR).Makingreasonableassumptionsto
restrictthepossibilityofmechanismsforE andRiscriticaltofacilitatesystematic
studyonthedynamicsofGNA.Herewemakethefollowingassumptions(Fig.3):
1. Nodestatesarebinary(0or1).
2. Nolinkstateisconsidered(i.e.,linkshomogeneouslytakeonlyonestateandit
willneverchange).
3. Links are undirected(i.e., every connectionbetween nodes is represented by a
pairofsymmetricallyplaceddirectedlinks).
4. TheextractionmechanismE alwaysselectsasubGNAby
a. randomlypickingonenodeufromtheentireGNA(Fig.3(a)),
b. takingallthedestinationnodesofitsoutgoinglinksL(u)(Fig.3(b)),and
t
c. producinga subGNA “induced”by these nodes{u}∪L(u), i.e., a subGNA
t
that includes all these nodes as well as all the links present between them
(Fig.3(c)).
5. ThereplacementmechanismRonlyreferstothestateofthecentralnodeuand
thelocalmajoritystate withintheinducedsubGNA.Ifthereareequalnumbers
of 0’s and 1’s within the subGNA, one of the two states is randomly chosen.
Thistwo-bitinformationwillbeusedtodeterminewhatwillhappentothelocal
configuration (Fig. 3 (d)). The following ten possible rewriting outcomes are
madeavailable(whichareextendedfrom[37]):
0) Thecentralnodeudisappears.
1) Everythingremainsinthesamecondition.
2) Thestateofthecentralnodeuisinverted.
3) Thecentralnodeudividesintotwowiththestatepreservedinbothnodes.
4) Thecentralnodeudividesintotwowiththestateinvertedinbothnodes.
5) Thecentralnodeudividesintotwowiththestateinvertedinonenode.
6) Thecentralnodeudividesintothreewiththestatepreservedinallthreenodes.
7) The central node u divides into three with the state inverted in all of three
nodes.
8) The central node u divides into three with the state inverted in two of three
nodes.
9) The central node u divides into three with the state inverted in one of three
nodes.
Incaseswherenodedivisionoccurs,thelinksthatwereconnectedtothecentral
nodeuisdistributedasevenlyaspossibletoitsdaughternodes(Fig.3(e)).
6. TheinitialconditionI consistsofasinglenodewithstate0.
10 HirokiSayamaandCraigLaramee
Fig.3 SimplifiedGNArewritingprocessusedfortheexhaustivesweepexperiments.Theextrac-
tionmechanismE(a)randomlypicksonenodeu,(b)takesallthedestinationnodesofitsoutgo-
inglinksLt(u),and(c)producesasubGNAinducedbythosenodes{u}∪Lt(u).Thereplacement
mechanism R(d)refers tothestateofthecentral node uintheselected subGNAandthelocal
majoritystatewithinittodeterminewhathappenstothelocalconfiguration,(e)producesanew
subGNAaswellasthecorrespondenceofnodesbetweentheoldandnewsubGNAsbasedonthe
choicemadein(d),andthen(f)embedsthenewsubGNAintotherestoftheGNA.
Notethattheabovemodelassumptionswillalwaysgenerateplanargraphsinwhich
thenodedegreesareboundeduptothreewheninitiatedwithasinglenode.There-
forealltheresultsshowninthischapteraretopologicallyplanar.
5.2 Methods
We carried out an exhaustivesweep of all the possible rewriting rules that satisfy
the assumptions discussed above. Since the extraction mechanism E is uniquely
defined,itisonlythereplacementmechanismRthatcanbevaried.HereRisdefined
as a function that maps each of the four possible two-bit inputs to one of the ten
possibleactions.Thereforethe numberofallthe possibleR’sis1022 =10000.To
indicateaspecificR,wewilluseits“rulenumber”rn(R)thatisdefinedby
rn(R)=a ×103+a ×102+a ×101+a ×100, (1)
11 10 01 00
where a is a numerical representation (numbers associated with each of the ten
ij
possibleactionsshownabove)ofthechoicethatRwillmakewhenthestateofthe
centralnodeuisiandthelocalmajoritystateis j.