Table Of Content

The Role of Redundancy in the Robustness of Random Boolean Networks Carlos Gershenson1, Stuart A. Kauffman2,3, IlyaShmulevich4 1VrijeUniversiteitBrussel,Krijgskundestraat33B-1160Brussel,Belgium 2InstituteforBiocomplexityandInformatics,UniversityofCalgary 2500UniversityDriveNW,Calgary,AlbertaT2N1N4,Canada 3SantaFeInstitute,1399HydeParkRoad,SantaFe,NM,USA87501 4InstituteforSystemsBiology,1441North34thStreet,Seattle,WA,USA98103-8904 6 [email protected] [email protected] [email protected] 0 0 2 Abstract itprovidesrobustnesstorecessivedetrimentalmutationsof n one gene of a pair. There is also substantial evidence for a Evolutiondependsonthepossibilityofsuccessfullyexplor- the hypothesis that growth in genetic regulatory networks J ingfitnesslandscapesviamutationandrecombination. With thesesearch procedures, exploration isdifficult in“rugged” occursprimarilyviatheduplicationofgenesandsubsequent 1 fitness landscapes, where small mutations can drastically mutations of one or both members of the duplicate pair 1 changefunctionalitiesinanorganism. RandomBooleannet- (Fosteretal.,2005;Lewin,2000). Thus,itwouldbedesire- ] works (RBNs), being general models, can be used to ex- abletoobtainabetterunderstandingofitsmechanisms. To O ploretheoriesofhowevolutioncantakeplaceinruggedland- achieve this, we used random Boolean networks (RBNs) scapes;orevenchangethelandscapes. A (Kauffman,1969; Kauffman,1993; Wuensche,1998; Inthispaper, westudytheeffectthatredundant nodeshave . Aldana-Gonza´lezetal.,2003; Gershenson,2004), a very n ontherobustnessofRBNs. Usingcomputersimulations,we popularmodelofgeneticregulatorynetworks. i havefoundthattheadditionofredundantnodestoRBNsin- l n creases their robustness. We conjecture that redundancy is Ourpresentwork wasinitially motivatedbyKauffman’s [ a way of “smoothening” fitness landscapes. Therefore, re- conjecture (Kauffman,2000, p. 195). This states that 1- dundancycanfacilitateevolutionarysearches. However,too bitmutationsto a minimalprogramwill changedrastically 2 much redundancy could reduce the rateof adaptation of an v evolutionaryprocess.Ourresultsalsoprovidesupportingev- theoutputoftheprogram,makingitindistinguishablefrom 8 idenceinfavourofKauffman’sconjecture(Kauffman,2000, noise. The conjecture points to the necessity of having 1 p.195). someredundancyto allow smoothtransitionsas a program 0 changesinanevolutionarysearch. 1 1 Introduction However, it is not obvious to obtain a minimal pro- 5 gram in a standard programming language such as C A system is robustif it continuesto functionin the face of 0 or Java. Still, we can consider cellular automata (CA) perturbations(Wagner,2005a). Thismakesrobustnessade- / n (vonNeumann,1966) or RBNs as programs. It has been sireable property in any system surrounded by a complex i shownthatcertainCA can performuniversalcomputations l environment. Living systems fall into this category. Sys- n (Gardner,1983; Cook,2004). SinceRBNs area wellstud- temsabletocopewithperturbationswillsurviveandrepro- : v duce. Moreover, robustness plays a key role in evolvabil- ied generalization of CA (Gershenson,2002), they seem i suitable candidates for studying the effect of redundancy X ity: itallowsthegradualexplorationofnewsolutionswhile in their robustness. We can measure the redundancy of a maintainingfunctionality. A small changein a fragile sys- r RBN,make1-bitmutationstothenetworks,andthenmea- a temwoulddestroyit. Thisisnotfavourediftheexploration suretheirrobustnesscomparingthestatespacesofthemu- methodconsistsofrandommutations. tantandthe“original”networks. There are several mechanisms that help a system to be robust (vonNeumann,1956), such as modular- It is not obvious how to measure redundancy, or ity (Simon,1996; WatsonandPollack,2005), degen- compressibility. Interpreting results from Kolmogorov, eracy (FernańdezandSole´,2004), distributed robustness Solomonoff,andChaitin (Chaitin,1975) we can say thatit (Wagner,2004),andredundancy.Here,wefocusonthislast is impossible to show that a program is “minimal” (inde- one.Itconsistsonhavingmorethanonecopyofanelement pendently of a fixed universal model of computation). We type. If one fails or changes its function, another can still decidednotto attemptto build “minimal”networks, butto performasexpected.Redundancyiswidespreadingenomes comparemoreandless redundantnetworks,andtryto find ofhigherorganisms(Nowaketal.,1997). A clearexample differences. isseenwithdiploidy,whereeachcellhastwocompletesets Thispaperisorganizedasfollows: Inthefollowingsec- of genes. This has severaladvantagesfor evolvability, e.g. tion,wepresentrandomBooleannetworks. Thenweintro- duce a method for adding redundant nodes in RBNs. We space transitionsusing normalisedHammingdistances(2). used this method in simulations to measure the robustness Our result: there was no difference. First, since fictitious ofRBNstopointmutations.Adiscussionofourresultsfol- inputshave no functionality, they do not affect the conver- lows,tofinallypresentconcludingremarks. genceofthenetworks.Second,itseemsthattheprobability ofmakingamutationtoaredundantlink(whichdidnothave Random Booleannetworks functionality) and making it functional is the same as do- A random Boolean network (RBN) ing the opposite(makinga functionallink non-functional). (Kauffman,1969; Kauffman,1993; Wuensche,1998; Therefore,theredundancyoflinksdoesnotaffectthestabil- Aldana-Gonza´lezetal.,2003; Gershenson,2004) has ityofnetworks. N nodes (consisting each of one Boolean variable) n s{tsat1e,so2f,e··a·c,hs nNo}dethsat icsandettaekremivnaeludebsyokf zrearnodoomr olynec.hoTshene H(A,B)= 1nX|ai−bi| (2) i i i nodes (connections) {s ,···,s }. In the classic model, i1 iki Werealizedthattheredundancyinnodeswasverydiffer- k =k ,∀i,j,soeverynodehasK inputs. i j entfromtheredundancyinlinks.1 Itwouldbecomputation- The way in which the k elements determine the value i allycomplicatedtogenerateanetworkwithsomeredundant of s is given by a Boolean function f(s ,···,s ). Each i i i1 iki nodes, and then trim them. Thereforewe did the opposite: combinationofinputs(thereare2ki)willreturn fi=1with RBNs weregeneratedrandomly,andthenredundantnodes aprobabilitypand fi=0withaprobability1−p. Oncethe wereadded. connectionsand functionshave been chosen randomly,the Themethodwedevelopedforaddingaredundantnodeto network structure does not change. In the classical model, aRBNisthefollowing: theelementsareupdatedsynchronously: 1. SelectrandomlyanodeX tobe“duplicated”. s (t+1)= f(s (t),···,s (t)) (1) i i i1 iki 2. AddanewnodeRtothenetwork(Nnew=Nold+1),with fi can be representedas a lookuptable, where the right- thesameinputsandlookuptableasX (i.e. kR=kX, fR= most column represents s i(t+1) determined by ki inputs fX),andoutputstothesamenodesofwhichX isinput: represented in the rest of the columns exhaustively com- binedin2ki rows. kinew=kiold∪kiR if ∃kiX,∀i (3) Sincethe numberofpossible statesisfinite (2N), sooner orlaterthenetworkwillreachastatethathasbeenalready 3. DoublethelookuptablesofthenodesofwhichX isinput visited. Because the dynamics are deterministic, the net- with the following criterion: When R=0, copy the old lookup table. When R=1, and X =0, copy the same work has reached an attractor. If we use RBNs as models valuesforallcombinationswhenX =1andR=0. Copy ofgeneticregulatorynetworks,attractorscanbeseenascell again the same values to the combinationswhere X =1 types(HuangandIngber,2000). andR=1.Inotherwords,makeaninclusiveORfunction The dynamics of “families” or ensembles inwhichX ORRshouldbeone,toobtaintheoldoutputs whenonlyX wasone (Kauffman,2004) of networks can be studied to find statistical properties common in networks independently of their precise functionalities. We will use this ensemble finew(s i1,···,s iX=0,s iR=0···,s iki)=fiold(s i1,···,s iX=0,···,s iki)  approach to study the effect of redundancy in families of ffiinneeww((ss ii11,,······,,ss iiXX==10,,ss iiRR==01······,,ss iikkii))==ffiioolldd((ss ii11,,······,,ss iiXX==11,,······,,ss iikkii))  if∃kiX,∀i RBNs. finew(s i1,···,s iX=1,s iR=1···,s iki)=fiold(s i1,···,s iX=1,···,s iki)  Introducing Redundancy Afterthis,RwillbearedundantnodeofX,andviceversa. WewillcallR,andotherredundantnodeswhichareadded Atfirstweattemptedtostudytheeffectofredundantlinksin tothe originalnetwork,“red” nodes. The “original”nodes RBNs. However,redundantlinksarefictitiouslinksi.e. the ofthe networkwillbe called“white” nodes. A diagramil- functionalitydoes notchange when they are removed. We lustratingtheinclusionofarednodeisdepictedinFigure1. beganwithanetworkwithadesiredpercentageoffictitious Lookuptablesshowhowthenodeoftherightmostcolumn inputs (this can be regulated with probability p of having 1’sinlookuptables),andthenmeasureditsstability. After- (in bold)at timet+1will be affected by the valuesof the othercolumnsattimet. wardswereducedthenumberoffictitiousinputs,measuring We implemented this algorithm in RBNLab the stability until there were no redundantlinks. The way (Gershenson,2005), and used this software laboratory we measured stability was: first, sensitivity to initial con- to measure the overlaps of state space transitions of 1-bit ditions, comparing if similar initial states convergedor di- verged. Second, making1-bitmutantsofthelookuptables 1This was inspired in redundancy of evolvable hardware ofthenetworks,andthencomparingtheoverlapofthestate (Thompson,1998) Figure1: ExampleofadditionofarednodetoaRBNN=3,K=2(seetextfordetails) mutant nets with “original” ones, as we add more and tables aredoubledeachtimeKisincremented.Thus,aone- more redundantnodes. The mutationsconsist in flipping a bitmutationwill have less effecton a networkwith higher random bit from the lookup table of a randomly selected connectivity.Toovercomethis,weplottedthevaluesofdSK node. We used normalised Hamming distances (2) to for the same results in Figure 3. We can see that a 1-bit measurethedifferencedSbetweenstatespaces: mutationmakesdSK ≃0.1fornetworkswithoutrednodes. Clearly the addition of red nodes increases the robustness 1 2n of the networks. The effect of red nodes is more evident dS= H(At+1,Mt+1) (4) 2nX i i forhigherK values,wherethenetworkdynamicsaremore i chaotic(Gershenson,2004). where A is the “original” network, and M is a 1-bit mu- Tobecertainthattherobustnessisgivenbytheadditionof tant of A. dS is calculated by computing one time step in arednode,andnotofanynode,weperformedsimulations bothnetworksforeachinitialstatei. Then,thedifferenceof with N =10. The average values of dS for one thousand thestatesthatbothnetworkstransitionediscalculatedwith networks are shown in Figure 4. We can quickly see that (2). Thisisaveragedforallinitialstates(2N),andtheresult a network with seven white nodes and three red nodes has reflectshowsimilarthetransitionsareinbothstatespaces.If a lower difference in state spaces than a network with ten dS=0,thereisnodifferencebetweenstatespaces,andthus whitenodesandnorednodes,especiallyforhighvaluesof the mutation had no effect. A higher dS reflects a greater K. effect of the mutation in the state space. There is no cor- We obtained results of RBNs with scale free topology relationofstate spaces, i.e. a mutationismaximallycatas- (Aldana,2003)(datanotshown).Weobservedasimilarten- trophic,whendS≃0.5. dency: as red nodes are introduced in a network, mutants A disadvantage of this method is that it is restricted to will have less differences in the state space, especially for small networks, since the state space is 2N. Therefore, for chaoticnetworksofhighconnectivity. each node that is added to a network, its state space is We also obtained results for different updating schemes doubled. Another option would be to have non-exhaustive (HarveyandBossomaier,1997; Gershenson,2002; explorations of state spaces, but this could be misleading. Gershensonetal.,2003) and both topologies, with similar Therefore,wedecidedtolimitoursimulationstosmallnet- results (data not shown). The precise effect of redundant works. nodeson dS does changewith the updatingscheme: in all casesdSdecreasesasredundantnodesareadded. SimulationResults We can appreciate the average results of ten thousand net- Discussion works of N =7, p=0.5 and different K values in Figure 2. WecanseethatdSdecreasesexponentiallyasrednodes Itisintuitivetoassumethatredundancywillincreasethero- areadded.WeshouldnotethatthedSforRBNswithoutred bustnessofasystem.Ifinaminimalprogramallnodesmat- nodesdecreasesasK increases. Thisisbecausethelookup ter,anymutationwillhavemoredrasticeffectsintheoutput Figure2:N=7,averagedSfor10000networks Figure3:N=7,averagedSK for10000nets Figure4:N=10,averagedSfor1000networks than in a non-optimalprogram. This has been seen exper- toastatespace,i.e. withnoeffectonthefunction,andthus imentally in evolvable hardware (Thompson,1998), where wouldbe of no use for evolvability. However,in ourRBN someredundancyallowsbetterevolutionofcircuits,buttoo approach,arednodecannotbeuseless,sinceitisconnected muchredundanyobstructsit. Still,thegoalofourworkwas to the rest of the network and thus affects its behaviour. to quantify the effect of node redundancy in robustness to Therefore, any red node will have a potential effect on the mutationsinRBN. functionofamutated network: thefunctionofthemutated Fictitious inputs do not increase the robustness because networkwillbeclosetothatoftheoriginalnetwork,foran- mutating such inputs can change the logic function. This othernodeisduplicatingthefunctionalitythatchangedwith is not the case for redundant nodes: if a node is mutated, themutation.Inotherwords,rednodesprovide“useful”re- the rest of the network keeps its (partial) function. In dundancy.Ontheotherhand,redundancygivenbyfictitious otherwords,mutationsinredundantnodesdonotpropagate linksin lookuptablesis indeed“useless”: addingfictitious throughthenetwork, whereasmutationsin fictitiousinputs links will have no effect on the mutated network function- do. ality. Still, ageneralmechanismfordecidingwethera cer- taintypeofredundancyisusefulornotseemstobeanopen From our results we can say that redundancysmoothens question. Inourcase,noderedundancydoeshelpanevolu- rough fitness landscapes (Kauffman,1993), making evolu- tionarysearchinruggedfitnesslandscapes. Towhichcases tionary search potentially feasible in rough fitness land- thisisapplicableremainstobeexplored. Forexample,ifa scapes.However,toomuchredundancyshouldbedetrimen- fitnesslandscapeisalreadysmooth,addingredundantnodes tal for evolvability, because evolutionary search becomes willbeuselesstoanevolutionarysearch. slower. WecanalsoconfirmrecentresultsbyAndreasWag- ner (Wagner,2005b) and noticethatrednodesincreasethe Returning to the original motivation of this work, if neutrality (Kimura,1983) of the network. A neutralmuta- Kauffman’sconjectureholds,arandommutationona“min- tionisonethathasnoeffectinthefunction(orphenotype). imal”RBNshouldproduceadS≃0.5.However,wecannot Aswecansee,redundancyofnodesdecreasestheprobabil- proveif a RBN cannotbereducedorcompressedindepen- ityofamutationcausingachangeinthefunctionality. Ifa dentlyofafixeduniversalmodelofcomputation. Thus,we mutationoccursinarednode,anothernodewilltakeitsrole cannot know if a RBN is minimal. What we have seen is toperformtheoldfunction.Fromthiswecansuggestpeople that“lesscompressible”RBNs,i.e. withnorednodes,have workingwithgeneticalgorithms:ifafitnesslandscapeistoo always a higher dS. As we add more red nodes, the RBN roughto make an evolutionarysearchon it, then addsome becomesmorecompressible,anddSdecreases,showingan redundancythatwillsmoothenit. Nevertheless,notalltypes increaseinrobustness. As we haveseen, the compressibil- ofredundancywillbeuseful(HarveyandThompson,1997). ityisdirectlyproportionaltotherobustnessofanetworkto Itcouldbearguedthat“useless”redundancycanbeadded randommutations. We couldextrapolateandinducethatif the compressibility could be minimal, then the robustness Gershenson, C. (2002). Classification of random Boolean would be also minimal. The minimal robustness would be networks. In Standish, R. K., Bedau, M. A., and Ab- givenwhenasinglemutationwouldcreatedS≃0.5,sowe bass, H. A., editors, Artificial Life VIII: Proceedings can concludethatthis correspondsto a minimalcompress- oftheEightInternationalConferenceonArtificialLife, ibility, i.e. a minimal program, even when in theory it is pages1–8.MITPress. notpossible to show this. ThisdoesnotproveKauffman’s Gershenson,C.(2004).Introductiontorandombooleannet- conjecture,butprovidesreasonableevidenceforitsvalidity. works. InBedau,M.,Husbands,P.,Hutton,T.,Kumar, Conclusions S.,andSuzuki,H.,editors,WorkshopandTutorialPro- ceedings,NinthInternationalConferenceontheSimu- We have presented an algorithm for introducingredundant lationandSynthesisofLivingSystems(ALifeIX),pages nodesinarandomBooleannetworks. Weusedthistoshow 160–173,Boston,MA. that redundancy of nodes increases the mutational robustness of RBNs. This may have advantages for evolvabil- Gershenson,C.(2005).RBNLab.http://rbn.sourceforge.net. ity, depending on the particular problem domain. How- ever, there are other issues that should be also considered Gershenson, C., Broekaert, J., and Aerts, D. (2003). Con- forevolvability,suchasdistributedrobustness,degeneracy, textual random Boolean networks. In Banzhaf, W., andmodularity. Therelationshipbetweenneutralityandre- Christaller, T., Dittrich, P., Kim, J. T., and Ziegler, dundancy demands further study. This is also the case for J., editors, Advances in Artificial Life, 7th European redundancyofnodesinlargernetworks. Conference, ECAL 2003 LNAI 2801, pages 615–624. Springer-Verlag. Ackwnoledgements Harvey, I. and Bossomaier, T. (1997). Time out of joint: WeshouldliketothankAndrewWuensche,AdrianThomp- AttractorsinasynchronousrandomBooleannetworks. son,MiguelGarvie,TatianaKalganova,CosmaShalizi,and In Husbands, P. and Harvey, I., editors, Proceedings Inman Harvey for interesting discussions and comments. of the Fourth European Conference on Artificial Life C.G.waspartlysupportedbytheConsejoNacionaldeCien- (ECAL97),pages67–75.MITPress. ciayTeconolg´ıa(CONACyT)ofMe´xico. Harvey,I.andThompson,A.(1997). Throughthelabyrinth References evolutionfindsaway: Asiliconridge. InHiguchi,T., Aldana, M. (2003). Boolean dynamics of networks with Iwata, M., andWeixin, L., editors, Proceedingsofthe scale-freetopology. PhysicaD,185(1):45–66. First InternationalConference on Evolvable Systems: From Biology to Hardware, volume 1259 of LNCS, Aldana-Gonza´lez,M.,Coppersmith,S.,andKadanoff,L.P. pages406–422.Springer-Verlag. (2003). Booleandynamicswithrandomcouplings. In Kaplan,E.,Marsden,J.E.,andSreenivasan,K.R.,ed- Huang, S. and Ingber, D. E. (2000). Shape-dependent itors,PerspectivesandProblemsinNonlinearScience. control of cell growth, differentiation, and apoptosis: ACelebratoryVolumeinHonorofLawrenceSirovich. Switching between attractors in cell regulatory net- SpringerAppliedMathematicalSciencesSeries. works. Exp.CellRes.,261:91–103. Chaitin,G.J.(1975). Randomnessandmathematicalproof. Kauffman,S.A.(1969). Metabolicstabilityandepigenesis ScientificAmerican,232(5):47–52. inrandomlyconstructedgeneticnets. JournalofTheo- reticalBiology,22:437–467. Cook, M. (2004). Universality in elementary cellular automata. ComplexSystems,15(1):1–40. Kauffman, S. A. (1993). The Origins of Order. Oxford UniversityPress. Fernańdez, P. and Sole´, R. (2004). The role of computation in complex regulatory networks. In Koonin, Kauffman,S.A.(2000). Investigations. OxfordUniversity E. V., Wolf, Y. I., and Karev, G. P., editors, Power Press. Laws,Scale-FreeNetworksandGenomeBiology.Lan- desBioscience. Kauffman,S. A.(2004). Theensembleapproachtounder- standgeneticregulatorynetworks. PhysicaA:Statisti- Foster,D.V.,Kauffman,S.A.,andSocolar,J.E.S.(2005). calMechanicsanditsApplications,340(4):733–740. Network growth models and genetic regulatory networks. arXivq-bio.MN/0510009. Kimura,M.(1983). TheNeutralTheoryofMolecularEvo- lution. CambridgeUniversityPress,Cambridge. Gardner,M.(1983). Wheels,Life,andOtherMathematical Amusements,chapter20-22. W.H.Freeman. Lewin,B.(2000). GenesVII. OxfordUniversityPress. Nowak, M. A., Boerlijst, M. C., Cooke, J., and Maynard Smith,J.(1997).Evolutionofgeneticredundancy.Na- ture,388:167–171. Simon, H. A. (1996). The Sciences of the Artificial. MIT Press,3rdedition. Thompson,A.(1998). HardwareEvolution:AutomaticDe- signofElectronicCircuitsinReconfigurableHardware by Artificial Evolution. Distinguished dissertation se- ries.Springer-Verlag. vonNeumann,J.(1956).Probabilisticlogicsandthesynthe- sis of reliable organismsfrom unreliable components. In Shannon, C. and McCarthy, J., editors, Automata Studies,Princeton.PrincetonUniversityPress. von Neumann, J. (1966). The Theory of Self-Reproducing Automata.UniversityofIllinoisPress.EditedbyA.W. Burks. Wagner, A. (2004). Distributed robustness versus redundancy as causes of mutational robustness. Technical Report04-06-018,SantaFeInstitute. Wagner,A.(2005a). RobustnessandEvolvabilityinLiving Systems. PrincetonUniversityPress,Princeton,NJ. Wagner,A.(2005b). Robustness, neutrality,andevolvabil- ity. FEBSLetters,579:1772–1778. Watson, R. A. and Pollack, J. A. (2005). Modular inter- dependencyin complexdynamicalsystems. Artificial Life,11(4):445–457. Wuensche, A. (1998). Discrete dynamical networks and theirattractorbasins. InStandish,R.,Henry,B.,Watt, S., Marks, R., Stocker, R., Green, D., Keen, S., and Bossomaier, T., editors, Complex Systems ’98, pages 3–21, University of New South Wales, Sydney, Aus- tralia.