Table Of Content

Control of Dynamics in Brain Networks Evelyn Tang Department of Bioengineering, University of Pennsylvania, PA 19104 Danielle S. Bassett Department of Bioengineering, Department of Electrical & Systems Engineering, University of Pennsylvania, PA 19104 7 (Dated: January 20, 2017) 1 0 The ability to effectively control brain dynamics holds great promise for the enhance- 2 ment of cognitive function in humans, and the betterment of their quality of life. Yet, n successfullycontrollingdynamicsinneuralsystemsischallenging,inpartduetotheim- mensecomplexityofthebrainandthelargesetofinteractionsthatcandriveanysingle a J change. Whilewehavegainedsomeunderstandingofthecontrolofsingleneurons,the control of large-scale neural systems—networks of multiply interacting components— 9 remains poorly understood. Efforts to address this gap include the construction of 1 tools for the control of brain networks, mostly adapted from control and dynamical systemstheory. Informedbycurrentopportunitiesforpracticalintervention,thesethe- ] oretical contributions provide models that draw from a wide array of mathematical M approaches. Wepresentintriguingrecentdevelopmentsforeffectivestrategiesofcontrol in dynamic brain networks, and we also describe potential mechanisms that underlie Q suchprocesses. Werevieweffortsinthecontrolofgeneralneurophysiologicalprocesses . with implications for brain development and cognitive function, as well as the control o of altered neurophysiological processes in medical contexts such as anesthesia admin- i b istration, seizure suppression, and deep-brain stimulation for Parkinson’s disease. We - conclude with a forward-looking discussion regarding how emerging results from net- q workcontrol—especiallyapproachesthatdealwithnonlineardynamicsormorerealistic [ trajectoriesforcontroltransitions—couldbeusedtodirectlyaddresspressingquestions inneuroscience. 2 v 1 3 CONTENTS 2. Structuraldriversofsynchrony: Graph 5 architectureandsymmetries 12 1 I. Introduction 1 B. Thecostofcontrollingspecifictrajectories 13 0 C. Empiricaltoolsforcontrolofspecificneural . II. Howdoesthebraincontrolitself? 2 dynamicsorpathways 14 1 0 III. Networkcontroltheory 4 VII. Emergingcontrolmethodswithpotentialutilityin 7 A. Controloflineardynamics 4 neuroscience 14 1 B. Keydrivernodes 4 A. Broadercontrolregimes 15 : C. Controlenergyandmetrics 5 1. Non-lineardynamics 15 v 2. Time-dependentcontrol 15 D. Applicationtobrainnetworks 6 Xi 3. Realisticcontroltrajectories 16 IV. Understandinghealthybrainfunctionthroughcontrol B. Exploitingsystemproperties 16 ar theory 6 1. Compensatoryperturbationsornoise 16 A. Networkcontrolasapartialmechanismforcognitive 2. Networktopology 16 control 7 VIII. Conclusion 17 B. Networkcontrolandcognitiveperformance 7 C. Evolutionofnetworkcontrolindevelopment 8 Acknowledgments 17 D. Openquestionsincontrolandcognition 8 References 17 V. Targetingtherapeuticinterventionstomaximize beneficialoutcomestopatients 9 A. Anesthesiatitration 9 B. Deep-brainstimulationforParkinson’sdisease 9 I. INTRODUCTION C. Non-invasivetranscranialstimulation 10 D. Seizuresuppressioninepilepsy 10 The brain displays a wealth of complex dynamics across various spatial and temporal scales (Betzel and VI. Controlofspecificneuraldynamicsorpathways 11 Bassett, 2016; Kopell et al., 2014). From 302 neurons A. Synchronyofneuralpopulations 11 1. Dynamicalcharacteristicsandclinicalrelevance 11 in the nematode worm C. elegans (Bentley et al., 2016; 2 Varier and Kaiser, 2011) to some 86 billion neurons in et al., 2011; Hagmann et al., 2008) or synapses at the the adult human (von Bartheld et al., 2016; Herculano- small scale. In functional brain graphs, the edges rep- Houzel et al., 2007), the units that drive brain func- resent synchronized dynamics that form functional links tion are large in their number but even more compli- (Achard et al., 2006; Stam, 2004) between these units. cated in their interactions. Far from the canonical mod- Our choice to focus on the control of brain networks els in statistical mechanics stemming from either crys- enables us to build a theoretical understanding regard- talline or random structure, the brain displays a hetero- ingbiologicalprocessesandassociateddynamicsthatoc- geneous pattern of interconnections (Bassett and Bull- cur across spatially distributed systems in neural tissue. more, 2016; Castellana and Bialek, 2014; Fraiman et al., Should the reader instead be searching for an excellent 2009) that fundamentally constrains the propagation of treatment of various control methods for single neurons activity. Understanding these dynamics remains of pri- or for ensembles of neurons, we direct them to the re- maryinterestinthefieldofneurophysics(GaoandGan- centtextbookbySchiff(2012),andtoreferencestherein. guli, 2015; Scott, 1977). An emerging and intriguingly For further details on emerging control technologies in tractable avenue for understanding the mechanisms of thebrain—especiallyinvasiveelectricalandopticalstim- thesedynamicsliesinthenotionofcontrol,orhowtoef- ulation at rapid timescales (milliseconds or below)—and fectivelyguideneuraldynamics. Howarebraindynamics associated modelling approaches, please see the recent controlled intrinsically in the awake, behaving animal? review by Ritt and Ching (2015). Can we harness natural principles of control in neural The remainder of this Colloquium is organized as fol- systems to better guide therapeutic interventions? lows. In Sec. II we draw inspiration for understanding The increase in available experimental neurotechnolo- control of brain networks by considering how the brain gies (Chang, 2015; Nag and Thakor, 2016; Patil and itself enacts intrinsic control. In particular, we briefly Thakor, 2016), as well as more sophisticated compu- discuss important computational paradigms of cognitive tational tools (Glaser and Kording, 2016; Marblestone control, a basic ability that each of us has to control et al., 2016) and theoretical models (Giusti et al., 2016), our neural activity and by extension our behavior. This hasrecentlymadeitpossibletotacklethisquestionfrom discussion motivates the introduction of network con- fundamentally new angles. While at present there is no trol theory in Sec. III, which offers a useful theoretical comprehensivetheoryofcontrolinthebrainthatwecan framework in which to probe control in brain networks refer to, the pursuit of such a theory remains critically constructed from neuroimaging data. We next turn in important, having pervasive implications for our under- Sec. IV to detailing a few examples of how we can use standingofhealthyneurophysiologicalprocesses,andour network control theory, or its extensions, to understand abilitytointervenewhenthosehealthyprocessesgoawry healthy brain function. In Sec. V, we describe the util- inneurologicaldiseaseandpsychiatricdisorders(Bassett ity of network control in targeting interventions when and Khambhati, 2017; Chen et al., 2014; Johnson et al., healthybrainfunctiongoesawry. WenextturninSec.VI 2013). Severalrecentmodelsproposenewwaystocontrol to modeling the controlled versus uncontrolled trajecto- neural activity and neural rhythms, and further provide ries of neural dynamics, and we close in Sec. VII by out- mechanistic insights into the rules by which brain dy- lining emerging frontiers at the intersection of dynami- namics are (and can be) guided. Hence, it is timely to calsystemstheory, controltheory, andcomplexsystems. discuss these emerging developments, and to seek to tie Throughout,wekeepneurosciencejargontoaminimum, themtogetherintoameaningfultheoreticalfieldthatcan although some terminology specific to the technique or be used to tackle current open questions in neuroscience context remains unavoidable. Our goal is to stimulate and medicine. discussion and encourage further work from physicists, Motivated by recent progress in understanding brain control theorists, practitioners and others in this excit- function from the perspective of interacting networks ing and rapidly developing field. (BassettandBullmore,2006,2009;BullmoreandSporns, 2012; Kaiser, 2011), we focus on systems-level control of either local neural circuits or whole-brain connectomes II. HOW DOES THE BRAIN CONTROL ITSELF? (Fornito and Bullmore, 2015; Sporns et al., 2005). Here we use the term “network” in the sense that is common While there may be many ways of tackling the ques- in network science (Newman, 2010). A brain network is tionofhowtocontrolbraindynamics,arguablyoneofthe a graph whose nodes represent units of the brain that simplest is to ask how the brain controls itself. Perhaps perform a specific function, like vision or audition (Bull- by understanding intrinsic mechanisms of control in the more and Bassett, 2011). At the large-scale, these units brain, we could harness that knowledge to inform ther- may be several centimeters of tissue, while at the small apeutic interventions for people with mental illness. In scale, these units may be individual neurons. In struc- considering this idea, it is useful to distinguish between tural brain graphs, the edges can represent structural external control, which is enacted on the system from links such as fiber bundles at the large scale (Bassett the outside, and internal control, which is a feature of 3 the system itself. In the brain, internal control processes include phenomena as conceptually diverse as homeosta- sis, which refers to processes that maintain equilibrium of dynamics (Nelson and Turrigiano, 2008; Nelson and Valakh,2015),andcognitivecontrol,whichreferstopro- cesses that exert top-down influence to drive the system betweenvariousdynamicalstates(BotvinickandBraver, 2015; Heatherton and Wagner, 2011). Here we focus on cognitive control because it is conceptually akin to the idea of extrinsic control: driving dynamics from one type to another. What can we learn from cognitive control that might help us to develop a theory for external control? To answer that question, we begin by turning to history. An early computational model that explained the production of decisions based on a given set of inputs was the perceptron (Freund and Schapire,1999;Rosenblatt,1958),asimpleartificialneu- ral network (Bishop, 1995; McCulloch and Pitts, 1943). FIG. 1 Model for adaptive cognitive control showing The perceptron and associated notions were developed distinct mechanisms between different brain regions. by proponents of connectionism (Medler, 1998), which Schematic of a neural network connecting the prefrontal cortex, which executes much of the “top-down” control actions, suggests that cognition is an emergent process of inter- to other brain regions. Another part of the brain – the ante- connected networks. The complexity of the connection rior cingulate cortex – serves as a conflict monitoring mech- architecture in these models was thought to support a anism that modulates the activity of control representations, complexity of brain dynamics, such as the separation of whileanadaptivegatingmechanismregulatestheupdatingof parallelneuralprocessesanddistributedneuralrepresen- controlrepresentationsinprefrontalcortexthroughdopamin- tationspropoundedbytheparalleldistributedprocessing ergic projections. From Botvinick and Cohen (2014). (PDP) model (Rumelhart et al., 1986). Notably, the PDP model offers conceptual explana- tions for the processes characteristic of cognitive control (Botvinick and Cohen, 2014). These ideas are built on the notion that the development of control systems in sors (Eisenreich et al., 2016; Haykin and Fuster, 2014). the brain (Chai et al., 2017) can be seen as responding Howexactlyinformationisprocessedonthesedistributed to the structure of naturalistic tasks, and therefore that systems remains an open question, but some promising control can be defined as the optimal parameterization modeling approaches include those that use Bayesian in- of task processing. Within such a parameterization, two ference, sparse-coding, and information entropy to char- specific features of cognitive control appear particularly acterize control (Haykin and Fuster, 2014). Specifically, critical: (i) its remarkable flexibility, which supports di- a few recent efforts draw heavily from the idea of prob- versebehaviors,and(ii)itsclearconstraints,whichlimit abilistic reasoning to formulate a model for risk control the number of control-demanding behaviors that can be – posited to be an overarching function of the prefrontal executed simultaneously. Addressing these two features, cortex – characterized by a closed-loop feedback struc- models inspired by the PDP approach allow for cogni- ture describing executive attention. tive control as instantiated in processes of selection from competing inputs or adaptation based on reward (Fig. 1). To briefly summarize, previous computational models These and related computational models emphasize of cognitive control have included the eclectic notions of the role of specific brain areas in cognitive control, in- neural networks, regional localization, distributed pro- cludingprefrontalcortex,anteriorcingulate,parietalcor- cessing, and information theory. Collectively, these no- tex, and brainstem. Yet, studying any of these areas in tionsmotivatetheconstructionofamodelortheorythat isolation will likely provide an impoverished undestand- explicitly builds on the emerging capability to measure ing of the system’s function. Indeed, Eisenreich et al. the brain’s true network structure to better understand (2016) argue that control in the brain is not localized to control. In the next section, we will describe recent de- smallregionsormodules,butisinsteadverybroadlydis- velopments in dynamical systems and control theory as tributed, enabling versatility in both information trans- applied to complex networks, whose application to the fer and executive control. Such a distributed – and even brain may offer explanatory mechanisms of neural dy- perhapsoverlapping–networkarchitecturecanalsooffer namics and provide insights into the distributed nature usefullyfuzzyboundariesbetweencontrollersandproces- of cognitive control. 4 III. NETWORK CONTROL THEORY Conceptually, it is interesting to ask the question whether and to what degree cognitive control (as de- finedbyneuroscientists)issimilartonetworkcontrol(as defined by physicists, mathematicians, and engineers). To address this interesting question, we must first define what it is that we mean by network control. Control- lability of a dynamical system refers to the possibility of driving the current state of the system to a specific FIG.2 Controllingasimplenetwork. Thissmallnetwork target state by means of an external control input, see can be controlled by an input vector uK = (u1(t),u2(t))T Kalman et al. (1963). Developments in engineering and (left),allowingustomovethenetworkwithinthestatespace, fromitsinitialstatetosomedesiredfinalstate(right). From physics have recently extended these ideas to the control Liu et al. (2011). of networks, as we describe in more detail below. A. Control of linear dynamics B. Key driver nodes We begin by describing a general framework for the Recent work from Liu et al. (2011) demonstrated that control of linear dynamics on a complex network. Con- the analytical framework described in the previous sec- sider a network represented by the directed graph G = tion could be used to study large, complex networks. In (V,E), where V and E are the vertex and edge sets, re- that study, the authors explored common patterns in a spectively. Letaij betheweightassociatedwiththeedge wide variety of networks from biological to man-made (i,j) ∈ E, and define the weighted adjacency matrix of and social. Under certain conditions in these weighted G as A = [aij], where aij = 0 whenever (i,j) (cid:54)∈ E. We and directed networks, the set of driver nodes capable of associate a real value (state) with each node, collect the guiding the dynamics of the entire system could be di- node states into a vector (network state), and define the rectlyestimatedfromthedegreedistribution. Sincethat map x : N≥0 → Rn to describe the evolution (network study,othershaveshownthatunderotherconditionsand dynamics) of the network state over time. A simple way in other networks, the degree distribution alone may not tobeginistodescribethenetworkdynamicsbyadiscrete provide enough information to adequately determine the time, linear, and time-invariant recursion set of driver nodes. Instead, that knowledge regarding the network’s structure must be complemented with an x(t+1)=Ax(t). (1) understanding of the network’s dynamics, or the state LetasubsetofnodesK={k ,...,k }beindependently equations at each node (Cowan et al., 2012). 1 m controlled, and let In these studies, networks are allowed to contain real- (cid:2) (cid:3) valued weights on each edge. However, for some real- B := e ··· e (2) K k1 km world networks, knowledge of the edge weights is uncer- be the input matrix, where ei denotes the i-th canonical tain. For such scenarios, a complementary framework is vector of dimension n. The network with control nodes providedbystructuralcontrollabilitywhichevaluatesthe K reads as controllability of binary networks (Kailath, 1980; Rein- schke, 1988). By studying the underlying “structure”, x(t+1)=Ax(t)+B u (t), (3) K K i.e. distinguishingmerelybetweenwhichedgesareabsent where u : N → R is the control signal injected into (zero)versus present(non-zero),thesemethodsallowthe K ≥0 the network via the nodes K (see Fig. 2). The network identificationofminimalstructuresorcontrolpointsthat (3) is controllable in T steps by the nodes K if, for every allowforfullcontrollabilityofthenetwork. Recentefforts statex ,thereexistsacontrolinputu suchthatx(T)= have extended these ideas to large-scale systems, and to f K x with x(0)=0 (Kailath, 1980). theproblemofidentifyingtheminimumnumberofnodes f Controllabilityofthistypeofsystemcanbeensuredby thatneedtobedriveninordertoachievestructuralcon- differentstructuralconditions(Kailath,1980;Reinschke, trollability (Pequito et al., 2016a). 1988). Forinstance,letCK,T bethecontrollabilitymatrix In recent work, Pequito et al. (2016b) extended the defined as notion of structural controllability to situations in which C :=(cid:2)B AB ··· AT−1B (cid:3). edges evolve dynamically, and they identified the mini- K,T K K K mum number of driven nodes for full controllability of Thenetwork(3)iscontrollableinT stepsbythenodesK the system. Their methods would appear particularly ifandonlyifC isoffullrowrank,whereT istypically relevant in situations like those observed in Khambhati K,T taken to be at least as large as the system size n. et al. (2015), who studied dynamic functional connectiv- 5 ity in epileptic patients, and faou. nd that the edges within b. Control input seizure-generating areas are almost constant over time, Control 8 Modal controllability: whereastheedgesoutsidetheseareasexhibithighervari- trajectory 7 Distant transitions ability over time. An important potential goal of control 6 wouldthenbetosteerfunctionontheseedgesawayfrom 5 pathological regimes (Pequito eDtiffauls.i,on2 0te1n6sbo)r .imaging Tractography 4 8 While network control and structural controllability 6 3 4 are particularly relevant concepts for brain network con- 2 2 trol,manyotherkeycontributionshavebeenmadetothe -02 1 -4 0 study of control in complex networks, which lie outside -6 Average controllability: 0 1 2 E3n4er5gy6 7 8 Connectivity Nearby transitions thescopeofthisarticle. WewishtoBrpaoini nntetiwnotrekr estedrebaedtw-een regions erstothefollowingreviewsthact.f ocusentirelyonnetwork d. FIG. 3 Energetic costs of controllabilitry metrics. cpstraeeooveinilniteCrotwosnhletbtnnoyoeo(taL2wlffsi0.uoe1crF4akto)n.prcdoaaFnrBotrtairercrvoumaillebaowiánrrseoidcfg(yo2menm0ndal controllability ae1pemt6rl0000hea)i....9999oxcl6789psdbsrssayoutcvsokctidehgimderasoessun,atnnstidyfhyenexaccnchroedernlcolteednrnnoeytt-,l PttstAhracgaoanaeslpttql aeuoib,n1112ani4680clwleilttuthhytdiiilseeage controllability deletaveatmhs32n12c2305ae..o50 d255r ldis.eb can rp(ea le<=2spr1 0c0gexot1..e1r2n4ta08t)in-rc 5osp cilrtlooaisopbtnoislssiotenyfercedaoarenblsitcysrrtoiiobclne(cs7oa)ntn.rtarAeno1112nvls4680eeitsrritgaorygnaetsleacgdnoiindess--- summary of the latest developmmoents. 12 ver22001 12 C. Control energy and metrics Mean 00..994515 20 IFnitdt2eivd5id puaarl aCmoe3ntn0riecc ftoormmwoef3h5WereK10e,TquaasMean alsiot0yc.5i1ias588t eadc h11w0i0e2iv0t he11d22λwm 121hi44n5e(nW 1e166vKe3r ,01T18x8)f. 2i20s03 5a n 2222e1 0igenvector Mean average controllability Agagee Average controllability identifies network nodes that, Another important area of work lies in the develop- on average, can steer the system into different states mentofmetricsthatcharacterizedifferentcontrolstrate- with little effort (i.e., input energy), see Fig. 3. The gies for real networks. Classic results in control theory average controllability in a network—formally defined as ensure that controllability of the network (3) from the Trace(W−1)—equalstheaverageinputenergyfromaset K set of network nodes K is equivalent to the controllabil- ofcontrolnodesandoverallpossibletargetstates(Marx ity Gramian WK being invertible, where et al., 2004; Shaker and Tahavori, 2012). This is mo- ∞ tivated by the relation Trace(W−1) ≥ N2/Trace(W ) (cid:88) K K WK = AτBKBTKAτ (4) (Summers and Lygeros, 2014), and the fact that WK is τ=0 close to singularity even for networks of small cardinal- and for A satisfying Schur stability. In practical appli- ity. ItshouldbenoticedthatTrace(W )encodesawell- K cations, controllable networks featuring small Gramian definedcontrolmetric,namelytheenergyofthenetwork eigenvalues cannot be steered to certain states because impulse response or, equivalently, the network H norm 2 the control energy is limited. This fact motivated (Kailath,1980). Intuitively,networknodeswithhighav- Pasqualettietal.(2014)toproposecertaincontrolstrate- eragecontrollabilityaremostinfluentialinthecontrolof giesandassociatedmetricsbasedonminimizingthecon- network dynamics over all possible target states. trol energy; these include average, modal, and boundary Modal controllability identifiesnetworknodesthatcan controllability. push the network activity into difficult-to-reach states, Todefinethesecontrolmetrics,wefirstletthenetwork which are those that require substantial input energy. be controllable in T steps, and let xf = x(T) be the To quantify modal controllability, we first note that the desired final state in time T, with ||xf||2 =1. Define the behavior of a dynamical system is fully determined by energy of the control input uK as the eigenvalues (modes) and eigenvectors of its system T matrix. Regarding controllability, the Popov-Belovich- (cid:88) E(u ,T)=||u ||2 = ||u (τ)||2, (5) Hautus test ensures that a system with matrix A is con- K K 2,T K 2 trollablebyaninputmatrixBifandonlyifallitsmodes τ=0 whereT isthecontrolhorizon. Theuniquecontrolinput arecontrollableor,equivalently,ifandonlyifthereexists thatsteersthenetworkstatefromx(0)=0tox(T)=x no left eigenvector of A orthogonal to the columns of B f with minimum energy is (Kailath, 1980) (Kailath, 1980). In particular, the ith mode is control- lablebythematrixBifandonlyifw B(cid:54)=0,wherew is u∗K(t)=BTK(AT)T−t−1WK−,1Txf (6) alefteigenvectorofAassociatedwithiitsith mode. Intiu- with t∈{0,...,T −1}. Then it can be seen that itively,networknodeswithhighmodalcontrollabilityare able to control all of the dynamic modes of the network, T−1 E(u ,T)= (cid:88)||u∗(τ)||2 =xTW−1 x ≤λ−1 (W ), and hence to drive the dynamics towards hard-to-reach K∗ K 2 f K,T f min K,T configurations. τ=0 (7) Boundary controllability identifies network nodes that 6 lie at the boundaries between network communities, as a. b. defined across all possible levels of hierarchical modular- ity in a network, and thus intuitively measures the ability to control the integration and segregation of network modules. Thismetricdependsonthechoiceofamethod fordetectingboundarycontrolpoints, forwhichanalgo- Diffusion tensor imaging Tractography rithm is proposed in Pasqualetti et al. (2014). This al- gorithmcanbealteredasneededforthephysicalsystem d. c. 8 6 under study, e.g., to enhance the accuracy in estimating 4 an initial partition of the network into communities, and 2 0 to sharpen or loosen the boundary point criteria. Intu- -2 -4 itively,networknodeswithhighboundarycontrollability -6 are able to gate information between different communi- Connectivity between Brain network ties, across topological scales in the network. regions Overall, these three metrics provide useful estimates FIG.4 Construction of a human brain structural net- for real systems especially when considering dynamics work. (a)Diffusionimagingmeasuresthedirectionofwater over the whole network. Further work could be done diffusion in the human brain. (b) From these data, white to investigate other scenarios such as dynamics in just matter streamlines can be reconstructed that connect brain partsofthenetwork,orhowdifferentpatternsofcommu- regions. (c) Adjacency matrix representation of the struc- nitystructurechangetheresultingcontrollability. These tural connectivity: entries denote the estimated strength of and more general questions about the relationship be- whitematterconnectivitybetweenbrainregions. (d)Result- tween network topology and the resulting dynamics re- ing brain network where nodes are brain regions and edges are the connection strengths between them. mainopenareasofstudy,whichwediscussinmoredetail at the end of this article. Usingnetworkscomposedofbetween83and1015nodes, D. Application to brain networks the authors study the three controllability metrics of av- To use these methods to answer questions in neuro- erage, modal, and boundary controllability (Pasqualetti science,wemustbeginbyconstructingnetworksbasedon et al., 2014) discussed in the previous section. While our knowledge of brain connectivity. At the large scale, these techniques have not yet been ubiquitously applied network nodes in the brain are often defined based on tonon-humanimaging(BadhwarandBagler,2015;Tang regional differences in cellular architecture (Brodmann, etal.,2012),themathematicsisgeneralizabletoanyesti- 1909; Glasser et al., 2016) or local gradients in fine-scale mate of structural connectivity in a neural system. Con- functional connectivity (Power et al., 2011; Yeo et al., ceptually,thisapproachsupportsthegeneralstudyofthe 2011). Connectivity between these nodes can be es- kinds of dynamics predicted by the constraints of struc- timated with emerging neurotechnologies. In humans, tural connectivity, particularly for the scenario in which one particularly powerful non-invasive probe of connec- a given brain region is acting as a control point for the tivity uses magnetic resonance imaging (MRI) to infer rest of the network. structural pathways in the brain (Wedeen et al., 2012) by exploiting molecular resonances of water molecules as they diffuse along white-matter tracts (Basser et al., 1994; Makris et al., 1997), see Fig. 4. By reconstruct- ingthepathwaysthatexistbetweenbrainregionsandby estimating the strengths of those pathways, a brain net- IV. UNDERSTANDING HEALTHY BRAIN FUNCTION work (weighted, symmetric graph) is obtained where the THROUGH CONTROL THEORY network edges are given by the inter-regional connection strengths (Bassett et al., 2011; Hagmann et al., 2008). Similar techniques can be used in rodents, cats, dogs, In this section, we explore the utility of network con- and non-human primates by way of a small-bore magnet trol theory for offering mechanisms of cognitive control, (Duong, 2010). Of course, tract-tracing techniques and providingexplanationsforindividualdifferencesincogni- other invasive methods are also a powerful way to im- tivecontrolacrosspeople,andcapturingtheevolutionof age structural pathways in non-human animals (Markov controlaswegrowfromchildrentoadults. Weclosethis et al., 2011; Okano and Mitra, 2015). section by discussing open questions in cognitive neuro- Recently,Guet al.(2015)appliednetworkcontrolthe- science that appear particularly amenable to extensions ory to such whole-brain structural networks in humans. of network control theory. 7 FIG.5 Cognitivecontrolhubsaredifferentiallylocatedacrosscognitivesystems. (a)Hubsofaveragecontrollability arepreferentiallylocatedinthedefaultmodesystem. (b)Hubsofmodalcontrollabilityarepredominantlylocatedincognitive control systems, including both the frontoparietal and cingulo-opercular systems. (c) Hubs of boundary controllability are distributed throughout all systems, with the two predominant systems being ventral and dorsal attention systems (Gu et al., 2015). A. Network control as a partial mechanism for cognitive 2015) are structurally predicted to optimally push the control systemintodistantstates,farawayontheunderlyingen- ergylandscape. Lastly,strongboundarycontrollerswere A simple question to ask about any theory is whether disproportionately located in regions implicated in at- or not it offers predictions of observed processes. One tention (Corbetta and Shulman, 2002), supporting their particularly straightforward and testable hypothesis is predicted role in gating (Eldar et al., 2013; Womelsdorf thatthecommoncontrolstrategiesstudiedincontroland and Everling, 2015) information between network com- dynamical systems theory are strategies that the brain munities. uses to control its own intrinsic dynamics. In a recent Thisstudyoffersapossiblemechanisticexplanationfor study, Gu et al. (2015) addressed this hypothesis by first how the brain might move between cognitive states that calculating the controllability strengths for each brain depends fundamentally on white matter microstructure. region, and then by identifying the preferences of each The work suggests that structural network differences brain region for different types of control. The authors between the default mode, cognitive control, and atten- found that strong average controllers, strong modal con- tionalcontrolsystemsdictatetheirdistinctrolesinbrain trollers, and strong boundary controllers were located in network function. While the results need to be validated quite different areas of the brain, see Fig. 5. in other species and data sets, the broad trends indicate the relevance of control theory for capturing canonical Intriguingly, the different sorts of controllers appeared concepts in cognitive control. to map on to the types of function that each brain region is thought to perform. For example, strong average controllersweredisproportionatelylocatedinthedefault modesystem,whichisaspatiallydistributedsetofbrain B. Network control and cognitive performance regionsthatareparticularlyactivewhenapersonissim- ply resting (Raichle, 2015). This is particularly inter- In the previous section, we reviewed evidence that no- esting because it suggests that areas of the brain that tionsfromnetworkcontrolappliedtoneuroimagingdata are active in the “ground state” are also areas that are can provide insight into the roles that brain regions may structurally predicted to optimally push the system into play in the control of neural dynamics. Here we ask the many local easily-reachable states, close by on the un- more specific question of whether the brain in one per- derlying energy landscape. Furthermore, strong modal son (or animal) might be optimized for a different type controllers were disproportionately located in cognitive of control than the brain in another person. That is, can control systems, including both the frontoparietal and controllabilitymetricsexplainwhycognitiveperformance cingulo-opercular systems. This is particularly interest- differs across individuals? ing because it suggests that the areas of the brain that While still a very open question, two recent studies are active during tasks that demand high levels of cog- suggest that indeed each brain displays a different pro- nitive control or task switching (Botvinick and Braver, file of control, and differences across people are corre- 0 0 0 0 0 0 .9 .9 .9 .9 .9 .9 14 5 6 7 8 9 5 2 8 0 cpttblsahatolotuismeueekdsddnpesyd.awtsariMoaniertmyhoithnrhedcedeeoaiiisffvnnplpteitdrhderreiyceuovdinlaiifiladcclactdeubabisuaoilrlllilniaytnsts,iynthotfhrhun(ioeemsmmietraaeraucangnotdIseehIg,istItnow.iMtCriootosni)ervcakdteaolhaclncgeccoaluonippbalgtaarenrtarcoeiefiittlontimitvraehmenlsoe..edaoct(nIarow2nylcn0oeato1arrnnk6poodes)-fl modal controllability 0000....99996789 253035Mean average Fitted parametric formcontrollability Individual Connectomeave223050 8p11112r< 246801=01 x0112.208A-154 gagee16 12..112558 20 22 1111224680 obtained from diffusion imaging, and they also test the ean 0.95 IFnitdteivdid puaarl aCmoentnriecc ftoormme 1100 112120 1144 A 116g6e 1188 2020 10 performance of subjects in cognitive control tasks that M0.94 0.5 measure the inhibition of behavior, the shifting of atten- 15 20 25 30 35 15 20 25 30 35 Mean average controllability per0 subject 1 2 tion,vigilance,andworkingmemorycapacity. Thestudy 1.5 1 .5 2 .5 5 reports key regional controllers in the brain whose con- FIG. 6 Controllability metrics are positively corre- trollability strength is correlated with task performance lated with age, with older youth displaying greater measuresacrossindividuals,thusprovidingasecondline average and modal control20lability than younger of evidence that network control may be a partial mech- youth. Each data point represents the average strength of controllability metrics calculated on the brain network of a anism for cognitive control in humans. 2 single individual, in a cohort of 8852 healthy youth from ages Turning from adults to children, Tang et al. (2016) 8 to 22. Brain networks were found to be optimized to sup- evaluated the controllability strength of brain regions as portenergeticallyeasytransitions(averagecontrollability)as 3 well as more general cognitive performance (not specific well as energetically costly ones (m0odal controllability). The tocognitivecontrol)inacommunity-basedsampleof882 colorbardenotestheageofthesubjects,illustratingasignif- healthyyouththatrangedinagefrom8to22years. The icant correlation between age and3the ability to support this 5 authors found that the relative strength of average con- diverse range of dynamics (Tang et al., 2016). 1 1 1 1 1 2 trollers in subcortical versus cortical regions (which are 0 2 4 6 8 0 the earliest evolving and latest evolving brain areas, re- spectively) is an important predictor of improved cogni- network control. These results suggest key neurophys- tive performance. This relationship held true even when iological changes that may be occuring during develop- accounting for differences in age across the cohort, sug- ment, driving the system towards an increasing capabil- gesting that it is a fundamental characteristic of human ity to traverse a larger surface of the energy landscape. brain structure and dynamics. It would be intersting in future to assess whether these metrics are altered in youth with neuropsychiatric disor- ders, or whether they could be used to predict transition to psychosis. C. Evolution of network control in development The identification of age-invariant relationships between controllability metrics and cognitive function begs D. Open questions in control and cognition the question of whether controllability metrics of brain networkschangewithage,eitherintheirmagnitudeorin Itisimportanttonotethatlinearmodelsofneuraldy- theirspatialdistribution. Toaddressthisquestion,Tang namics (Fernández Galán, 2008; Honey et al., 2009) for etal.(2016)studiedthecontrollabilitymetricsofaverage use in network control theory have both advantages and controllability and modal controllability in 882 healthy disadvantages. Their advantage is that one has access youth from 8 to 22 years, and quantified a single value to a wide array of theoretical observations that can of- of controllability for a person as given by the average fer intuitions about the system’s (controlled) dynamics, of controllability strengths across all brain regions. This particularlyaround anoperatingpoint(Gu et al.,2015). coarse-graining of the data enabled the authors to study Thedisadvantageisthattheycannotspeaktoneuralpro- howbrainnetworksarefacilitateenergeticallyeasytran- cesses that transition from one dynamical regime (limit sitions (average controllability) as well as energetically cycles, fixed points, attractors) to another (Deco and costly ones (modal controllability). Jirsa, 2012; Golos et al., 2015; Muldoon et al., 2016a). They found that brain networks are highly optimized Inthesecases, developingadditionalmethodsforcontrol to support a diverse range of possible dynamics (as com- of nonlinear systems may be necessary. paredwithrandomizedversionsofthenetworks)andthat One simple scenario in which limit cycles – or transi- this range of supported dynamics increases with age, see tions between them – may be particularly important for Fig. 6. Seeking to investigate structural mechanisms processes of cognitive control is that of human decision- that support these changes, the authors simulate net- making(ChandandDhamala,2016;Chandet al.,2016). work evolution with a set of growth rules, to find that For example, oscillatory activity in specific brain regions all brain networks – from child to adult – become in- has been linked to rational versus irrational decision- creasingly structured in a manner highly optimized for makinginataskthatrequiresfinancialjudgements(akin 9 to gambling). Sacr et al. (2016) studied a group of human subjects in which multiple depth electrodes were implanted in deep brain structures as a part of routine presurgical evaluation for medically refractory epilepsy. By recording the local field potentials at each of these electrodes, the authors were able to monitor the activity of neuronal ensembles in the precuneus and show that high-frequency activity (70-100 Hz) increased when irrational decisions were made. This and similar studies in other areas of higher-order cognitive function that depend upon synchronized oscillatory activity in neuronal ensembles (Bassett et al., 2009; Kopell et al., 2000) suggest the possibility that control strategies could be de- vised that use brain stimulation to alter the frequency FIG.7 Burstsuppressionphenomenology. (a)Atypical of neuronal synchrony to modulate cognitive processes. recording of burst suppression from a human subject anes- Such a possibility will depend on accurately extending thetized with propofol – a type of general anesthesia. The linear control models to nonlinear ones, isolating the dy- bursts manifest concurrently across the scalp (here, shown namics relevant for the cognitive process of interest, and for left and right frontal electrodes). (b) Spectrogram for a localizing the region that is most impacted. frontalelectrodeduringdeep,butnotburst-suppression,general anesthesia. (c) At a deeper level of general anesthesia, burst suppression is achieved (the spectrogram clearly displays epochs of quiescence). From Ching et al. (2012b). V. TARGETING THERAPEUTIC INTERVENTIONS TO MAXIMIZE BENEFICIAL OUTCOMES TO PATIENTS debilitating side effects. In this section, we broaden our focus from linear mod- Building on their earlier biophysical model, Ching els of network control in order to more generally discuss et al. (2013) suggest the real-time computation of emerging engineering approaches for the control of brain the brain’s burst suppression state from the electroen- dynamicsinthecontextofclinicalmedicine. Weseparate cephalogram (see Fig. 7), which indicates a state of our discussion into methods for modulating conscious- highly reduced electrical and metabolic activity (Ching ness via anesthesia administration, methods for ongoing et al., 2012b). This is done using an on-line parame- monitoring and treatment of Parkinson’s disease, meth- ter estimation procedure and proportional-integral con- ods for non-invasive stimulation, and methods for the troller. The technique has already been validated in ro- control of transient epileptic seizures. These topics are dents, where it can be used to successfully monitor and in no way meant to be comprehensive of the field, but controltheburstsuppressionstate. Translatingthiswork simply to highlight important directions of clinical rel- into humans will require more extensive computational evance. Where appropriate, we point out connections estimation of model parameters and empirical validation to the network control literature, and opportunities for over periods of several hours. further synergies between medicine and control theory. B. Deep-brain stimulation for Parkinson’s disease A. Anesthesia titration High-frequency deep brain stimulation (DBS), com- Anesthesia is used in medical institutions to modulate monly used to treat Parkinson’s disease, is one of the consciousness through drugs during surgery, potentially oldest examples of successful dynamical manipulation of byalteringdistributedcircuitry(Croneet al.,2016). Ac- brain function to alleviate clinical symptoms. Yet, it re- curatelytitratingthelevelsofanestheticforeachperson, mainsunclearexactlyhowandwhyitworkssowell. Con- andateachtimepointduringthesurgery,iscriticallyim- trol and systems theory approaches are useful for mod- portantforthehealthandsurvivalofthepatient. Recent elling the underlying circuitry to understand the mecha- efforts seek to optimize this titration using a closed-loop nismsbywhichdeep-brainstimulationaffectsbehavioral system (Ching et al., 2013), where the challenge is to phenotypes (Santaniello et al., 2015; Tass et al., 1998; maintain a medically-induced coma by delivering propo- Wilson and Moehlis, 2016). fol via an intravenous catheter or pump. Using a com- Recent work has highlighted the network-level mecha- puter to control this delivery system, precise amounts of nismsofthediseaseddynamics,andthecontrolnecessary anesthetic can be chosen, administered, and adapted in to treat them. For example, Santaniello et al. (2015) a time-dependent manner, potentially reducing the in- move from localized functions to the relevant circuitry, cidence of propofol overdose which is accompanied by positing that DBS increases the regularity of firing pat- 10 terns in the basal ganglia, thereby decreasing symptoms of Parkinson’s disease (Chiken and Nambu, 2014). The authors suggest that high-frequency stimulation of 130 Hz in DBS is effective because it is a resonant frequency oftheoverallcortico-basalganglia-thalamo-corticalloop. The authors explore the effects of different stimulation conditions by simulating hundreds of biophysically real- istic neurons from different regions of the circuitry that FIG. 8 Closed-loop stimulation for seizure suppres- are thought to have very different functions. Their re- sion in a rat. Recordings from channels a,b and c in the sults suggest a loop-based reinforcement model, where cortexarefilteredforspikedetection,wheresignalsexceeding DBSproximallyordistallydoesnotindividuallyaccount the predetermined amplitude threshold are detected. These thresholded signals are used to trigger transcranial electric forresultingpatternchanges,butinsteadreliesonacom- stimulation,whichisappliedthroughthescalp. FromBerényi bined impact across the circuit. This observation could et al. (2012). inform the choice of stimulation frequency and location when using DBS clinically (Johnson et al., 2013). While identifying the resonant frequency of a critical invasive methods of brain stimulation are becoming in- circuit may provide a useful target for control, other creasingly feasible. The most common is that of tran- mechanisms may also exist, and it is possible that in- scranial magnetic (electric) stimulation, which is the ap- terventions targeting more than one mechanism could plication of a magnetic (electric) field through the scalp be more effective than targeting one mechanism alone. for a short period of time (Bikson et al., 2016). While Other candidate mechanisms include coupling between the effects of transcranial stimulation tend to be diffuse, peripheral tremor rhythms, and the phase locking of the they have demonstrated utility in treating depression activity of primary and secondary motor areas. For ex- andotherneurologicalandpsychiatricdisorders(Kedzior ample,Tasset al.(1998)proposetwotechniquestoiden- et al., 2016). Indeed, the ubiquitous use of non-invasive tify the relative phase locking between two MEG sig- stimulation makes it critical to build mechanistic mod- nals, thereby detecting synchronization of neuronal ac- els that provide a deeper understanding of the effects of tivityandmappingitsrelationshiptoperipheraltremors. stimulation (Johnson et al., 2013), and rules by which Otherattemptstouncovermechanismsincludetheinves- stimulation parameters and location can be optimized tigationofentrainmentanddesynchronizationdynamics, to enhance beneficial impacts, and mitigate detrimental both seen in populations of neurons, as a result of DBS. ones. Wilson and Moehlis (2016) study a population of model One study directly bridges mathematical models of neurons and the effects of stimulation, to observe under- nonlinearneuraldynamicsandthepredictionsofnetwork lying low-dimensional patterns that can illuminate col- controltheoryinthecontextofsuchexogeneousstimula- lective processes in spiking neurons. The simplicity of tion. Muldoon et al. (2016b) consider the effects of elec- that particular model affords theoretical insight into a tricalstimulationtoaspecificbrainregionusingamodel potential mechanism that governs DBS. of non-linear oscillators connected by a coupling matrix Once the optimal mechanism(s) have been identified, estimated from actual diffusion imaging data (Fig. 4). a tantalizing goal is the use of control theory to create a BysimulatingdynamicsinthisnetworkofWilson-Cowan closed-loopsystemformoreeffectivetreament. Holtand oscillators, they can test for different regimes of desired Netoff (2014) identify their goal for DBS as the suppres- functionaloutcomessupportedbythenetwork—iftheef- sionofpathologicalfrequenciesthatoccurduringParkin- fectsofstimulationremainfocalorspreadglobally—and son’s disease. They simulate the physiology of the basal compare these with the predictions from network con- ganglia to create a mean-field model of the closed-loop trol theory using the controllability metrics described in system, which allows for the tuning of stimulation pa- III.C. Importantly, their results validate linear network rameters based on patient physiology. This setup pro- control predictions over eight subjects and more gener- vides significant advantages over the current method of allyprovideamodelthatcanbeusedortestedinclinical trial-and-error tuning, which is based on the clinician’s settings, in order to strengthen the connection between past experience. If such a model can be empirically vali- theory and clinical practice. dated, it would be an important step towards improving theefficacyofDBSforpatientswithParkinson’sdisease. D. Seizure suppression in epilepsy C. Non-invasive transcranial stimulation Both invasive and non-invasive stimulation methods have been considered for the treatment of medically in- While such invasive monitoring and stimulation tractable epilepsy. Both types of interventions would paradigmsarenotaccessibletomosthumans,othernon- seemtobepreferabletothecurrentclinicalpracticeofre-