Feedforward Architectures Driven by Inhibitory Interactions

3.1 MB·

by  Yazan N. Billeh

Feedforward Architectures Driven by Inhibitory Interactions
Yazan N. Billeh1,∗ and Michael T. Schaub2,†
1Computation and Neural Systems Program, California Institute of Technology‡
2ICTEAM, Universit´e catholique de Louvain§
Directed information transmission is a paramount requirement for many social, physical, and
biological systems. For neural systems, scientists have studied this problem under the paradigm of
feedforward networks for decades. In most models of feedforward networks, activity is exclusively
driven by excitatory neurons and the wiring patterns between them while inhibitory neurons play
only a stabilizing role for the network dynamics. Motivated by recent experimental discoveries
of hippocampal circuitry and the diversity of inhibitory neurons throughout the brain, here we
illustratethatonecanconstructsuchnetworkseveniftheconnectivitybetweentheexcitatoryunits
in the system remains random. This is achieved by endowing inhibitory nodes with a more active
7 role in the network. Our findings demonstrate that feedforward activity can be caused by a much
1 broader network-architectural basis than often assumed.
0
2
I. INTRODUCTION in most feedforward networks. While some studies exist
n
that specifically account for intra- and interlayer inhi-
a
J The ability to reliably propagate signals in a targeted bition [3, 6, 24–26], information is still mediated by a
8 mannerisessentialfortheoperationofmanynaturalsys- cascade of excitatory neurons. However, as experimen-
1 tems, and a necessary building block to establish many tal evidence suggests, the nervous system likely uses a
further computational mechanisms. Prototypical models combinationofmethodstotransmitinformation[27,28].
C] for such targeted information transmission within a neu- An important natural question is thus if embedded ex-
ral substrate are feedforward networks, which have been citatory pathways are necessary for feedforward propa-
N
consideredintheliteraturefordecades. Inthesemodels, gation or whether one can construct networks in which
o. the basic paradigm is to group nodes (neurons) into sep- therearenopreferredexcitatory-to-excitatorypathways,
i aratelayers,eachofwhichreceivesexcitatoryinputfrom but inhibitory neurons play a pivotal role for the prop-
b
the preceding layer, and projects excitatory connections agation of activity between layers. In the following we
-
q tothesubsequentlayer(seereviews, [1,2]). Thethuses- demonstrate, via numerical experiments and brief ana-
[ tablishedforward-directedexcitatorypathwaysguidethe lytical considerations, that such feedforward processing

1 activity sequentially through the layers. A large number is indeed possible with two exemplary circuits.
v of variations of this scheme have been considered, such
5 asembeddingfeedforwardarchitecturesinrandomlycon-
0 nected networks to examine their effect on the overall II. RESULTS
9 network dynamics and signal propagation [3, 4]. Fur-
4 ther, feedforward networks have been shown to propa- A. Cross-coupled feedforward networks
0
gate firing rates [5, 6], synchrony/pulse packets [7–12],
.
1 combinations of firing rates and synchronous spiking [2], Inarecentexperimentalstudyofthehippocampus[29]
0
and even the ability to gate activity transmission [13]. it was demonstrated that excitatory neurons in the CA3
7
However, within the paradigm of feedforward networks, regionareunabletodriveexcitatoryneuronsinareaCA2
1
inhibitory units (neurons) play merely a balancing role: duetostrongfeedforwardinhibition. However,whenthis
:
v they ensure that the network remains stable, either sep- strong inhibition of CA2 is alleviated, CA3 can indeed
i
X arately for every layer or globally. exciteCA2excitatorycellstoelicitactionpotentials[29]
Interestingly, recent work in neuroanatomy has re- (Fig. 1a). An interesting feature of this finding is that
r
a vealed an enormous diversity of inhibitory neurons [14– the directionality in the interaction between CA2 and
22]. Moreover, specific plasticity rules for different sub- CA3 appears to be dictated by the connections between
types of inhibitory neurons [23] add further to their di- excitatory and inhibitory neurons, rather than a conse-
verse and heterogeneous connectivity profiles. In this quenceofunidirectionalexcitatoryconnectionstargeting
light it would be strongly surprising if inhibitory neu- CA2. Indeed, excitatory connections between CA2 and
ronsservethecortexonlyinahomogeneous,passiverole CA3 are reciprocal [30, 31] – yet there is still a directed
whenitcomestoinformationpropagation,asisassumed propagation towards CA2 [29]. Stated differently, the
targetedactivationofCA3iscontrolledbyanexcitatory-
inhibitory-excitatory pathway.
We sought to leverage this targeted activation mech-
∗ [email protected]
† [email protected] anism by cascading this connection motif using leaky-
‡ Current Address: AllenInstituteforBrainScience,Seattle integrate-and-fire (LIF) networks (see Materials and
§ CurrentAddress: InstituteforData,Systems,andSociety,Mas- Methods). The result is a circuit with uniform connec-
sachusettsInstituteofTechnology tivity among excitatory neurons with no preferred direc-
2
a b c
strong weakened on ght
inhibition inhibition eur Wei
LTD ... c N c
ynapti napti
s y
CA3 CA2 CA3 CA2 ost S
P
Layer i-1 Layer i Layer i+1
Presynaptic Neuron
Figure1. Crosscoupledfeedforwardnetworks. (a)SchematicofthefindingofRef.[29]. Inhibitorylongtermdepresssion
(LTD) lowers the feedforward inhibition of CA2, allowing information transfer from CA3 to CA2. (b) Schematic of proposed
network architecture. Note that only connections that are not identically distributed in the rest of the network are displayed
for visual clarity. The feedback between excitatory and inhibitory neurons drive the feedforward activity, while connections
between alike neurons remain uniform (see text). (c) Synaptic weight matrix of an example network with five groups. Note
that the network contains connections between all neuron types. Panel b only emphasizes the dominant pathways altered in
the ccFFN architecture.
tion, whichisneverthelessabletopropagatefeedforward tioofcorrespondingsynapticweights,respectively),that
activityduetothespecificcrosscouplingbetweentheex- modulates the amount of feedforward structure (see Ma-
citatoryandinhibitoryneurons. Wetermthiscircuitrya terials and Methods). For simplicity, we kept all these
cross-coupledfeedforwardnetwork(ccFFN).Aschematic ratios equal. Note, however, that feedforward activity
canbefoundinFig.1bthatemphasizesthemainproper- canbeobservedbychangingonlytheweightsorthecon-
ties of the network only and does not include all connec- nectivity probabilities separately (for a related observa-
tions. For that, Fig. 1c is a weight matrix of a network tion,see[32]). Usingthissimplesetting,byvaryingQas
instantiationwith5groups(withthelastgroupconnect- our only parameter, we can alter the overall feedforward
ing to the first) which illustrates the full connectivity of structure. Note,Q=1correspondstothecasewherethe
ccFFNs. network is perfectly uniform. By increasing Q to values
The behavior of ccFFNs can be explained by the fol- largerthan1theleveloffeedforwardstructureincreases.
lowing rationale: (i) the excitatory neurons in each layer ThenetworksimulatedinFig.2aconsistsof5layersof
aremorestronglycoupledtothegroupofinhibitoryneu- neuronswithafeedforwardratioofQ=2.6. Toillustrate
rons in their own layer relative to other inhibitory neu- thatincreasingQindeedresultsinincreasedfeedforward
rons; (ii)anactivityincreaseofsuchanexcitatorygroup activity,wecalculatedthecross-covariance(Fig. 2b)and
thus triggers elevated activity in the corresponding in- thePearsoncorrelationcoefficient(Fig. 2c),betweenthe
hibitory neurons; (iii) this inhibitory group of neurons firingpatternsoftheneurons,averagedoverthedifferent
targets the subsequent layer of excitatory neurons more layers.
weakly relative to other excitatory neurons; (iv) the re- Fig. 2b shows the average cross-covariance functions
duced inhibition (relative) of the subsequent excitatory withinthesamelayerforthetwoconditionsofQ=1and
group leads to increased excitatory activity in the subse- Q=2.6, averaged over 10 realization of the network. To
quent layer, while the activity in the initial layer returns getasmoothestimate,weconvolvedthespike-trainofev-
to baseline; (v) by cascading this cross-coupling motif, ery neuron with a Gaussian signal of standard deviation
elevated activity of excitatory neuron groups propagates 5ms. For every neuron pair, the convolved signal (fi(t))
through the circuit. was then used to calculate the pairwise cross-covariance
φ =cov[f (t+τ),f (t)]:
A simulation of a network with such a ccFFN topol- ij i j
ogyisshowninFig.2a. Notethat,toeliminatetransient (cid:90)
effects from particular driving inputs we connected the φ (τ)≈ [f (t+τ)−µ(f )][f (t)−µ(f )]dt, (1)
ij i i j j
last layer with the first layer, thus establishing a circular
pathwaywithaself-sustainedforwardpropagationofac- whichweaveragedoverallneuronsinsidethesamelayer:
tivity. Importantly,inadditiontothepropagationofthe Here µ(·) denotes the mean of the signal. While there
excitatory activity, we observe that the inhibitory neu- is no apparent temporal structure in the networks with
rons’activityprogressesfromonegrouptothenext. This Q = 1, there is a clearly visible increased synchrony in
emphasizesthepivotalroleplayedbyinhibitoryunitsfor the networks with high feedforward ratio Q=2.6, as in-
the observed dynamics, which is clearly beyond simply dicated by the large peak at zero lag. Moreover peaks
balancing the network. appearing at a lag of ±δ corresponding to the repetition
For our simulations, to describe the statistical periodofthefiring,resultingfromthethecirculartopol-
strength of the aforementioned connectivity motifs, we ogy we imposed.
have defined a ‘forward activity’ parameter Q, which is To investigate the tendency for each layer to fire in
simply the ratio of the connection probabilities of the unison further, we computed the Pearson correlation co-
‘targeted’ vs. ‘non-targeted’ neuron groups (or the ra- efficient of the convolved spike-trains of all neuron pairs
3
Figure 2. Firing pattern characterization of cross coupled feedforward networks . (a) Example raster plot with 5
groupsthatshowthepropagationofactivitybetweenlayers. Observethatinhibitoryneuronsalsoshowfeedforwardpropagating
activityandthatthefinalgroupconnectsbacktothefirst(circulararrangement)andhencetheactivitypropagatesindefinitely
(b) Group-averaged cross-covariance for parameter ratio values Q=1 and Q=2.6. For Q=1 (no feedforward structure) the
firingisclearlynotsynchonousanddoesnotdisplayanypattern. Incontrast,withimposedcross-coupledfeedforwardstructure
(Q = 2.6), there is a peak indicating strong synchronous firing inside each layer. The second peak at time ±δ indicates the
periodically repeating firing pattern (see also (c)). (c) The average Pearson correlation coefficient within layers as a function
ofQ. Thelargerthefeedforwardratio,Q,thegreaterthecorrelationoffiringwithinlayers. Errorbarsarestandarddeviations.
(d)Averagecross-covarianceoftheneuralfiringpatternsinlayeriwithneuronsinallotherlayersj forQ=2.6. Observethat
the time lag of the peaks are arranged consecutively, illustrating the orderly feedforward progression between groups.
for varying levels of Q. We plot the average correlation andshouldnotbereducedtoasingletypeofcircuitry,we
coefficient within each group and layer in Fig. 2c. For present here a second network architecture which takes
each value of Q, 10 network realizations were simulated. inspirationfromthespecificityofinhibitoryneurons’con-
As can be seen, the larger the value of Q, the more syn- nectivity patterns [17, 18]. In disinhibition motifs, cer-
chrony there is within groups. This synchronous firing tain subtypes of interneurons inhibit other interneurons
of groups is not decoupled but propagates along layers which normally suppress connected excitatory neurons.
as can be seen in Fig. 2d. There we plot the average Such a disinhibition cascade can thus lead to an increase
cross-covariance of a layer relative to all other layers (c.f in firing rates in excitatory neurons which are normally
Fig. 2b). As the regular shifts in the cross-covariance in- suppressed. Interestingly, long range disinhibition has
dicate, there is indeed a clear consecutive progression of been reported to propagate network activity [33, 34],
activityfromonelayertothenext. Wecanthusconclude although we stress the architecture presented here does
thattheratioQdirectlyinfluencesthefeedforwardactiv- not impose spatial constraints.
ity propagation in ccFFNs, demonstrating that directed
Inspired by these experimental findings, we con-
information transmission is possible without an imposed
structed a network model in which disinhibitory motifs
excitatory-to-excitatory pathway in the network.
enable the feedforward propagation of spiking activity.
We hereafter call this architecture a disinhibitory feed-
forwardnetwork(dFFN).Adiagramofthewiringscheme
B. Disinhibitory feedforward networks for a dFFN is shown in Fig. 3a that highlights the key
features of the network and does not include all connec-
tions. For that, Fig. 3b shows a weight matrix instantia-
The aforementioned cross-coupling of excitatory and
tionwith5groups(withthelastgroupconnectingtothe
inhibitory neurons is not the only arrangement possi-
first) which illustrates the full connectivity of dFFNs.
ble to create targeted feedforward activity driven by in-
hibitory units. To illustrate that our finding is general The functionality of this circuit can be explained
4
schematically as follows: (i) each layer comprises a func- C. A stylized linear rate model for disinhibitory
tional group of excitatory and inhibitory neurons more feedforward networks
strongly connected to each other than to the rest of the
network; (ii) inhibitory neurons in one layer target pref- Inordertogainsomefurtherinsightintotheoperation
erentially the inhibitory neurons in the subsequent layer of the disinhibitory feedforward network, it is insightful
(disinhibition);(iii)thus,whentheactivityinthepreced- toconsiderastylizedlinearratemodel[32]. Letusdenote
inglayerincreases,theinhibitoryneurons’activityinthe thegroupofexcitatory(inhibitory)neuronsinlayer1by
next layer will decrease. (iv) This in turn allows the ex- E (I ), and the corresponding firing rates by r (r );
1 1 E1 I1
citatoryneuronsinthenextlayertoincreasetheirfiring; see Figure 4a for a schematic. Note that without loss
(v) after a short delay, the inhibitory neurons increase of generality we may assume that the firing rates are
in activity again, as they receive input from the excita- measuredrelativetosomebaselineactivity,andcanthus
tory neurons in their layer, which have elevated activity be positive or negative. We can then write down the
asaresultoftheirdisinhibition. Thiseventuallyreduces following coupled rate equations for layer 1 (equivalent
the total firing back to baseline in the group – however, equations for layer 2 not shown):
not without the activity moving to the next group via
disinhibition again; (vi) by cascading this motif, every drE1 =−β r −r , (2)
upstream layer is activated and the information propa- dt E1 E1 I1
gates. We implemented dFFNs with varying number of drI1 =−β r +r −r , (3)
layers confirming that the above circuitry results in di- dt I1 I1 E1 I2
rectedinformationpropagation(Fig.3a). Onceagainwe
where β denotes the effective self-coupling of the indi-
observe feedforward signal propagation for both the ex-
vidual groups, which is a compound of a constant decay
citatory and inhibitory neurons (Fig. 3c). As before, to
(leak) term and the coupling of the neurons within the
avoid boundary effects, we used a circular network lay-
same group. For simplicity we assume here that there
outinwhichthelastgroupconnectsbacktothefirstand
is no self-excitation term for the excitatory neurons, and
hence the activity keeps propagating.
hence the network has an excess of inhibition (not bal-
anced). The magnitude of the parameter β will control
how much dissipation there is for each neuron. If we set
β = 0.5, which is the minimal requirement for a stable
system, the dominant modes are associated with purely
Similar to the ccFFNs, the level of feedforward struc- imaginary eigenvalues and correspond to (phase-shifted)
ture was control by a parameter Q that determines the wave-functions.
ratio of connection probabilities and ratio of weights in If the network is now excited by an instantaneous in-
the network to realize a dFFN (see Materials and Meth- put, such as an input to E1 at t=10 as shown in Figure
ods). The displayed network in Fig. 3c corresponds to 4b, indeed a propagation of activity can occur along the
a 5-layer network with Q = 2.6. When comparing the networkasoutlinedabove. Notethatthegeneralfeatures
average cross-covariance function of such a network with ofthisfindingarenotanartifactofthechosenparameters
uniformly connected networks Q = 1, we can again see or model size, but can be generalized to arbitrarily sized
an increased synchrony and a propagation of activity re- networks. The forward propagating activity is indeed a
sulting from the dFFN architecture (Fig. 3d). This is consequenceofthedescribedarchitecture: theparticular
further demonstrated by the increasing Pearson correla- inhibitory connection profiles drive the dynamics.
tion coefficient of neurons within layers as a function of
Q (Fig. 3e). Finally, we plot again the average cross-
covariance between different layers in Fig. 3f. Although III. DISCUSSION
thesamebehaviorisqualitativelyseenofthesignalprop-
agating along the cascaded layers of the dFFN topol- We have demonstrated, via LIF network simulations,
ogy, this appears to be far less pronounced than in the that one can construct networks in which feedforward
ccFFN architecture. The reason for this effect can be activity is propagated even if connections between ex-
explained by inspecting the raster plot further (Fig. 3c). citatory neurons are kept completely random. This is
Note that, in the ordered rastergram, the lowest group achieved by endowing the inhibitory neurons with an
can fire again even though the cascade that emanated active role in the feedforward propagation, which has
from it previously has not reached the top group yet. not been considered so far in the literature. Two cir-
For instance, the firing of a new cascade at layer i and cuit layouts were proposed demonstrating that both in-
a previous cascade at layer i+3 can temporally over- hibitory an excitatory neurons can display feedforward
lap and this co-alignment results in the multiple peaks dynamics simultaneously. We remark that the ‘layers’
in the cross-covariance. Hence, the network propagates discussed within both of these network types should not
multiple signals concurrently, or stated differently, the be taken too literally. While within our wiring schemes
network’s architecture is able to process multiple signals certain subpopulations are targeted (statistically), there
simultaneously effecting the cross-covariance measure. is still a (biased) all-to-all connectivity probability
5
a b
on ht
... eur eig
N W
Postynaptic2 Synaptic2
Layer i-1 Layer i Layer i+1
c d Presynaptic2Neuron
2000 ×10-3
e 6
D c
on2I1500 arian 4
ur1000 v
e o
N C 2
-
500 s
s
o 0
r
C
0 0.1 0.2 0.3 0.4 -90 -45 0 45 90
e f
Time28sp Lag28msp
-3
×10
ation2Coef. 00.1.15 Covariance 246 LLLLaaaayyyLyeeeearrrry2222eiiiiWWWWr34212i
el 0.05 s-
r s
or o 0
C 0 Cr
1 1.4 1.8 2.2 2.6 -90 -45 0 45 90
Parameter2Ratio Lag28msp
Figure 3. Disinhibitory feedforward network. (a) Schematic of dFFN architecture, in which feedforward activity propa-
gates by the cascading of several disinhibitory structural motifs. For clarity, not all connections are shown in the schematic;
importantly,excitatorytoexcitatoryconnectionsarerandomlyconnected(seetext). (b)Synapticweightmatrixofanexample
networkwithfivegroups. Notethatthereareconnectionsbetweenanytypesofneurons,asdescribedintheMethodsection(a
only emphasizes the pathways altered in the dFFN architecture). (c) Example raster plot of a network with 5 layers showing
forward propagation (Q=2.6). The inhibitory neurons again display forward propagating activity. Note that the final group
connectsbacktothefirst(circulararrangement)andhencetheactivitypropagatesindefinitely. (d)Cross-covariancefunctions
forthecasesofQ=1andQ=2.6(seeFig.2). (e)TheaveragePearsoncorrelationcoefficientwithinlayersasafunctionofQ.
The larger the feedforward ratio, Q, the greater the correlation of firing within layers. Error bars are standard deviation. (f)
Average cross-covariance function between different layers (see also Fig. 2). Note that due concurrently propagating multiple
cascades, the crosscovaraince between different groups display multiple peaks (see text).
throughout. Circuits like discussed here might thus be cationsthatasimilarmechanismtodirectactivitymight
found within one cortical column, for instance. While be implemented in canonical cortical microcircuits [43].
long range projections in mammalian brains tend to be The traditional view of these ubiquitous circuits is that
excitatory,topologiesasdiscussedheremightalsobeim- inputs from layer 4 (L4) drive layers 2/3 (L2/3) that
plementedbylongrangeexcitatoryconnectionstargeting thenexciteslayer5(L5). However,asPlutaandcowork-
local inhibitory neurons (see Fig. 1a). Moreover, long- ers [43] have shown, the picture is likely to be more in-
range inhibitory projections have been reported [17, 34– tricate: in particular it appears that L4 first suppresses
42]. However, we emphasize that our models are not de- L5 while driving L2/3, and only afterwards L5 shows el-
pendent on projection lengths and thus are not limited evated activity [43] — a finding that shows parallels to
to a particular scenario by construction. our proposed ccFFN mechanism.
Thefirstcircuitwasinspiredbythehippocampalarchi- The second circuit was inspired by the disinhibitory
tecturerecentlyuncoveredbetweenCA3andCA2where role of interneuron subtypes [17, 18, 44] in addition to
the propagation of a signal between CA3 to CA2 pyra- thesubtypespecificplasticityrules[23]. Suchdiscoveries
midal cells is governed by the amount of inhibition CA2 and the advancement of connectomics in uncovering the
interneurons impose on their CA2 pyramidal cells [29]. diversity of neuronal cell types and their corresponding
Interestingly, there exist some further experimental indi- connectivityrulesinfluencedourproposedwiringscheme
6
inhibitory neurons. Note that although the constant in-
put term is supra-threshold, balanced inputs guaranteed
an average sub-threshold membrane potential [32, 45].
In the model, the network coupling is captured by
the sum in (4), which describes the input to neuron i
from all other neurons in the network. Here W denotes
ij
the weight of the connection from neuron j to neuron
i (W = 0 if there is no connection). After a presy-
ij
naptic spike of neuron j, the synaptic inputs gE/I(t) are
j
increased step-wise (gE/I → gE/I +1) instantaneously,
j j
Figure 4. Inhibitory connection profiles drive dynam- and then decay exponentially according to:
ics. (a) Network architecture of stylized linear model. See
text for dynamical equations. (b) Example simulation for
β =0.5 and a pulse of activity given to E at t=10 causing
1
a permanent oscillation in all nodes of the network.
dgE/I
τ j =−gE/I(t), (5)
E/I dt j
of dFFNs.
Overall, the recently discovered diversity of interneu-
with time constants τ = 3 ms for an excitatory in-
rons suggests that they play a much more vital role for E
teraction, and τ = 2 ms if the presynaptic neuron is
thedynamicsthansimplyactingasabalancingdevicefor I
inhibitory. For all networks described in the following,
the network. We believe that new experimental discov-
thetotalconnection-strengthperneuronwaskeptequiv-
eries call for a reassessment of the role of inhibitory neu-
alent to an unstructured, balanced network displaying
ronsinmodelsorneuronalcircuits,andwouldencourage
asynchronous activity, with p = p = p = 0.5,
scholars to assign them more active and functional roles. EI IE II
p = 0.2, w = w = −0.042, w = 0.0115, and
Here we have demonstrated how such a functional role EE EI II IE
w = 0.022. Here, p and w stand for the connection
could be shaped in feedforward networks, but possibili- EE
probabilityandconnectionweight,respectively. Thefirst
ties for such roles could clearly go far beyond.
subscript denotes the destination and the second super-
script denotes the origin of the synaptic connection, and
E,I stand for an excitatory or inhibitory neuron, respec-
IV. MATERIALS AND METHODS
tively.
Simulations were performed in MATLAB (2012b
or later) and code can be found at github.com/
CellAssembly/inhibitory-feedforward.
B. Cross-coupled feedforward networks (ccFFN)
A. Model of spiking neurons
To construct ccFFNs we kept excitatory-to-excitatory
To illustrate our ideas, we have used leaky-integrate-
and inhibitory-to-inhibitory connections uniform as out-
and-fire (LIF) networks, stylized models of neural net-
linedabove. Wedividethenetworkintolayersconsisting
works, which act like pulse-coupled oscillators. Us-
of both inhibitory and excitatory units and connected as
ing a time step of 0.1ms we numerically integrated the
outlined in Fig. 1b,c: Excitatory neurons in layer i are
non-dimensionalizedmembranepotentialofeachneuron,
statistically biased to target the inhibitory neurons in
which evolved according to:
their own layer with a weight ratio W = wi /wnot[i],
IE IE IE
dVi(t) = 1 (µ −V (t))+(cid:88)W gE/I(t), (4) comparedtotheinhibitoryneuronsintherestofthenet-
dt τ i i ij j work. Similarly, the inhibitory neurons within a layer i
m
j targettheexcitatoryneuronsinthenext layeri+1,more
withafiringthresholdof1andaresetpotentialof0. All weaklyaccordingtotheratioW =(wi+1/wnot[i+1])−1.
EI EI EI
networks comprised N = 2000 units, with an excitatory Inadditiontomodifyingtheweights,wecontroltheanal-
to inhibitory neuron ratio of 4 : 1 (1600 excitatory, 400 ogous ratio of connections probabilities R and R .
IE EI
inhibitory). The input terms µ were chosen uniformly Note that W ,R are defined with an inverse ratio,
i EI EI
intheinterval[1.1,1.2]forexcitatoryneurons,andinthe i.e., a higher ratio means a weaker targeting correspond-
interval [1,1.05] for inhibitory neurons. Membrane time ingtoastrongerfeedforwardstructure. Tomodulatethe
constants for excitatory and inhibitory neurons were set embeddedfeedforwardlevelinthenetworks,wecanthus
to τ =15 ms and τ =10 ms, respectively, and the re- vary the ratios W ,W ,R ,R , while keeping the
m m IE EI IE EI
fractoryperiodwasfixedat5msforbothexcitatoryand average weights and number of connections constant.
7
C. Networks driven by disinhibitory structure ACKNOWLEDGMENTS
(dFFN)
We are thankful for discussions with Christof Koch,
In dFFN networks, excitatory-to-excitatory connec-
Costas Anastassiou, and Mauricio Barahona and com-
tions remain again uniform. Every layer in this net-
ments from Jean-Charles Delvenne and Renaud Lam-
work is composed of groups of exctiatory and inhibitory
biotte. YNB wishes to thank the Allen Institute
units as shown in Fig. 3a,b. Similar to above, we can
founders, P. G. Allen and J. Allen, for their vision, en-
control the imposed feedforward level with the ratio pa-
couragement and support. Most of this work has been
rameters WIE = wIiE/wInEot[i], WEI = wEi I/wEnoIt[i], and performed while MTS was at the Universit´e catholique
W =wi+1/wnot[i+1] or the analogous ratios of connec- de Louvain. MTS acknowledges support from the ARC
II II II
tions probabilities R ,R ,R . Again, for simplicity and the Belgium network DYSCO (Dynamical Systems,
IE EI II
we set all six parameters equal to Q and vary them con- Control and Optimisation) and an F.S.R. fellowship of
currently. the Universit´e catholique de Louvain.
[1] T.Vogels,K.Rajan, andL.Abbott,AnnuRevNeurosci [26] J.Kremkow,L.U.Perrinet,G.S.Masson, andA.Aert-
Annual Review of Neuroscience, 28, 357 (2005). sen, J Comput Neurosci 28, 579 (2010).
[2] A.Kumar,S.Rotter, andA.Aertsen,NatRevNeurosci [27] A. Reyes, Nat Neurosci 6, 593 (2003).
11, 615 (2010). [28] K.Vincent,J.S.Tauskela, andJ.-P.Thivierge,Frontiers
[3] C. Mehring, U. Hehl, M. Kubo, M. Diesmann, and in Computational Neuroscience 6 (2012), 10.3389/fn-
A. Aertsen, Biol Cybern 88, 395 (2003), 6th Dynamic com.2012.00086.
Brain Forum, Breisach, Germany, Sep 11-14, 2001. [29] K. Nasrallah, R. A. Piskorowski, and V. Chevaleyre,
[4] A. Kumar, S. Rotter, and A. Aertsen, J Neurosci 28, eneuro 2(4), 1 (2015).
5268 (2008). [30] K. Kohara, M. Pignatelli, A. J. Rivest, H.-Y. Jung,
[5] M.vanRossum,G.Turrigiano, andS.Nelson,JNeurosci T. Kitamura, J. Suh, D. Frank, K. Kajikawa, N. Mise,
22, 1956 (2002). Y.Obata,I.R.Wickersham, andS.Tonegawa,NatNeu-
[6] T. Vogels and L. Abbott, J Neurosci 25, 10786 (2005). rosci 17, 269 (2014).
[7] M.Diesmann,M.Gewaltig, andA.Aertsen,Nature402, [31] M. Llorens-Martin, L. Blazquez-Llorca, R. Benavides-
529 (1999). Piccione,A.Rabano,F.Hernandez,J.Avila, andJ.De-
[8] H.CateauandT.Fukai,NeuralNetworks14,675(2001). Felipe, Frontiers in Neuroanatomy 8, (2014).
[9] W. Kistler and W. Gerstner, Neural Comput 14, 987 [32] M.T.Schaub,Y.N.Billeh,C.A.Anastassiou,C.Koch,
(2002). and M. Barahona, PLoS Computational Biology 11,
[10] V. Litvak, H. Sompolinsky, I. Segev, and M. Abeles, J e1004196 (2015).
Neurosci 23, 3006 (2003). [33] S.Melzer,M.Michael,A.Caputi,M.Eliava,E.C.Fuchs,
[11] A. Aertsen, M. Diesmann, and M. Gewaltig, Journal of M. A. Whittington, and H. Monyer, Science 335, 1506
Physiology-Paris 90, 243 (1996). (2012).
[12] M.Gewaltig,M.Diesmann, andA.Aertsen,NeuralNet- [34] A. Caputi, S. Melzer, M. Michael, and H. Monyer,
works 14, 657 (2001). Macrocircuits, Current Opinion in Neurobiology 23, 179
[13] T. P. Vogels and L. F. Abbott, Nat Neurosci 12, 483 (2013).
(2009). [35] G.Buzsaki,C.Geisler,D.Henze, andX.Wang,Trends
[14] A. Kepecs and G. Fishell, Nature 505, 318 (2014). in Neurosciences 27, 186 (2004).
[15] T. Klausberger and P. Somogyi, Science 321, 53 (2008). [36] A. Alonso and C. Kohler, Neuroscience Letters 31, 209
[16] J. S. Isaacson and M. Scanziani, Neuron 72, 231 (2011). (1982).
[17] L. Roux and G. Buzsaki, Neuropharmacology 88, 10 [37] A. T. Lee, D. Vogt, J. L. Rubenstein, and V. S. Sohal,
(2015). The Journal of Neuroscience 34, 11519 (2014).
[18] K. D. Harris and G. M. G. Shepherd, Nat Neurosci 18, [38] A. Alonso and C. Kohler, J. Comp. Neurol. 225, 327
170 (2015). (1984).
[19] Z. J. Huang, Neuron 83, 1284 (2014). [39] T. F. Freund and M. Antal, Nature 336, 170 (1988).
[20] H.Taniguchi,FrontiersinCellularNeuroscience8(2014), [40] R. Tomioka, K. Okamoto, T. Furuta, F. Fujiyama,
10.3389/fncel.2014.00008. T. Iwasato, Y. Yanagawa, K. Obata, T. Kaneko, and
[21] S. R. Olsen, D. S. Bortone, H. Adesnik, and N. Tamamaki, European Journal of Neuroscience 21,
M. Scanziani, Nature 483, 47 (2012). 1587 (2005).
[22] D.Bortone,S.Olsen, andM.Scanziani,Neuron82,474 [41] S. Jinno, T. Klausberger, L. F. Marton, Y. Dalezios,
(2014). J. D. B. Roberts, P. Fuentealba, E. A. Bushong,
[23] S.X.Chen,A.N.Kim,A.J.Peters, andT.Komiyama, D. Henze, G. Buzsa´ki, and P. Somogyi, The Journal
Nat Neurosci 18, 1109 (2015). of Neuroscience 27, 8790 (2007).
[24] Y. Aviel, C. Mehring, M. Abeles, and D. Horn, Neural [42] A. I. Gulyas, N. Hajos, I. Katona, and T. F. Freund,
Comput 15, 1321 (2003). European Journal of Neuroscience 17, 1861 (2003).
[25] T. Tetzlaff, M. Buschermohle, T. Geisel, and M. Dies- [43] S. Pluta, A. Naka, J. Veit, G. Telian, L. Yao, R. Hakim,
mann, Neurocomputing 52-4, 949 (2003). D. Taylor, and H. Adesnik, Nat Neurosci 18, 1631
8
(2015). [45] A. Litwin-Kumar and B. Doiron, Nat Neurosci 15, 1498
[44] M. Karnani, J. Jackson, I. Ayzenshtat, J. Tucciarone, (2012).
K.Manoocheri,W.Snider, andR.Yuste,Neuron90,86
(2016).