Associative pattern recognition through macro-molecular self-assembly

3.7 MB·

by  Weishun Zhong

Associative pattern recognition through macro-molecular self-assembly
Weishun Zhong1, David J. Schwab2, Arvind Murugan1
1 Department of Physics and the James Franck Institute, University of Chicago, Chicago, IL 60637
2 Department of Physics, Northwestern, Evanston, IL 60000
Weshowthatmacro-molecularself-assemblycanrecognizeandclassifyhigh-dimensionalpatterns
in the concentrations of N distinct molecular species. Similar to associative neural networks, the
recognition here leverages dynamical attractors to recognize and reconstruct partially corrupted
patterns. Traditional parameters of pattern recognition theory, such as sparsity, fidelity, and ca-
pacity are related to physical parameters, such as nucleation barriers, interaction range, and non-
equilibrium assembly forces. Notably, we find that self-assembly bears greater similarity to contin-
7 uousattractorneuralnetworks,suchasplacecellnetworksthatstorespatialmemories,ratherthan
1 discrete memory networks. This relationship suggests that features and trade-offs seen here are
0
nottiedtodetailsofself-assemblyorneuralnetworkmodelsbutareinsteadintrinsictoassociative
2
pattern recognition carried out through short-ranged interactions.

b
e
F Algorithms to recognize patterns in high-dimensional When programmed with examples of idealized patterns,
signals have made remarkable breakthroughs in the last such neural networks can then “associate” a new pre-
4
decade, identifyingobjectsincompleximagesandvoices sented pattern with one of the idealized patterns, even
2
innoisyaudio[1,2]. Thesealgorithmsarefundamentally if the presented pattern were corrupted or incomplete.

] different from earlier algorithms in that they often simu- Suchnetworkshavebeenextensivelystudiedinstatistical
n
late a strongly interacting many-body dynamical system physics, theoretical neuroscience and computer science
n
(i.e. aneuralnetwork)andexploititsemergentcomputa- [9–11]. They has been used for tasks such as handwrit-
-
s tional ability. Such emergent pattern recognition ability ing recognition, and has served as a conceptual starting
i
d [3]raisesthequestionofwhetherthedynamicsofnatural point for more complex models of pattern recognition.
. or engineered physical systems can directly show similar Weshowherethatpatternrecognitioninself-assembly
t
a behavior. Pattern recognition would be of use to living bears a strong similarity to continuous attractors mod-
m
organisms, which must often scan environmental signals els of neural networks [12–19]; these networks code for
- forpatternsthatmightbefuzzy,ill-defined,andcanonly a continuum of states in short-ranged interactions and
d
be learned through examples rather than through a set thus differ from the the original models of associative
n
ofdefinitions[4–6]. Similarly,fuzzyrecognitionbuiltinto memory that code point attractors in long-ranged inter-
o
c theinternalphysicaldynamicsofsyntheticmaterialslike actions. In fact, we show that self-assembly is mathe-
[ sensorswouldallowtoreducetherelianceonfragileelec- maticallycloselyrelatedtoassociativememoryin“place

tronic or neural systems for intelligent signal processing. cell” network models [15, 17, 18, 20] of the hippocam-
2
Such an engineered ‘smart’ material can exploit its dy- pus that store representations, or maps, of multiple spa-
v
9 namics to respond differently to, say, the chemical envi- tial environments through which an animal may want to
6 ronmentofChicagotopsoilthantoPeoriatopsoil, even move. We show that the storage of memories and their
7 if the individual chemical environments are themselves retrieval via pattern recognition in self-assembly are re-
1 complex and highly variable and the differences are hard lated to analogous processes in the place cell network.
0
to define. A particularly intriguing aspect of the connection be-
.
1 Here, we show that the self-assembly dynamics of tween self-assembly and neural networks is that it re-
0 N interacting molecular species can recognize patterns lies on an extended quasi-particle approximation for the
7
in those N concentration levels. In the right win- neural network. For N binary neurons, this approxima-
1
dow of physical parameters like temperature, binding tion reduces the configuration space from 2N to a low
:
v energy and chemical potentials, even corrupted or in- dimensional attractor manifold of size N, correspond-
Xi complete patterns in the concentrations of N molecular ing to the quasi-particle center of mass. Such a re-
speciesareclassifiedintooneofseveralpredefinedclasses duction to a collective coordinate and its role in com-
r
a through the assembly of designated structures. We use plex computational abilities has generated much interest
a model recently investigated in [7], where multiple self- [12, 13, 15, 17, 18, 20, 21].
assembling behaviors were programmed into a one set of Thus our mapping between self-assembly dynamics
self-assemblingDNAmolecules. Intuitivelyspeaking,the andneuralnetworksdynamicshelpsisolateexactlywhich
patternrecognitiondescribedhereexploitsthephysicsof aspectsofeachmodelareessentialtopatternrecognition
selective nucleation. In particular, we show that nucle- ability. We hope that such an abstracted understand-
ation is sensitive to how closely correlations in concen- ing of requirements will stimulate further work on find-
trationspacereflectthephysicalproximityofthosesame ingphysicalsystems[7,22–28]whosedynamicscanshow
species in self-assembled structures. the kind of learning behavior that has been successfully
Therobustpatternrecognitionwefindhereisreminis- demonstrated over the last decade in machine learning
centof“associativememory”[3,8,9]inneuralnetworks. research.
2
50
14 212 ... ... 109
813 1435 ... ... 2343
m = 0 ... ... ... ... ... =
1 5
... ... ... ... ...
2 1932 ... ... 1817
50
734 1781 ... ... 35
1932 636 ... ... 1201
m = 0 ... ... ... ... ... =
2 5
... ... ... ... ...
2012 17 ... ... 1905
50
943 212 ... ... 902
38 1031 ... ... 636 FIG. 2. A concentration pattern (cid:126)c has high overlap χ with
m3 = 50 ... ... ... ... ... = mcoenmtiogruyoums iniftthheeasrpraecnigeesmweintthmh.igThhceonpcaetntetrranti(cid:126)conshionwn(cid:126)c haares
... ... ... ... ... significant overlap χ≈1 with the Leo memory but low over-
lapwiththeEinsteinandcatmemories. Herewevisualizethe
942 125 ... ... 1153 overlapχforeachmemoryusingablurringmaskdescribedin
the appendix. Pixels are assigned their true gray scale value
if the average concentration of species in their neighborhood
FIG.1. Astoredmemoryconsistsofa2-dimspatialarrange-
of area a∗ = 100 is high and a random value if the average
ment of N molecular species. Here, we show three distinct
concentrationislow. Thusaclearregion,likeLeo’sface,rep-
memories made of the same N = 2500 species arranged as
resents high overlap χ ≈ 1 and high average concentration.
50×50grids. Eachspeciesoccursexactlyonceineachmem-
The high concentration species are scattered throughout the
ory. Wewillchoosetorepresentthesememoriesbygrayscale
Einsteinorcatmemories,resultinginlowaverageconcentra-
images by associating each species with a unique gray scale
tions, low overlap and thus a blurry image.
value. Hence different memories correspond to different gray
scaleimagesinwhicheachgrayscalevalueisrepresentedex-
actly once. Thus the three images here are permutations of
the set of concentration patterns that must be classified
the same N = 2500 pixels. (Note: Many computer systems
can only display 256 distinct grayscale values.) as being of type mα.
• A concentration pattern is defined as a concen-
tration vector(cid:126)c in which some number s of species have
a high concentration c and the other N −s species
I. PATTERN RECOGNITION IN high
are at c . See Fig.2. For simplicity, we will restrict to
SELF-ASSEMBLY low
the stored patterns to be made of binary concentrations
c or c while describing our work.
We assume we have N distinct species of macro- high low
• Overlap between pattern and memory The
molecules. We begin with a few definitions:
overlapχ isameasureofwhetherthespecieswithhigh
aα
• A memory is defined as a 2-dimensional arrange-
concentrationsinpattern(cid:126)c arephysicallyproximate(or
a
ment of these N species as a grid. Fig.1 shows three ex-
contiguous) in the memory m . We define χ (x) to
α a,α
amplesofsuchmemories-referredtothe‘Leo’,‘Einstein’
be the fraction of species in a square region of area q
and ‘cat’ memories, made of N =50×50=2500 species
centered at a position x in memory m that have high
α
of molecules. As we will see below, memories play two
concentrations in (cid:126)c . Thus, if χ (x) = 1, all species
a a,α
roles in pattern recognition; (1) memories represent the
in a region around x in m have high concentrations in
α
physical2-dimensionalstructurethatisself-assembledin
pattern(cid:126)c . We define the overlap χ ,
a a,α
response to a concentration pattern. Thus memories m
α
representthediscreteclassesthatenvironmentalconcen- χ ≡max χ (x) (1)
a,α x a,α
trationpatternsareclassifiedinto. Agoodpatternrecog-
nizer should produce only one self-assembled memory in To gain intuition, in Fig.2, we show the local over-
responsetoanexternalconcentrationpattern;(2)thear- lap χ (x) visually using a blurring mask, described in
a,α
rangementoftheN speciesineachmemorym specifies the Appendix; a blurry region indicates low overlap and
α
3
FIG. 3. Classification of patterns through self-assembly. Each row in (a) represents one concentration pattern shown in three
representations as in Fig.2. The corresponding row in (b) shows the resulting self-assembly outcome. Concentration patterns
(cid:126)c ,(cid:126)c with overlap with Leo or the cat respectively more than other memories assemble only that memory almost exclusively
1 2
(E = 3kT,µ = 1.7,µ = 1.9). Patterns(cid:126)c and(cid:126)c have overlap with Leo and the cat are not as significantly different.
0 high low 3 4
When assembly is carried out with µ = 1.5 as in (cid:126)c , a chimeric mosaic of Leo and the cat is assembled, resulting in a
high 3
failuretoclassify(row3). However,in(cid:126)c ,thehighconcentrationisreducedtoµ =1.7;nucleationslowsdownforboththe
4 high
Leo and the cat but more so for the cat and only the Leo memory is evoked. Pattern(cid:126)c has an overlap of χ=0.8 with both
5
Einstein and Leo; we find a mosaic of Leo and Einstein and classification fails. Finally pattern (cid:126)c has no significant overlap
6
with any memory, giving no nucleation on the time scale of our simulations. (Note: One typical self-assembly outcome from
Gillespie simulations combined digitally with image of monomers. For time stamps of the assemblies shown, see Appendix.)
hence lower concentrations than the average while an in- entirely blurry.
focusregionindicatesaregionofhighconcentration. For Note that the overlap defined here serves as a general-
example,thespecieswithhighconcentrationsinthepat- izationoftheusualdotproducttothecaseofcontinuous
tern in Fig.2a are all localized to a part of Leo’s face in attractors [9, 17]; it is computed by restricting to local
the 2-dim arrangement corresponding to Leo’s memory. regions of size q of the memory. See the Appendix for a
For x in that part of Leo’s face, χa,α(x) = 1 and thus discussion of the appropriate choice of area q in comput-
χa,α = 1; we say the pattern (cid:126)ca matches Leo’s memory ing the overlap; we use q =8×8 in Fig.2.
mα perfectly. On the other hand, these concentrations • A concentration pattern(cid:126)ca is said to evoke a mem-
arewidelydistributedoverthecatmemory, asseenfrom orym if, whenspeciesaresuppliedatconcentration(cid:126)c ,
α a
the bar charts in Fig.2. Consequently, the cat image is a structure with molecules arranged as in memory m is
α
4
self-assembled. energy barrier to nucleation. We find [7],
(cid:18) (cid:19)
E
Γ(χ )=exp − 0
aα 2−(cid:104)µ(cid:105) /E
m 0
Storing memories
(cid:18) (cid:19)
E
=exp − 0 (4)
f +(δµ)χ
To create self-assembly dynamics that can recognize aα
concentration patterns, we first ‘program’ the given set where δµ ≡ (µ −µ )/E is a measure of the dy-
low high 0
ofidealizedmemories,suchasthoseinFig.1,intothein- namic range of concentrations and f ≡2−µ /E sets
low 0
teractions of N species. Inspired by associative memory the growth rate of nucleated structures. Consequently, a
found in neural networks [3, 9], we will take the binding pattern (cid:126)c will result in assembly of memories m and
a β
affinities between the N molecular species to reflect spa- m in the ratio Γ(χaβ). Assuming(cid:126)c was to be classified
tialrelationshipinthememories;thatis,weassumethat α Γ(χaα) a
as m because χ > χ for all stored memories ι, we
two molecular species have high affinity for each other if α aα aι
define the error rate in classification as,
they occur next to each other in one of the memories
(cid:80)
Γ(χ )
(cid:40)−E , if i,j are neighbors in any memory η = ι(cid:54)=α aι (5)
Jij = 0 (2) Γ(χaα)
0, otherwise
where the sum is over all stored memories except α.
Note that a given species i will generally have distinct With this definition of error rate η, we see that clas-
neighbors in different memories; hence the above pre- sification fundamentally relies on the nucleation rate Γ
scriptioncreatespromiscuousinteractions. Suchamodel showing large variation as a function of the overlap χ.
ofpromiscuousinteractionswasstudiedin[7]asamodel
of self-assembly with multiple target structures. While
analogous to self-assembly polymorphs, we are working Response to patterns
in the limit of many distinct species. Consequently [7]
found that the number of different polymorphs that can What is the response of a soup of promiscuously in-
be programmed in has a sharp limit, analogous to the teracting particles (defined by Eqn. 2) to different con-
sharplimitonthenumberofpatternsthatcanbestored centration patterns (cid:126)c? How frequently are patterns in-
in large associative neural networks [3]. Here, we will be correctly classified? How do the error rate η and time to
interested in how such a promiscuous soup of molecules classify Γ depend on parameters of the theory?
responds to various concentration patterns. To answer these questions, we simulated self-assembly
for a set of illustrative cases of concentration patterns
showninFig. 3. Ineachcase, wetookasetofN species
with interactions as defined by Eqn. 2 and set the con-
Recognition through nucleation
centrations to be a chosen pattern (cid:126)c. We assume for
simplicity that concentrations are held fixed during self-
To understand how the promiscuous soup defined in
assembly,asinagrandcanonicalensemble. Wesimulate
Eqn.2 can classify concentration patterns, we need to
self-assemblyusingtheGillespiealgorithmdetailedinthe
understand the selective nucleation of different memo-
Appendix and report on the mix of assembled structures
ries by a concentration pattern. We find that a pattern
produced.
(cid:126)c is much more likely to nucleate a memory m with
a α • Pattern (cid:126)c in Fig. 3 was taken to perfectly match
1
high overlap χ than a memory m with low overlap
aα β a part of the Leo memory (χ = 1.0). That is, the con-
χ <χ .
aβ aα centrations of s=364 species that form a clump around
To see this quantitatively, it is convenient to work Leo’s left eye were taken to be c and the rest c .
high low
with chemical potentials µ = −logc of species instead The high concentration species were randomly spread
of concentrations c. The average chemical potential of out and not contiguous in the other memories. We ob-
all species in a square region of area q at position x for served rapid nucleation and subsequent assembly of the
memory mα due to pattern(cid:126)ca is given by, Leo memory exclusively.
• Pattern (cid:126)c in Fig. 3 imperfectly matches the cat
2
(cid:104)µ(cid:105)(x)=µlow+χaα(x)(µhigh−µlow) (3) memory (χ = 0.7) and poorly matches the others. Such
an imperfect pattern is nevertheless correctly classified
Since χ = max χ (x) and ∆µ ≡ µ −µ < 0, by self-assembly since the Leo and Einstein overlaps are
aα x aα high low
the region of the memory with lowest average chemical much lower (χ = 0.28,0.30). However, the nucleation
potential (and thus greatest propensity to nucleate) has rate Γ is lower than for(cid:126)c since χ=0.7 has significantly
1
chemicalpotential(cid:104)µ(cid:105) =µ +χ (µ −µ ). The lower average concentration in the nucleation region rel-
m low aα high low
rate Γ of nucleating a seed in such a region can be calcu- ative to χ = 1.0. That is, the time to classify has in-
lated from classical nucleation theory by computing the creased.
5
• Patterns (cid:126)c and (cid:126)c , which are structurally identi- increases. Thus,wecantradeofflowerclassificationtime
3 4
cal, have higher overlap χ =0.9 with Leo than with the 1/Γ for higher classification error η and vice-versa.
r
cat χ = 0.69. But if we speed up nucleation by setting To understand this trade-off, in Fig 4 we made a scat-
µ = 1.5 for (cid:126)c , we find a high error rate. Many cat terplotofclassificationtime1/Γ versustheerrorrateη
high 3 r
memoriesarenucleatedatsitesontheboundaryofgrow- astheparameters,E andµ,arevaried. Wefindafunda-
0
ing Leo memories (‘heterogeneous nucleation’). These mentalerror-timeperformancebound;givenafixedtime,
errors can be eliminated almost entirely by increasing theerrorrateisboundedfrombelow. Infact,wecande-
µ = 1.7 for (cid:126)c . Increasing µ lowers nucleation rive an analytic formula for this bound from Eqns.4 and
high 4 high
rates for both Leo and the cat but the relative rate in- 5 by extremizing over parameters; we find that
creases. Thuswetradelongerclassificationtimeforlower
∆χ
error rate. This speed-accuracy tradeoff is discussed in η ≥ Γrχw (7)
more detail below.
• Pattern (cid:126)c has near identical overlap χ = 0.8 with where ∆χ = χ −χ and χ is the largest overlap (i.e.,
5 r w r
both the Leo and Einstein memories. Such an ambigu- overlapwiththe‘right’memory)andχ isthenexthigh-
w
ous pattern can exist only because the Einstein and Leo estoverlap(i.e.,overlapwiththedominant‘wrong’mem-
memories themselves were designed resemble each other ory).
in a region near Einstein’s shirt and Leo’s face. Pattern Thus, the minimum achievable error rate always falls
(cid:126)c has high concentrations in such a region, and conse- off if the classification time is increased (i.e., nucleation
5
quently,pattern(cid:126)c evokesamixtureofLeoandEinstein. rate Γ is decreased). But the rate at which it drops off
5 r
• Finally, pattern (cid:126)c involved s high concentration ismuchslowerforapatternthathassimilaroverlapwith
6
species that were not significantly grouped together in different memories (i.e., with small ∆χ). Thus patterns
any memory. Such a pattern has low overlap with all withcloseroverlapnecessarilytakelongertodistinguish.
memoriesanddoesnotevoke(i.e. nucleate)anymemory
within the duration of the simulation.
Pattern sparsity sets dynamic range of concentrations
Error rates can be lowered through slower nucleation The requirement that a random pattern with s high
concentration species not evoke a random memory im-
Asmentionedwhendiscussingtheresponsetopatterns poses a constraint on the dynamic range of concentra-
(cid:126)c3 and (cid:126)c4, nucleation dynamics can be accelerated by tions chigh and clow.
lowering µ . However, lowering µ beyond a point To see this, note that random patterns and memo-
high high
becomes counter-productive in terms of error rate; e.g., ries will generically have overlap χ ∼ s/N. Due to the
if µ ≈µ , all patterns(cid:126)c are indistinguishable from underlying discrete nature of our structures, the nucle-
high low
uniformconcentrations. Iftimetonucleatethestructure ation rate Γ effectively reaches its highest value when
were no concern, what is the optimal value of µhigh that f +(δµ)χ ∼ 0.5 (or equivalently (cid:104)µ(cid:105)m ∼ 1.5E0); at this
would give the smallest error rate? From Eqn. 4 and point, the critical nucleation seed has area ∼ 2×2 and
5, we find that to the minimize the error rate in distin- thus the nucleation barrier effectively disappears. Com-
guishing two patterns with overlaps χ (with the ‘right’ biningthisinequalitywiththerequirementthattheseed
r
memory) and χw (with the ‘wrong’ memory), we should grows, fgrowth >0, we find that
choose
µ /E >2−N/2s.
high 0
(cid:18) (cid:19)
1
µlow =µhigh 1+ √χrχw (6) bTohuunsd, tohnecsphiagrhs.ity 1− Ns of the pattern(cid:126)ca sets an upper
Trade-off: Time vs error vs overlap difference Capacity: Maximal number of patterns
We also saw that while (cid:126)c and (cid:126)c were the same pat- Finally, we have only considered three patterns in our
3 4
tern structurally, the rapid dynamics in response to (cid:126)c simple example and found that the self-assembly prop-
3
resulted in a proliferation of errors. This illustrates a erly forms the intended structure in the right regimes of
fundamental trade-off between error rate and classifica- χ,µ,E .
0
tion time. We can attempt to speed up classification However, as shown in [7], once the number of pro-
(i.e.,increasethenucleationrateΓ ofthe‘right’memory grammedmemoriesexceedsathreshold,m ,thecapacity
r c
with highest overlap) by increasing the high concentra- of the system, promiscuous interactions lead to a combi-
tion (i.e., lowering µ ) or lowering the binding energy natorial explosion of chimeric structures. The capacity
high
E (or equivalently raising the temperature). However, m was found to scale with the size of the memories as
0 c
this causes the nucleation rate to increase for all memo- N1−2/z where z is the coordination number of the struc-
rieswithanyoverlapχand,infact,theerrorrateη then ture. Even a pattern with perfect overlap χ = 1 with,
6
Structure 1 Self-assembly Structure 2
interactions
5 3 4 2 3 2
1
1 3 5 4
2 1 4
6
6 5 6
place field map
3 2 3
5 4 2 3
1 1 3 5 4
2 1 4
6 6 5 6
Place cell network
Environment 1 Environment 2
FIG. 5. Programming self-assembling particles with multi-
FIG. 4. Trade-off between classification time and classifica- pletargetstructuresisanalogoustoprogrammingaplacecell
tionerrorbasedonnucleationrates(Eqns.4,5). Weevaluated neuralnetworkwithmultiplespatialenvironments. Bothsys-
classificationtimeΓ anderrorη forpattern(cid:126)c ofFig.3with tems are programmed with internal representations (by tun-
r 3
overlapχ=0.9withLeoandχ=0.69withthecatformany ing particle interactions or neuronal connections) of external
valuesofbindingenergyE andchemicalpotentialµ . We spatial relationships (between particles or place fields). The
0 high
findanabsolutelowerboundonerrorforanyfixedclassifica- self-assembling particles can then coherently grow any select
∆χ/χcat storedstructurefromaseed;theneuralnetworkcan‘mentally
tion time (red line) given by η = Γ (See Eqn.7 for a
r explore’ [20] an environment selected by initial conditions by
derivation). If E is constrained to be a fixed value as would
0 traversing through it.
be common experimentally [29, 30], the trade-off curves pa-
rameterized by µ are shown by lines of different colors.
high
The black dot shows the minimal error for a fixed E if time
0
were immaterial. here,thesespuriouspatternsareainevitableconsequence
ofusingshortrangedinteractionstoprogramcontinuous
attractors instead of point attractors. We discuss this
say Leo, cannot faithfully evoke the Leo memory near or
distinction further in the conclusions.
past this capacity transition; chimeric combinations of
different stored memories will be assembled in response.
II. RELATIONSHIP BETWEEN
Inverse design: memories from patterns SELF-ASSEMBLY AND PLACE CELL NEURAL
NETWORKS
In the above discussion, we started with the notion of
memories–a2dspatialarrangementofN species–which We now show that the self-assembly model is able to
definedawholesetofconcentrationpatternsthatmatch recognizeconcentrationpatternsbecauseofaremarkable
it. Instead,ifoneisgivenasetofconcentrationpatterns similarity to place cell neural networks that store spa-
(cid:126)c , how can one design memories and thus interactions tial maps of environments. Relying on a simple model
a
J suchthateachpatternevokesauniquememorystruc- of place cells based on recent experiments, we show the
ij
ture? connection at two levels; (1) At the level of structure
We can define a memory m for each pattern by tak- of the interactions, both models create an internal rep-
α
ing s of the species that are at high concentrations and resentation of external spatial relationships. (2) At the
designastructurethatputsthosesspeciestogetherasa level of dynamics, pattern recognition in self-assembly is
contiguousregioninarandompermutationinonepartof related to a collective coordinate approximation of place
the structure. Species with low concentrations could be cell dynamics.
placed in a random arrangement surrounding this seed.
With such a prescription, each given concentration pat-
tern (cid:126)c would reliably evoke memory m since the odds
a i
ofthesameseedoccurringinanothermemorym would A. Structure: Internal representation of external
β
spatial relationships
be negligible below the capacity threshold.
However, note that memory m will necessarily be
α
evoked by patterns unrelated to (cid:126)c since patterns local- As shown in Fig 5, in self-assembly, the affinities be-
a
ized in other regions of m will also evoke it. These tween different species code for the spatial proximity of
α
other unavoidable patterns associated with m are rem- species in different target structures. In particular, for
α
iniscent of spurious patterns in neural network models; each of the α = 1,...,m target structures, we assume
7
that the interaction matrix between species is, species in the different stored structures. In the hip-
pocampus, the synaptic connectivity Jtot codes for the
(cid:40) ij
1, if |f (i)−f (j)|<d spatial relationships of place fields in the different stored
Jα = α α (8)
ij 0, otherwise environments. Infact,[35]showedthatthegeometryand
topology of stored environments can be mathematically
where f (i) is the spatial location of species i in struc- reconstructed from the pooled matrix Jtot.
α ij
ture α and d is the interaction range. We take the total
interaction matrix to be,
(cid:88) B. Dynamics : Inhibition and attractors
Jtot = Jα
ab ab
α
Despitethesimilarityofinteractionmatrices,thestate
Recent neuroscience experiments reveal a similar pic- space of these systems is in principle very different. The
ture[31–33]forhowmammalsrepresentmemoriesofspa- energy of a self-assembled structure depends on the spa-
tial environments in the hippocampus. When a rodent tial ordering of different species that occur in the struc-
hasbeenexposedtoaspatialenvironment(e.g.,aroom), ture; on the other hand, a state of the neural network is
placecellsinthehippocampusareeachassigneda‘place determined by which set of neurons is firing and there is
field’, i.e., a small spatial region of that environment. no corresponding ordering information. How could the
A place cell fires only when the rodent is in that cell’s states and dynamics map on to each other?
‘place field’ region of the environment. As the rodent
Collective coordinate of the neural network
moves through the environment, place cells with nearby
Long-range global inhibition found in the hippocampus
place fields fire concurrently when the rodent is in the
limits the total number of neurons that can be active
overlap region of two place fields. Thus place cells with
at once. Earlier theoretical work [15, 17, 18, 20, 38]
nearby place fields might strengthen their synaptic con-
showedthatthebalancebetweenshort-rangedexcitation
nections in a Hebbian fashion (cells that ’fire together,
and long-range inhibition causes neural activity to con-
wire together’) while connections between cells with dis-
denseintoacontiguous‘clump’ofsomefixedsizel,i.e.,l
tant place fields are not facilitated. (‘Place cells’ may
neuronswithplacefieldsinoneregionofspacefirewhile
arise as effective modes of ’grid cells’ in the entorhinal
all others are off. Such a stable clump — a collective co-
cortex but we will use the effective place cell description
ordinate for neural activity — corresponds to a specific
here.)
rodent position in the physical environment. This con-
Thus after being exposed to one environment α, place
straint is analogous to working at fixed magnetization in
cells in the hippocampus will have connectivity,
a ferromagnetic Ising model; the firing neurons condense
(cid:40) into a clump of activity to minimize its surface area.
1, if |f (i)−f (j)|<d
Jα = α α (9) Theclumpcanbedriventomovearoundthenetwork,
ij 0, otherwise
whichisinterpretedasmentalexplorationofaspatialen-
vironment. SeeFig. 7. Thedescriptionofneuralactivity
where now f (i) is the location of the center of neuron
α in terms of such a collective coordinate, e.g., the center
i’s place field in environment α and d sets the range of
of mass of the clump, simplifies the problem, reducing
interactionsbetweenplacecells. Thus, Eqn.(9)saysthat the configuration space from the 2N possible states of N
place cells whose place fields in environment α overlap
neurons to N possible center of mass states of the clump
are wired together.
along the continuous attractor [15].
Exposing the rodent to a different environment [34],
Simple model of two memories in 1-dimension
say β, creates another seemingly random assignment of
Whileourfocusintheprevioussectionwasnucleation,
placefieldsforthesameplacecellsthatappeartobearno
here we study the similarities in growth dynamics of
relationship to place field arrangements in environment
self-assembly to clump motion in attractor neural net-
α Exposure to environment β will create a new set of
works. To gain intuition, we contrast the dynamics of
neuronal connections between place cells, in addition to
self assembly and place cells when they both encode the
those created by environment α. We assume [17, 18,
samepairof1-dimensionalstructuresorenvironments.As
20] that the connections between place cell neurons after
shown in Fig.6, the first memory in both cases is set to
being exposed to m environments are simply a sum of
1−2−...−N without loss of generality. The second
each individual environment’s contribution, i.e.
memory is a random permutation but with a region of
(cid:88) somesizethatisincommonwithmemory1. Weassume
Jtot = Jα (10)
ij ij that interactions extend over q = 4 nearest neighbors
α
(range q sets parameter d in Eqns.8 and 9).
In both constructions, Eqn.8 and Eqn.9, the internal That is, in both cases, the total interaction matrix
interaction matrix reflects external spatial relationships. is the sum of interaction matrices corresponding to two
In self-assembly, the affinity matrix Jtot between chem- memories (i.e., two structures made of the same species
ij
ical species codes for the spatial relationships of those or two spatial environments covered by the same place
8
FIG. 6. Self-assembly and neural networks programmed with two memories (red and blue) with high resemblance in a region
leads to similar failure to classify due to chimeric structures or mental teleportation. (a) A fragment of the programmed blue
structure,growingthroughself-assemblycantransitiontogrowingastheredstructurewhengrowingthrougharegionofhigh
resemblance shown in (c). The result is a half blue-half red chimeric structure. (b) If neural activity is initialized as a clump
at one end of the blue environment, the rat should be able to mentally explore the blue environment by driving the clump
from left to right in environment 1 (e.g., path planning [36, 37]). However, the clump can transition to being a clump in
environment2(red)whenpassingthrougharegionofhighresemblance(stateC).(d)Theprobabilityofchimerictransitionsin
self-assemblyandteleportationinneuralnetworksgrowswiththeforcef usedforretrievalandwithlocalresemblancebetween
stored memories (circles - self-assembly, crosses - neural networks, solid - barrier crossing formula Eqn.11)
cells): neouslydestroytheclumpinoneenvironmentandreform
the clump in the other environment? Similar transitions
Jtot =J(1)+J(2) have been seen experimentally in rats [39] and in simu-
ab ab ab
lationswithoutforces[16,17]. Detailsofoursimulations
The central question is whether dynamics based on
including a force are in the Appendix.
Jtot caneffectivelyshowbehaviorscorrespondingtoonly
ab We find that the probability of a teleportation event,
one of the pooled matrices, say J(1), chosen by initial shownascrossesinFig.6disdeterminedbyplacefieldre-
ab
conditions. semblanceintheoverlappingregionaswellasthedriving
Forself-assembly,weaskifwecanselectivelygrowone force,incloseagreementwiththeself-assemblyresultsfor
of the stored structures (blue) from a seed belonging to the same pair of 1-d permutations (circles in Fig. 6d).
thatstructureusingasupplyoffreecomponentsasshown We are able to accurately capture such
in Fig. 6. The two structures programmed in possess chimeric/teleportation behavior of both systems by
a region of overlap with strong similarity in a region of modeling chimeric/teleportation transitions as a barrier
lengthl=4. Atfirst,theself-assemblyprocesswillbegin crossing process under a driving force. For the case of
to grow the first structure. But if the common region is near-perfect resemblance (black circles and crosses in
comparable to the range of interactions (q = 4), growth Fig. 6(d)), we find
may switch and continue growing as the red structure,
therebyproducingablue-redchimera. Wecomputedthe p ef +eB
trans ∼ (11)
probability of a chimera for different degrees of resem- 1−p ef +1
trans
blance and as a function of the driving force completing
the reaction. See circles in Fig. 6. (Details of the 1d where B is the energy barrier along the chimeric transi-
self-assembly model are provided in the appendix.) tion(i.e.,segmentCGinFig.6),ingoodagreementwith
For place cell networks, we ask whether the rodent’s simulationsofbothneuralnetworksandself-assembly. B
neuralactivityclumpcantraverse(i.e,‘mentallyexplore’ is the energy barrier along the chimeric transition (i.e.,
[20])oneenvironment(say,blueinFig.6b),fromoneend segment CG in Fig. 6) due to imperfect resemblance; for
to the other, without being affected by the other. Or the black curve, we used a resemblance with only one
might a ‘mental teleportation’ event in a region of high mismatch at the end of the common region in Fig.6(c)
resemblance of place field arrangements (Fig.6c) sponta- and thus B = E (one missing bond). B = 0 if the
0
9
resemblance is perfect over a region of size q. (a) a interaction
Thisexampleoftwomemoriesindicatesthattheclump range q=4
... ...
of neural activity is analogous to the growing tip of the
2 3 4 5 6 7 2 3 4 5 6 7 a
self-assembling structure. We explore this relationship
growing tip growing tip
quantitatively below.
Fluctuationspectrumofgrowingstructuresand
clump of firing
the neural clump
neurons
The place cell network model and self-assembly bear 1 2 3 4 5 6 7 8 9 10 ... a ...
strong similarity because transformations of the growing
tip of length ∼ q of a structure can be mapped to low
energy transformations of an activity clump of size l∼q 1 2 3 4 5 6 7 8 9 10 ... a ...
with the same composition as the growing tip. We find
that the mapping works to the extent that the collective (b)
coordinate approximation is appropriate. 9
As shown in Fig. 7, all the transformations of self- 8
assembly map onto transformations of the clump in a
7
one-to-one manner, and the energies match up as well.
6
For example, the free energy change in adding a species
5
a to a growing tip is equal to the free energy change in
4
turning on neuron a and turning off the weakest neuron
3
in the clump; both transformations have a free energy
2
change ∆F = −f +wE where w is the number of zero
0
1
energybondsmadebythenewmolecule(neuron)awith
the growing structure (moving clump) and f is the driv- 0 0 0..5 1 1.5
ing force on the clump / on assembly (see Appendix).
However, the neural activity has many other transfor-
mations that make it discontiguous and thus do not cor- FIG. 7. (a) All transformations of the growing tip of a self-
respond to addition of molecules in self-assembly. But, assembling structure and the low energy transformations of
by definition, these fluctuations break the collective co- the clump of place cell activity can be put in a 1−1 corre-
spondence. Theclumpalsohashigherenergytransformations
ordinate approximation and they are generally of higher
which do not correspond to any transformation of the grow-
energy. As a result, such energetically expensive trans-
ing tip of structures. (b) We computed the histogram of free
formations(whichleavetheattractormanifold)arelikely
energy changes along trajectories of length k = 6, starting
tobefollowedbytransformationswhichsubsequentlyre-
at the same state, for both systems (place cells in blue, self-
storecontiguitytotheclump(andthusreturnthesystem
assembly in red). The low energy trajectories of the clump
to the attractor manifold). are matched by trajectories of self-assembly. (Driving force
To quantify such corrections to the collective coordi- f =0.05 in these simulations.)
nateapproximation,wenumericallycomputethefreeen-
ergy change associated with all possible sequences of k
molecule additions and k neuron flips – i.e., we found limit of self-assembly [40–45] describes macromolecular
thefreeenergychangesforalltrajectoriesoflengthk for complexassembly[46]and,inparticular,syntheticDNA
bothsystems. Forlowk ∼1,2,wefindthattheclumpin- brick assemblies [29, 30] where each DNA strand is used
deed has several transformations that do not correspond exactlyonceinastructure. However,suchanassumption
to any transformation of the tip and are only of slightly is not essential to our model; as long as multiple copies
higher energy. However, as k ∼ O(q), we find that such of a species occur far enough apart in the memories, the
extra transformations are pushed to significantly higher robustpatternrecognitionseenherewillcontinuetohold
energies; the histogram of free energy changes for tra- [7].
jectories of length k = 6 are shown in Fig. 7(b) (blue - Similarly, recording experiments on the hippocampus
place cells, red - self-assembly). We see that the low en- show that that the same place cell might have multi-
ergy part of the spectrum matches up between the two ple place fields in a given environment. Much like with
systems. self-assembly, the behavior of place cell network models
remains unchanged provided the place fields of a given
place cell are widely separated in a given environment
III. SIMILARITIES AND DIFFERENCES [16, 20].
Non-specific interactions
Multiplicity of species and place fields In self-assembly, we assumed that we can code arbi-
The self-assembled structures we considered here had traryspatialarrangementsspecifiedbymemoriesm into
α
only one species of each type (e.g., in Fig. 1, each pixel the chemical affinities between N species and that there
has a unique gray scale value). Such a heterogeneous are no other undesired non-specific interactions between
10
the N species. While DNA sequence-based interactions exclusive assembly of the corresponding structure. How-
[29, 30] are highly programmable, non-specific interac- ever, we found that low error rates in classification sets
tionscanneverbecompletelyavoided. Theeffectofsuch a fundamental lower bound on classification time (i.e.,
interactionswasshowntobesmallprovidedthestrength nucleation time) that is independent of physical param-
of non-specific interactions was below a threshold loga- eters and only depends on how similar different memo-
rithmic in the number of species ∼logN [7, 40, 42, 43]. ries are. Within this bound, we found that classification
In place cell networks, sufficiently large heterogeneity timescanbereducedattheexpenseofhighererrorrates
in the strengths of neural connections has been shown bychangingparametersliketemperature,bindingenergy
to destabilize continuous attractors and break it into a and concentrations.
series of discrete attractors [13, 14]. We then related such pattern recognition ability to
Classification vs memory storage place cell neural networks. We emphasize that the rela-
Wereliedonnucleationinself-assemblytoclassifycon- tionshipworksontwodifferentlevels;(1)thesimilarityof
centration patterns. The condensation of the activity place field maps and the chemical affinity map shown in
bump in place cell networks into a single environment’s Fig. 5 (2) a dynamical connection that relies on a quasi-
attractorservesasananalogousclassifierofinputs. How- particle approximation of neural activity. Only the low
ever, continuous attractors in neural networks are used energystatesoftheneuralnetworkcanberelatedtoself-
often as short-term memory storage devices in addition assembly as shown in Fig. 7.
to being a classifier; the precise position of the activ- The nature of patterns recognized here, based on con-
ity clump within the attractor manifold is interpreted as tinuous attractors, is fundamentally different from the
storing a continuous number [13] that can later be re- original case of point attractors [3]. A 2-dim memory
trieved or reset to another state. An analogous function m designedtorecognizeagivenpattern(cid:126)c willnecessar-
α a
does not exist in self-assembly. ily recognize a further two-parameter family of patterns
Self-assembly as the path integral of neural ac- that correspond to other subsets of the 2-dim memory.
tivity Forexample, anysubsetoftheimageofLeocorresponds
The relationship between neural networks and self- to a distinct pattern (cid:126)c with perfect overlap χ = 1 with
assembly is fundamentally a relationship between short the Leo memory and will recall it just as easily. There
trajectories in the two systems; moving the neural activ- is no way to avoid this continuous degeneracy and hence
ity bump from point A to B corresponds to growing the no way to create a self-assembly response dedicated to a
structure between these two states. givenpattern(cid:126)c thatisnotevokedbyanyotherpattern.
a
However, a major difference is that while the bump’s (In the neural network, this degeneracy implies that the
final state at B has no memory of its past (e.g., being at clump of activity can moved around within a stored en-
A), the self-assembled structure encodes the whole his- vironment with no energy cost.)
tory of the bump’s movement in its internal structure. In contrast, in the original model of Hopfield [3], each
For example, in Fig.6, the final state of the teleported patternisstoredasadiscretepointattractorandthereis
clump in the red environment has no memory of under- no continuous degeneracy of states. However, the origi-
goingtheteleportation;however,thecorrespondinghalf- nalmodelofHopfieldwasininfinitedimensions;i.e.,any
blue half-red chimeric structure, contains a record of the neuron could connect to any other, forming a fully con-
neural bump’s trajectory. nected network. As shown in [47–49], short-ranged finite
Sucharecordofhistorylimitsthelengthoftrajectories dimensionalcounterpartsoftheoriginalmodelhavenon-
over which the mapping is faithful due to interference extensive O(N0) capacity and cannot store an extensive
with the past trajectory. This restriction is most easily number of patterns.
apparentin2dimensions;aself-assemblingstructurehas Thus robust pattern recognition with short ranged in-
a ring-like growing front. Growth in a local region of teractions requires us to go beyond point attractors to
thatfrontbysmallamountsisstillin1-1correspondence continuous attractors. As noted in [16] while compar-
with bump moving across that growing front. However, ingthephasediagramsofpointandcontinuousattractor
interference between larger growth from different parts models, continuous attractors are more robust to inter-
of the front cannot be captured by the dynamics of the ference between stored patterns than point attractors.
neural bump. Intuitively, storing a single continuous attractor makes
a bigger demand of the system than storing a point at-
tractor since a whole set of states needs to be stabilized
IV. CONCLUSIONS fortheformer. Thus,withshortrangedinteractions,one
can still have an extensive capacity when storing contin-
Wehaveinvestigatedpatternrecognitionthroughself- uous attractors [7] but not with point attractors [47–49].
assembly; the dynamics of self-assembly is able to clas- Consequently,thedegeneracyinpatternrecognitionseen
sifypatternsintheconcentrationsofN molecularspecies here is intrinsic to self-assembly with short-ranged inter-
intodiscretecategoriesbyself-assemblingdifferentstruc- actions and cannot be removed through minor modifica-
tures. Evenpatternsthatonlymodestlyresembleanide- tions of the model.
alized memory are correctly identified and lead to near- We note that it is possible to interpolate between the