Stochastic Search Genetic Algorithm Approximation of Input Signals in Native Neuronal Networks

128 Pages·2013·1.87 MB·English

Stochastic Search Genetic Algorithm
Approximation of Input Signals in
Native Neuronal Networks

Andrei Anisenia

A thesis submitted to the
Faculty of Graduate and Postdoctoral Studies
in partial fulfillment of the requirements for the
MSc degree in Computer Science
with specialization in Bioinformatics

School of Electrical Engineering and Computer Science

Faculty of Engineering
University of Ottawa

© Andrei Anisenia, Ottawa, Canada, 2013
Abstract
The present work investigates the applicability of Genetic Algorithms (GA) to the
problem of signal propagation in Native Neuronal Networks (NNNs). These
networks are comprised of neurons, some of which receive input signals. The
signals propagate though the network by transmission between neurons. The
research focuses on the regeneration of the output signal of the network without
knowing the original input signal. The computational complexity of the problem is
prohibitive for the exact computation. We propose to use a heuristic approach called
Genetic Algorithm. Three algorithms are developed, based on the GA technique.
The developed algorithms are tested on two different networks with varying input
signals. The results obtained from the testing indicate significantly better
performance of the developed algorithms compared to the Uniform Random Search
(URS) technique, which is used as a control group. The importance of the research
is in the demonstration of the ability of GA-based algorithms to successfully solve
the problem at hand.
ii
Acknowledgements
The research thesis was done under the supervision of Prof. Dimitrios Makrakis in
the Faculty of Engineering, University of Ottawa. I would like to thank all the people
who have helped me along the way.
Firstly, I wish to express my heartfelt thanks to my supervisor Prof. Dimitrios
Makrakis for his guidance and encouragement throughout my studies at the
University of Ottawa. Without his knowledge, advice and mentoring the completion
of this research would have been impossible. I am very grateful to him for his
sincere sympathy and invaluable support.
I would like to acknowledge Prof. Franz Oppacher for his deep knowledge of genetic
algorithms, which provided inspiration for the development presented in the current
work. A special mention has to be made to Grigory Chudov, my friend, for his help
with computational issues. I would like to thank the School of Electrical Engineering
and Computer Science at the University of Ottawa for providing all the necessary
assistance to complete my work.
I would like to express special gratitude to my wife, Tatiana Kolechkina, for her
patience and unconditional support throughout my studies and also for invaluable
help with Matlab.
iii
Notations
Abbreviations:
NNN Native Neuronal Network
ANN Artificial Neural Network
GA Genetic Algorithm
BFS Breadth First Search
URS Uniform Random Search
SGA Simple GA
FIGA Fixed Input GA
FGA Fusion GA
CISGA Complex Input Search GA

Terms:
Neuron A type of biological cell, capable of transmitting electrical
signals along its cell membrane.
Potential The difference in electric charge between the two sides of a
neuron’s membrane. It changes in time and moves between the
adjacent membrane areas, therefore is usually interpreted as
“moving” potential.
Dendrite Branching structure of a neuron cell, which can receive
incoming electrical signals, transformed into potentials, moving
towards the other parts of a neuron (soma, axons).

iv
Soma (cell body)
Central part of a neuron, where potentials arriving from

dendrites are summed.
Axon Branching structure of a neuron cell, which transmits potentials

away from the soma of a neuron.
Axonal terminal
The end of axonal branch where potentials arrive. This part of
a neuron forms contacts or synapses with the neighbouring
neuron’s dendrites, allowing for transmission of electrical
signals between the neurons.
Synapse A point of contact between two neuron cells. It consists of an
axonal terminal of the first neuron, short extracellular gap
between the membranes of the two neurons and the dendrite’s
membrane of the second neuron.
Neurotransmitter
A molecule released from the axonal terminal of a neuron at the
synapse that is able to bind a dendrite’s membrane of another
neuron, causing generation of a potential in the dendrite’s
membrane.
Synaptic strength
Value given to each synapse, indicating the power of the
electrical signal that the synapse is able to generate.
Local potential
A potential initialized at a dendrite, propagating towards soma.
These potentials decay as they propagate.

v
Spike (/ global potential / action potential)
Potential actively generated by a neuron inside the soma,
propagating away from the soma along the axons. These
potentials retain the same strength as they propagate until
reaching axonal terminals.
Excitation threshold
Value of potential required to be reached or passed by the
potential reaching the soma in order for a spike to be
generated. Local potentials arriving from the dendrites
contribute towards reaching this value.
Refractory period
A period of time during which no spike can be generated in a
neuron. It immediately follows the previous spike, generated at
the neuron.
Interference
Effect of signal propagation interruption in NNNs caused by the
neurons, which enter refractory period after spiking, and
therefore are unable to transmit other incoming signals.

vi
Contents
Abstract ......................................................................................................................ii
Acknowledgements ................................................................................................... iii
Notations ...................................................................................................................iv
Contents ................................................................................................................... vii
List of tables .............................................................................................................. x
List of figures ............................................................................................................ xii
Chapter 1: Introduction .............................................................................................. 1
1.1 Research motivation .................................................................................... 1
1.2 Background on neuronal networks ............................................................... 2
1.2.1 Single neuron ........................................................................................ 2
1.2.2 Neuronal signal generation laws ............................................................ 4
1.3 Definition of the research problem ............................................................... 7
1.3.1 Computation in neuronal networks ........................................................ 7
1.3.2 Direct simulation versus prediction ........................................................ 8
1.4 Motivation for Considering Use of Genetic Algorithm and Related
Background ........................................................................................................... 9
1.5 GA encoding for the prediction problem in NNNs ....................................... 11
1.6 Thesis focus ............................................................................................... 12
1.7 Thesis contribution ..................................................................................... 13
1.8 Thesis outline ............................................................................................. 14
Chapter 2: Literature review .................................................................................... 15
2.1 Fundamental works .................................................................................... 15
2.2 Structural analysis of NNNs ....................................................................... 18
vii
2.3 Utilization of signal propagation in NNNs ................................................... 23
2.4 Application of GA techniques in NNN domain ............................................ 25
Chapter 3: Model of signal propagation in NNNs .................................................... 28
3.1 Neuronal network model ............................................................................ 28
3.2 Model parameters ...................................................................................... 31
3.3 Simulation algorithm ................................................................................... 35
3.4 Simplifications and assumptions of the model............................................ 38
3.5 Complexity of the computation ................................................................... 39
3.6 Minor simulation algorithm improvements .................................................. 41
Chapter 4: GA-based algorithms ............................................................................. 43
4.1 Requirements ............................................................................................. 43
4.2 Uniform Random Search ............................................................................ 44
4.3 Template GA .............................................................................................. 45
4.4 Simple GA .................................................................................................. 50
4.4.1 Population size and Number of generations ........................................ 50
4.4.2 GA encoding of solutions ..................................................................... 50
4.4.3 Fitness function ................................................................................... 51
4.4.4 Crossovers and Mutations ................................................................... 53
4.5 Fixed Input GA ........................................................................................... 57
4.5.1 Population size and Number of generations ........................................ 60
4.5.2 GA components implementation .......................................................... 61
4.5.3 Scalability issue ................................................................................... 62
4.6 Fusion GA .................................................................................................. 63
4.6.1 Population size and Number of generations ........................................ 65
4.6.2 GA components implementation .......................................................... 66
viii
4.6.3 Scalability issue ................................................................................... 67
Chapter 5: Generation of input for testing ............................................................... 68
5.1 Overview .................................................................................................... 68
5.2 Auditory cortex model network ................................................................... 69
5.3 Pulse shaped input sequences .................................................................. 70
5.4 Input complexity ......................................................................................... 72
5.5 Complex Input Search GA.......................................................................... 75
5.5.1 Population size and Number of generations ........................................ 75
5.5.2 GA components implementation .......................................................... 76
5.6 Test set generated by CISGA .................................................................... 77
5.7 Modified Feedback Network ....................................................................... 77
Chapter 6: Simulation results .................................................................................. 81
6.1 Collected statistics overview ...................................................................... 81
6.2 Input complexities ...................................................................................... 82
6.3 Auditory cortex and pulse input with frequency equal 12 ........................... 86
6.4 Auditory cortex and pulse input with frequencies 6 and 3 .......................... 91
6.5 Auditory cortex and CISGA generated input .............................................. 96
6.6 Modified feedback network and CISGA generated input ............................ 99
6.7 Outperformance of GA-based algorithms over URS ................................ 102
Chapter 7: Conclusions and future work ............................................................... 105
7.1 Research contributions ............................................................................ 105
7.2 Future research directions........................................................................ 107
References ............................................................................................................ 109

ix
List of tables
Table 6.1. Runtimes of the test sets………………………………………………….....80
Table 6.2. Mean and Standard Deviation values for input linearity levels of the
auditory cortex network with different inputs…………………………………………...81
Table 6.3. Mean and Standard Deviation values for input linearity levels of CISGA
generated input……………………………………………………………………………82
Table 6.4. Mean and Standard Deviation values for the number of output bit
mismatches for the auditory cortex network with pulse shaped input having
frequency 12……………………………………………………………………………….85
Table 6.5. Mean and Standard Deviation values for the number of input node
mismatches for the auditory cortex network with pulse shaped input having
frequency 12……………………………………………………………………………….85
Table 6.6. Mean and Standard Deviation values for the number of output bit
mismatches for the auditory cortex network with pulse shaped input having
frequency 6………………………………………………………………………………...89
Table 6.7. Mean and Standard Deviation values for the number of input node
mismatches for the auditory cortex network with pulse shaped input having
frequency 6………………………………………………………………………………...89
Table 6.8. Mean and Standard Deviation values for the number of output bit
mismatches for the auditory cortex network with pulse shaped input having
frequency 3………………………………………………………………………………...91
Table 6.9. Mean and Standard Deviation values for the number of input node
mismatches for the auditory cortex network with pulse shaped input having
frequency 3………………………………………………………………………………...91
Table 6.10. Mean and Standard Deviation values for the number of output bit
mismatches for the auditory cortex network with CISGA generated input………… 93
x