Parallel Processing on VLSI Arrays: A Special Issue of Journal of VLSI Signal Processing

135 Pages·1991·5.75 MB·English

by  Josef A. Nossek (auth.)

PARALLEL PROCESSING
ON VLSI ARRAYS
edited by
Josef A. Nossek
Technical University of Munich
A Special Issue of
JOURNAL OF VLSI SIGNAL PROCESSING
Reprinted from JOURNAL OF VLSI SIGNAL PROCESSING
VoI. 3, Nos. 1-2 (1991)
SPRINGER SCIENCE+BUSINESS MEDIA. LLC
Contents
Special Issue: Parallel Processing on VLSI Arrays
Guest Editor: losef A. Nossek
Introduction ............................................................ Josef A. Nossek 5
Numerical Integration of Partial Differential Equations Using Principles of Multidimensional Wave
Digital Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alfred Fettweis and Gunnar Nitsche 7
Signal Processing Using Cellular Neural Networks ........... L.o. Chua, L. Yang and K.R. Krieg 25
Nonlinear Analog Networks for Image Smoothing and Segmentation .............. A. Lumsdaine,
· ........................................................ J. L. llYatt, Jr. and l. M. Elfadel 53
A Systolic Array for Nonlinear Adaptive Filtering and Pattern Recognition ........ J.G. McWhirter,
· ..................................................... D.S. Broomhead and T.J. Shepherd 69
Control Generation in the Design of Processor Arrays ............ Jurgen Teich and Lothar Thiele 77
A Sorter-Based Architecture for a Parallel Implementation of Communication Intensive Algorithms
· ................................................................... Josef G. Krammer 93
Feedforward Architectures for Parallel Viterbi Decoding ...... Gerhard Fettweis and Heinrich Meyr 105
Carry-Save Architectures for High-Speed Digital Signal Processing ............... Tobias G. Noll 121
Library of Congress Catalogiog-in-Publicatioo Data
Parallel processing on VLSI arrays I edited by losef A. Nossek.
p. cm.
"A Special issue of lournal ofVLSI signal processing."
"Reprinted from lournal of VLSI signal processing, voI. 3, nos.
1-2 (1991)."
Based on papers presented at the International Symposium on
Circuits and Systems held in New Orleans in May 1990.
ISBN 978-1-4613-6805-2 ISBN 978-1-4615-4036-6 (eBook)
DOI 10.1007978-1-4615-4036-6
1. Parallel processing (Electronic computers) 2. Integrated
circuits--Very large scale integration. I. Nossek, losef A.
II. International Symposium on Circuits and Systems (1990: New
Orleans, La.)
QA76.58.P378 1991
004'.35--dc20 91-16484
CIP
Copyright 1991 by Springer Science+Business Media New York
Originally published by Kluwer Academic Publishers in 1991
Softcover reprint ofthe hardcover Ist edition 1991
AII rights reserved. No part of this publication may be reproduced, stored in a retrieval
system or transmitted inany formor by any means, mechanical, photo-copying, recording,
or otherwise, without the prior written permission of the publisher, Springer Scien
ce+Business Media, LLC.
Printed on acid-free paper.
Introduction
Guest Editor:
JOSEF A. NOSSEK
This is a special issue of the Journal of VLSI Signal Processing comprising eight contributions invited for publica
tion on the basis of novel work presented in a special session on "Parallel Processing on VLSI Arrays" at the
International Symposium on Circuits and Systems (ISCAS) held in New Orleans in May 1990. Massive parallelism
to cope with high-speed requirements stemming from real-time applications and the restrictions in architectural
and circuit design, such as regularity and local connectedness, brought about by the VLSI technology are the key
questions addressed in these eight papers. They can be grouped into three subsections elaborating on:
• Simulation of continuous physical systems, i.e., numerically solving partial differential equations.
• Neural architectures for image processing and pattern recognition.
• Systolic architectures for implementing regular and irregular algorithms in VLSI technology.
The paper by A. Fettweis and O. Nitsche advocates a signal processing approach for the numerical integration
of partial differential equations (PD Es). It is based on the principles of multidimensional wave digital filters
(MDWDFs) thereby preserving the passivity of energy dissipating physical systems. It is particularly suited for
systems ofPDEs involving time and finite propagation speed. The basic ideas are explained using Maxwell's equa
tions as a vehicle for the derivation of a multidimensional equivalent circuit representing the spatially infinitely
extended arrangement with only very few circuit elements. This is then transformed into an algorithm along the
principles available for the design of MDWDFs. The attractiveness of the approach is in offering massive parallelism,
requiring only local interconnections and inheriting all the robustness properties of WDFs including finite word
length effects.
The next three papers are concerned with neural architectures. The first two of them are relying on nonlinear
analog circuits as basic functional units for collective analog computing.
The paper by L.0. Chua, L. Yang and K.R. Krieg describes Cellular Neural Networks (CNNs) which combine
some features of fully interconnected analog neural networks with the nearest neighbor interactions found in cellular
automata. It is shown how CNNs are, on the one hand, well suited for VLSI-implementation because of their
local interconnections only, and, on the other, can perform global image processing tasks because of their dynamics.
This is a very interesting and promising branch of neural networks and a lot of research has already been initiated
by the very first publication.
In the paper by A. Lumsdaine, J.L. Wyatt Jr. and I.M. Elfadel, a series of nonlinear analog networks based
on resistive fuses is developed for a very difficult early vision task, i.e., image smoothing and segmentation. These
circuits, which are well suited for VLSI implementation, can automatically solve smoothing and segmentation
problems because their solutions are characterized by an extremum principle.
The last neural network paper by J.O. McWhirter, D.S. Broomhead and TJ. Shepherd describes a systolic array
for nonlinear adaptive filtering and pattern recognition. It consists of a RBF (radial basis function) preprocessor
and a least squares processor behaving, in many respects, like a neural network of the feed-forward multilayer
perceptron (MLP) type.
The highly parallel and pipelined architecture offers the potential for extremely fast computation and is much
more suitable for VLSI -design than the MLP-architecture.
This leads us to the remaining group of contributions focusing on systolic architectures to support piecewise
regular algorithms (PRA), irregular algorithms and regular ones as well as describing algorithmic transformations
and proper number representations to allow efficient VLSI implementations.
The paper by J. Teich and L. Thiele deals with the control generation for the mapping of PRAs onto regular
processor arrays. A systematic procedure is proposed for an efficient design of and control generation for configur
able processor arrays coping with time-and space-dependent processing functions and interconnection structure.
A set of rules is provided to minimize the control overhead.
6
The paper by IG. Krammer proposes an architecture based on a sorting memory to cope with the communication
problem encountered while executing irregular algorithms on a regularly and locally connected processor array.
The sorter-based architecture is a very flexible and efficient one to perform global data transfer operations such
as needed, e.g., in sparse matrix computations. If the interconnection structure is restricted, although being global,
the efficiency can be increased by exploiting these restrictions. The described architecture requires only .IN proc
essors for a problem of size N. Therefore, it offers a very interesting and attractive solution for communication
intensive algorithms.
The paper by G. Fettweis and H. Meyr elaborates on the Viterbi algorithm (VA), which poses a very hard
problem for VLSI realization because of the nonlinear recursion involved. This nonlinear recursion is algebraically
transformed into an M-step recursion. By choosing M large enough, a pure feed-forward signal flow is possible
leading to a regular parallel architecture consisting of identical cascaded modules. This is of special interest for
high speed communication systems design.
The paper by T.G. Noll exploits the potentials of the redundant carry-save number representation for various
high speed parallel signal processing tasks. Many problems such as: overflow effects, testability, and optimized
pipelining schemes are addressed. It gives an overview of the work carried out by the author over several years
in the industry. It will be of special interest to anyone involved in the actual design of high speed signal processing
VLSI-circuits, both on the architectural and circuit module level.
I would like to thank all the authors for submitting their excellent work to this special issue, which reveals
interesting interrelations between various branches of parallel processing, spanning from the numerical simulation
of physical systems, analog and digital neural networks, to systolic architectures for high-speed digital signal proc
essing including the mapping of piecewise regular, irregular and nonlinear recursive algorithms. I would also like
to thank all the reviewers for their cooperation and the Editor-in-Chief, Dr. Earl Swartzlander, for his support.
Numerical Integration of Partial Differential Equations Using Principles of
Multidimensional Wave Digital Filters
ALFRED FETTWEIS AND GUNNAR NITSCHE
Ruhr-Universitaet Bochum, Lehrstuhl fiter Nachrichtentechnik, Postfach 10 21 48, D-4630, Bochum 1, Germany
Received June 15, 1990; Revised December 18, 1990.
Abstract. Physical systems described by partial differential equations (PDEs) are usually passive (due to conser
vation of energy) and furthermore massively parallel and only locally interconnected (due to the principle of action
at proximity, as opposed to action at a distance). An approach is developed for numerically integrating such PDEs
by means of algorithms that offer massive parallelism and require only local interconnections. These algorithms
are based on the principles of multidimensional wave digital filtering and amount to directly simulating the actual
physical system by means of a discrete passive dynamical system. They inherit all the good properties known to
hold for wave digital filters, in particular the full range of robustness properties typical for these filters.
In this paper, only the linear case is considered, with particular emphasis on systems of PDEs of hyperbolic
type. The main features are explained by means of an example.
1. Introduction successful [6], [7]. Such a possibility implies in par
ticular that all stability problems that may originate
Partial differential equations play an important role in from the unavoidable numerical inaccuracies can be
many scientific disciplines. Since analytic solutions can fully solved. More generally, since stability is only one
be obtained only in a few particularly simple situations, aspect of the more general problem of robustness [4],
numerical methods of integration have received a very [8], simulations by means of MD WDFs may be ex
large amount of attention ever since computers for ad pected to behave particularly well with respect to this
dressing such a task have become available. Already general criterion. Robustness in the sense used here is
the number of books on this topic is so large that it defined to designate the property of guaranteeing that
does not seem appropriate to attempt listing them in the strongly nonlinear phenomena induced by the
this paper. unavoidable operations of rounding/truncation and
Obviously, numerical integration always implies overflow correction cause only a particularly small
some form of discretization of the original continuous change in behavior compared to that one would obtain
domain problem, and it is thus understandable that ap if all computations were carried out with infinite preci
proaches based on multidimensional (MD) digital signal sion. For testing robustness, a number of individual
processing are potential candidates [1], [2]. Among such criteria have been proposed, and these can all be
approaches, it appears to be particularly attractive to satisfied by means of passive MD WDFs [4], [8].
investigate the possibility of directly simulating the In addition to robustness, a number of other prop
original continuous-domain physical system by means erties can a priori be expected to be achievable by the
of a discrete passive dynamical system. This way, one approach to be outlined in this paper. These concern
may expect to be able to preserve e.g., the natural aspects such as parallelism, nonconstant parameters,
passivity that physical systems have due to conserva boundary conditions, types of equations, specialized
tion of energy, and thus to carry out the simulation by computers [9] etc. This will be discussed in some more
means of passive MD wave digital filters (WDFs) [3], detail in Section 2.
[4], [5], especially as corresponding approaches in the In the subsequent sections, a direct approach for
one-dimensional case have already proved to be very achieving our goal will be presented. Another approach
8 Fettweis and Nitsche
is based on types of sampling obtained by appropriate related problems. Indeed, passIvIty and in
rotations [4], [5], [10]; it will be described in more cremental passivity [3], [18], [19] are the most
detail in a forthcoming further paper. Applications to powerful means available for finding satisfactory
linear problems such as those arising in the analysis solutions to such problems.
of electrical or acoustical phenomena will be offered. (ii) Passive simulation is greatly facilitated by start
Extensions to nonlinear problems in these fields ap ing from the original system ofP DEs. In partic
pear not to be problematic since the same has been ular, elimination of dependent variables should
demonstrated to hold in the one-dimensional case [7]. be avoided, and this excludes the widely used
It is to be expected that the basic ideas described in principle of first deriving a global POE by
this paper can also be applied in some other and more eliminating all dependent variables except one.
difficult fields in which partial differential equations Such a global POE cannot characterize the
are of vital interest. passivity of a system, as is already the case for
The method is particularly suitable for solving wave the global ordinary differential equations en
propagation problems (hyperbolic problems), but can countered in one-dimensional (1-0) problems.
also be a;>plied to other situations. In the case of Max Such a global ordinary differential equation
well's equations in 2 spatial dimensions, it leads to relates indeed directly the output variable to the
algorithms similar to those obtainable by the method input variable and corresponds in a simple way
based on the use of the transmission line (unit element) to the transfer function to be considered, and it
concept [11]-[14], but the formulation of the algorithm is well-known that any transfer function of a
can be derived in a much more straightforward way, passive system can also be synthesized by an ac
and this is especially true for 3 spatial dimensions. Con tive system.
trary to known finite-difference methods, e.g., the Yee (iii) Physical systems (i.e., at least all those that are
algorithm [15], [16], no stability problems arise in the of engineering relevance) are by nature massively
case of nonconstant parameters, and reflection-free parallel and only locally interconnected. This is
boundary conditions can be fulfilled in a very simple a way of expressing that all these physical systems
manner. A quite different point of view characterizes are subject to the principle of action at proxim
the methods that work entirely in the frequency domain, ity rather than that of action at a distance. Thus,
e.g., the Finite Integration Technique (FIT) [17]. the behavior at any point in space is directly in
Clearly, the topic to which this paper is devoted is fluenced only by the points in its immediate neigh
rather vast, and the present text can only constitute a borhood, and, since propagation speed is finite,
first introduction. any change originating at time to at any specific
point in space can cause changes at any other
point only at time > to. This inlIerent massive
2. General Principles parallelism and exclusively local interconnectiv
ity represents an extremely desirable feature and
As mentioned in the Introduction, the basic principle should be preserved in the simulation.
is to obtain a discrete simulation of the actual physical (iv) The simulation should preferably be done by
system described by a system of partial differential means of the best approximation achievable in
equations (POEs). This amounts to replacing the the MO frequency domain (say in the spatio
original system of POEs by an appropriate system of temporal frequency domain, wave numbers being
difference equations in the same independent physical called, in the terminology adopted here, frequen
variables (e.g., spatial variables and time) as those oc cies, or spatial frequencies), assuming the equa
curring in the original POEs, or in independent tions to be linear and to involve only constant
variables obtained from the former by simple transfor parameters. This ensures, under appropriate con
mations. Specific aspects that are of relevance in do ditions, a particularly good approximation in
ing so are listed hereafter: time and space and amounts to adopting the so
called trapezoidal rule of integration. The latter
(i) Physical systems are usually passive (contractive)
aspect remains valid also in the nonconstant and
due to conservation of energy. The simulation
even the nonlinear case.
should preserve this natural passivity since this
(v) Instead of using original quantities such as
opens the possibility of solving all stability-
voltage, current, electromagnetic field quantities,
Numerical Integration of Partial Differential Equations 9
pressure, velocity, displacement, etc., one should other physical variables) will lead to fewer
adopt corresponding so-called wave quantities multiplications and easier ways of guaranteeing
(also frequently simply called waves, especially passivity; they will therefore be preferred
in the context of circuit theory [20]). In the case wherever possible [3].
of an electric port characterized by a voltage, u, (vii) If a simulation by a passive MD-WDF circuit
and a current, i, this amounts to assigning to the is obtained, numerical instabilities that otherwise
port a suitably chosen, but otherwise arbitrary, could occur due to linear discretization, i.e., to
port resistance, R, and to define a forward wave, discretization in space and time, are fully ex
a, and a backward wave, b, e.g., either as voltage cluded. In the case of nonlinear PDEs this may
waves by means of hold only if power waves are adopted.
(viii) If an MD WDF is passive under ideal condi
a = u + Ri, b = u - Ri (1) tions, i.e., under the assumption that all com
putations are carried out with infinite precision,
or as power waves by means of
it can be built in such a way that passivity and
a = (u + Ri)/(2YR), b = (u - Ri)/(2YR).(2) usually also incremental passivity remain
guaranteed even if the strongly nonlinear effects
Closely related to the use of waves is the descrip are taken into account that are due to the
tion of physical systems by means of scattering unavoidable operations implied by the need for
matrices. rounding/truncation and overflow correction.
Such an approach corresponds to adopting a This way, complete robustness of the algorithm
basic principle of wave digital filtering, as used carrying out the numerical integration can be en
also in the MD case. Just as for WDFs, this prin sured, i.e., it can be ensured that the behavior
ciple is quite essential for obtaining a directly of this algorithm under finite-arithmetic condi
recursible, thus explicitly computable, passive tions (including the particularly annoying
simulation. For understanding this, recall that overflow aspects, which could otherwise e.g.,
waves and scattering matrices are concepts of even lead to chaotic behavior) differs as little
fundamental, universal importance for describ as possible from the one that would be obtained
ing physical systems. Using these concepts in the case of exact computations [8].
amounts, e.g., to distinguishing clearly between Note that the term passivity must be inter
input quantities (incident waves) and resulting preted in a somewhat wider sense than what has
reflected and transmitted output quantities, thus conventionally been done [3]. Thus, it is suffi
to distinguishing explicitly between cause and cient that the MD WDF circuit can be embed
effect and hence to making explicit use of the ded in a suitable way in a passive circuit. Or
causality principle. It is precisely this principle else, consider a I-D algorithm onto which the
which is essentially behind the principle of com MD-WDF algorithm can be mapped in view of
putability, i.e., behind the fact that in order to its recursibility. It is sufficient that this 1-D
be able to carry out computations in a sequen algorithm corresponds, in whatever way, to a
tial machine it must be possible to give a con passive I-D WDF [21].
secutive ordering in which the required opera (ix) The preservation of properties such as massive
tions must be performed. parallelism and exclusively local nature of the
(vi) In many cases there is no need to carry out ex interconnections, which are inherent to the
plicitly the change from the original variables original physical problem, is of interest not only
to the wave quantities. All what is needed is to for physically implementing the algorithm (in
obtain, by means of a suitable (and usually very particular for gaining speed by increasing hard
elementary) analogy, an MD passive electric cir ware and for enabling the use of systolic-array
cuit [3] and to apply to this circuit known prin type arrangements), but is quite essential from
ciples for deriving corresponding MD WDFs. a more basic viewpoint. It makes it possible, in
The choice between voltage waves and power deed, to allow very easily for arbitrary varia
waves is relatively irrelevant and should be made tions (e.g., in space) of the characteristic
according to suitability. In many cases, voltage parameters of the physical system and also for
waves (or corresponding waves in the case of arbitrary boundary conditions.
10 Fettweis and Nitsche
(x) Since the approach simulates directly the steady-state in the case of a sinusoidal or
behavior of the actual physical system, assumed complex-exponential excitation.
of course to obey causality, the approach is par (xvi) Usual digital filters are linear, and application
ticularly suitable for time-dependent problems of the present approach is thus easiest in the case
implying propagation over finite distances in of linear problems. However, extension to
nonzero time, thus for problems of hyperbolic nonlinear problems is possible, in particular
type. along lines similar to those already used suc
(xi) Problems of elliptic type as they occur for deter cessfully for ordinary differential equations [7].
mining equilibrium states can be dealt with in (xvii) The approach is particularly suitable as basis for
different ways. One possibility is to adopt developing specialized computers [9] that involve
suitable starting values and then to solve the massively parallel processing and that are con
dynamic problem until equilibrium is reached. ceived for numerically solving specific classes
In order to ensure convergence, one should of of PDEs. Such computers would consist of large
course introduce suitable losses that have no ef numbers of similar (or even identical) and
fect once the equilibrium is reached. As an ex similarly programmable (or even identically pro
ample, if the equilibrium state of a set of elec grammable) individual processors. These proc
trically charged conductors is to be computed, essors could be interconnected in form of
these conductors may be made strongly lossy systolic-array-type arrangements and have essen
since this causes dissipation only as long as cur tially to carry out only additions/subtractions and
rents are still flowing, i.e., as long as the multiplications. Thus, the individual processors
equilibrium is not yet reached. In other prob may simply be digital signal processors, possibly
lems, such losses may even be of a type that does even of simplified type. Furthermore, due to the
not occur in the actual physical system. inherent advantageous properties ofWDFs, these
(xii) Problems of parabolic type imply infinite prop digital signal processors may be built with
agation speed and thus normally imply idealiza shorter word-lengths for coefficients and signal
tions of what is physically obtainable. Hence, parameters (data).
they can suitably be modified in such a way that
any propagation will occur at finite speed. This
modified problem is then again amenable to our 3. Direct Approach by Frequency-Domain Analysis
approach.
(xiii) Multigrid methods are known to be attractive for 3.1. Representation by Means of an Equivalent Circuit
numerical integration of PDEs [22]. Alternative
multigrid methods can also be used in relation
We will explain the basic ideas by means of a concrete
with the present approach. For determining an
example. For this, we first choose the equation of a set
equilibrium as explained under item (xi), one
of conducting plates (possibly lossy) separated by a
may start out with a very coarse grid. The equi
dielectric (possibly also lossy); in this case the variables
librium computed this way may be used as initial
involved are indeed electric, which is easiest to under
value for a computation with a denser grid, etc.
stand if one wants to establish the analogy with the elec
(xiv) The multirate principle of digital signal process
tric basis ofWDFs. The equations to be considered are
ing [23], which is known to involve operations
such as interpolation (e.g., by first applying zero I ail + ril + au = fl(t) (3a)
stuffing) and decimation (by dropping of samp at3 atl
ling points), should be applicable to the present
approach, in particular in order to allow for grid 1ai-2 + Tl.2 + -au = fz(t) (3b)
densities that are nonuniform in space. No details at3 at2
for making use of this possibility have, however, ail ai au
been worked out so far. - + - 2 + c - + gu = Nt) (3c)
(xv) The approach can be modified in order to deter atl at2 at3
mine steady-state solutions in an alternative way. where t3 corresponds to time, tl and t2 are the two
In particular, it is not required to compute the spatial coordinates, i I and i2 the current densities in the
complete time behavior in order to determine the direction of tl and t2, respectively, u the voltage between
Numerical Integration of Partial Differential Equations 11
ret)
the two plates, and a given excitation (forcing func
tion), with
(4)
The problem thus described is three-dimensional (3-D),
at least in the terminology of digital signal processing,
since it comprises 2 spatial dimensions, fl and f2, and
time, f3• For a more conventional notation, ft. f2, and
f3 should be replaced by Xt. X2, and f, respectively. For
the parameters l, c, r, and g, we may assume
l > 0, c > 0, r 2!: 0, g 2!: O. (5a,b,c,d) P351-PI'3 P351-PI'3
®
These parameters may be constants or functions of tl
E3 PI '3
and t2; they may also be functions of f3' but such a
dependence is of more limited practical importance, and
independence of f3 somewhat simplifies certain later P3 iC'i-61-52)+g,} PJ~-P!3 P3il-~)+'
discussions.
In this section we assume that l, c, r, and g are con
13 Pz~
stants. Let us solve (3) for a 3-D steady-state behavior
of the form
Fig. 1. a) Circuit representing (8) and (9).
b) Circuit equivalent to that of (a).
where It. I" 13, Et. E2 and E3 are complex constants
(complex amplitudes) while r3 is a positive, but other
wise arbitrary, constant. Furthermore,
p = (Pt. P2, P3)T
where PI, P2, and P3 are arbitrary complex constants.
We may interpret 11,12,13 and Et. E2, and E3 as com
plex amplitudes, while Pt. P2, and P3 are complex fre
quencies (more specifically: complex wave numbers
in the case of PI and P2)' Substituting (6) and (7) in
(3), we obtain the set of algebraic equations
(P3l + r)II + Plr3I3 = Et. (8a)
(P3l + r)I2 + P2r3I3 = E2 (8b) Fig. 2. a) A symmetric two-port in T-configuration.
b) Its equivalent lattice representation comprising the canonic
Plr3I1 + P2r3I2 + (P3C + g)r;I3 = E3 (9) impedances Z I and Z II.
c) A simplified representation of b).
where the last equation has actually been multiplied d) A so-called Jaumann equivalent of b).
by r3' The equations (8) and (9) can be interpreted in
form of the circuit of figure la, which in turn is Z' = Za' Z" = Za + 2Zb, (10)
equivalent to that of figure lb, 01 and O2 being arbitrary and figure 2c a simplified representation of the circuit
positive constants. of figure 2b. Applying this equivalence to figure lb,
Consider next the well-known transformation of we obtain the circuit of figure 3 where Z;, Z;" Z~, and
Z;
figure 2, where figure 2a represents a symmetric two are given by
port in T-configuration, figure 2b its equivalent lattice Z; = P301 - Plr3, Z;' = P301 + Plr3, (11)
representation comprising the canonic impedances Z'
and Z" given by Z~ = P302 - PZr3, Z~' = P302 + P2r3; (12)