290 IEEE TRANSACTIONS ON NEURAL NETWORKS. VOL. I . NO.

4, DECEMBER 1990

The Stone-Weierstrass Theorem and Its Application to Neural Networks

NEIL E. COTTER

Abstract-This paper describes neural network architectures based
on the Stone-Weierstrass theorem. Exponential functions, polyno-
mials, partial fractions, and Boolean functions are used to create net-
works capable of approximating arbitrary bounded measurable func-
tions. A “modified logistic network,” satisfying the theorem, is
proposed as an alternative to commonly used networks based on logis-
tic squashing functions.
I. INTRODUCTION
2 ) Separability: For any two points xI # x2 in D,there
is a n f i n 5 such t h a t f ( x , ) f f ( x 2 ) .
3) Algebraic Closure: Iffand g are any two functions
+
in 5, then f g and uf bg are in 5 for any two real num-
bers a and b.
Then 5 is dense in C ( D ) , the set of continuous real-
valued functions on D . In other words, for any E > 0 and
any function g in C(D ) , there is a function f in 5 such
that 1 g ( x ) - f ( x ) 1 < E for all x E D.

I N the past several years, researchers have shown that

infinitely large neural networks with a single hidden
layer are capable of approximating all but the most path-
ological of functions [ 11-[4]. This powerful result assures
us that neural networks have two desirable properties: 1)
larger networks can produce less error than smaller net-
works; and 2) there are no nemesis functions that cannot
be modeled by neural networks.
The Stone-Weierstrass theorem from classical real
analysis can be used to show that certain network archi-
tectures possess the universal approximation capability.
Hornik et al. [4] introduced this theorem as the basis of
their proofs and considered existing architectures that fail
to satisfy the theorem’s hypotheses. In this paper, we fo-
cus instead on new architectures satisfying the theorem.
This ensures that our networks have the two desirable
properties listed above.
By employing the Stone-Weierstrass theorem in the de-
sign of our networks, we also guarantee that the networks
can compute certain polynomial expressions: if we are
given networks exactly computing two functions, fand g,
then a larger network can exactly compute a polynomial
expression offand g . We will see how the larger network
can be explicitly determined from the smaller ones. Our
discussion begins with the theorem and a pragmatic de-
scription of some abstract terminology.
11. STONE-WEIERSTRASS THEOREM AND TERMINOLOGYin physical systems.
Theorem (Stone- Weierstrass) [5], [6]: Let domain D
be a compact space of N dimensions, and let 5 be a set
of continuous real-valued functions on D, satisfying the
following criteria:
1) Identity Function: The constant function f ( x ) = 1 Lusin [6].
is in 5.
Manuscript received January 24, 1989; revised June 15, 1990. This pa- main D,then given 6 > 0 there is a continuous real-val-
per was supported by the National Science Foundation under Grant EET-
88 10478.
The author is with the Department of Electrical Engineering, University
of Utah, Salt Lake City, UT 841 12.
IEEE Log Number 9038255.
A. Compact Spaces
In applications of neural networks, the domain on which
we operate is almost always compact. It is a standard re-
sult in real analysis that every closed and bounded set in
!JIN is compact [6]. The unit hypercube, that we assume
for the domain from now on, is the prototypical example
of such a compact space:
D = [0, 1IN. (1)
Any region obtained by smoothly distorting (but not tear-
ing) the unit hypercube is also a suitable compact space
to which the Stone-Weierstrass theorem can be applied.
B. Measurable Functions and Lp Spaces
The Stone-Weierstrass theorem states conditions that
guarantee that a network can approximate continuous
functions. Nevertheless, many interesting functions, in-
cluding step functions, are discontinuous. These func-
tions are members of the set of bounded measurable func-
tions. This set includes continuous functions, bounded
functions that have a countable number of discontinuities,
and all other bounded functions that are likely to be en-
countered in practice. Nonmeasurable functions, such as
white Gaussian noise, are typified by having discontinu-
ities at uncountably many points in their domain. Such
functions are abstractions, however, that are unrealizable
C. Convergence Almost Everywhere
The Stone-Weierstrass theorem can be extended to
bounded measurable functions by applying a theorem of
Theorem (Lusin): If g is a measurable real-valued func-
tion that is bounded almost everywhere on a compact do-
ued functionfon D such that the measure of the set where
f i s not equal to g is less than 6

1045-9227/90/1200-0290$01.OO 0 1990 IEEE

IEEE TRANSACTIONS ON NEURAL NETWORKS. VOL. I. NO. 4. DECEMBER 1990
In other words, the minimum total volume of open spheres
required to cover the set wheref # g is less than 6.
Results from real analysis allow us to strengthen this
result of convergence in measure to convergence almost
everywhere, and we conclude that continuous functions
can approximate bounded measurable functions.
An alternative formulation is to consider the space
Lp [ D ] consisting of all real measurable Lebesque-inte-
grable functions whose Lpn o m is finite on domain D:
(3)
where the norm for 1 Ip < 00 is
Because of their practical meanings, it is convenient
to use the Lp spaces: for p = 1, we obtain the set of finite
area functions on domain D ,and forp = 2, we obtain the
set of finite energy functions on domain D. B~~~~~~these
descriptions are so easily understood, we state our theo-
rems in terms of Lp spaces.
When we say that a set 3 of functions is dense in
Lp[ D 1, we will mean that for any f i n Lp[ D 1, we can find
a sequencef, of functions in 5 such that
It is always possible to find a sequence of continuous
functions satisfying this condition: For such a sequence it
is also possible to conclude that f n converges to f almost
everywhere.
The practical consequence of the preceding discussion
is that an infinitely large neural network can model any
Lp[D] function at all but isolated points. A finite net-
work, however, might only accurately model such func-
tions over a subset of the domain.
This kind of problem also occurs with finite Fourier se-
ries, manifesting itself as Gibb’s phenomenon. The 18%
overshoot that occurs at step discontinuities decays away
over some interval, causing significant error if too few
terms are included in the Fourier series. In the limit of
infinite terms, the error is confined to a set of measure
zero and might be inconsequential. Users of finite neural
networks might observe analogous phenomena. They may
be confident, however, that errors can be reduced if they
use a larger network and possess a suitable learning al-
gorithm.
D. Advantages of Satisfying the Stone- Weierstrass
Theorem
A network that satisfies the Stone-Weierstrass theorem
can compute the weighted sum af + bg and the product
f g of functions f and g computed by smaller networks.
Conversely, we can separate a polynomial expression into
smaller terms that can be approximated by neural net-
works. These networks may be recombined to approxi-
mate the original polynomial. We shall describe the pro-
29 1
cedure for constructing a network that computes sums,
and the proof of Theorem 1 doubles as a procedure for
constructing a network that computes products. With these
procedures, we can explicitly determine synaptic weights
for a network approximating a polynomial.
In comparison, networks employing sigmoidal squash-
ing functions, including the popular logistic [7] and hy-
perbolic tangent functions [8], are unable to exactly com-
pute a product of sigmoids. Using logistic squashing
functions, for example, we can trivially calculate logistics
along each of two axes
Nevertheless, no finite-sized “logistic network” can ex-
actly compute f g . Nor does adding layers to the network
resolve this problem. In this case, we find it difficult to
exploit structural information about the function we wish
to approximate.
111. APPLYING THE STONE-WEIERSTRASS THEOREM
A. Generic Network Architecture
A tree structure, in which many neurons on one layer
feed a single neuron on the next layer, is a generic archi-
tecture for networks satisfying the Stone-Weierstrass
theorem. An example of a tree-structured network is
shown in Fig. 1. The tree structure has one or more hid-
den layers followed by a linear output neuron. We assume
that an arbitrarily large number of neurons are present in
each hidden layer.
In hidden layers, we will employ a variety of squashing
functions. Since several of the squashing functions are
unbounded, we use the term “squashing function”
loosely.
Before verifying that the generic architecture satisfies
the Stone-Weierstrass theorem, we observe that the ap-
proximation capabilities of our networks are unaffected by
various modifications to the tree structure.
1) Because tree-structured networks can be embedded
in totally connected networks, our theorems extend to to-
tally connected networks.
2) If the range of the function being approximated is
appropriately bounded, we may include an invertible con-
tinuous squashing function in the output neuron. We ob-
tain this result by observing that, prior to squashing, we
have a continuous function and a linear output neuron.
(For this argument to be valid, the range of the approxi-
mated function must be properly contained in the range
of the squashing function.)
3) N copies of a network, placed side by side, can com-
pute continuous N-dimensional vector-valued functions.
In other words, single-output networks generalize to mul-
tiple outputs.
4) Preprocessing of inputs via any continuous inverti-
ble function does not affect the ability of an architecture
to approximate continuous functions.
292 IEEE TRANSACTIONS ON NEURAL NETWORKS. VOL. I, NO. 4. DECEMBER 1990

represent f g as a sum of functions. Thus, the key to sat-

isfying the Stone-Weierstrass theorem is to find functions
that transform multiplication into addition so that prod-
ucts can be written as summations. There are at least three
generic functions that accomplish this transformation: ex-
x1
ponential functions, partial fractions, and step functions.
We use these squashing functions in the next section to
XN construct networks.
Iv . NETWORKS
SATISFYING THE STONE-WEIERSTRASS
THEOREM
A. Decaying-Exponential Networks
Fig. 1 . Tree structured network for modified logistic network. Each box
Exponential functions are basic to the process of trans-
represents a neuron which computes a weighted sum of inputs and passes forming multiplication into addition in XWeral kinds of
the result through the function shown in the box. networks:
exp (XI) exp (4= exp (XI + 4 . (8)
B. Identity Function
The first hypothesis of the Stone-Weierstrass theorem Decaying-exponential networks use this identity explic-
requires that our neural network be able to compute the itly:
Theorem I : Let 3 be the set of all functions that can
identity function f ( x ) = 1. An obvious way to compute
be computed by arbitrarily large decaying-exponential
this function is set all synaptic weights in the network to
networks on domain D = [ 0 , 1 ] N :
zero except for a unitary threshold input to the last sum-
mation unit. A more seamless way to compute this func-
tion is to use a penultimate-layer squashing function
whose output value is one for an input value of zero. Then
I / N \ \
setting all synaptic weights to zero results in an output
value equal to one. This eliminates the need for an extra- = C
i= 1
w,exp (,- C
n=l winxn):wir winE %). (9)
neous threshold input. With the exception of the modified
logistic network, all the networks we discuss are of this Then 3 is dense in L p [ D ] ,for 1 Ip < 03.
tY Pe. Proof: Letfand g be two functions in 3 wherefis
as shown in (9) and g is as shown in (10)
C. Separability
J \
The second hypothesis of the Stone-Weierstrass theo-
rem requires that our neural network be able to compute
functions that have different values for different points.
Without this requirement, the trivial set of functions { f : Then the productfg may be written as a single summation
f ( x ) = n, n E % } would satisfy the Stone-Weierstrass as follows:

(-il
theorem. Separability is satisfied whenever a network can IJ

compute strictly monotonic functions of each input vari- fg = C

able. All of our networks have this capability.
D. Algebraic Closure-Additive
The third hypothesis of the Stone-Weierstrass theorem
requires that our neural network be able to approximate
sums and products of functions. If the network can com-
pute either of two functions, f a n d g , we construct a net-
work that computes af +
bg as follows: embed the net-
works forfand g in a larger network and, at the output
neuron, scale the synapses by a and b . Since the output
unit is linear, we obtain af +
bg. The resulting network
also has a tree structure as before.
E. Algebraic Closure-Multiplicative
Modeling the product f g of two functions is the last
capability we must demonstrate before we can conclude
that the Stone-Weierstrass theorem applies to a network.
Because the output unit is assumed to be linear, we must
1=1 Wl exp W&) (11)
where w, = wi wj and w,,,= win win. +
Hence, f g is in 3,and the network satisfies the Stone-
Weierstrass theorem. It follows that 3 is dense in Lp[ D ],
f o r 1 s p < 03.
Corollary I : In the decaying-exponential network, hid-
den-layer weights winmay be restricted to integer or non-
negative integer values. (The latter case yields bounded
outputs from the decaying-exponential neurons when in-
puts are nonnegative. This is the motivation for using
negative rather than positive exponentials.) Alternatively,
if a threshold input is included in the hidden layer, output
layer weights wi may be restricted to integer values. Re-
stricting weights on both layers to integer values, how-
ever, would violate the hypothesis that the network com-
putes all functions af +
bg where a and b are real
numbers. Note that adding a threshold input on the first
layer is equivalent to multiplying wi by a real constant.
IEEE TRANSACTIONS ON NEURAL NETWORKS. VOL. I ; NO. 4. DECEMBER 1990
B. Fourier Networks
Fourier networks, introduced by Gallant and White 131,
implement Fourier series in network form. Fourier series
satisfy the Stone-Weierstrass theorem by the familiar tri-
gonometric identity that transforms multiplication into
addition:
cos (A + B ) = cos A cos B - sin A sin B. (12)
This identity is merely a disguised form of (8) obtained
from complex exponentials. Hence, Fourier networks are
also based on exponentials.
A clever twist suggested by Gallant and White is to chop
sinusoids into half-cycle sections and add flat tails. The
result is a sigmoidal cosine squasher. A sinusoid is ob-
tained by shifting and summing cosine squashers. We
shall refer to a slightly modified version of the cosine
squasher as a "cosig" function.
By showing that Fourier networks mimic Fourier se-
ries, Gallant and White proved an equivalent form of the
following theorem and corollary. We note that Fourier se-
ries, and hence Fourier networks, satisfy the Stone-
Weierstrass theorem. The cosig network, described by
Theorem 2, computes a superset of functions computed
by Fourier networks.
Theorem 2: Let 3 be the set of all functions that can
be computed by arbitrarily large cosig networks, on do-
main D = [o, 1 l N :
f-
i= 1 3 form of network presented here, synaptic weights serve
where cosig( ) is a cosine squasher
1 works on domain D = [0, 1 l N :
(0 X I --
2
cosig (x) = -- <x <0 (14)
2 I
x 2 0.
Then 3 is dense in L p [ D ] ,for 1 5 p < 00.
Corollary 2: The Fourier network, which is a restricted
form of the cosig network, has the same approximation
capabilities as the cosig network. In the Fourier network,
in order to yield sinusoids corresponding to a Fourier se-
ries, first layer weights winare restricted to fixed integer
values, and thresholds 8; are restricted to values of form
q / 4 where q is an integer.
An interesting feature of a cosig function is that it has
zero derivative outside the interval [ - $, 01. Conse-
quently, a backpropagation learning algorithm would al-
ter only a small subset of neurons for each training pattern
presented to the network. This might be advantageous in
293
preventing the learning of new knowledge from corrupt-
ing previously stored information.
We also observe that learning in a cosig network might
yield something unlike a Fourier series. The first layer
might not compute sinusoidal functions. Another possi-
bility is that the network might compute sinusoids but al-
ter their frequencies. This scenario is equivalent to find-
ing a representation for a function in terms of sinusoids
that are not harmonically related. These variations on
Fourier series might be an efficient way to use computa-
tional hardware.
C. Mod$ed Sigma-Pi and Polynomial Networks
A third kind of network based on exponentials is the
polynomial or modified sigma-pi network. We observe
that powers of x provide another example of (8)
xnxm= exp ( n In x ) exp ( m In x )
= exp [ ( n + m ) In x ] =x"'~. (15)
Rumelhart et al. [ 7 ] have described a sigma-pi neuron
that computes sums of products of selected inputs. A sin-
gle layer of such neurons is incapable of approximating a
function of the form f ( x l , - -
, x N ) = x,4, where q is
large, unless the network allows inputs to be multiplied
by themselves. The modified sigma-pi or polynomial net-
work, described by the next theorem, explicitly provides
the capability of approximating terms of the form xf' * * *
$ where qi is any real number. As has been noted by
Barron and Barron [9] and Hornik et al. [ 4 ] ,this kind of
network satisfies the Stone-Weierstrass theorem. In the
as exponents for the inputs. We can achieve a similar re-
sult by placing a hidden layer of sigma (summation) neu-
rons before the pi (product) neuron.
Theorem 3: Let 3 be the set of all functions that can
be computed by arbitrarily large modified sigma-pi net-
N >
= c wi IT x?:
i=l n=l w;, winE $3
Then 3 is dense in L p [ D ] ,for 1 Ip < 00.
Corollary 3: In the modified sigma-pi or polynomial
network, the synaptic weights winin the hidden layer may
be restricted to integer or nonnegative integer values. A
standard polynomial network is obtained in this way.
D. Exponentiated-Function Networks
A fourth kind of network based on exponentials is the
exponentiated-function network. This network is obtained
by preprocessing inputs to a modified sigma-pi network.
If g is the preprocessing function, the first layer of the
sigma-pi network computes polynomial functions of the
form g ( X l ) " ' ' * ' g(xN)nN.
Theorem 4: Let 5 be the set of all functions that can

294
be computed by arbitrarily large exponentiated-function
networks, on domain D = [ 0 , 1 I N :
f
I N
= c wi
i=l n=l
g(x,)w"':g E C[O, 11 invertible,
w;, Win E 3
1 .
Then 3 is dense in L p [ D ] ,for 1 Ip < 00.
Corollary 4: In the exponential-function network, the
synaptic weights winin the hidden layer may be restricted
to integer or nonnegative integer values.
We note that iff is a logistic function or similar sig-
moid, thenf" is also a sigmoid. Thus, if we wish to com-
pute a single-variable function, we may use a network of
neurons with special sigmoidal squashing functions and
satisfy the Stone-Weierstrass theorem.
E. Partial Fraction Networks
Partial fractions are an example of nonexponential
functions that translate multiplication into addition
1 1
( K G ) (Kx)
1 of two such functions. Since a perceptron with properly
+ (A)
(-).
One might suppose that a similar identity holds when
wix is replaced by Er=, winxn.With this change, how-
ever, the product no longer translates into a finite sum.
Attempts to adapt the partial fraction method to multiple
variables fail, and one may conclude that partial fractions
afford a clever way of computing X " but not x " P . Never-
theless, we could use partial fractions in a network that
computes functions of single variables
M values of zero and one. The hyperplanes define bounda-
F. Modijied Logistic Network
Combining the partial fraction idea with exponentials
produces a network that computes arbitrary functions of
more than one variable. Furthermore, the combination of
partial fractions and exponentials is similar to a logistic
function. The resulting modified logistic network, shown
in Fig. 1, obeys the Stone-Weierstrass theorem:
Theorem 5: Let 3 be the set of all functions that can
be computed by arbitrarily large modified logistic net-
works, on domain D = [0, 1lN:
I I
IEEE TRANSACTIONS ON NEURAL NETWORKS. VOL. I . NO. 4. DECEMBER 1990
3 = if(.'. * ' ,X N )
K
i= I k= I
Then 3 is dense in L p [ D ] ,for 1 Ip < 03.
(17)
Corollary 5: In the modified logistic network, the syn-
aptic weights wiknin the first hidden layer may be re-
stricted to integer or nonnegative integer values. Also,
threshold weights may be added in either the output layer
or first hidden layer, and nonnegative real or integer
weights wikmay be included in the penultimate layer.
The set of functions computed by a modified logistic
network (with logistic squashing function appended at the
output) is a superset of the set of functions computed by
a logistic network. Furthermore, the backward error prop-
agation weight update rules for the modified logistic net-
work are identical to those of a standard logistic network.
Hence, the modified logistic network may be substituted
for a logistic network when one wishes to take advantage
of multiplicative closure.
G. Step Functions and Perceptron Networks
In the 1940's, McCulloch and Pitts [lo] showed that a
network of neurons with stepped squashing functions,
later called perceptrons, could compute any Boolean logic
function. Although its input may be real valued, a net-
work of perceptrons computes only binary functions. We
may, therefore, use an AND gate to compute the product
(18) chosen threshold and synaptic weights is an AND gate,
we can construct a perceptron network that computes fg
from networks that compute f and g.
If we add a linear output neuron to a perceptron net-
work, we have a network that computes step functions
(meaning stair-step functions with multiple steps).
Lippmann [ 111 offers a heuristic description, paraphrased
here, of how such a network with two hidden layers can
approximate an arbitrary function f.
First Hidden Layer: Computes hyperplanes, each of
which divides the input space into half-spaces assigned
ries of convex regions in the input space. Neurons are
added to the first layer until the value off is nearly con-
stant over each convex region.
Second Hidden Layer: Computes AND functions that
determine which convex region an input point lies in.
Output Layer: Multiplies each output from the second
hidden layer by the value off in the corresponding convex
region. In other words, the output of the second hidden
layer identifies the convex region, and the synaptic weight
in the output layer is the value off.
In this heuristic argument lies a proof that, if two net-
works can approximate f and g , we can construct a net-
work that computes fg.

IEEE TRANSACTIONS ON NEURAL NETWORKS. VOL. I . NO. 4. DECEMBER 1990
First Hidden Layer: Includes the hyperplanes for both
f and g.
Second Hidden Layer: Computes AND functions for
every intersection of convex regions from f and g.
Output Layer: Multiplies each output from the second
hidden layer by the value offg in the corresponding con-
vex region.
We cannot apply the Stone-Weierstrass theorem to the
aforementioned network because of a mathematical diffi-
culty: the perceptron network computes step functions
rather than continuous functions. Step functions, how-
ever, are dense in the set of measurable functions by the
following lemma [6].
Lemma ZV-(Step Functions): Iff is an almost every-
where bounded and measurable function on a compact
space, then given E > 0, there is a step function h such
that
If-hl < E (21)
except on a set of measure less than E .
Intuitively, the idea here is that any continuous function
can be accurately approximated by a step function if the
steps are sufficiently small. This procedure introduces yet
another source of error for finite networks. Similar errors
occur in digital-to-analog converters that produce stair-
step approximations of bandlimited functions. In this case,
suitable low-pass filtering removes the error. The same
procedure removes errors for a perceptron network ap-
proximating a bandlimited function.
Although the Stone-Weierstrass theorem does not ap-
ply to our network, there is an advantage to having shown
that we satisfy algebraic closure: we can explicitly con-
struct networks that compute certain polynomial expres-
sions. This is partial compensation for the lack of a mul-
tilayer perceptron learning algorithm [ 121.
V . CONCLUSION
In the first part of this paper, we observed that errors
which approach zero in the limit of large networks may
be significant in finite networks. Two possible types of
errors are overshoot and slow convergence. A third type
of error may arise in two-layer networks: the network may
be unable to efficiently approximate the product of two
functions.
I
295
These difficulties are partially resolved by requiring
networks to satisfy the hypotheses of the Stone-Weier-
strass theorem, as do the networks presented in this paper.
These networks are derived by transforming multiplica-
tion into addition. The transformation is accomplished via
three types of squashing functions: exponentials, partial
fractions, and step functions. Networks based on these
functions satisfy the algebraic closure hypotheses of the
Stone-Weierstrass theorem, allowing us to explicitly con-
struct networks that compute polynomial expressions of
functions computed by smaller networks.
A combination of partial fractions and exponentials
produces the modified logistic network. This network is
similar to a logistic network but satisfies the Stone-
Weierstrass theorem directly. The modified logistic net-
work is suggested as an alternative to the logistic network
in situations where one wishes to compute polynomial
expressions.
REFERENCES
111 G. Cybenko, “Approximations by superpositions of a sigmoidal
function,” Marh. Contr., Signals, Syst., vol. 2 , pp. 303-314, 1989.
121 K. Funahashi, “On the approximate realization o f continuous map-
pings by neural networks,” Neural Networks, vol. 2 , pp. 183-192,
1989.
[3] A . R. Gallant and H. White, “There exists a neural network that does
not make avoidable mistakes,” in Proc. IEEE Inr. Conf. Neural Net-
works, San Diego, CA, July 24-27, 1988, vol. I , pp. 657-664.
[4] K. Hornik, M. Stinchcombe, and H . White, “Multilayer feedfonvard
networks are universal approximators,” Neural Networks, vol. 2, pp.
359-366, 1989.
[5] L. V . Kantorovich and G. P. Akilov, Functional Analysis, 2nd ed.
Oxford: Pergamon, 1982.
[6] H. L. Royden, Real Analysis, 2nd ed. New York: Macmillan, 1968.
171 D . E. Rumelhart, G. E. Hinton, and R. J . Williams, Parallel Dis-
fributed Processing: Explorations in the Microstructures of Cogni-
rion. Cambridge, MA: M.I.T. Press, 1986.
[8] J . J . Hopfield, “Neurons with graded response have collective com-
putational properties like those of two-state neurons,” in Proc. Nar.
Acad. Sci. USA, vol. 81, pp. 3088-3092, 1984.
[9] A. Barron and R. Barron, “Statistical learning networks: A unifying
view,” presented at the Symp. Interface: Statistics Comput. Sci.,
Reston, VA, 1988.
[IO] W. S . McCulloch and W. Pitts, “A logical calculus of the ideas im-
manent in nervous activity,” Bull. Math. Biophys., vol. 5 , pp. 115-
133, 1943.
[111 R. P. Lippmann, “An introduction to computing with neural nets,”
IEEE ASSP Mag., pp. 4-22. Apr. 1987.
[12] M. Minsky and S. Papert, Perceprrons. Cambridge, MA: M.I.T.
Press, 1969.