A Nonlinear Time-Varying Channel Equalizer

Using Self-organizing Wavelet Neural Networks

Cheng-J ian Lin* Chuan-Chan Shih' Po-Yueh Chen'
* Department of Computer Science and Information Engineering
Chaoyang University of Technology
+ Department of Networking and Communication Engineering
Chaoyang University of Technology
168 Gifeng E. Rd., Wufeng, Taichung County, 413 Taiwan, R. 0. C.
E-mail: cjlin@mail.cyut.edu.tw
Abstract
This paper describes the self-organizing wavelet neural
network (SOW") for nonlinear time-varying channel
equalizers. The SOWNN model has a four-layer structure
which is comprised of an input layer, a wavelet layer, a
product layer and an output layer. The derivative online
learning algorithm involves two kinds of leaming. The
structure learning is performed to determine the network
structure and the parameter learning is to adjust the shape
of the wavelet bases and the connection weights of a
SOWNN. The proposed equalizer is enhanced in order to
handle the highly nonlinear functionality. Computer simu-
lation results show that the bit error rate of the SOWNN
equalizer is very close to that of the optimal equalizer.
Keywords:
Time-valying channel, Wavelet neural network, Equalizer,
Additive white Gaussian noise, Back-propagation
1. lntroduction
During the past few years, applications of high-speed
communication are required and fast increasing. High level
equalizers become more desirable to cope with the need for
high-speed data transmission. Nonlinear distortion becomes
a major factor which limits the performance of a communi-
cation system. Another important factor is the structure of
the adaptive equalizers. The received signal is often irn-
paired by the channel inter-symbol interference (ISI), the
additive white Gaussian noise (AWGN) [ I ] and the effects
of time-varying channels [ 2] . All these effects are nonlinear
and complex problems. Nevertheless, adaptive equalizers
are used in digital communication system receivers to miti-
gate the effects of nonideal channel characteristics.
It is well known that a nonlinear filter is required for
satisfactorily controlling an equalization plant. The nonlin-
ear channel equalizer design may be regarded as a problem
of nonlinear function approximation. Among all different
nonlinear techniques, methods based on artificial neural
network (ANN) have been grown into a popular research
topic in recent years [3]-[5].
Gibson, Siu and Cowan proposed a multilayer percep-
tron (MLP) based equalizer [ 6] . The MLP structure is less
sensitive to learning gain variation and capable of converg-
ing to a lower value of mean square error. Despite of pro-
viding considerable performance improvements, MLP
equalizers are still problematic in terms of their conver-
gence performance and complex structure. A suitable ap-
proach to overcome the disadvantages of global approxi-
mation networks is the substitution of the global activation
function with localized basis functions. From a function
representation perspective, a radial basis function (RBF)
network is a scheme that represents a function using locally
supported functions. One of the typical hcti ons used by
RBF is the Gaussian function. The locality of the functions
makes RBF more suitable for learning functions with local
variations and discontinuities. This means that RBF can
represent any function that is in space spanned by the hasis
functions [3, 71.
Following the concept of locally supported basis func-
tions such as RBF, a class of Wavelet Neural Networks
(WNN) which originates from wavelet decomposition in
signal processing has become more popular lately [SI.
Wavelets are a class of basic elements with oscillations of
effectively finite duration that makes them look like "little
waves". The self-similar, multiple resolution nature of
wavelets offers a natural framework for the analysis of
physical signals and images. Concrete theory has been fully
developed to guarantee the global stability of neural net-
based communication schemes as well [SI. In comparison
with the Gaussian function, wavelets are suitable to depict
functions with local nonlinearities and fast variations be-
cause of their intrinsic properties of finite support and self-
similarity. The wavelet-related analysis has attracted much
attention from engineers and mathematicians [9]. Further-
more, Daubechies [lo] have demonstrated the existence of
orthonormal wavelet bases. It improves the rates of con-
vergence for the approximation using wavelet-based net-
works. With wavelet's specific features, the wavelet hasis
function network is considered more efficient than the con-
ventional neural networks in approximating nonlinearities
appeared in servo systems. Recently, wavelets have also
0-7803-8359-1/04/$20.00 02004 IEEE 2089
been combined with the neural network to create a wavelet
basis function network. A special class of functions, known
as wavelets, possess good localization properties while the
functions are simple orthonormal bases. Thus, the WNN
can be applied to highly nonlinear systems.
In this paper, a Self-organizing Wavelet Neural Net-
work (SOWNN) model is proposed. It is a four-layered
network structure comprised of an input layer, a wavelet
layer, a product layer, and an output layer. Based on the
self-learning ability, the on-line stmcture/parameter learn-
ing algorithm is performed concurrently i n the S OW". In
the structure learning scheme, degree measure method is
used to find the proper wavelet bases and to minimize the
number of wavelet bases generated from the input space. In
the parameter learning scheme, the Back-Propagation (BP)
method is applied to adjust the shape of wavelet functions
and the connection weights in SOW". Finally, the pro-
posed SOWNN equalizer is applied to a time-varying
channel for comparison with Bayesian and MLP equalizers.
The paper is organized as follows. In Section 2, the
SOW" structure and functionality is described. The
learning algorithm is presented in Section 3. Simulation
results and discussions can be found in Section 4. Section 5
gives the conclusion.
2. The Structure of SOWNN
The structure of the proposed SOWNN model is
shown in Figure I . It is designed as a four-layer structure
which is comprised of an input layer, a wavelet layer, a
product layer, and an output layer. The input data in the
input layer of the network is x=[ xI , x2, ..... %I 2
where n is the number of dimensions. The input data are
directly transmitted into the wavelet nodes in the wavelet
layer. For the discrete wavelet transform, mother wavelet
((X) with dilation of a and translation of b is described as
follow:
These functions constitute an orthonormal basis of a func-
tional space (namely, LZ(Yid)). In this paper, the Mexican-
hat function (see Figure 2) is adopted as the mother wave-
let. It is the second derivate of the Gaussian function and
expressed as follow:
Therefore, the activation function of the j th wavelet node
connected with the i th input data is represented as:
i =l , . . , n j =l , . . , nr ( 3)
Using this equation, we can easily adjust the parameter a
and b appropriately. In equation (3), n is the number of
input-dimensions and m is the number of the wavelets.
Each wavelet in the product layer is labeled II, i.e., the
product of the j th multi-dimensional wavelet with n input
dimensions of Xi . The relation is defined as
(4)
According to the theory of multi-resolution analysis
(MRA), for any/ EL*(%), it can be regarded as a linear
combination of wavelets at different resolution levels. For
this reason, the functionfis expressed as
m
Y( X) =f ( x ) = Wj Wj ( X) ( 5 )
j =l
Where v j =[I,v,,~, ,___, w,] is used as a nonlinear
transformation function of hidden nodes, and
wj =[ wI , w2, _. ., w, ] is used as the adjustable weighting
parameters to provide the function approximation. Note
that, in addition to Wj , the dilation and translation factors
a and b may also be regarded as adjustable parameters.
Applying Eq. ( 5 ) , we can expression the SOW" model-
ing function Y.
Input Wavekt PI c dKt output
Figure 1: The architecture of the SOW" model
2090
Figure 2: Translated and dilated Mexican-hat Iunction
3. The Learning Algorithm for SOWNN
The proposed approach allows SOW" model to
grow during training by gradually increasing the number of
prototypes that represent the feature vectors in the training
set. The prototypes play the role of the centers of the wave-
let basis functions. The degree measure method is used to
decide the number of wavelet bases in the wavelet layer
and the product layer. On the other hand, the BP learning
algorithm is used for adjusting the center, dilation, and the
connection weight parameters. The details of the algorithm
are presented below.
3.1 The Structure Learning Scheme
Initially, there is only one wavelet base in the
SOW" model. The first task is to decide when we need a
new wavelet base to be glowing. For each incoming pattern
x i , the firing strength of a wavelet base can he regarded as
the degree of the incoming pattern belonging to the corre-
sponding wavelet base. Based on this concept, the firing
strength obtained from equation (4) in the product layer can
be used as the degree measure
p, =) y j l j =1, ...., q
(6)
Where q is the number of existing wavelet bases. Accord-
ing to the degree measure, the new wavelet base increased
in accordance with the new input data xi. We set the
threshold value to be Tas follows:
T =max F,
11jQ
( 7)
If there is an Fj which is greater than T, then a new wave-
let base is generated, where T is a pre-specified threshold
that should decay during the learning process. The decay of
threshold value limits the size of the SOWNN model.
bq+l =xi (8)
aq+l =0.1 (9)
wq+] =random value (10)
Where xi is the new incoming data; the connection weight
wq+, of the output layer is selected fiom the range be-
tween -1 and l randomly and the dilation avtl is set to the
0.1 to obtain a higher firing strength for the input value xi .
3.2 The Parameter Learning Algorithm
The well-known back-propagation learning algorithm
is used for the supervised learning to find the output errors
of the node in each layer and perform parameter adjustment.
The goal is to minimize the error function
e =A x ) - Y" 4
(11)
1 2
and E =-e
2
Where y d( t ) is desired output, y( t ) is the model output
and E is the cost function. The parameter learning algo-
rithm based on back-propagation is performed as follows:
Assuming that w is the connection weight of the output
layer parameter in a node, the generally used learning rule
is
wj(t +1) =wj ( t ) +Awj (13)
Wherer] is the leaming rate.
According to the Chain rule, AW can be decomposed as
Similarly, the updating laws of a, and b, are shown as
follows:
a,(t +1) =a,(t) +Aa, (15)
b,(t +1) =b,(t) +Ab,
Where
And
209 I
2
X, - b,
2
=qewj4j exp(- I *
5
--
uy *( x j -by)(3 - (xi - by)*u;2)
4. Illustrative Examples
In this section, an example of a communication system
is given to demonstrate the validity of the presented
SOW". Because High speed communication channels
are oRen impaired by the channel Inter-symbol Interference
(ISI), the Additive White Gaussian Noise (AWGN) and the
effects of time-varying channels, we design a SOW"
nonlinear channel equalizer to reduce those noise and dis-
tortion.
Figure 3: Discrete-time model of a data transmission system
4.1 A digital communication system with AWGN
A discrete time model of a digital communication
system is depicted in Figure 3. The transmission input sig-
nal x(n) is a sequence of statistically independent random
binary symbols taking valuesx(n) E {-l,l}. The nonlinear
channel output is described as
s ' ( n) =H( n ) @x ( n ) (21)
where H(n) is the time-varying channel characteristic and
@ is the convolution. The task of the equalizer is to ob-
serve the information sequence s[nJ,s[n-l]. ..., s[n-m+IJ
and reconstruct the transmitted signal x ' [ n - d] (where
the term m is the equalizer length and the equalizer order
can be considered as m-1). The goal of design is that
greater speed and higher reliability can be achieved. The
equalizer provides a decision function as
x' [n -4 =f{+)} (22)
The observed signal s[n] is the channel output s' [ n] cor-
rupted by AWGN e(t) , that is
s [ n] =s ' [ n ] +e[ n ] (23)
It is passed through a decision device to provide an esti-
mate of the transmitted signal (d is the delay associated
with the equalizer decision).
Theperformance of the equalizer is determined by the
probability of misclassification with respect to certain sig-
nal-to-noise ratio (SNR). With the assumption of inde-
pendent identically distributed (i.i.d.) sequence, the SNR
can be defined as
4.2 The Bayesian channel equalizer
The Bayesian decision theory provides the optimal solution
to the general decision problem. Therefore, the optimal
symbol-by-symbol equalizer can be formed from the
Bayesian probability theory and is termed a Bayesian or
maximum a posteriori probability (MAP) equalizer. Each
channel state has equal occurrence probability. The channel
states can be partitioned into two classes. The decision out-
put of a Bayesian equalizer can be expressed as
The minimum error probability decision can be rewritten as
where oe denotes the standard deviation (std) of
the Gaussian additive noise e@). For equiprobable symbols,
the coefficients pj ( 2z u; ) T in f , ( s( n) ) become redun-
dant and can be removed. This gives rise to the following
simpler form of the optimal decision function.
-m
where C,; and C; refer to the channel states which are +I
and -1 respectively. They represent the estimates of the
noise-free received signal vector. Equation (27) can be
further simplified as
Where T=+l i f ci Ec,?and y=- l i f $E $.
Therefore, we can make the optimal decision boundary
base on this function. Formthis point of view, the equalizer
can be regarded as a classifier and the communication
problem can be considered as a classification problem. In
order to get the better performance, we must expend more
cost. If there are 1000 samples of s(n). every input data will
be processed with those 1000 samples. The compute is very
complex and time-costing inBayesian Equalizer. Based on
2092
Table 1 Channel States for A Time-Varying Channel Model With Binary Symbols, d=O
7 1 - I I - I I -al (n)-aZ(n)-0.9[-al (n)-aZ(n)1"3
8 1 - I I - I I - I 1 - ~i ( ~) - a~( ~) - 0. 9~- ~1( ~) - ~2( n) 1"3 I -aI(n)-aZ(n)-O.P[-al (n)-aZ(n)]"3
the proposed SOW", wecan obtain the correct signal by a
simple mathematical equation. For the purpose of graphical
display, the equalizer order is chosen to be m=2. Let the
nonlinear time-invariant channel transfer function be
s' ( n) =a&) +a,x(n - 1)
4~. 9[ u, x( n) +U& -I)? +e(k) (29)
Where a, =1 and az =OS as shown in Figure 4(a). The
channel states are plotted in Figure 4(c). Since we assume
the channel is time-varying, a, , a, are two time-varying
coefficients. Those time-varying coefficients are generated
by passing the white Gaussian noise through a Butteworth
low-pass filter (LPF). The example is centered at U, =I and
a,=0.5 and the input for the Butterworth filter is a white
Gaussian sequence with standard deviation (std) p. Apply-
ing the function provided by the Matlab Signal Processing
Toolbox, we can generate a second-order lowpass digital
Butteworth filter with cutoff frequency p4. l . The main
... ...~..., ~ ..................
.;..... i... ,!.
Figure 4: For the channel (a) time-invariant channel with
a, -1 and a2=0.5. (b)An example of a timevarying channel
with p4.l. (c) Channel states (noise free) of time invariant
channel. (d) Channel states (noise free) of the time-varying
channel.
Figure 5: Time-varying channel, data clusters, SNR=lOdb,
p =0.1, 1000 samples of s(n), and decision boundary.
Figure 6 Comparison of bit-error-rate curves for tbe Bayes-
ian, SOW", MLP equalizer, in time-varying channel with
p =0.1.
2093
Figure I: Comparison of bit-error-rate curves for the Bayes-
ian, SOW , MLP equalizer, in time-varying channel with
SNR=ZOdB, p =0.04 to 0.32.
Adjusting the time-varying coefficients, the coefficients
and the corresponding channel states are plotted in Figure
4(b) and (d) respectively. Notice that the channel states are
eight clusters instead of eight individual points. Applying
equation (28), we can determine the optimal decision
boundary. In Figure 5, the shaded region is the region
where the transmitted signal is classified as 1. The other
region is where it is classified as -1. This way of decisions
making is optimal in the sense that it produces the mini-
mum average error probability or bit error rate. As shown
in the caption, The simulation conditions for Figure 5 are:
noise of IOdb, 1000 samples ofs(n) and p =0.1.
4.3 Comparison of bit-error-rate
We now use the SOWNN as an equalizer to solve the
communication time-varying channel problem with p= 0.1.
To see the actual bit-error-rate (BER), a realization of lo6
points of sequence x(n) and e@) is applied to test the BER
of a trained SOWNN equalizer. The resulting BER curve
of the SOWNN equalizer under different SNR values be-
tween 2dB to 16dB is shown in Figure 6. In the next ex-
periment, we fixed SNR at 20 dB and ran simulations for
eight different /3 values ranging from /3=0.4 to p =0.32,
with step size 0.04 (set d =0). The result is shown in Fig-
ure 7.
According to the results shown in Figure 7, we com-
pare the performance of our model with that of the other
existing methods. The MLP is a simple and popular method.
Its basic idea is to represent the input vectors with a smaller
set of prototypes that provide a good approximation to the
input space. The Bayesian equalizer is more complex and a
near optimal method for communication channel equalizers.
Figure 6 shows all the BER curves for the optimal Bayes-
ian equalizer, the S OW equalizer and the MLP equal-
izer. Simulation results show that the proposed SOW
model always outperfom the MLP and has better per-
formance near Bayesian in high signal noise rate (SNR).
5. Conclusion
We have implemented a new elegant SOWNN equal-
izer whose performance is very close to that of the optimal
Bayesian with substantial reduction in computational com-
plexity. The proposed SOW can elevate the converging
speed for data communication using nonlinear time-varying
equalizers and others applications.
Acknowledgement
This research was supported by National Science
Council of the R.O.C. under grant NSC 92-2213-E-324-
002.
Reference
[l ] S. K. Patra and B. Mulgrew, Fuzzy techniques for
adaptive nonlinear equalization, Elservier Science
Signal Processing, Vol. 80, pp.985-1000,2000.
[2] Qilian Liang and J erry M. Mendel, Equalization of
nonlinear time-varying channels using type-2 fuzzy
. - - ..
adaptive filters, IEEE Transactions on Fuzzy Systems,
Vol. 8, No. 5 , October 2000.
Sheng Chen, Bernard Mulgrew, and Peter M. Grant,
A clustering technique for digital communications
channel equalization using radial basis function net-
works, IEEE Transaction on Neural Networks, Vol, 4.
No, 4. J uly 1993.
J agdish C. Patra and Ranendra N. Pal A functional
link artificial neural network for adaptive channel
equalization, Elsevier Science B. V. Signal Processing,
Vol. 43, pp.181-195, 1995.
G. Kechriotis and E. S. Manolakos, Using recurrent
neural networks for adaptive communication channel
equalization, IEEE Transaction on Neural Networks,
Vol. 5, No. 2, 1994, pp. 267-278.
G. J . Gibson and C. G. N. Cowan, On the decision
regions of multilayer perceptrons, Proceedings of the
IEEE, Vol. 78, pp.1590-1594, October 1990.
B. Mulgrew, Applying Radial Basis Function, IEEE
Signal Processing Magazine , pp. 53-63, MARCH
1996.
P. Cristea, R. Tuduce, and A. Cristea Time Series
Prediction with Wavelet Neural Networks, Proceed-
ings of IEEE Neural Network Applications in Electri-
cal Engineering, 2000, pp. 5-10,
1. Zhang, G. G. Walter, Y. Miao, and W. N. W. Lee,
Wavelet Neural Networks for Function Learning,
IEEE Trans. Signal Processing, Vol. 43, ~1485-1497,
J une 1995.
[ 1011.. Daubechies, The wavelet transform, time-
frequency localization and signal analysis, IEEE
Transforms on Information Theoty, Vol. 36, No. 5, pp.
961-1005, 1990.
2094