AbstractIn this paper, to mitigate the tradeoff between fast

convergence rate, low steady-state misadjustment and good

tracking ability, a novel, easy to implement, time-varying
step-size normalized least mean square (NLMS) algorithm-based
transversal filters is presented in system identification
applications. By utilizing the system input power and
cross-correlation between the input signal and estimated error,
the new variable step-size scheme can reduce the effect of the
system noise on the performance without the priori knowledge of
system noise power, especially for variable system noise.
Experimental results in context of system identification illustrate
that the propose algorithm is superior to other existing
algorithms in terms of convergence speed, misadjustment and
tracking ability.
Index Termsadaptive filters, normalized LMS (NLMS),
variable step-size, power estimation.
I. INTRODUCTION
DAPTIVE filtering is now widely used in many signal
processing application areas such as system identification,
control, noise cancellation and communications[1][2]. As we
all known, the normalized least-mean-square (NLMS)
algorithm[1] has become the most popular adaptive algorithm,
attributing to its simplicity , stability and easy to implement.
However, it must take into account a tradeoff among low excess
mean-square error (EMSE), fast convergence rate and good
tracking ability, due to the effect of the fixed step-size. Much
effort from the following aspects has been directed to relieve
this problem.
One way employs a variable step-size (VSS) scheme in
NLMS algorithms [3-9], based on the fact that the step-size
decreases as the magnitude of feedback error (the difference
between the desired output and adaptive filter output) decreases.
However, these algorithms still have defects in practical
applications. In [3], the update of step was subject to
disturbance of the noise, especially in the high signal-to-noise
(SNR) environment. References [4-6] eliminate this
disturbance such that the steady-state error is reduced, but the
tacking ability [4][6] is poor in vary-time system, [5] need

This work was partially supported by National Science Foundation of P.R.
China (Grant: 61271340 and 61071183), the State 863 projects
(Grant:2011AA110501), the Sichuan Provincial Youth Science and
Technology Fund (Grant: 2012JQ0046), and the Fundamental Research Funds
for the Central Universities (Grant: SWJTU12CX026 and SWJTU11ZT07)
Haiquan Zhao and Yi Yu are with the School of Electrical Engineering at
Southwest Jiaotong University, Chengdu, 610031, China. (e-mail:
hqzhao@home.swjtu.edu.cn).
*
Corresponding author.
know the power of the input signal and the tuning of
parameters[6] is very difficult to control in real-world
applications. To solve the problem, some relatively tuning-free
nonparametric algorithms were proposed [7-9], but their
performance depends on the priori knowledge of the system
noise power.
Another approach is the normalized least mean-fourth
(NLMF) algorithm by modifying the cost function [10-13],
which consists of minimizing the fourth power of the error.
However, the proposed NLMF algorithm in [10] is instability,
due to its stability is not independent of the input signal
power[11]. Considering a variable step-size NLMS algorithm
an global stability LMF (GS-LMF) algorithm was presented
[12][13]. However, the convergence rate of this algorithm
speeded up as SNR decreases, which is unfavorable in practical
application.
To better solve this inherent problem of the NLMS
algorithm, this paper presents a new VSS-NLMS algorithm,
which employs the idea of [14]. By using the system input
power and cross-correlation between the input signal and
system estimated error, the novel variable step-size strategy can
reduce the effect of the system noise on the performance
without the priori knowledge of system noise power, especially
for variable system noise. Simulations in the context of system
identifications show that the proposed algorithm performs
much better than the existing algorithms [7-9], with fast
convergence, low misadjustment and good tacking ability.
II. THE ADAPTIVE FILTER MODEL
( ) x n ( ) d n
( ) y n
( ) e n

( ) n
o
w
( ) n w
Fig. 1. Block diagram of an adaptive system for system identification.
A well known adaptive system model for system
identification is depicted in Fig. 1. The desired response
( ) d n of the adaptive filter is given as
( ) ( ) ( )
T
o
d n n n = + w x (1)
Haiquan Zhao
*
, Member, IEEE, Yi Yu
Novel adaptive VSS-NLMS Algorithm for
System Identification
A
2013 Fourth International Conference on Intelligent Control and Information Processing (ICICIP)
June 9 11, 2013, Beijing, China
978-1-4673-6249-8/13/$31.00 2013 IEEE 760

where n is time index, superscript T indicate transposition of
matrix or vector,
o
w denotes the weight coefficient vector of
the unknown system with the memory length M, i.e.,
[ ]
0 1 1
T
o o o oM
= w w w w (2)
and ( ) n x is the input vector of the system at n time
[ ] ( ) ( ) ( 1) ( 1)
T
n x n x n x n M = + x (3)
Note that ( ) n is the system noise which is zero-mean
Gaussian white noise with variance
2

, and is independent of
( ) n x .
In this brief, we assume that all signals are real-valued. Then,
the update of weight vector ( ) n w by employing the
conventional NLMS algorithm with a fixed step-size of is
described as follows
( ) ( )
( 1) ( )
( ) ( )
T
e n n
n n
n n
+ = +
+
x
w w
x x

(4)
where ( ) e n denotes the system output error
( ) ( ) ( ) e n d n y n = (5)
the output ( ) y n is expressed as
( ) ( ) ( )
T
y n n n =w x (6)
and 0 > is a very small regularization parameter to avoid
numerical divergence of algorithm. The step-size is
bounded by zero and two for guaranteeing the stability of the
algorithm. However, the choice of the step-size reflects a
tradeoff among steady-state error, the speed and tracking ability,
i.e., a small step-size gives small steady-state error but also a
longer convergence time constant and poor tracking ability.
III. NOVEL VSS-NLMS ALGORITHM
In order to solve the inherent problem of the conventional
NLMS algorithm, keeping the view in [14], we propose a new,
easy to implement, NLMS algorithm with a time-varying
step-size ( ) n which is represented as follows
2
( )
max max min
( ) ( )
c
n
n e



= + (7)
where
min max
0 < < are chosen to provide a minimum level
of tracking ability and to ensure stability of the algorithm,
respectively. And the parameter is a positive number which
adds some flexibility of algorithm design.
In (7),
2
( )
c
n is the power of the true instantaneous error
( ) c n which can be obtained by subtracting system noise ( ) n
from the error ( ) e n , i.e.,
( ) ( ) ( ) ( ) ( )
T
o
c n e n n n y n = = w x (8)
To greatly reduce the disturbance of system noise ( ) n with
respect to the update of step-size,
2
( )
c
n can be obtained by
effective power estimation technology [9], and as described
below:
2
2
1
( ) ( ) ( )
( )
T
c ex ex
x
n n n
n

(9)
where
2
( )
x
n is the power of the input signal ( ) x n ,
( )
ex
n denotes the cross-correlation between input vector
( ) n x and the error ( ) e n , and is defined by
( ) [ ( ) ( )] ( ) ( )
ex xx
n E e n n n n = = x R w (10)
where [ ] E is the mathematical expectation, ( )
xx
n R is the
autocorrelation matrix [ ( ) ( )]
T
E n n x x of the input signal ( ) n x ,
and ( ) n w is the difference between
o
w and ( ) n w .
Since the values of
2
( )
x
n and ( )
ex
n in (9) are exact and
not available in practice, they can be estimated by using the
exponential recursive weighting algorithm.
2 2 2
( ) ( 1) (1 ) ( )
x x
n n x n = + (11)
( ) ( 1) (1 ) ( ) ( )
ex ex
n n e n n = + x (12)
where is a exponential weighting factor which is generally
set to (0.9,1) for obtaining superior estimation
performance of
2
( )
c
n . As previously mentioned, the proposed
VSS-NLMS algorithm is summarized as Table I .
TABLE I THE PROPOSED VSS-NLMS ALGORITHM
Initialization 2
(0) 0, (0) 0, (0) 0
ex x
= = = w
Parameters
max min
, , , ,
Loop For n = 1,2,3,
( ) ( ) ( )
T
y n n n = w x
( ) ( ) ( ) e n d n y n =
( ) ( 1) (1 ) ( ) ( )
ex ex
n n e n n = + x
2 2 2
( ) ( 1) (1 ) ( )
x x
n n x n = +
2
2
1
( ) ( ) ( )
( )
T
c ex ex
x
n n n
n

=
2
( )
max min max
( ) ( )
c
n
n e



= +
( ) ( )
( 1) ( ) ( )
( ) ( )
T
e n n
n n n
n n
+ = +
+
x
w w
x x

End
IV. PERFORMANCE ANALYSIS
A. Convergence Analysis
In the early stage of training of the proposed algorithm, due
to the system mismatch,
2
( )
c
n is generally large value. So then
2
( )
c
n
e

is approximately equal to zero, and the adaptive filter
gets a large step-size
max
( ) n from (7) to result in
speeding up the convergence. When the algorithm starts to
converge,
2
( )
c
n becomes smaller such that
2
( )
c
n
e


becomes increasingly close to one, and then the step-size ( ) n
gets smaller. When the adaptive filter converges to the
761

optimum solution,
2
( )
c
n is pretty close to zero, leading to a
constant step-size
min
( ) n = to obtain a lower steady-state
error. The parameter which reflects the degree of the
step-size with respect to change of real error ( ) c n , and then
controls the EMSE of the algorithm, will be investigated in the
next section. Note that the contribution of system noise to
( ) n is balanced as we utilize
2
( )
c
n in (11).
B. EMSE analysis
We can firstly assume the misadjustment is small and the
regularization parameter is much smaller. And then utilizing
the result of [15-16], we can obtain the misadjustment of the
conventional NLMS with a constant step-size , which is
expressed as
min
2
ex

= =

(13)
where
min
denotes the minimum MSE, and
ex
represents the
steady-state value of the EMSE. Since
0
lim 1
x
x
e x

= , the
step-size in (7) converges to
2
2
max max min
max min min
( ) ( )
( )
( )(1 )
( )
c
c
=
=
+
+

(14)
Substituting (14) and
2
min
= into (13), we have
2
( )
2 ( )
ex

=

(15)
Because
2
( )
ex c
, (15) can be simplified as follows
2 2
min
0
ex ex
a b

+ + = (16)
where
2
max min min
2 ( ) b

= + (17)
max min
( ) a = (18)
Then, the quadratic discriminant of (16) is calculated by
2 2
2 2
min
max min min min
4
2 )] 8 [( ) (
b a

=
+ =
(19)
Since 0 , the parameter satisfies
2 2
2 2
min min
max max min min
( 2 ) ( 2 )
(0, ] [ , )
( ) ( )

+


(20)
By solving the quadratic equation (16), we can obtain the
EMSE
ex
of the proposed VSS-NLMS algorithm as follows:
2 2 2 2
min min
4 4
min{ , }
2 2
ex
b b a b b a
a a
+
=

(21)
We also study the system distance that the Euclidean norm is
defined as the difference between ( ) n w and
o
w , i.e.,
2
( ) ( )
o
MSD n E n

=


w w (22)
When the conventional NLMS algorithm with step-size
(0, 2) and the regularization parameter 0 > converges to
the steady-state, the system distance is approximated as [16].
2
2
( )
(2 ) 2
x
M
MSD
M



=
+
(23)
Assuming that
2
(2 ) 2
x
M >> , then
2
2
( )
(2 )
x
MSD

(24)
Substituting (13) and (21) into (24) yields
2
1
( )
ex
x
MSD

(25)
V. SIMULATIONS
In the simulation of system identification, the unknown
system is assumed to be FIR with a length of 10, and its weight
vector
o
w is randomly generated, except for occurring sudden
change at 1000 point. Initial weights of all algorithms are set to
be zero, and the choice of other parameters is shown as Table II
[7-8]. We use the normalized misalignment (in
dB),
10
2 2
20log ( ( ) / )
o o
n w w w , to measure the
performance of the algorithms. All the simulation plots are
result of one running.
TABLE II SIMULATION PARAMETERS OF THE STEP-SIZE UPDATE FORMULAS OF
FOUR VSS ALGORITHMS
Algorithms Parameters
NPVSS[7]:
2 5
0.985, 0.01, 10
v

= = =
NVS-NLMS [8]:
2 5
0.985, 0.01, 10
v

= = =
NEW-NPVSS [9]: 0.995, 0.01, 0.1
th
= = =
Our algorithm:
max min
0.01, , 0.0001 1.2 = = =
0.975, =
20 =
A. Effect of varying
To understand the effect of varying for EMSE of the
proposed method, we use five different values (5, 15, 20, 30,
and 50) of in this experiment. The input signal is a white
Gaussian process with
2
1
x
= and system noise power
is
2
0.01

= . Fig. 2 shows the misalignment curves of the

proposed algorithm with varying . It is clearly observed that
the proposed algorithm in the range 15-30 has the advantages of
both fast convergence rate and low steady-state error. Therefore,
in the simulation results that are presented in Fig. 2, optimal
value of is set to 20.
762

0 200 400 600 800 1000 1200 1400 1600 1800 2000
-45
-40
-35
-30
-25
-20
-15
-10
-5
0
5
10
Iteration number
M
i
s
a
l
i
g
n
m
e
n
t
(
d
B
)
5
15
20
30
50

Fig. 2 Effect of varying for system noise power
2
0.01 =

.
B. Case 1: white Gaussian input
0 200 400 600 800 1000 1200 1400 1600 1800 2000
-50
-40
-30
-20
-10
0
10
20
Iteration number
M
i
s
a
l
i
g
n
m
e
n
t
(
d
B
)
NPVSS [7]
NEW-NPVSS [9]
NVS-NLMS [8]
Proposed algorithm

Fig. 3 Misalignment of four VSS-NLMS algorithms for system noise power
2
0.01 =

.
In this case, the input signal is the same to the first
experiment. The comparable results with other existing
algorithms [7-9] are shown for system noise power
2
0.01

= in Fig. 3. Fig. 4 depicts the evaluated result of these

algorithms in the scenario that system noise changes as follows:
Noise power is 0.01 in iteration number (0, 1000), and increases
to 0.09 at 1000.
0 200 400 600 800 1000 1200 1400 1600 1800 2000
-40
-30
-20
-10
0
10
20
Iteration number
M
i
s
a
l
i
g
n
m
e
n
t
(
d
B
)
NPVSS [7]
NEW-NPVSS [9]
NVS-NLMS [8]
Proposed algorithm

Fig. 4 Misalignment of four VSS-NLMS algorithms for system varying noise.
Obviously, it has shown from Figs. 3 and 4 that the proposed
algorithm has lower misalignment than others. Moreover, it has
good tracking ability when the change of the weight vector of
unknown system occurs. Furthermore, it is observed that the
propose algorithm still outperforms other algorithms [7]-[9]
when system noise power increases from 0.01 to 0.09.
Moreover, it does not need to know system noise power.
C. Case 2: AR(1) input
In this experiment, the input signal is an AR(1) process
generated by filtering a white Gaussian noise with
2
1
x
=
through a first order system
1
1/ (1 0.5 ) z

[7-8]. The results

are given in Fig. 4 (system noise power
2
0.01

= ), and Fig. 5
which describes the evaluated result of these algorithms in the
scenario that system noise changes as follows: Noise power is
0.01 in iteration number (0, 1000), increases to 0.09 at 1000.
0 200 400 600 800 1000 1200 1400 1600 1800 2000
-45
-40
-35
-30
-25
-20
-15
-10
-5
0
5
10
Iteration number
M
i
s
a
l
i
g
n
m
e
n
t
(
d
B
)
NPVSS [7]
NEW-NPVSS [9]
NVS-NLMS [8]
Proposed algorithm

Fig. 5 Misalignment of four VSS-NLMS algorithms for system noise power
2
0.01 =

.
The results obtained from Figs. 5 and 6 have shown that the
proposed algorithm still has less misalignment than the
algorithms in [7-9] when all the algorithms converge to the
steady-state with the same convergence rate for related input
signals ( e.g., AR(1) input ), and it can well track the changes of
unknown system weight vector. Although the proposed method
doesnt need know system noise power, the proposed method
has faster convergence rate under the condition of same
misalignment.
763

0 200 400 600 800 1000 1200 1400 1600 1800 2000
-45
-40
-35
-30
-25
-20
-15
-10
-5
0
5
10
Iteration number
M
i
s
a
l
i
g
n
m
e
n
t
(
d
B
)
NPVSS [7]
NEW-NPVSS [9]
NVS-NLMS [8]
Proposed algorithm

Fig. 6 Misalignment of four VSS-NLMS algorithms for system varying noise.
As a result, the main reason we achieve a better performance
is fact that the algorithms in [7-8] depend on the understanding
of the system noise power, and only apply to unchanged system
noise power environment, while the proposed algorithm by the
new variable step-size strategy is not very sensitive to noise
power change, especially for variable system noise. Moreover,
the proposed algorithm can further reduce the misadjustment
by utilizing the system input power and cross-correlation
between the input signal and system estimated error.
VI. CONCLUSION
By utilizing the system input power and cross-correlation
between the input signal and system estimated error to control
step-size update, a novel, easy to implement, VSS-NLMS
algorithm is proposed for system identification in this paper.
The proposed algorithm outperforms existing VSS-NLMS
algorithms in terms of convergence speed, misadjustment, and
tracking ability, especially for variable system noise.
Theoretical analysis and simulation results have demonstrated
the proposed method has good performance.
REFERENCES
[1] S. Haykin, Adaptive Filter Theory, 4th ed. Upper Saddle River,
NJ:Prentice-Hall, 2001.
[2] Y. Huang, J. Benesty, and J. Chen, Acoustic MIMO Signal Processing,
Boston, MA: Springer, 2006.
[3] R. H. Kwong, E. W. Johnston, A variable step size LMS algorithm,
IEEE Trans. Signal Processing, vol. 40, pp. 16331642, July 1992.
[4] T. Aboulnasr and K. Mayyas, A robust variable step size LMS-type
algorithm: Analysis and simulations, IEEE Trans. Signal Processing,
vol. 45, no. 3, pp. 631639, Mar. 1997.
[5] M.H. Costa, J.C.M. Bermudez, A noise resilient variable step-size LMS
algorithm, Signal Processing,88(3), pp. 733-748, 2008.
[6] T. Li, M. Shi, Q. Yi, An Improved Variable Step-Size LMS Algorithm,
Wireless Communications, Networking and Mobile Computing ( WiCOM
), Wuhan, pp. 1-4, 2011.
[7] J. Benesty, H. Rey, L.R. Vega, S. Tressens, A nonparametric VSS NLMS
algorithm, IEEE Signal Process. Lett., 13 (10), pp581584, 2006.
[8] J. Liu, X. Yu, H. Li, A nonparametric variable step size NLMS algorithm
for transversal filters, Applied Mathematics and Computation, vol. 217,
pp. 7365-7371, 2011.
[9] M. A. Iqbal and S. L. Grant, Novel variable step size NLMS algorithms
for echo cancellation, in Proc. IEEE Int. Conf. Acoust., Speech,Signal
Process. (ICASSP), pp. 241-244, 2008.
[10] E. Eweda and A. Zerguine, New insights into the normalization of the
least mean fourth algorithm, Signal, Image, Video Process., 2011,
10.1007/s11760-011-0231-y.
[11] N. Bershad and J. C. M. Bermudez, Mean-square stability of the
normalized least mean fourth algorithm for white Gaussian Inputs, Digit.
Signal Process., vol. 21, no. 6, pp. 694-700, 2011.
[12] Eweda Eweda , Global Stabilization of the Least Mean Fourth
Algorithm, IEEE Trans. Signal Processing, 60(3), pp.1473-1477, 2012.
[13] Eweda Eweda and Neil J. Bershad, Stochastic Analysis of a Stable
Normalized Least Mean Fourth Algorithm for Adaptive Noise Canceling
With a White Gaussian Reference, IEEE Trans. Signal Processing,
vol.60, no.12, pp.6235-6244, 2012.
[14] D. J. Park, B. E. JUN, J. H. KIM, Fast tracking RLS algorithm using
novel variable forgetting factor with unity zone, Electron. Lett., vol. 27,
pp. 21502151, 1991.
[15] B. Farhang-Boroujeny, Adaptive Filters: Theory and Applications, New
York: Wiley, 1998.
[16] Hsu-Chang Huang and Junghsi Lee, A New Variable Step-Size NLMS
Algorithm and Its Performance Analysis, IEEE Trans. Signal
Processing, vol.60, no.4, pp.2055-2060, 2012.
764