4266 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 64, NO.
5, MAY 2017
Circuit Implementation of Data-Driven TSK-Type
Interval Type-2 Neural Fuzzy System With
Online Parameter Tuning Ability
Chia-Feng Juang, Senior Member, IEEE, and Kai-Jie Juang
AbstractThis paper proposes a new circuit for im-
plementing a reduced-interval type-2 neural fuzzy system
using weighted bound-set boundaries (RIT2NFS-WB) with
online tuning ability. The antecedent and consequent parts
of the RIT2NFS-WB use interval type-2 fuzzy sets and
Takagi-Sugeno-Kang (TSK) rules with interval combination
parameters, respectively. In the software implementation,
the structure and parameters of the RIT2NFS-WB are
learned through firing-strength-based rule generation and
gradient descent algorithms, respectively. The software-
designed RIT2NFS-WB is then transferred to a circuit
implementation with online parameter-tuning ability; the
hardware version is called the RIT2NFS-WB(HL). The
RIT2NFS-WB(HL) is characterized by its online tuning abil-
ity with updatable consequent and weighting parameters.
To the best of our knowledge, the RIT2NFS-WB(HL) is
the first TSK-type interval type-2 neural fuzzy circuit with
online parameter tuning ability in the literature. To take
advantage of the inherent parallel processing property of
the rules, a parallel processing technique is utilized in the
RIT2NFS-WB(HL) to achieve computational speedup. The
RIT2NFS-WB(HL) is applied to examples of online system
modeling and sequence prediction to demonstrate the
systems functionality.
Index TermsFuzzy chips, neural fuzzy systems (NFSs),
online learning, type-2 fuzzy systems.
I. INTRODUCTION
ITH the increasing complexity and scale of industry
W processes, data-driven techniques have emerged as a
promising approach to addressing various system problems,
such as modeling, monitoring, and control, through the uti-
lization of process data instead of physical models [1], [2]. In
data-driven approaches, two main issues are the selection of an
agent and its learning through process data. Neural fuzzy sys-
tems (NFSs) represent a popular data-driven approach in which
a fuzzy system functions as an agent and is trained using neural
networks. Although early works on NFSs focused on the use of
Manuscript received January 15, 2016; revised April 12, 2016; ac-
cepted May 4, 2016. Date of publication May 30, 2016; date of current
version April 10, 2017. This work was supported by the Ministry of
Science and Technology, Taiwan, under Grant MOST 104-2221-E-005-
039-MY2.
The authors are with the Department of Electrical Engineering, Na-
tional Chung-Hsing University, Taichung 402, Taiwan (e-mail: cfjuang@
dragon.nchu.edu.tw; ryjasonut@hotmail.com).
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TIE.2016.2574300
0278-0046 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.
type-1 fuzzy rules [3], interval type-2 NFSs have attracted more
attention in recent years and have been shown to outperform
their type-1 counterparts in several data-driven applications
[4][12]. When implemented, interval type-2 NFSs are substan-
tially more computationally expensive than are type-1 NFSs
because of the formers inherent interval operations, making
their implementation more difficult. Considering the demands
of online real-time computation and low-cost implementation
in several real applications of data-driven techniques, this paper
proposes the circuit implementation of interval type-2 NFSs.
Many type-1 fuzzy chips have been proposed [13][19] and
successfully applied to real processes such as water bath tem-
perature control [17], and robot control [18]. To handle time-
varying processes, several type-1 neural fuzzy chips with on-
chip parameter learning abilities have been proposed [20][25].
In contrast to the relatively mature techniques for implementing
type-1 fuzzy chips, hardware implementation of interval type-2
fuzzy systems (IT2FSs) is in its start-up stage mainly because
of the complex computations performed by the system, espe-
cially the type reduction operation. Table I shows a comparison
of different circuit implementations of IT2FSs. A typical type
reduction operation, such as the KarnikMendel (KM) proce-
dure [26], requires an iterative procedure to find the system ex-
tended outputs and is costly in hardware implementations [27].
To reduce hardware implementation costs, a hardware imple-
mentation of the type-reduction operation using lookup tables
[28] or simplified, closed-form operations have been proposed
[29][32]. Hardware implementation using a closed-form op-
eration to approximate the actual extended outputs based on
WuMendel uncertainty bounds was proposed in [29]. A more
economic hardware implementation of a reduced-interval type-
2 NFS using weighted bound-set boundaries (RIT2NFS-WB),
which uses fewer adders, multipliers, and dividers than the chip
in [29], was proposed and called RIT2NFS-WB(H) in [31]. The
implementation of a simplified type-reduction operation using
the average of two type-1 fuzzy systems was proposed in [30].
These simplified type reduction operations and their hardware
implementations demonstrate the tradeoff between chip imple-
mentation cost and the difference between the reduced outputs
and the normal system outputs. Regarding the implemented
types of rules, IT2FS chips with the Mamdani type [27][30],
the TakagiSugenoKang (TSK) type with crisp combination
parameters [32], and the TSK type with interval combination
parameters [8], [31] have been proposed. Among these three
JUANG AND JUANG: CIRCUIT IMPLEMENTATION OF DATA-DRIVEN TSK-TYPE INTERVAL TYPE-2 NEURAL FUZZY SYSTEM
TABLE I
COMPARISON OF DIFFERENT CIRCUIT IMPLEMENTATION METHODS FOR IT2FSS
Studies [27] [28] [29] [30] [33] [31]
Rule type Mamdani Mamdani Mamdani Mamdani Mamdani TSK (interval parameters) TSK (interval parameters) TSK (crisp parameters) TSK (interval parameters)
Type reduction Normal Simplified Simplified Simplified Simplified Simplified
Normalization Term Yes No Yes Yes No Yes
Flexible No No No No No Yes
Parameter Learning No No No No Yes No
types of rules, hardware implementation of the last type is more
complex because of the additional circuits required to imple-
ment signed interval multiplications and additions.
The system parameters in the aforesaid hardware-
implemented IT2FSs [8], [27][31] are fixed, i.e., these chips do
not possess online learning ability. Recently, an interval type-
2 neural fuzzy chip with on-chip incremental learning ability
(IT2NFC-OL) was proposed in the literature [33]. The IT2NFC-
OL uses Mamdani-type rules, where there are only two tunable
consequent parameters for each rule. To reduce the design ef-
fort and implementation cost, a simplified type-reduction op-
eration using a simple weighted sum operation without output
normalization terms (i.e., no dividers) is used in the IT2NFC-
OL. However, this simplification operation causes the values of
the consequent parameters in the rules and their meaning to be
different from those with a normalization term [27][31].
This paper proposes a new circuit for the implementation of an
interval type-2 NFS with online parameter learning ability. The
motivations for proposing the new circuit are explained as fol-
lows. First, in several applications of fuzzy systems (FSs), such
as fuzzy sequence prediction in power electronics [34], [35],
fuzzy control [23], [24], [36], and neural fuzzy modeling in sig-
nal processing [25], high-speed real-time operation ability has
been shown to be necessary. In addition, regarding handling the
unmolded or changing dynamics of underlying processes, these
applications have also shown that NFSs with online parameter
learning ability are necessary. Concerning the implementation
platform, the hardware implementation of NFSs benefits from
smaller platform size, lower power consumption, lower cost, and
higher execution speed, in contrast to software implementations
[17], [18], [23][25]. Second, interval type-2 NFSs consisting
of TSK-type rules with interval combination parameters have
been shown to outperform those consisting of Mamdani-type
rules and type-1 NFSs [4], [31]. Therefore, the circuit imple-
mentation of a TSK-type interval type-2 neural fuzzy chip with
online tuning ability is demanding. To this end, this paper in-
troduces a new learning circuit into the RIT2NFS-WB(H) to
endow it with online tuning ability; the new circuit is called
RIT2NFS-WB(HL).
In the RIT2NFS-WB(HL), all the TSK-type consequent
parameters and a weighting parameter are tunable, en-
abling it to handle data-driven time-varying processes. The
RIT2NFS-WB(HL) employs a parallel processing technique for
computational speedup and is flexible so that it is applicable to
interval type-2 NFSs with different numbers of inputs and rules
without requiring circuit redesign. Unlike the IT2NFC-OL that
implements interval type-2 NFSs with Mamdani-type rules
without output normalization terms, the RIT2NFS-WB(HL)
4267
[8] [32] This paper
Simplified Simplified Simplified
No Yes Yes
No No Yes
No No Yes
implements the more complex systems with TSK-type rules
with the general normal normalization terms. Because of
the additional inclusion of the normalization and TSK-type
consequent-parameter-tuning operations, three pipeline stages
are proposed in the RIT2NFS-WB(HL) instead of two such
stages used in the IT2NFC-OL. The additional pipeline stage
is mainly dedicated to processing the tuning operation so that
the computation time in each stage can be reduced.
The contributions of the RIT2NFS-WB(HL) are twofold.
First, almost all previous hardware circuits implement type-1
FSs/NFSs or interval type-2 FSs, which are relatively simple
in terms of circuit design. Although the IT2NFS-OL chip im-
plements IT2NFS, it considers only the Mamdani-type rules
with the shortcomings discussed aforesaid and is simpler in
terms of implementation compared to TSK-type IT2NFSs. To
the best of our knowledge, the RIT2NFS-WB(HL) is the first
TSK-type interval type-2 neural fuzzy circuit with online tuning
ability in the literature. Second, in addition to the new circuits
proposed to implement the complex functions in a TSK-type
IT2NFS, another characteristic of the RIT2NFS-WB(HL) is its
flexibility in applications. The RIT2NFS-WB(HL) allows the
user to transfer an initial interval-type fuzzy model to hardware
with different numbers of rules and inputs without redesigning
the circuits, which provides flexibility and convenience of use.
The IT2NFC-OL and most interval type-2 circuits discussed in
Table I use a fixed structure, in which circuit redesign is neces-
sary if the number of inputs or rules changes. The RIT2NFS-
WB(HL) is applied to different data-driven examples to verify
its performance.
This paper is organized as follows. Section II introduces
the functions in the RIT2NFS-WB. Section III introduces the
structure and parameter learning in the RIT2NFS-WB. Section
IV introduces the designed circuit modules in the RIT2NFS-
WB(HL), specifically the circuits implementing the online pa-
rameter tuning module. Section V presents the simulation re-
sults of the RIT2NFS-WB(HL). Finally, Section VI provides
the conclusions.
II. RIT2NFS-WB OPERATIONS
This section describes the inputoutput functions of the
RIT2NFS-WB. Each rule in the TSK-type RIT2NFS-WB is
of the following form [31]:
Rule j: If x1 is Aj1 and . . . and xn is Ajn
  n  
then y is cj0 sj0 , cj0 + sj0 + cji sji , cji + sji xi
i=1
j = 1, . . . , M (1)
4268 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 64, NO. 5, MAY 2017

where x1 , . . . , xn are input variables scaled to lie within the 
M M
range [1, 1]; y is an output variable; Aji , i = 1, . . . , n, are yr(0) = fj wrj IU , yr(M ) = f j wrj IL (11)
interval type-2 fuzzy sets; [cji sji , cji + sji ], sji 0 is a com- j =1 j =1
bination interval; and M is the number of rules. Fuzzy set Aji where
uses a Gaussian primary membership function (MF) with width
ij and an uncertain center [m1 ji , m2 ji ], and the output of the MF 1 1
IL = M , IU = M . (12)
is an interval set [A j , A j ], given as follows: f j j
j =1 j =1 f
i i

2

1 x mj1 i III. STRUCTURE AND PARAMETER LEARNING

exp
i
xi < m1 ji


,

2 ij A. Rule Generation


A j (xi ) = 1, mj1 i xi mj2 i

The RIT2NFS-WB uses an average firing-strength-based
i


2

1 x mj2 i method to generate a new rule according to training data. The

exp i
, xi > mj2 i number of existing rules at time t is denoted as M (t). For each

2
ij 
piece of incoming data x = (x1 , . . . , xn ), the RIT2NFS-WB
(2) finds
 
J = arg max F j x

and (13)

2 1j M (t)

1 x mj2i mj1i mj2i

i + where F j = (fj + f j )/2 is the average firing strength. If

exp , x
2 ij i 2
F J ( x) is smaller than a prespecified threshold th (0, 1) or


A j (xi ) =
2 .

1 x mj1i mj1i + mj2i
M (t) = 0, then a new rule is generated. For a new rule, the

i

exp
i
, xi > initial uncertain centers and widths of its composing antecedent

2 ij 2
interval type-2 fuzzy sets in each input variable xi are assigned
(3) as follows:
By using the algebraic product function, the firing strength,  
M (t)+1 M (t)+1
F j , is an interval given as follows: m1 i , m2 i = [xi (t) 0.1, xi (t) + 0.1] (14)
 
F j = f j , fj (4) and

where
fi x e d = 0.5, M (t) = 0


n 
n M (t )+ 1   J 2
0 . 5
fj = A j and f j = A j .
i =

n m 1 i + m 2 Ji .
i
(5)
2
1
xi , M (t) > 0
i 2
i=1 i=1 i= 1

The consequent value in rule j is an interval [wlj , wrj ], given (15)

as follows:
n 
n B. Parameter Learning
wlj = cji xi |xi | sji , x0 := 1 (6)
Parameter learning is performed using the gradient descent
i=0 i=0
algorithm to minimize the following error function:
and
1

n 
n E = [y(t + 1) y d (t + 1)]2 (16)
wrj = cji xi + |xi | sji . (7) 2
i=0 i=0 where y(t + 1) and y d (t + 1) denote the system and desired
To avoid the use of a computationally expensive iterative outputs, respectively. The update equations of the consequent
procedure, such as the KM procedure [26], to compute the parameters cji and sji are given as follows:
system output y, RIT2NFS-WB uses the following simplified
E
equation based on weighted bound-set boundaries: cji (t + 1) = cji (t) (17)
cji (t)
y = yl + (1 )y r (8)
E
where [0, 0.5] is a weighting parameter that is online tunable sji (t + 1) = sji (t) (18)
and sji (t)
   
(0) (M ) where is a learning coefficient and other terms are defined by
yl = min yl , yl , y r = max yr(0) , yr(M ) (9)
(19) and (20), shown at the bottom of the next page.
where The update equations of the weighting parameter are given
as follows:

M 
M
= f j wlj IL , yl = fj wlj IU (10)
(0) (M )
yl E
(t + 1) = (t) (21)
j =1 j =1 (t)
JUANG AND JUANG: CIRCUIT IMPLEMENTATION OF DATA-DRIVEN TSK-TYPE INTERVAL TYPE-2 NEURAL FUZZY SYSTEM 4269

where
E     an output y and updates the consequent and weight parameters.
= y y d yl y r . (22)
After the update, if is smaller than 0, then it is set to 0.
Similarly, if is greater than 0.5, then it is set to 0.5. Detailed
update equations of the antecedent parameters ij , m1 ji , and
m2 ji can be found in [31].
IV. CIRCUIT IMPLEMENTATION
Given a training dataset, a RIT2NFS-WB is built using a
software implementation of the structure and parameter learn-
ing algorithms. The software-designed RIT2NFS-WB, denoted
as RIT2NFS-WB(S), is then implemented in the RIT2NFS-
WB(HL) with the same number of rules and without an addi-
tional circuit-implemented structure-learning circuit. Because
tuning the antecedent parameters is substantially more complex
than tuning the consequent and weighting parameters, only the
latter is implemented in the RIT2NFS-WB(HL) to endow it with
the online parameter tuning ability without heavy implementa-
tion cost. The RIT2NFS-WB(HL) is designed to be flexible so
that it can implement IT2FSs with a different number of in-
put variables n and rules M without redesigning the circuits.
Fig. 1 shows the architecture of the RIT2NFS-WB(HL), which
contains six modules: selection, antecedent, consequent, out-
put processing, output, and learning. The implementation of
the six modules is divided into three stages (T1 , T2 , and T3 ).
Fig. 1 shows that stage T1 computes the selection, antecedent,
and consequent modules. Because the computations of the an-
tecedent firing strengths in (5) and the consequent values in
(6) and (7) are independent in the RIT2NFS-WB(HL), the an-
tecedent and consequent modules that compute these values are
implemented in parallel in the same stage. This parallel compu-
tation structure avoids the additional time spent on computing
the TSK-type consequent values when using a serial computa-
tion structure and, therefore, helps to reduce the execution time.
Stage T2 computes the output processing module, and stage T3
computes the output and learning modules. The circuit sends
The first five modules compute the system inputoutput func-
tions in (2)-(12). The circuits in these five modules follow those
in the RIT2NFS-WB(H) [31] with the additional inclusion of
input/output buses to receive/send signals from/to the learning
module. The online tuning ability of the RIT2NFS-WB(HL)
is achieved by introducing the new learning module into the
RIT2NFS-WB(H), whose parameters are fixed. Section IV-A
briefly describes the functions of the first five modules and their
connections to the learning module. Detailed implementation
circuits of the five modules can be found in [31]. Section IV-
B details the circuits in the new learning module proposed to
endow the original RIT2NFS-WB(H) with online tuning ability.
A. Circuit Implementation of InputOutput Functions
For the first five modules, the selection module allows the
user to specify the number of rules M and input variables n for
different applications. The function is implemented by sending
out the input number control signal In [7:0] and the rule num-
ber control signal Rule [7:0] to the multiplexers in the other
modules. The antecedent module computes the firing strength
pair f j and fj in (5), where the exponential function in (2) and
(3) is implemented using a lookup table with a resolution of
10 and 8 bits in the input and output, respectively. A total of
M antecedent modules are activated by the rule number control
signal, and these modules run in parallel to find all M firing
strength pairs, as shown in Fig. 1. The consequent module com-
putes the consequent values wlj and wrj in (6) and (7). Like the
arrangement of the antecedent modules, a total of M conse-
quent modules are activated by the rule number control signal,
and these modules run in parallel to find all consequent values.
The RIT2NFS-WB(HL) uses additional input buses in the con-
sequent module to receive the updated consequent values from
the learning module at each clock cycle. The output process-
ing module computes the four values in (10) and (11), namely,

(0) (M )

(y y d )f j IL xi , yl = yl and y r = yr







(y y d )[f IL + (1 )fj IU ] xi ,
j
yl = yl
(0)
and y r = yr
(0)
E
=
cji


(y y d )[fj IU + (1 )f j IL ] xi , yl = yl
(M ) (M )
and y r = yr





(M ) (0)
(y y d ) fj IU xi , yl = yl and y r = yr
(19)
(0) (M )

(y y d )(1 2)f j IL |xi | , yl = yl and y r = yr







(y y d )[f IL + (1 )fj IU ] |xi | ,
j
yl = yl
(0)
and y r = yr
(0)
E
= (20)
sji


(y y d )[fj IU + (1 )f j IL ] |xi | , yl =
(M )
yl and y r =
(M )
yr





(M ) (0)
(y y d )(1 2)fj IU |xi | , yl = yl and y r = yr
4270
Fig. 1. Architecture of the RIT2NFS-WB(HL).
Fig. 2. Submodules proposed to implement part of the products in (19)
and (20).
(0) (M ) (0) (M )
yl , yl , yr , and yr , in parallel. This module uses two di-
viders to implement IU and IL in (12). The RIT2NFS-WB(HL)
introduces two additional buses in this module to send IU and
IL to the learning module. The output module first computes yl
and y r in (9) using minimum and maximum circuits and then
computes the final system output y using (8). The results of the
minimum and maximum circuits are also sent to the learning
module.
B. Circuit Implementation of the Learning Module
The learning module implements the consequent param-
eter update equations in (17)-(20) using the submodules in
Figs. 25 and the weighting parameter update equations in (21)
and (22) using the submodule in Fig. 6. The following details
the circuits in each submodule.
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 64, NO. 5, MAY 2017
Fig. 3. Submodule that computes the final outputs in the four conditions
in (19) and (20).
Submodule in Fig. 2: This submodule finds the partial prod-
ucts in (19) and (20). The upper (lower) half circuit computes
IL f j (IU fj ), followed by its multiplication with the four
values , (1 ), (1 2), and in parallel. Among the
four multiplication terms, only the product value in the last term
is negative; therefore, the technique of 2s complement is used
in the last product term. The upper and lower circuits compute
the eight multiplication terms in parallel.
Submodule in Fig. 3: This submodule computes the multipli-
cations of the ten outputs from the submodule in Fig. 2 with the
error value E = y y d and xi (or |xi |) to obtain all the product
terms for the eight conditions in (19) and (20). The left four cir-
cuits in Fig. 3(a)(d) compute the four cases of the partial value
E/cji in (19). For example, the output signal case1(cj )xi
in Fig. 3(a) denotes the term value (y y d )f j IL xi in the first
JUANG AND JUANG: CIRCUIT IMPLEMENTATION OF DATA-DRIVEN TSK-TYPE INTERVAL TYPE-2 NEURAL FUZZY SYSTEM
Fig. 4. Submodule proposed to determine E/cji and E/sji ,
i = 0, . . . , n of rule j in parallel.
Fig. 5. Submodule proposed to find the updated consequent values in
(17) and (18).
Fig. 6. Submodule proposed to find the updated weight parameter
(t + 1) in (21) and (22).
case in (19). Similarly, the right four circuits in Fig. 3(a)(d)
compute the four cases of the partial value E/sji in (20).
Sub-module in Fig. 4: This submodule determines which of
the four cases in (19) and (20) holds and sends all the final values
of E/cji and E/sji , i = 0, . . . , n, of rule j in parallel to
the next module according to the specified values of n and M .
The maximum number of allowed input variables is denoted as
n. This submodule contains three types of multiplexers, called
MUX-1, MUX-2, and MUX-3, that determine the final outputs.
The In signal from the selection module controls MUX-1 to
provide the circuit with the flexibility of learning with a specified
number of inputs without redesigning the circuit. The condition
4271
In[i] = 0 (i = n + 1, . . . , n) means that the input variable xi
is not used; thus, the output of the MUX-1 is set to 16 b0. There
are a total of eight MUX-1s to account for the four cases of
E/cji and E/sji . Based on the result of the minimum and
maximum circuits in the output module, two control signals are
defined. The control signals based on the result of the minimum
(0) (M )
circuit are signal yl and signal yl , the values of which are
given as follows:
1, yl = yl(0)
(0)
signal yl = ,
0, otherwise
1, yl = yl(M )
(M )
signal yl = (23)
0, otherwise.
Similarly, the control signals based on the result of the max-
(0) (M )
imum circuit are signal yr and signal yr , the values of
which are given as follows:
1, y r = yr(0)
(0)
signal yr = ,
0, otherwise
1, yr = yr(M )
(M )
signal yr = (24)
0, otherwise.
The four control signals are sent to MUX-2 to determine
which of the four cases in (19) and (20) holds. The Rule[j]
signal from the selection module controls MUX-3 to provide
the circuit with the flexibility of learning with different specified
numbers of rules. The condition Rule[j] = 0 means that the
jth rule is not used; thus, the output of MUX-3 is set to 16 b0.
Su-module in Fig. 5: This submodule computes the final con-
sequent parameter update equations in (17) and (18). This sub-
module first finds the product of the learning rate with the
partial differentiation terms E/cji and E/sji using the bit
right-shift operation and then adds the result to cji (t) and sji (t)
to obtain cji (t + 1) and sji (t + 1), respectively.
Submodule in Fig. 6: This sub-module computes the update
of in (21) and (22). The difference between yl and y r received
from the output module is multiplied by the error value E to ob-
tain the partial differentiation term E/ in (22). The product
of E/ with the learning rate is implemented using the bit
right-shift operation, and the result is added to (t) to obtain the
final (t + 1) value in (21). Two cascaded comparators com-
pare the computed (t + 1) value with 0 and 0.5 to endure that
(t + 1) [0. 5]. Circuits for updating cji (t), sji (t), and (t) are
implemented in parallel to obtain a computational speedup.
V. SIMULATIONS
This section presents the simulation results of the RIT2NFS-
WB(HL). The utilized software tools are the electrical design
automation software tool of ISE and ModelSim SE6.5. This
section presents three examples of the simulation results of
RIT2NFS-WB(HL). When the RIT2NFS-WB(HL) is specified
4272 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 64, NO. 5, MAY 2017

TABLE II
PERFORMANCES OF THE SOFTWARE-IMPLEMENTED RIT2NFS-WB WITH/WITHOUT ONLINE LEARNING AND VARIOUS MODELS
FOR THE SERIES-A PREDICTION PROBLEM IN EXAMPLE 1

Model category Other models Type-1 NFSs IT2NFSs

Models PRMS[38] NN[38] SONFIN[3] SVM-FM[8] T2FLS-G[8] SEIT2FNN[4] T2NFS-T1(A)[8] RIT2NFS-WB(S) RIT2NFS-WB(SL)
Rule number - - 9 31 5 5 5 5 5
Test RMSE 14.226 24.6 5.938 9.707 7.16 5.766 2.985 2.627 2.614

TABLE III
TEST RMSES OF THE SOFTWARE- AND CIRCUIT-IMPLEMENTED
RIT2NFS-WB WITH/WITHOUT ONLINE LEARNING IN EXAMPLE 2

Software Circuit

Implementations RIT2NFS RIT2NFS RIT2NFS RIT2NFS

Models -WB(S) -WB(SL) -WB(H) -WB(HL)

k = 1 to 200 0.0241 0.0146 0.0251 0.0211

k = 201 to 400 0.0934 0.0362 0.0922 0.0460
k = 401 to 600 0.1784 0.0447 0.1771 0.0564

utilized for comparison used the same number of rules and

Fig. 7. Test prediction result of the real time series using the RIT2NFS-
WB(SL) in Example 1.
to have a maximum of seven input variables and seven rules, the
total number of gate counts is 621 164. With the inclusion of
the online update of the parameters, the number of fuzzy logic
inferences per second (FLIPS) is 7.07 million. For RIT2FS-
WB(H) with the same number of inputs and rules, the number
of FLIPS is 24.107 million. As expected, the execution speed of
the RIT2NFS-WB(HL) is lower than that of the RIT2FS-WB(H)
because of the additional inclusion of the learning module in
stage three.
Example 1: (real time-series prediction). This example con-
siders the prediction of the SanteFe real-time series (series A)
measured from a real physical system [37]. Five past values
y(t 5), . . . , y(t 1), were used to predict y(t). As in [8],
[38], 1000 samples were used, 90% and 10% of which were
used for training and testing, respectively. In the software im-
plementation, 1000 iterations were performed for training. Five
rules were generated when th was set to 0.2. Table II shows
the test root-mean-square-error (RMSE) of RIT2NFS-WB(S),
where all system parameters in RIT2NFS-WB(S) are fixed. For
comparison, Table II also shows the test RMSE of the software-
implemented RIT2NFS-WB with online consequent and weight
parameter learning, as with the RIT2NFS-WB(HL), and the sys-
tem is denoted as RIT2NFS-WB(SL). The RIT2NFS-WB(SL)
produced a smaller test error than did the RIT2NFS-WB(S),
indicating the effect of the online learning operation. Fig. 7
shows the prediction result of the test data using the RIT2NFS-
WB(SL), where the predicted outputs were very close to the
actual values.
For comparison, Table II also shows the reported results ob-
tained using different models, including type-1 and type-2 NFSs,
applied to the same prediction problem [8], [38]. The IT2NFSs
training iterations as the RIT2NFS-WB(S). The result showed
that both the RIT2NFS-WB(S) and RIT2NFS-WB(SL) achieved
smaller test RMSEs than did all the models used for the com-
parison.
The RIT2NFS-WB(S) has been shown to outperform dif-
ferent type-1 and interval type-2 NFSs in other applications
[31]. Therefore, the following examples focus on presenting
the online tuning ability and functionality of the RIT2NFS-
WB(HL) and the accuracy difference between software- and
circuit-implemented RIT2NFS-WB models.
Example 2: (online system modeling). This example con-
siders the online modeling of a nonlinear plant guided by the
following difference equation:
yd (k)
yd (k + 1) = + u3 (k) (25)
1 + yd2 (k)
where u(k) = sin(2k/100) and yd (1) = 0. In the software im-
plementation, the inputs of the RIT2NFS-WB(S) are yd (k) and
u(k), and the desired output is yd (k + 1). Training patterns
were generated with u(k) = sin(2k/100), k = 0, . . . , 200,
and training was performed for 100 iterations. Seven fuzzy rules
were generated when th was set to 0.1. Table III shows the
test RMSEs of the RIT2NFS-WB(S) and RIT2NFS-WB(SL)
when using the 200 training data as test data. As in Example
1, the RIT2NFS-WB(SL) produced a smaller test error than did
the RIT2NFS-WB(S), indicating again the effect of the online
learning operation.
In the circuit implementation, the external inputs were speci-
fied as (n)2 = 010 and (M )2 = 111. Table III shows the test
RMSEs of the RIT2NFS-WB(H) and RIT2NFS-WB(HL). Fig. 8
shows the modeling result obtained using the 200 test data points
(k = 1, . . . , 200) for the software and circuit implementations
with/without online learning. The RIT2NFS-WB(HL) produced
a smaller test error than did the RIT2NFS-WB(H) because of
the on-chip learning ability. Table III and Fig. 8 show that the
JUANG AND JUANG: CIRCUIT IMPLEMENTATION OF DATA-DRIVEN TSK-TYPE INTERVAL TYPE-2 NEURAL FUZZY SYSTEM 4273

Fig. 9. Modeling results of the RIT2NFS-WB(H) and RIT2NFS-WB(HL)

Fig. 8. Modeling results (k = 1, . . . , 200) of the (a) software- and when (a) k = 201, . . . , 200 and (b) k = 401, . . . , 600 in Example 2.
(b) circuit-implemented RIT2NFS-WB with/without the learning ability
in Example 2.
TABLE IV
TEST RMSES FOR DIFFERENT NUMBERS OF RULES IN THE SOFTWARE-
test error of the RIT2NFS-WB(HL) is similar, although larger, IMPLEMENTED RIT2NFS-WB WITH/WITHOUT ONLINE LEARNING IN
to that of the RIT2NFS-WB(SL). The performance degradation EXAMPLE 3
is mainly due to resolution lost in the computation and the finite
floating point representation accuracy. th 0.05 0.3 0.5 0.55 0.6
Rule number 3 5 7 10 12
To further test the learning ability of the RIT2NFS-WB(HL), RIT2NFS-WB(S) 0.11869 0.11674 0.11625 0.11591 0.1165
the plant model is assumed to be time varying after the time RIT2NFS-WB(SL) 0.11823 0.11627 0.11593 0.11560 0.1162
step at k = 200; the time-varying plant model is described as
follows:

y d (k ) 3
201 < k 400 where = 17 and x(0) = 1.2 The four past values x(t
1+y d2 (k ) + u (k) + 0.1,
yd (k + 1) = . 18), x(t 12), x(t 6), and x(t) were used to predict the

y d (k ) future value x(t + 85). A total of 3000 patterns generated from
2
1+y d (k )
+ u 3
(k) + 0.2, 401 < k 600
time steps t = 201 to 3200 were used for training, and the 3000
(26)
patterns generated from t = 5001 to 8000 were used for testing.
Table III shows the RMSEs of the software- and circuit-
In the software implementation, 100 iterations were performed
implemented RIT2NFS-WB with/without online learning. Fig.
for training. To show the influence of the number of rules on
9 shows the test outputs of the different implementations. An ob-
the prediction performance, Table IV presents the test RMSEs
vious difference between the outputs of the RIT2NFS-WB(H)
of the RIT2NFS-WB(S) and the RIT2NFS-WB(SL) with dif-
and the actual outputs is observed because of the change in
ferent numbers of rules generated from setting different values
the plant with time. In contrast to the RIT2NFS-WB(H), the
of th . As in Examples 1 and 2, the RIT2NFS-WB(SL) pro-
RIT2NFS-WB(HL) obtained more similar values to the actual
duced smaller errors than did the RIT2NFS-WB(S) for differ-
plant outputs, which verify the online tuning ability of the cir-
ent numbers of rules. Table IV shows that the initial increase
cuit. Similar to the test result for k = 1, . . . , 200, Table III shows
in the number of rules tends to improve the test performance.
that the test error of the RIT2NFS-WB(HL) is similar, although
However, when the number of rules is too large, the test perfor-
larger, to that of the RIT2NFS-WB(SL), which verifies the cor-
mance may degrade. The reason for this may be that a larger
rect functionality of the designed circuits.
IT2FS needs a larger number of training iterations to converge to
Example 3: (online chaotic sequence prediction). This ex-
an optimal parameter set or even when the IT2FS is well trained,
ample presents the prediction of the following MackeyGlass
the larger model may encounter the overtraining problem.
chaotic time series [31]:
Table V shows the test RMSEs of the software- and circuit-
dx(t) 0.2x(t ) implemented RIT2NFS-WB with/without online tuning abil-
= 0.1x(t) (27)
dt 1 + x10 (t ) ity when the number of rules was 7. The RMSE difference
4274
TABLE V
TEST RMSES OF THE SOFTWARE- AND CIRCUIT-IMPLEMENTED
RIT2NFS-WB WITH/WITHOUT ONLINE LEARNING
IN EXAMPLE 3
Implementations Software
RIT2NFS RIT2NFS RIT2NFS
Models -WB(S) -WB(SL) -WB(H)
t= 0.11625 0.11593 0.11704
5001, . . . , 8000
TABLE VI
TEST RMSES OF RIT2NFS-WB(H) AND RIT2NFS-WB(HL) WHEN
DIFFERENT DISTORTING VALUES ARE ADDED TO ALL THE CONSEQUENT
PARAMETERS IN THE INITIAL MODEL IN EXAMPLE 3
Models Distorting RIT2NFS
Values -WB(H)
t = 5001, . . . , 8000 0.01 0.11846
0.05 0.16197
0.1 0.27350
between RIT2NFS-WB(HL) and RIT2NFS-WB(SL) is smaller
than 0.001, which again shows the correct functionality of the
designed circuits. The RIT2NFS-WB(HL) produced a smaller
test error than did the RIT2NFS-WB(H). However, the differ-
ence was small due to the well-trained initial model transferred
from the RIT2NFS-WB(S) and the time-invariant property of
the series.
To further demonstrate the effect of the online tuning ability
in the RIT2NFS-WB(HL), it is assumed that the initial con-
sequent parameters in the RIT2NFS-WB(HL) are distorted by
adding a distorting value to the consequent parameters of every
rule. The distorting values of 0.01, 0.05, and 0.1 are tested. For
this initially corrupted model, Table VI shows the test RMSEs
of the RIT2NFS-WB(H) and RIT2NFS-WB(HL), where the lat-
ter achieves a substantially smaller error than does the former
for different distorting levels. Fig. 10(a) and (b) shows the pre-
diction outputs from t= 5001 to 6000 and from t = 7001 to
8000, respectively, of the RIT2NFS-WB(H) and the RIT2NFS-
WB(HL) with a distorting value of 0.1. Fig. 10(a) shows that,
at the start of the test, the prediction values from both the
RIT2NFS-WB(H) and the RIT2NFS-WB(HL) are significantly
larger than the actual values; however, the prediction values from
the RIT2NFS-WB(HL) tend to be closer to the actual values
due to its online tuning ability. Fig. 10(b) shows that, after online
tuning with 2000 time steps, the predicted values obtained by
the RIT2NFS-WB(HL) are close to the actual values, whereas
the prediction error of the RIT2NFS-WB(H) remains large.
VI. CONCLUSION
This paper has proposed a new circuit for the implementa-
tion of an interval type-2 NFS chip with online tuning abil-
ity. The proposed RIT2NFS-WB(HL) is different from most
existing type-2 fuzzy chips, whose parameters are fixed and
do not possess any online tuning ability. The learning mod-
ule proposed in the RIT2NFS-WB(HL) tunes all the interval
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 64, NO. 5, MAY 2017
Circuit
RIT2NFS
-WB(HL)
0.11688
RIT2NFS
-WB(HL)
0.11777
0.13241
0.17675
Fig. 10. Prediction values of the RIT2NFS-WB(H) and the RIT2NFS-
WB(HL) (a) in the first 1000 test samples (t = 5001 to 6000) and (b) in
the 2001 to 3000 test samples (t = 7001 to 8000) in Example 3.
combination parameters in the consequent of a TSK-type rule
and the weighting parameter in the weighted bound-set bound-
ary operation, where the operation is used to simplify the type-
reduction operation and reduce the circuit implementation cost.
The RIT2NFS-WB(HL) with online tuning ability is suitable for
handling data-driven problems with time-varying characteris-
tics, the property of which has been verified through simulations
of process modeling and sequence prediction. The limitation of
the RIT2NFS-WB(HL) is that the structure should be learned in
advance using software implementation, after which the fixed
structure can be transferred to the circuit implementation. In
the future, the implementation of the structure learning ability
into the RIT2NFS-WB(HL) will be studied so that the circuit
can online evolve not only system parameters but also system
structure to handle time-varying processes. The application of
the RIT2NFS-WB(HL) to a real process will be studied as well.
REFERENCES
[1] S. Yin, H. Gao, and O. Kaynak, Data-driven control and process mon-
itoring for industrial applicationsPart II, IEEE Trans. Ind. Electron.,
vol. 62, no. 1, pp. 583586, Jan. 2015.
[2] S. Yin, X. Li, H. Gao, and O. Kaynak, Data-based techniques focused
on modern industry: An overview, IEEE Trans. Ind. Electron., vol. 62,
no. 1, pp. 657667, Jan. 2015.
[3] C. F. Juang and C. T. Lin, An on-line self-constructing neural fuzzy
inference network and its applications, IEEE Trans. Fuzzy Syst., vol. 6,
no. 1, pp. 1232, Feb. 1998.
[4] C. F. Juang and Y. W. Tsao, A self-evolving interval type-2 fuzzy neural
network with on-line structure and parameter learning, IEEE Trans. Fuzzy
Syst., vol. 16, no. 6, pp. 14111424, Dec. 2008.
[5] R. H. Abiyev and O. Kaynak, Type 2 fuzzy neural structure for identi-
fication and control of time-varying plants, IEEE Trans. Ind. Electron.,
vol. 57, no. 12, pp. 41474159, Dec. 2010.
JUANG AND JUANG: CIRCUIT IMPLEMENTATION OF DATA-DRIVEN TSK-TYPE INTERVAL TYPE-2 NEURAL FUZZY SYSTEM
[6] G. D. Wu and P. H. Huang, A vectorization-optimization-method based
type-2 fuzzy neural network for noisy data classification, IEEE Trans.
Fuzzy Syst., vol. 21, no. 1, pp. 115, Feb. 2013.
[7] Y. Y. Lin, J. Y. Chang, and C. T. Lin, A TSK-type-based self-evolving
compensatory interval type-2 fuzzy neural network (TSCIT2FNN) and its
applications, IEEE Trans. Ind. Electron., vol. 61, no. 1, pp. 447459, Jan.
2014.
[8] C. F. Juang and W. S. Jang, A type-2 neural fuzzy system learned through
type-1 fuzzy rules and its FPGA-based hardware implementation, Appl.
Soft Comput., vol. 18, pp. 302313, May 2014.
[9] C. J. Kim and D. Chwa, Obstacle avoidance method for wheeled mobile
robots using interval type-2 fuzzy neural network, IEEE Trans. Fuzzy
Syst., vol. 23, no. 3, pp. 677687, Jun. 2015.
[10] A. K. Das, S. Sundaram, and N. Sundararajan, A self-regulated interval
type-2 neuro-fuzzy inference system for handling non-stationarities in
EEGsignals for BCI, IEEE Trans. Fuzzy Syst., to be published, doi:
10.1109/TFUZZ.2016.2540072.
[11] A. Mohammadzadeh, O. Kaynak, and M. Teshnehlab, Two-mode indirect
adaptive control approach for the synchronization of uncertain chaotic
systems by the use of a hierarchical interval type-2 fuzzy neural network,
IEEE Trans. Fuzzy Syst., vol. 22, no. 5, pp. 13011312, Oct. 2014.
[12] M. A. Khanesar, E. Kayacan, M. Reyhanoglu, and O. Kaynak, Feedback
error learning control of magnetic satellites using type-2 fuzzy neural
networks with elliptic membership functions, IEEE Trans. Cyber., vol. 45,
no. 4, pp. 858868, Aug. 2015.
[13] H. Eichfeld, T. Kunemund, and M. Menke, A 12b general-purpose fuzzy
logic controller chip, IEEE Trans. Fuzzy Syst., vol. 4, no. 4, pp. 460475,
Nov. 1996.
[14] M. J. Patyra, J. L. Grantner, and K. Koster, Digital fuzzy logic con-
troller: design and implementation, IEEE Trans. Fuzzy Syst., vol. 4, no. 4,
pp. 439459, 1996.
[15] A. Gabrielli and E. Gandolfi, A fast digital fuzzy processor, IEEE
MICRO, vol. 17, no. 1, pp. 6879, Jan.-Feb. 1999.
[16] G. Ascia, V. Catania, and M. Russo, VLSI hardware architecture
for complex fuzzy system, IEEE Trans. Fuzzy Syst., vol. 7, no. 5,
pp. 553569, Oct. 1999.
[17] C. F. Juang and J. S. Chen, Water bath temperature control by a recur-
rent fuzzy controller and its FPGA implementation, IEEE Trans. Ind.
Electron., vol. 53, no. 3, pp. 941949, Jun. 2006.
[18] S. Sanchez-Solano, A. J. Cabrera, I. Baturone, F. J. Moreno-Velo, and
M. Brox, FPGA implementation of embedded fuzzy controllers for
robotic applications, IEEE Trans. Ind. Electron., vol. 54, no. 4, pp. 1937
1945, Aug. 2007.
[19] A. H. Zavala and O. C. Nieto, Fuzzy hardware: A retrospective and
analysis, IEEE Trans. Fuzzy Syst., vol. 20, no. 4, pp. 623635, Aug.
2012.
[20] C. W. Lin, J. S. Wang, and C. C. Yu, "Synchronous pipeline hardware
design for a neuro-fuzzy circuit with on-chip learning capability," Lecture
Notes Artif. Intell., vol. 4682, pp. 192201, Aug. 2007.
[21] I. del Campo, K. Basterretxea, J. Echanobe, G. Bosque, and F. Doc-
tor, System-on-chip development of a neurofuzzy embedded agent for
ambient-intelligence environments, IEEE Trans. Syst., Man, Cybern. B,
Cybern., vol. 42, no. 2, pp. 501512, Apr. 2012.
[22] G. Bosque, I. Campo, and J. Echanobe, Fuzzy systems, neural networks
and neuro-fuzzy systems: A vision on their hardware implementation
and platforms over two decades, Eng. Appl. of Arti. Intell., vol. 32,
pp. 283331, Jun. 2014.
[23] C. F. Hsu, C. J. Chiu, T. T. Lee, and J. Z. Tsai, FPGA-based intelligent
power regulator IC design for forward DC-DC converters, in Proc. IEEE
Int. Conf. Intell. Comput. Intell. Syst., 2009, pp. 833837.
[24] N. K. Quang, Y. S. Kung, and Q. P. Ha, FPGA-based control archi-
tecture integration for multiple-axis tracking motion systems, in Proc.
IEEE/SICE Int. Symp. Syst. Integr., 2011, pp. 591596.
[25] M. Elloumi, M. Krid, and D. S. Masmoudi, Hardware implementation of
neural-fuzzy network based image denoising approximation, in Proc. 1st
Int. Conf. Image Process. Appl. Syst., 2014, pp. 15.
[26] J. M. Mendel, Uncertain Rule-Based Fuzzy Logic System: Introduction
and New Directions. Upper Saddle River, NJ, USA: Prentice-Hall, 2001.
[27] R. Sepulveda, O. Montiel, O. Castillo, and P. Melin, Embedding a high
speed interval type-2 fuzzy controller for a real plant into an FPGA, Appl.
Soft Comput., vol. 12, no. 3, pp. 988998, Mar. 2012.
[28] C. F. Juang and Y. W. Tsao, A type-2 self-organizing neural fuzzy system
and its FPGA implementation, IEEE Trans. Syst. Man Cyber. B, Cybern.,
vol. 38, no. 6, pp. 15371548, Dec. 2008.
4275
[29] M. A. Melgarejo and C. A. Pena-Reyes, Implementing interval type-2
fuzzy processors, IEEE Comput. Intell. Mag., vol. 2, no. 1, pp. 6371,
Feb. 2007.
[30] R. Sepulveda, O. Montiel, O. Castillo, and P. Melin, Modeling and simu-
lation of the defuzzification stage of a type-2 fuzzy controller using VHDL
code, Control Intell. Syst., vol. 39, no. 1, pp. 3340, Mar. 2011.
[31] C. F. Juang and K. J. Juang, Reduced interval type-2 neural fuzzy
system using weighted bound-set boundary operation for computation
speedup and chip implementation, IEEE Trans. Fuzzy Syst., vol. 21, no. 3,
pp. 477491, Jun. 2013.
[32] M. D. Schrieber and M. Biglarbegian, Hardware implementation and
performance comparison of interval type-2 fuzzy logic controllers for
real-time applications, Appl. Soft Comput., vol. 32, pp. 175188, Jul.
2015.
[33] C. F. Juang and C. Y. Chen, An interval type-2 neural fuzzy chip with on-
chip incremental learning ability for time-varying data sequence prediction
and system control, IEEE Trans. Neural Netw. Learning Syst., vol. 25,
no. 1, pp. 216228, Jan. 2014.
[34] C. Fleischer, W. Waag, Z. Bai, and D. U. Sauer, Self-learning state-of-
available-power prediction for lithium-ion batteries in electrical vehicles,
in Proc. IEEE Conf. Veh. Power Propulsion, 2012, pp. 370375.
[35] H. T. Lin and T. J. Liang, Prediction of state of charge for Li-Co batteries
with fuzzy inference system based fuzzy neural networks, in Proc. 1st
Int. Conf. Future Energy Electron., 2013, pp. 891896.
[36] F. J. Lin and P. H. Chou, Adaptive control of two-axis motion control
system using interval type-2 fuzzy neural network, IEEE Trans. Ind.
Electron., vol. 56, no. 1, pp. 178193, Jan. 2009.
[37] A.S. Weigend and N.A. Gersehnfield, Time Series Prediction: Forecasting
the Future and Understanding the Past. Reading, MA, USA: Addison-
Wesley, 1994.
[38] S. Singh, Noise impact on time-series forecasting using an intelli-
gent pattern matching technique, Pattern Recognit., vol. 32, no. 8,
pp. 13891398, Aug. 1999.
Chia-Feng Juang (M99SM08) received the
B.S. and Ph.D. degrees in control engineering
from National Chiao-Tung University (NCHU),
Hsinchu, Taiwan, in 1993 and 1997, respectively.
Since 2001, he has been with the Department
of Electrical Engineering, NCHU, Taichung, Tai-
wan, where he became a Full Professor in 2007
and has been a Distinguished Professor since
2009. He has authored or coauthored 7 book
chapters, over 90 journal papers (including over
50 IEEE journal papers), and over 100 confer-
ence papers. His current research interests include computational in-
telligence (CI), chip implementation of CI techniques, intelligent control,
computer vision, and evolutionary robots.
Dr. Juang received the Youth Automatic Control Engineering Award
from the Chinese Automatic Control Society (CACS), Taiwan, in 2006;
the Outstanding Youth Award from the Taiwan System Science and En-
gineering Society, Taiwan, in 2010; the Excellent Research Award from
NCHU, Taiwan, in 2010; the Outstanding Youth Award from the Taiwan
Fuzzy Systems Association, Taiwan, in 2014; and the Outstanding Au-
tomatic Control Engineering Award from CACS, Taiwan, in 2014.
Dr. Juang is an Associate Editor of the IEEE TRANSACTIONS ON FUZZY
SYSTEMS, IEEE TRANSACTIONS ON CYBERNETICS, Asian Journal of Con-
trol, and Journal of Information Science and Engineering and an Area
Editor of the International Journal of Fuzzy Systems.
Kai-Jie Juang received the B.S. degree in au-
tomatic control engineering from Feng Chia
University, Taichung, Taiwan, China, in 2010.
He is currently working toward the M.S.
degree in the Department of Electrical En-
gineering, National Chung-Hsing University,
Taichung, Taiwan.
His research interests include type-2 fuzzy
systems, neural fuzzy systems, and fuzzy chips.