2.3 Conclusions
A novel representation of hierarchical Takagi-Sugeno fuzzy models has been presented in this section.
The hierarchical structure is represented by a binary fuzzy tree. The model tree is identified by a supervised clustering algorithm embedded to a greedy incremental model building algorithm. The analysis of the clusters can be used for the selection of the relevant scheduling variables that are needed to handle the nonlinearity of the system. The presented fuzzy regression tree induction method is eval-uated in some well known synthetic (Fried, 3DSin) and real life (Abalone, MPG, Kin8nm) benchmark applications. Simulation results shown that the identified hierarchical TS-FS models are effective for the identification of linear/nonlinear systems. When compared to other fuzzy model identification (FMID, ANFIS, EM) and regression tree building (CART, GUIDE, SECRET, SECRET(0)) algo-rithms, the hierarchical TS fuzzy model exhibits competing results with a high accuracy and smaller size of architecture. The preliminary results are encouraging.
Chapter 3
Fuzzy Clustering for System Identification
In this chapter we deal with fuzzy model identification, especially by dynamical systems. In practice, there is a need for model-based engineering tools, and they require the availability of suitable dynamical models. Consequently, the development of a suitable nonlinear model is of paramount importance.
Fuzzy systems have been effectively used to identify complex nonlinear dynamical systems. In this chapter we would like to show how effectively clustering algorithm can be used in every step of the identification of compact Takagi-Sugeno fuzzy models to represent single-input single-outputand also multiple-input multiple-output dynamical systems.
Until this point, the order of the input-output model is assumed to be known. However, it is often not known beforehand and has to be determined somehow. Selecting the order of an input-output model of a dynamical system can be the key step toward the goal of system identification. The false nearest neighbors algorithm (FNN) is a useful tool for the estimation of the order of linear and nonlinear systems. Advanced FNN uses nonlinear input-output data based models. To increase the efficiency of this method, we present a clustering-based algorithm. Clustering is applied to the product space of the input and output variables. The model structure is then estimated on the basis of the cluster covariance matrix eigenvalues. The main advantage of the proposed solution is that it is model-free. This means that no particular model needs to be constructed in order to select the order of the model, while most other techniques are ‘wrapped’ around a particular model construction method.
This saves the computational effort and avoids a possible bias due to the particular construction method used.
The main approach mentioned above can be used not only for input-output systems but au-tonomous systems as well. It is a challenging problem by chaotic systems and it can be the key step toward the analysis and prediction of nonlinear and chaotic time-series generated by these sys-tems. The embedding dimension can be selected by the presented clustering based algorithm which is very similar to the algorithm described in Chapter 2. However, it is advantageous to think the whole procedure over again because of the difference of the contextures. The intrinsic dimension of the reconstructed space can then be estimated based on the analysis of the eigenvalues of the fuzzy cluster covariance matrices (the ’flatness’ of the clusters), while the correct embedding dimension is inferred from the prediction performance of the local models of the clusters. The main advantage of the proposed solution is that three tasks are simultaneously solved during clustering: selection of the embedding dimension, estimation of the intrinsic dimension, and identification of a model that can be used for prediction.
3.1 Data-Driven Modeling of Dynamical Systems
In this section, before the introduction of fuzzy models of dynamical systems, the widely used dynam-ical model structures will be reviewed.
When the information necessary to build a fundamental model of dynamical processes is lacking or renders a model that is too complex for an on-line use, empirical modeling is a viable alternative.
Empirical modeling and identification is a process of transforming available input-output data into a functional relation that can be used to predict future trends. While fundamental physical process models are mostly developed in continuous time, computer-based process control systems operate in discrete time: measurements are made and control actions are taken at discrete time instants. In addition, the available input-output data used for model identification are only available in discrete time instants. Hence, the identification of a discrete input-output dynamical model is based on the observed inputs {u(k)}k and outputs{y(k)}k [158],
{u(k)}k = [u(1), u(2), . . . , u(k)], (3.1) {y(k)}k= [y(1), y(2), . . . , y(k)], (3.2) Our aim is to find a relationship between past observations and future output
ˆ
y(k) =g({y(k−1)}k,{u(k−1)}k, θ) +e(k) (3.3) where θ is a finite-dimensional parameter vector and the additive term e(k) accounts for the fact that the next outputy(k) will not be an exact function of the past data. However, the goal is that e(k) remains small. Instead of using the whole previous input-output sequence, {u(k−1)}k and {y(k−1)}k, a finite-dimensional regression vector has to be used. This can be done if (3.3) is written as a concentration of two mappings: one that takes the increasing number of past inputs and outputs and maps them into a finite dimensional vector φ(k) of a fixed dimension,
φ(k) =ϕ({y(k−1)}k,{u(k−1)}k, η) (3.4) whereφ(k) input vector of the model is a subset of the previous input and output variables, generated by theϕ(.) model: whereηdenotes the parameter vector of theϕfunction that generates the regressors of thef(.) model
ˆ
y(k) =f(φ(k), θ). (3.5)
Following this scheme, the nonlinear system identification of dynamical systems involves the fol-lowing tasks [116]:
• Structure selectionHow to choose the nonlinear mappingf(·) and the regression vectorφ(k)?
• Input sequence designDetermination of the input sequence u(k) which is injected into the plant to generate the output sequencey(k) that can be used for identification (see [111] for more details).
• Noise modelingDetermination of the dynamic model which generates the noise e(k).
• Parameter estimationEstimation of the model parameters from the dynamic plant datau(k) andy(k) and the noisee(k).
• Model validation Comparison of the output of the plant and the model based on data not used in model development.
As the structure selection task has the largest effect on the performance of the model, a small review about regressor selection for dynamic systems is provided in the following subsection.
As the nomenclature of nonlinear dynamical models is based on the terminology used to categorize linear input-output models, in the following the linear empirical model structures are discussed that can be summarized by the general family [116, 158]
A(q)y(k) = B(q)
F(q)u(k) +C(q)
D(q)e(k) (3.6)
where q denotes the shift operator. For instance, A(q) is a polynomial in q−1. This model can be given in a “pseudo-linear” regression form
ˆ
y(k) =θTφ(k) (3.7)
where the regressors, i.e., the components ofφ(k), can be given by
• u(k−i), i= 1, . . . , nb, control signals (associated with the B polynomial)
• y(k−i), i= 1, . . . , na, measured process outputs (associated with theApolynomial),
• yˆu(k−i) simulated outputs from pastu(k) (associated with theF polynomial),
• e(k−i) =y(k−i)−y(kˆ −i) prediction errors (associated with theC polynomial),
• eu(k−i) =y(k−i)−yˆu(k−i) simulation errors (associated with theD polynomial).
Based on these regressors, different types of model structures can be constructed. For instance, the simplest linear dynamical model is the finite impulse response (FIR) model
ˆ
y(k) =B(q)u(k) +e(k) =b1u(k−1) +· · ·+bnu(k−nb) +e(k). (3.8) In this equation, the corresponding predictor ˆy(k) =B(q)u(k) is thus based on the
φ(k) = [u(k−1), . . . , u(k−nb)] regression vector.
Other special cases of (3.6) are known as the Box — Jenkins (BJ) model (A = 1), the AR-MAXmodel (F = D = 1), the output-error (OE) model (A = C = D = 1) and the ARX model (F =C=D= 1).
Following this nomenclature of linear models, it is natural to construct similar nonlinear model structures:
• NFIR, Nonlinear Finite Impulse Response models, in this case the regressor vector is composed as
φ(k) = [u(k−1), . . . , u(k−nb)]
• NARX, Nonlinear AutoRegressive with eXogenous input models, which use regressors φ(k) = [y(k−1), . . . , y(k−na), u(k−1), . . . , u(k−nb)]
• NOE, Nonlinear Output Error Models, which use
φ(k) = [ˆy(k−1), . . . ,y(kˆ −nb), u(k−1), . . . , u(k−nb)]
• NBJ, Nonlinear Box–Jenkins models, where the regressors are past inputs, past estimated puts, estimation errors using past outputs, and the estimation errors using past estimated out-puts
φ(k) = [ˆy(k−1), . . . ,y(kˆ −na), u(k−1), . . . , u(k−nb), εu(k−1), . . . , εu(k−nu), ε(k−1), . . . , ε(k−ne)]
In the soft-computing community, the NARX model is called a series-parallel model, while the NOE is referred to as a parallel model [132]. The NOE, and NBJ models are recurrent models, because they use the estimated output that constitutes a feedback. This makes the identification of these models difficult. Because the NARX model structure is non-recursive, its parameters are easy to estimate. Therefore, the NARX model is frequently used for identification. The identified NARX or series-parallel model is often used and tested as an NOE or parallel model, when the past outputs of the nonlinear model are used instead of the real plant outputs. By using this approach a multi-step-ahead prediction can be made, using the former predictions of the system. This procedure is often called ’free run’ simulation. This ’free run’ is a very rigorous test of the predictive power of the model, because in this way small errors can accumulate to major ones.
Many general empirical model structures, especially those employing a linear parameterization, lead to a large number of parameters to be estimated. An excessive number of unknown coefficients leads to an ill-conditioned estimation problem causing numerical difficulties and high sensitivity to noise and other identification errors. In the following some special model structure will be presented to avoid this dimensionality problem:
Lowering of the regressor dimension through input projection: Theϕ(.) model (3.4) is used to transform the original input-space of the dynamical model into a lower-dimensional space.
This transformation is parameterized by an η parameter vector and can be designed based on a priori knowledge about the dynamics of the system. The projection can also be determined on the basis of certain statistics computed using the data. Proposed methods are mostly extensions of the principal component analysis [48] and partial least squares. For instance, in the PLS neural network proposed by Qin and McAvoy, one dimensional linear projections are decided one at a time based on the residuals resulting after fitting a neural network to the projected data [147]. Methods to obtain nonlinear projections have also been published, but the computational complexities associated with these extensions have made it very difficult to apply them to realistic problems [48]. However, based on prior knowledge, semi-physical regressors can also be defined [170, 171].
ANOVA decomposition: The other option to reduce the search-space of the identification problem is to decompose the general structure into a collection of simpler sub-structures. For instance, consider the Gabor — Kolmogorov or ANalysis Of VAriance (ANOVA) decomposition of a general nonlinear function
f(φ) =f0+ Xn i=1
fi(φi) + Xn i=1
Xn j=i+1
fi,j(φi, φj) +· · ·+f1,2,...,n(φ1, . . . , φn) (3.9) where thef(φ) function is simply an additive decomposition of simpler subfunctions; in whichf0is a bias term andfi(φi), fi,j(φi, φj), . . .represent univariate, bivariate,. . .components. Any function and hence any reasonable dynamical system can be represented by this decomposition. Most of the systems certain input interactions will be redundant and hence components in the ANOVA decomposition can be ignored resulting in a more parsimonious representation.
By using this approach, the following well-known model classes can be constructed.
• NAARX Nonlinear Additive AutoRegressive models with eXogenous inputs models are defined
as [142] where the functions fi and gi are scalar nonlinearities. As can be seen, this model does not permit “cross terms” involving products of input and output values at different times.
• Volterra modelsare defined as multiple convolution sums ˆ
• Polynomial ARMA modelsare superior to Volterra series models in the sense that the number of parameters needed to approximate a system is generally much less with polynomial models [78] because of the use of previous output values.
ˆ Block-oriented modeling: Nonlinear effects encountered in some industrial processes, such as distillation columns, pH-neutralization processes, heat-exchangers, or electro-mechanical systems, can be effectively modeled as a combination of a nonlinear static element and a linear dynamic part [58, 145]. Because of the static nature of the nonlinearities, the problem of the nonlinear behavior of the system can be effectively removed from the control problem, allowing the use of simple linear algorithms instead of computationally intensive nonlinear programming ones [63, 136, 137].
According to the order of the static and dynamic blocks, three main block-oriented model structures can be defined.
• Hammerstein models
A special case of the NAARX model is theHammerstein model, where the same static nonlin-earityf is defined for all of the delayed control inputs [55, 58]:
ˆ
As it is depicted in Figure 3.1 this model can be obtained as a series combination of a memoryless nonlinearity and linear dynamics.
Figure 3.1: Hammerstein model structure.
• Wiener models
When the nonlinear element follows the linear block (see Figure 3.2), the resulting model is called the Wiener model [56, 183] that can be formulated as
ˆ y(k) =f
na
X
i=1
aiy(k−i) +
nb
X
j=1
bju(k−j)
+e(k). (3.14)
Figure 3.2: Wiener model structure.
• Feedback block-oriented models
The basic feedback block-oriented model structure as defined by the diagram shown in Figure 3.3 consists of a linear dynamic model in the forward path with a static nonlinearity in the feedback path [145].
Figure 3.3: Feedback-block-oriented model structure.
This model is also a member of the NAARX model family:
ˆ y(k) =
na
X
i=1
ai(y(k−i)) +
nb
X
j=1
bju(k−j) +
nb
X
j=1
f(y(k−j)) +e(k). (3.15)
The previously presented empirical model structures are frequently applied in the soft-computing community. Artificial neural networks are the most popular framework for nonlinear empirical model development [82, 132]. Usually, these models are applied in the NARX structure, although techniques based on Hammerstein [56, 63, 162] and Wiener models [56] and polynomial ARMAX models also have been presented [24, 78].