Linear-in-parameters Models

Genetic Programming for System Identification

Because nonlinear dynamical models play very important role in chemical process engineering, it is important to deal with the structure identification of these models. One of the most preferred structure identification method is Genetic Programming (GP) which is a data-driven symbolic optimization al-gorithm. Genetic Programming selects potential solutions from a given space of possible structures using evolutionary technique to find a minima (or max-ima) of a given cost function.

Although GP is an effective algorithm for the identification of model tures, it tends to generate overparameterized models when it is used for struc-ture identification of dynamical models. Because the model transparency is very important from the aspect of practical usefulness, it is important to find a balance between accuracy and model transparency. Relatively little research has been done on this problem. The main goal of this chapter is to propose a method that eliminates superfluous model terms during structure identifica-tion in such a way that the model preserves its accuracy.

This chapter is organized as follows. In Sect. 3.1 linear-in-parameters mod-els and the main idea of the proposed method will be introduced. In Sect. 3.2 the bases of the structure identification algorithm will be presented, and in Sect. 3.3 the proposed method is discussed. Finally, in Sect. 3.4 presents the application examples.

3.1 Linear-in-parameters Models

3.1.1 Introduction to Linear-in-parameters Models

Data-driven identification of model structure cannot be separated from the identification of model parameters, because it is not possible to determine how good and accurate a given model structure is without the parameters.

So to evaluate potential model structures, one has to identify the parameters for these models. Unfortunately, if a model is nonlinear, the identification of

its parameters requires nonlinear optimization algorithm. In most cases data-driven model structure identification leads to very difficult and ill-conditioned nonlinear optimization problems with numerical difficulties and high sensitiv-ity to noise. Hence, even if one finds a good model structure, this model structure may turn out to be useless due to the model parameter identifica-tion step. For example, it is possible that the modeler chooses a poor model structure instead of a good one because he or she is not able to identify the parameters of the ’better’ model.

The very first step of model structure identification is the selection a model family that contains the set of candidate model structures. Consequently, it is worth to select such a model family which does not suffer from the above described difficulties. For this purpose this chapter proposes the appli-cation of linear-in-parameters models. Linear-in-parameters models are quite widespread in process engineering, e.g. let us consider the following well-known model classes:

• NAARX Nonlinear Additive AutoRegressive models with eXogenous in-puts models are defined as [48]

ˆ where the functions f_i and g_i are scalar nonlinearities. As can be seen, this model does not permit ’cross terms’ involving products of input and output values at different times.

• Volterra models are defined as multiple convolution sums ˆ

• Polynomial ARMA models are superior to Volterra series models in the sense that the number of parameters needed to approximate a system is generally less than with polynomial models [49] because of the use of previous output values.

3.1 Linear-in-parameters Models 37 Generally, linear-in-parameters models are formulated as

ˆ y(k) =

M i=1

p_iF_i(x(k)), (3.4)

where F₁, . . . , FM are nonlinear functions (they do not contain parameters), p₁, . . . , p_M are model parameters, ˆy(k) is the model output at k-th time in-stant. x(k) is the regressor-vector at the k-th time instant and it consists of u input, y output ande error values:

x(k) = (u(k−nd−1),· · ·, u(k−nd−nu), y(k−nd −1),· · ·,

y(k−n_d −n_y), e(k−n_d−1),· · ·, e(k−n_d−n_e)), (3.5) where nd is the dead-time, nu, ny and ne are the input-, output- and error-orders. Structure identification of linear-in-parameters models includes two types of problems:

• Identification of model order, namely finding appropriate M, nd, nu, ny

and n_e values (integer values).

• Identification of F₁, . . . , FM nonlinear model equations (symbolic opti-mization).

This chapter will deal with these types of identification problems in the fol-lowings.

The great advantage of linear-in-parameters models is that the linear Least Squares (LS) method can be used for parameter identification, which is much less computationally demanding than nonlinear optimization algorithms. The LS method minimizes the square error between measured output and calcu-lated output, i.e. minimizes the

χ² = cost function, where N is the number of data-points and M is the number of regressors. The optimal p = [p₁, . . . , pM] parameter vector, where χ² is minimal, can be calculated by

p =

3.1.2 Orthogonal Least Squares Method

The Orthogonal Least Squares (OLS) method [50,51] is an effective algorithm to determine which terms are significant in a linear-in-parameters model. The OLS method introduces error reduction ratio, which provides the decrease in the variance of output by a given term. The compact matrix form of linear-in-parameters models (3.4) is the following:

y=F·p+e, (3.9)

where F is the regression matrix (3.8), p is the parameter vector, e is the error vector. The OLS method transforms columns of F matrix into a set of orthogonal basis vectors in order to inspect the individual contributions of each term.

The OLS method assumes that the F regression matrix can be orthog-onally decomposed as F = WA, where A is an M ×M upper triangular matrix (it means Ai,j = 0 if i > j) and W is anN×M matrix with orthogo-nal columns in the sense that W^TW =D diagonal matrix. (N is the length of y vector, M is the number of regressors.) The orthogonal decomposition can be done by standard mathematical packages such as MATLAB. Then one can calculate the OLS auxiliary parameter vector as

g =D⁻¹W^Ty, (3.10)

The output variance (y^Ty)/N can be explained as y^Ty=

M i=1

g_i²w^T_i wi+e^Te, (3.11)

thus the err (error reduction ratio) of F_i can be expressed as [err]ⁱ = g_i²w_i^Tw_i

y^Ty . (3.12)

This ratio offers a simple mean for ordering the terms, so it can be easily used to select the significant model terms.

3.1.3 Model Structure Identification for Linear-in-Parameters Models

The problem of model structure identification for linear-in-parameters models is to find the model order and the proper set of nonlinearFi functions of (3.4).

To attack this problem, two approaches can be distinguished:

• The first approach generates all of the possible model structures and then selects the best.