In this section the solvability of the following infinite horizon DARE will be examined in detail assuming no pre-stabilization of the system (2.1).
P∞ =Q+ATP∞
I+BR−1BTP∞
−1
A Q=CTQ1C+CTQ2C
C =
I−CrT CrCrT−1
Cr
(2.24)
The DARE contains a state weighting Q and an input weighting R matrix. Usually, Q ≥ 0 and R > 0 are the given conditions for DARE solvability. In output tracking problems, the role of Q is slightly changed. It represents the combination of output tracking error weighting matrix Q2 ≥ 0 and an additional matrix Q1 ≥ 0. Q2 weights the tracking error related to the tracking outputs ykr = Crxk. The Q matrix satisfies to be positive semi-definite but this is not a sufficient condition for DARE solvability if one considers open-loop systems with the poles of A on the unit circle. Such systems are called type of integral (Type – I) often covering real physical phenomenons and therefore these are strictly related to real applications [BRSB08].
In (2.6) Cr ∈ Rr×n means that max(rank(Cr)) = r (r ≤ n and so, it can have r linearly independent rows and also columns). Considering this maximum rank, (2.6) can be transformed with a permuting similarity transformation (ˇxk=TSxk) to achieve a form in whichCr is the following:
Cˇr=
V V M
where ykr = ˇCrxˇk (2.25) In (2.25) the V block contains the r linearly independent columns of Cr and so, it is a full rank quadratic matrix (V ∈Rr×r). M is an appropriately selected r x (n−r) matrix which constructs the other linearly dependent columns ofCr from V.
In special cases, when the number of states affecting the controlled output (yrk) equals with it’s dimensionr,M = 0 because all of the linearly dependent columns in Cr are zero vectors.
In the rest of this chapter, A, B, Cr will denote the matrices of the transformed system equations ˇA=TSATS−1, Bˇ =TSB, Cˇr =CrTS−1 respectively. Now, the sufficient conditions for DARE solvability, and the possible problems for type–I open-loop systems will be examined considering these matrices.
Usually, the conditions given to obtain a solvable DARE are Q ≥ 0 and R > 0 considering a DARE in the following form (which is equivalent to (2.24)):
P∞=ATP∞A+Q−ATP∞B BTP∞B +R−1
BTP∞A (2.26) However, textbook [14] p. 135 gives two, more accurate conditions which have to be satisfied to solve equation (2.26). These are the followings:
1. Condition 1: The pair A, BR−1/2
has to be stabilizable.
2. Condition 2: The pair Q1/2, A
should not have unobservable modes on the unit circle
Now examine these two conditions for type–I open-loop systems (or for systems having poles on the unit circle). The first condition is guaranteed to be satisfied, if the system matrices combined with the inverse square root of the input weight satisfy the Kalman rank condition:
rank
BR−1/2 ABR−1/2 . . . An−1BR−1/2
=n (2.27)
However,R−1/2 can be moved out from the brackets which results in the following condi-tion:
rank
B AB . . . An−1B
R−1/2
=n (2.28)
The original condition forRwas its positive definiteness, which also means that the matrix is full rank and invertible. So, its inverse square root is also full rank, which means that it does not modify the rank of the matrix product in (2.28). From this, it is obvious that, if the pair (A, B) is controllable (it satisfies the Kalman rank condition) the pair
A, BR−1/2
is also controllable.
However, we only assumed the stabilizability of the pair (A, B). Multiplication of B by a full rank matrix preserves its rank and also the properties of the pair (A, B). So, the stabilizability condition (Condition 1), is satisfied by any plant with stabilizable (A, B) pair. This means that, the first DARE solvability condition is satisfied in these cases.
The satisfaction ofCondition 2 is not so straightforward. The Kalman rank condition can not be used to decide about unobservable modes on the unit circle, instead another condition (given in [14] p. 354) should be used. This is the following:
An eigenvalue (mode) λ of A is observable (considering the pair Q1/2, A ) iff rank
A−λI Q1/2
=rank(Λ) =n (2.29)
So, the second solvability condition can be checked testing (2.29) for every eigenvalue of the DT state matrixA on the unit circle. To point out the problems, assume that the A matrix is transformed into Jordan canonical form with transformation matrix T and contains the examined λ eigenvalue in its first Jordan block (this can be again achieved with transformation by permuting matrices). Assume also that, theQmatrix is permuted similarly (but preserve the notationQ). In this way the eigenvalue observability condition becomes:
rank(Λ) = rank
T(J−λI)T−1 T T−1Q1/2T T−1
=n (2.30)
where:
J =
λ 1 0 0 . . . 0 0 λ 1 ... ...
0 0 λ 0 . . . 0
0 X
Here λ was assumed to be on the unit circle and have a multiplicity of three, but the method works for any multiplicity. X means the bidiagonal matrix constructed from the other Jordan blocks. Considering the above Jordan canonical form, the rank condition will be the following (from (2.30)):
rank
T 0
0 T
0 1 0 0 . . . 0 0 0 1 ... ... 0 0 0 0 . . . 0
0 X
T−1Q1/2T
T−1
=n (2.31)
This shows that the rank of A− λI will be rank(A)−1 = n−1 if rank(A) = n (assume this best case) because another zero column (and also row) is generated. The matrix in the rank condition is 2n×n so its rank will be below n if it has one or more linearly dependent column orn+ 1 or more linearly dependent row.
The original condition for Q is its positive semi definiteness which means that, it can be a zero matrix. At first, examine this case. If Q is a zero matrix, then Λ will have at least one zero column so, its rank is below n and the Q1/2, A
pair has unobservable mode on the unit circle. In this case Λ has at leastn+ 1 zero rows so, its row rank (i.e.
the number of linearly independent rows) is also belown. This means that, Qshould not be zero matrix (and usually it isn’t). The question is that, how close has to be Q to a positive definite matrix (how many zero and nonzero eigenvalues are allowed) to satisfy Condition 2.
Of course, Condition 2 depends on the system matrix A. If one examines (2.31) it is obvious that, if Q has even one nonzero row different from all the rows of J −λI then Λ has less than n+ 1 linearly dependent rows (if rank(A) = n) so, its row rank is ≥ n.
This means that only its column rank (i.e. the number of linearly independent columns) can be belown. Now assume Q in the following form:
Q= 0 0
0 Y
where Y diagonal with elements in R
(2.32)
Examine T−1Q1/2T in (2.31). At first, consider the left multiplication with T−1 this results in:
0 Z1
0 Z2
T
where Z1 and Z2 are f ull blocks
(2.33)
Now the column rank of the matrix in (2.31) depends on the T matrix which is unpredictable without the knowledge of A (will the first column of T−1Q1/2T result as zero, or not?). This means that the assumption (2.32) can be relaxed to consider any nonzeroQmatrix. Experiences show that, if Qhas a lot of zero columns (or equivalently rows because Q is symmetric), then the probability of loosing column rank in (2.31) is high and this violatesCondition 2.
However, if Q > 0 holds then T−1Q1/2T will be full rank and the pair Q1/2, A will not have unobservable modes on the unit circle.
As a summary, it can be stated that,Condition 1 is satisfied by the considered system class, meanwhile the satisfaction ofCondition 2 highly depends on the selection of the Q matrix. In the following, the weighting strategy with Q1 and Q2 (originally proposed in [2]) will be examined, taking special attention to the positive definiteness of Q.
Considering Q1 = 0 which means Q = CrTQ2Cr where Cr ∈ Rr×n, Q2 ∈ Rr×r (it weights the output tracking error) and so, Q ∈ Rn×n. If Q2 > 0 it is a full rank (rank(Q2) = r) invertible matrix, but Cr has only rank r. Matrix multiplication can not increase the rank of the product, which means thatQ will be a matrix with rankror
belowr. So, it is a singular matrix and not positive definite. This can probably result in an unsolvable DARE mainly for open-loop systems having many poles on the unit circle.
Another problem can be the lack of constraints on states unaffected byQ=CrTQ2Cr. This is obvious for designs withM = 0 (Cr =
V 0
) whereQ results as:
Q=
VTQ2V 0
0 0
The unaffected states will not be constrained by the controller, which can result in poor system performance and even in system damage.
[2] proposes a weighting strategy with nonzero Q1 which solves both of the problems (DARE solvability and state weighting) for the class of systems considered here and with output tracking design characterized byM = 0. This will be pointed out in the following.
Examine the structure of the combined state weighting matrix:
Q=CTQ1C+CrTQ2Cr (2.34) Above, it is pointed out that, Q >0 guarantees DARE solvability so, if (2.34) gives a positive definite matrix, the DARE in (2.24) will be solvable (for R >0).
Qis the state weighting matrix in LQ optimal control, so it characterizes the quadratic term for the states (or state tracking errors here) in the functional: xTkQxk. However, xk can be partitioned into two parts considering the partitioning of Cr = ˇCr in (2.25):
xkT =h
xIkT xIIk T
i (2.35)
Assume that,Q1 andQ2 are positive definite matrices, but partition Q1 in the same way asxk:
Q1 =
Q1,11 0 0 Q1,22
Q1,11>0 Q1,22 >0 (2.36) In (2.34) C and Cr are rank deficient matrices so, Q will be only a positive semidefinite matrix if Q1 and Q2 are positive definite and one assumes general matrices. However, C, Cr and Q1 have special structure (as assumed) so, one has the possibility to have a positive definite resultant Q matrix. From these preliminaries considering (2.25) and by evaluatingC and (2.34) one gets (with algebraic manipulations) three different matrices (for details see Appendix 8.6) with M3 =I+MTM:
Q1,11 = M3−TMTQ1,11M M3−1 ≥0 Q1,22 = M3−TQ1,22M3−1 >0
Q2 = VTQ2V >0
Q1,11+Q1,22can be denoted asQ1 >0. It can be verified that the quadratic expression xTkQxk can be equivalently written in the following form by considering (8.22) and (2.35):
xTkQxk =h
xIkT xIkTM −xIIk T xIIk TMT i·
Q2 0 0 Q2 0 Q1 Q1 0 0 Q1 Q1 0
Q 0 0 Q
·
xIk MTxIk
−xIIk M xII
(2.37)
In this way, the positive definiteness of Q needs the positive definiteness of the matrix on the right hand side of (2.37) (if one considers xk with arbitrary nonzero xIk and xIIk components). This property can be examined using the Sylvester criterion, which states that, a matrix is positive definite if all the determinants of its leading principal submatrices are positive. Examine the determinant of the 3 x 3 quadratic block of the matrix given in (2.37) using the lemma for the determinant of a block matrix and considering the properties: Q1 >0 andQ2 >0.
DET =
Q2 0 0 0 Q1 Q1 0 Q1 Q1
= det Q2
det Q1
·
det Q1−
0 Q1
"
Q−12 0 0 Q−11
# 0 Q1
!
= 0
(2.38)
The determinant in (2.38) is zero, because the last term contains the determinant of a zero matrix. This means that, we have found a leading principal submatrix which has a non positive determinant. So, the matrix given in (2.37) and also Q can not be positive definite in a general case (M 6= 0).
However, one can also examine the special case when M = 0 (Cr = V 0
). In this case Q1,11= 0, Q1,22=Q1,22>0,Q2 =VTQ2V >0 and Q1 =Q1,22 =Q1,22>0. Finally, the expression forxTkQxk (in (2.37)) can be simplified to:
xTkQxk =h
xIkT xIIk T i·
Q2 0 0 Q1,22
· xIk
xIIk
(2.39) This contains a positive definite matrix which is equal withQ and so, guarantees DARE solvability. Note that, here Q2 weights the states considered in yrk (see the structure of Cr above) and Q1,22 weights the states unaffected byQ2.
The results of this examination can be summarized in the following theorem:
Theorem 1 (Sufficient weighting for DARE solvability) Consider a discrete time, linear, time invariant system with state equation: xk+1 =Axk+Buk, and controlled output ykr = Crxk (xk ∈ Rn, uk ∈ Rm, ykr ∈ Rr, A ∈ Rn×n, B ∈ Rn×m, Cr ∈ Rr×n). Assume that, the pair(A, B) is stabilizable and all the n−r linearly dependent columns of Cr are zero (this means that, Cr can be transformed into the form given in (2.25) with M = 0).
If all the above conditions are satisfied, the following weighting strategy guarantees DARE solvability (irrespective of the structure of A) and makes it possible to weight the states considered and not considered in the controlled output separately:
State weighting matrix: Q=CTQ1C+CrTQ2Cr
where: Q2 >0
Q1 >0 diagonal C =I−CrT CrCrT−1
Cr
Input weighting matrix: R > 0
Remark: The system matrices A, B, Cr do not have to be transformed to achieve Cr = V 0
(see (2.25)) but in the case they are transformed, theQ1 matrix can be partitioned into Q1,11 and Q1,22 as in (2.36). In this case, Q1,11 = 0 can be set, because C =
0 0 0 I and so, Q1,11 does not have effect on Q. Note that, in this case Q1,22 weights the states not considered in the controlled output. The transformation means the reordering of system states, so in the untransformed case this concept can also be used by nulling out the elements of Q1 where C has zero diagonal entries.
In the following, the examination will be extended to non diagonal Q1 matrices.
Q1 =
Q1,11 Q1,12
QT1,12 Q1,22
>0 (2.40)
The positive definiteness of Q1 implies constraints on its blocks considering block matrix determinant and Sylvester criterion (a matrix is positive definite if all the deter-minants of its leading principal submatrices are positive).
det(Q1,11)>0 ⇒ Q1,11>0 det(Q1) = det(Q1,11)
| {z }
>0
det Q1,22−QT1,12Q−11,11Q1,12
| {z }
>0
>0 Q1,22−QT1,12Q−11,11Q1,12
| {z }
≥0
>0 ⇒ Q1,22 >0
(2.41)
Consider now the construction of Q in a similar way as in (2.37). The same steps as presented in Appendix 8.6 lead to:
Q1,11=M3−TMTQ1,11M M3−1 ≥0 Q1,22=M3−TQ1,22M3−1 >0 Q2 =VTQ2V >0
Q1 =Q1,11+Q1,22 >0
Q1,12=MTM2−TQ1,12M3−1+M3−TQT1,12M M3−1
(2.42)
The matrix in (2.43) contains the same first 3x3 block as in (2.37) which violates the Sylvester criterion. This means that in the general case (M 6= 0) even non diagonal Q1
and Q2 do not guarantee DARE solvability. However, in the special case when M = 0, the same expression as in (2.39) results and this guarantees the solvability.
xTkQxk=h
xIkT xIkTM −xIIk T xIIk TMT −xIkTMi
·
Q2 0 0 Q2 M2−TM QT1,12M M3−1
0 Q1 Q1 0 0
0 Q1 Q1 M3−TQT1,12M2−1 Q1,12
Q2 0 M2−TQ1,12M3−1 Q2 0 M3−TMTQ1,12MTM2−1 0 QT1,12 0 0
·
·
xIk MTxIk
−xIIk M xIIk
−MTxIk
(2.43) Remark: In the derived multi-step LQ optimal tracking solution the pre-stabilization of the system solves the possible problems with DARE solvability because the pre-stabilized Φ system matrix will not have eigenvalues on the unit circle. However,Q1 can be further used to weight the dynamics of states unaffected by the controlled output weighting Q2.