**2.1 Literature Synopsis**

**2.1.6 Validating Quality of Multi-objective Search and Optimisation**

Various stochastic search and optimisation methods based on stochastic gradient descent variations like simulated annealing, taboo search; swarm intelligence methods like ant colonies, bacterial search; and evolutionary algorithms like genetic algorithms have been successfully used for fuzzy system optimisation [32], [16], [2], [9]. Genetic algorithms are efficient, well studied simple stochastic optimisation methods of proven convergence, capable of global multi-objective search and optimisation of versatile complex systems. Capabilities and performance characteristics of properly formed GAs are defined in [27] so the effect on the GA performance becomes obvious if any supplementary operator – like a new vector comparison or a new niching operator is included into a simple GA.

This paper will prove the validity of a new general vector comparison operator, suitable for applications in any mathematical or engineering field, including stochastic search, and evolutionary algorithms. For a relation to be a proper strict inequality relation it must have the properties of irreflexivity, antisymmetry and transitivity. I shall

mathematically prove these traits by analysing basic set cardinality properties of my proposed vector comparison operators. These new vector comparison methods provide more information when comparing two vectors than the classic Pareto-based comparison, thus the GA is faster, more efficient in its search.

Many benchmark problems were designed to test the effectiveness of a multi-objective GA. Of the most interest to us, thus what I will use is the analytically designed GA hard multi-objective function set presented in [79]; I have also used their generalisation to 4, 8 and 16 objectives. To have a clear, familiar simple baseline I have also implemented a single objective GA, which minimises the sum of 42, 84, 168 and 336 parameters – since the 2, 4, 8 and 16 multi-objective problems use this many parameters in total.

**2.1.6.1 ****Simple Two Objective Optimisation Problem **

According to [79] a simple two objective optimisation problem can be defined as:

Minimize f1(x) = f1(x1, x2,…, xm),

Minimize f2(x) = g(xm+1,…, xN)^{.}*h(f*1(x1,…, xm),g(xm+1,…, xN)). (15)

Figure 4.

Four hyperbolic lines (f1f2 = c) with c*1** < c**2** < c**3** < c**4* are shown.

The most simple case is where *f*_{1} = x_{1 }and *g(x*_{2}) (>0) is a function of *x*_{2} only, and h =
*g/f*1. The first objective function *f*1 is a function of *x*1 only and the function *f*2 is a
function of both x1 and x2. In the function space (that is, a space with (f1; f2) values), the
above two functions obey the following relationship: f1(x1; x2) ** ^{.}** f2(x1; x2) = g(x2).

For a fixed value of g(x2) = c, a f1 – f2 plot becomes a hyperbola (f1*f*2 = c) as shown for a

objective f* _{1}*, along a Pareto-front like f

_{1}

*f*

_{i}= c

*, for i=2,3,4,5. Results of the optimisation that correspond to the smallest achievable c are the global, Pareto-optimal solutions.*

_{i}For the general case of (15) the Global Pareto-optimal solutions are: 0 x1 1 and xi = 0 for i = 2, 3,…,N. In [79] the proposal is N=20.

**2.1.6.2 ****Deceptive Multiobjective Optimisation Problem **

A deceptive *g function is defined over binary alphabets, thereby making the search *
space discontinuous. Let us say that the following multiobjective function is defined
over l bits, which is a concatenation of N substrings of variable size *l*i such that sum(l*i**) *

*= l: *

*Minimise f*_{1}* = 1 + u(l*_{1}*),*

*Minimise f**2** = sum(g(u(l**i**)) / (1+u(l**1**)), * (16)
where u(l*1**) is the unitation of the first substring of length l*1.

The first function f_{1} is a simple onemin problem, where the absolute minimum solution
is to have all 0s in the first sub-string. A one is added to make all function values strictly
positive.

Figure 5.

10 000 randomly generated individuals and the true deceptive two objectives Pareto-front.

The function g(l_{1}) is defined as g(u(l_{1})) = 2 + u(l_{1}); if u(l_{1}) < l_{1}, or g(u(l_{1})) = 1; if u(l_{1}) =
*l*1.

This makes the true attractor (with all values of one in the substring) to have worst
neighbours and with a function value g(l_{1}) = 1 and the deceptive attractor (with all 0s in

the substring) to have good neighbours and with a function value g(0) = 2. Since, most
of the substrings lead towards the deceptive attractor, GAs may find difficulty to
converge to the true attractor (all 1s) as in Figure 5. Since each *g function has two *
minima (one true and another deceptive), there are a total of 2^{N}-1 local minima, of
which only one is global. Global Pareto-optimal solutions are: 0 u(x1 li and u(xi) = li

for i = 2, 3,…,N. In [79] the proposal is N=20.

**2.1.6.3 ****Multimodal Multiobjective Problem **

When the function g(x2) is multimodal with local x2 and global x2 minimum solutions,
the corresponding two objective problem also has local and global Paretooptimal
solutions corresponding to solutions (x_{1}; *x*_{2}) and (x_{1}; *x*_{2}), respectively. The
Paretooptimal solutions vary in x1 values.

Figure 6.

10 000 randomly generated individuals and the true multi-modal 2 objectives Pareto-front.

We create a modal, two objective optimisation problem by choosing a
multi-modal g(x_{2}) function:

Minimize f1(x) = f1(x1, x2,…,xm),

Minimize f_{2}(x) = g(x_{m+1},…,xN)^{.}*h(f*_{1}(x_{1},…,xm),g(x_{m+1},…,xN)),
*h = g/f*_{1, }where

*g(x*_{2},…,xN ) = 1+ 10(N-1) + sum(x_{i}^{2} – 10cos(2x_{i})). (17)
Global Pareto-optimal solutions are: 0 x 1 and x = 0 for i = 2, 3,…,N as in Figure 6.

To highlight the specifics, where the difficulty lies with optimisations along a multi-modal Pareto front, please observe the magnified part of the search space of the solution distribution in Figure 7.

Figure 7.

A magnified portion of the true multi-modal Pareto-front.

**2.1.6.4 *** Convex and Nonconvex Paretooptimal Fronts *
We choose the following function for h:

Minimize f1(x) = f1(x1, x2,…,xm),

Minimize f_{2}(x) = g(x_{m+1},…,xN)^{.}*h(f*_{1}(x_{1},…,xm),g(x_{m+1},…,xN)), where

*h(f*1, g) = 1 – (f1/g)^{}, if f1g; and 0 otherwise. (18)
With this function, we may allow 0f1, *g may be any function 0<g. The global *
Paretooptimal set corresponds to the global minimum of g function. The parameter is
a normalisation factor to adjust the range of values of functions *f*_{1} and *g. To have a *
significant Paretooptimal region, may be chosen as *f*1,max/gmin, where *f*1,max and
*g*_{min} are the maximum value of the function *f*_{1} and the minimum (or global optimal)
value of the function g, respectively.

The above function can be used to create multiobjective problems having convex Paretooptimal set by setting 1.

Global Pareto-optimal solutions are: 0 x1 1 and xi = 0 for i = 2, 3,…,N as in Figure 8.

In [79] the proposal is to use *N=20 *in equation (18), which results in a suffitienty
versatile search space.

Figure 8.

10 000 randomly generated individuals and the true convex two objectives Pareto-front.

Note that when > 1, the resulting Paretooptimal front is nonconvex. It is important to note that when > 1 is used, the classical weighted sum method cannot find any intermediate Paretooptimal solution by using any weight vector.

Although there exist other methods (such as perturbation method or goal programming method), they require problem knowledge and, moreover, require multiple application of the single objective optimiser.

Global Pareto-optimal solutions are: 0 x1 1 and xi = 0 for i = 2, 3,…,N as presented in
Figure 9. In [79] the proposal is to use *N=20 in equation (18), which results in a *
suffitienty versatile search space.

Figure 9.

10 000 randomly generated individuals and the true non-convex two objectives Pareto-front.

**2.1.6.5 ****Discontinuous Paretooptimal Front **

We have to relax the condition for h being a monotonically decreasing function of *f*1 to
construct multiobjective problems with a discontinuous Paretooptimal front. In the
following, we show one such construction where the function h is a periodic function of
*f*_{1}:

Minimize f1(x) = f1(x1, x2,…,xm),

Minimize f_{2}(x) = g(x_{m+1},…,xN)^{.}*h(f*_{1}(x_{1},…,xm),g(x_{m+1},…,xN)),

*h(f*1, g) = 1 – (f1/g)^{}- (f1/g)sin(2qf1) (19)
where *g may be any function 0<g. The parameter q is the number of discontinuous *
regions in an unit interval of f1. Since the h (and hence f2) function is periodic to x1 (and
hence to f1), we generate discontinuous Paretooptimal regions.

Global Pareto-optimal solutions are: 0 x1 1 and xi = 0 for i = 2, 3,…,N as in Figure 10. In [79] the proposal is N=20.

Figure 10.

10 000 randomly generated individuals and the true discontinuous two objectives Pareto-front along the non-dominated region.

**2.1.6.6 ****Biased Search Space **

The function *g makes a major role in introducing difficulty to a multiobjective *
problem. Even though the function g is not chosen to be a multimodal function nor to
be a deceptive function, with a simple monotonic g function the search space can have
adverse density of solutions towards the Paretooptimal region. Consider the following
function for g:

Minimize f1(x) = f1(x1, x2,…,xm),

Minimize f_{2}(x) = g(x_{m+1},…,x_{N})^{.}*h(f*_{1}(x_{1},…,x_{m}),g(x_{m+1},…,x_{N})),

*h = g/f*1, where (20)

*g(x*_{m+1},…x_{N}) = g_{min} + (g_{max} - g_{min} ) ((sum(x_{i}) – sum(x_{i,min})) / (sum(x_{i,max}) – sum(x_{i,min})))^{}
where gmin and gmax are minimum and maximum function values that the function g may
take. The values of *x*_{i,max} and x_{i,min} are minimum and maximum values of variable x_{i}. It
is important to note that the Paretooptimal region occurs when g takes the value g_{min}.
The parameter controls the biasness in the search space. If < 1 then the density of
solutions increases while getting further away from the Paretooptimal front.

Global Pareto-optimal solutions are: 0 x1 1 and xi = 0 for i = 2, 3,…,N as in Figure 11. In [79] the proposal is N=20.

Figure 11.

10 000 randomly generated individuals and the true biased two objectives Pareto-front.

Random search methods are likely to face difficulty in finding the Paretooptimal front in the case with close to zero, mainly due to the low density of solutions towards the Paretooptimal region.

Although multiobjective GAs, in general, will progress towards the Paretooptimal
front, a different scenario may emerge. Although for values of greater than one, the
search space is biased towards the Paretooptimal region, the search in a multiobjective
GA with proportionate selection and without mutation or without elitism is likely to
slow down near the Paretooptimal front. In such cases, the multiobjective GAs may
prematurely converge to a front near the true Paretooptimal front. This is because the
rate of improvement in *g value near the optimum (x*20) is small with 1.

Nevertheless, a simple change in the function g with a change in suggested above will change the landscape drastically and multiobjective optimisation algorithms may face difficulty in converging to the true Paretooptimal front.

**2.1.6.7 ****Generalisation of Two Objectives to Four Objectives **

The idea is to simply introduce a third function *f*3 that is similar to the first one *f*1 but
having different domains. The fourth function f_{4} is chosen to be similar to the second
function f_{2}, but also having different domains. Thus a general 4 objective problem is:

Minimize f_{1}(x) = f_{1}(x_{1}, x_{2},…, x_{m}),

Minimize f2(x) = g(xm+1,…, xN)^{.}*h(f*1(x1,…, xm),g(xm+1,…, xN)),

Minimize f3(x) = f1(xN+1, xN+2,…, xN+m), (21)
Minimize f4(x) = g(xN+m+1,…, xN+N)^{.}*h(f*3(xN+1,…, xN+m),g(xN+m+1,…, xN+N)),

where function f_{1}, g and h can be chosen identically as in equations (15)-(20). With the
same approach we can duplicate the four objective problems to eight competing
objectives and further on to 16 competing objectives.