The class of Genetic Algorithms

A Genetic Algorithm (GA) is a global stochastic search algorithm used for the solution of dicult problems by the application of the principles of evolutionary biology and com-puter science. GAs use techniques such as inheritance, mutation, natural selection and recombination derived from biology. In GAs, populations of abstract representations of candidate solutions evolve toward better solutions. The evolution starts with a popula-tion containing random individuals and happens in generapopula-tions, in which stochastically selected individuals are modied (via recombination and mutation) to form the popula-tion of the next iterapopula-tion. The genes of the chromosomes contain the informapopula-tion where each gene represents a property.

In the following, the base concepts of GAs are presented along with their most com-mon operators and applications. The introductory material is neither exhaustive nor comprehensive. Its main purpose is to lay the ground for the base of assessment of the new methods presented in this thesis.

2.7.1 Base Concepts

Whenever a GA is congured to solve a particular problem, each individual in the pop-ulation of the algorithm represents a candidate solution of the problem at hand. The chromosome, which can be considered as the blueprint of the individual, stores the in-formation about the solution. How the data about the solution is encoded depends on the genetic representation of the problem. The chromosome is made up of genes, which can take several values, called alleles. One type of encoding is the binary representation, in which the state of each gene is either 0 or 1, making the chromosome equivalent to a bitstring. The allele aects the expression of a particular trait. The locus of a gene is its position in the chromosome. The genotype is the specic allele makeup of the individual.

The phenotype is the observable characteristic of the individual, which is the set of all traits. These concepts are shown in Figure 2.5.

During the course of the evolution, individuals are selected for reproduction and for survival. These are the two kinds of selection mechanism the GA is working with. The former (selection for reproduction) selects individuals according to their goodness (which is represented by their tness values), and in most of the cases, creates new individuals by applying the recombination and mutation operators on the selected chromosomes.

Recombination creates osprings by exchanging the genetic information of the parents, while mutation randomly changes the alleles of the newly created osprings. The latter (selection for survival) selects the individuals for the next generation from the set of newly created osprings and from the current generation.

2.7.2 Representation

The encoding of the problem is the mapping of the phenotype to the genotype, while decoding is the inverse operator, which calculates the object parameters (parameters of phenotype) from the genotype. The genotype encodes the genetic information of the

2.7 The class of Genetic Algorithms 27

Figure 2.5: A population with four individuals, with basic GA terminology highlighted. The genotype (binary string encoding) and phenotype (an unsigned integer represented at locus 1-2, and a signed integer represented in two's complement form at lo-cus 3-5) of the individuals are shown, as well as the calculated tness values ((signed integer)(unsigned integer)), for each individual.

individual, it is the representation of the problem. Crossover and mutation operators act on the genotype. In case every object parameter in the phenotype has a unique state in the genotype, then the representation is called direct, otherwise indirect. Traditional representations are the following: binary representation, real-coded representation, per-mutation encoding, value representation, tree representation. These are presented in some detail in Appendix A.2.1.

Usually binary encoding (Appendix A.2.1.1) is used for the representation of 0-1 KPs (dened in Section 3.3.1), where each bit of the binary string represents whether a specic item (identied by the locus of the bit) is in the knapsack or not. For bounded and unbounded KPs (where there are limited or unlimited copies of the same item available, respectively) integer encoding is used, meaning that each allele is an unsigned integer value that represents the number of a particular item in the knapsack.

2.7.3 Evaluation

Evaluation of the individuals is done by calculating an objective-score for each individual.

In case of single-objective optimization, the objective-score is a real number, in case of multi-objective optimization, the objective score is a vector of real numbers (as there is an objective score for each objective). The tness of the individual, which is a real number, is calculated from the objective score(s) with a possibility of taking the other individuals of the population into account. The objective score (or considering multi-objective optimization, each multi-objective score) is problem specic and therefore it should not be modied with the aim of enhancing the convergence of the evolution process.

However, the mapping of the objective score(s) to a tness value makes it available to

adjust the goodness of an individual for selection, as selection only takes the tness value as parameter. Considering the multi-objective case, which will be detailed in Section 2.8, the objective-score to tness mapping is anRⁿ→Rmapping, that takes the n-vector of objective-scores and represents it in a single tness value, thus hiding the multi-objective nature of the problem from the selection operator. During the following part of the section, only the single-objective case is considered.

The type of the objective-score to tness mapping is either scaling or ranking. In case of scaling, the tness is a linear or non-linear function of the objective-score, while in case of ranking, the population is sorted according to the objective-score, and each individual is assigned a tness value according to its position or rank in the population (two individuals with the same objective-score will get the same rank, but not the same position). A more detailed introduction to scaling and ranking is given in in [48].

Note that in many cases, the objective-score to tness value mapping is the identity function (f(x) =x), with the tness value being equal to the objective-score.

2.7.4 Operators 2.7.4.1 Selection

As it was mentioned earlier, in the general EA, two kinds of selection happen: selection for reproduction and selection for survival. The rst selects the individuals from the population for ospring creation, while the second selects the individuals of the new population. This section presents the selection techniques which select individuals for reproduction.

Selection for survival is a GA variant specic algorithm, and thus, it is usually de-tailed together with the evolution process of that particular GA. The default is to select each newly created ospring to survive into the next generation, and have all the individ-uals of the current population unselected. Algorithms which work this way are termed generational GAs, while those that not replace the whole population in each iteration are called steady-state GAs. In the following, the concepts of normalized tness and relative tness from [48] are presented here.

Normalized tness of a parent individual is the fraction of the population's total num-ber of ospring the given parent is assigned to produce in the next selection step.

The value of the normalized tness is in the interval [0,1], and is often seen as a probability value, which is true for some implementations of selection algorithms.

Relative tness of an individual is the tness of the individual normalized by the average tness of the population. The relative tness value gives a direct indication on how much the individual is better or worse than the current average individuals.

The most widely used selection algorithms are the following: roulette wheel selection, tournament selection, truncation selection, linear ranking selection, exponential ranking selection. These are presented in details in Appendix A.2.2. A more detailed description of selection schemes is presented in [49].

2.7 The class of Genetic Algorithms 29

2.7.4.2 Crossover

Crossover is a special form of recombination, used in GAs to create two osprings from two parents by exchanging the genetic information stored in the parent chromosomes.

Crossover is responsible for exploring the search space by making big jumps in it through the exchange of the alleles of parent individuals. There are many variants of crossover (three of the most popular are shown in Figure 2.6). Which one is preferred depends on the encoding and on the problem.

In binary encoding, the most common crossover operators are single point crossover, two point crossover, n-point crossover, uniform crossover and arithmetic crossover.

(a) One point crossover (b) Two point crossover (c) Uniform crossover Figure 2.6: The most commonly used crossover operators for binary representations

In single point crossover, shown in Figure 2.6(a), one crossover point is randomly selected (in this example it points to the4^th gene) and the two osprings are created by interchanging the bits of the parents after the crossover point. Analogously, in two point crossover (Figure 2.6(b)), the genes of the chromosomes are interchanged before and after the crossover points. N-point crossover works by interchanging genetic information be-tween odd intervals (determined by crossover points2k−1and2k, wherek= 1, . . . , n/2 ) and keeping them between even intervals. In uniform crossover, shown in Figure 2.6(c) bits are randomly copied from the parents. Arithmetic crossovers perform arithmetic op-erations on parents to form the new osprings, for example, with AND or OR operators.

In shue crossover, chromosomes are shued before single-point crossover occurs, and they are consequently deshued after [50]. In reduced surrogate crossover, one of the previously presented crossover operators are applied, but only to the non-identical bits of the chromosomes. This guarantees that the osprings created using reduced surrogate crossover will dier from their parents.

2.7.4.3 Mutation

The mutation operator in GAs is responsible for maintaining the genetic diversity of the population. Mutation is the only operator that can introduce new alleles in the population. Considering the binary representation, the most commonly used mutation

operator is the bit inversion, which randomly selects bits from the bitstring and inverts them. Besides that only mutation can introduce new alleles, it also helps avoiding local minima, by preventing the individuals from becoming too similar. Mutation is also responsible for exploiting promising areas of the search space, which is done by performing small alterations on some or all of the alleles.

2.7.5 Feasibility of Osprings

The crossover and mutation operators may render the ospring individuals into infeasible solutions of the problem they encode. The phenomena of infeasible osprings are frequent in the eld of constrained optimization and also in problems which use permutation encoding (Appendix A.2.1.3).

Basically, there are two ways to have feasible ospring: rst, by not rendering them infeasible, which can be achieved by employing feasibility-preserving operators [51], or second, by applying repair algorithms. Popular repair schemes are the Lamarckian and Baldwinian methods, which are frequently applied in EAs solving KPs. Both are greedy repair methods. Lamarckian repair modies the genetic information of the individual, while Baldwinian repair is executed on the y at each tness evaluation, keeping the genetic information of the individual intact. The comparison of the two methods are given in [52]. More on the feasibility of solution regarding constraint handling is given in Appendix A.5.

2.7.6 Formal denition of GAs

An abstract formalization of the EA concepts (representation, evaluation of individuals, operators) presented in the former sections are given in the following. A framework developed by Merkle and Lamont in 1997 [53], and by Bäck in 1996 [1], is presented here in Denition 8 and in Algorithm 1 to formalize and mathematically describe evolutionary algorithms and GAs (as GAs belong to the class of EAs). The denition of the formal framework is presented in [11]. The formalism gives us a more precise specication on what EAs are and how they work.

Before giving the formalization of EAs (see Denition 8), Merkle and Lamont dene the following concepts: decoding function, tness function, population transformation, random population transformation, evolutionary operator, recombination operator, mu-tation operator, selection operator. The denitions of these concepts are given in Ap-pendix A.3 to increase the lucidity of Denition 8, which is quite compact and details of it are hardly apprehensible without the complementary denitions. A more simple outline of the EA is given in Algorithm 2 to make the discussion of EAs more apprehensible.

Denition 8 (Evolutionary Algorithm) Let I be a non-empty set (the individual space), {µ⁽ⁱ⁾}_i∈N a sequence in Z⁺ (the parent population sizes), {µ⁰⁽ⁱ⁾}_i∈N a sequence

2.7 The class of Genetic Algorithms 31 sample spaces for random events associated with the evolutionary operators (see Appendix A.3 and [53] for details) are denoted withΩ. Then the algorithm shown in Algorithm 1 is called an Evolutionary Algorithm.

For the sake of clarity and readability, a simplied, and thus very shallow outline of EA is presented in Algorithm 2. This simple description hides the attributes of the evolu-tionary operators (selection, recombination, mutation) behind function calls (for example like recombine-individuals for recombination). Only input and output parameters of these functions are shown in this simplied description.

Algorithm 1 Evolutionary Algorithm Outline

χ∈ {true, false} {is a design parameter which is decided and set at compile-time} t:= 0;

Algorithm 2 Evolutionary Algorithm Outline (simple form) t:= 0;

initialize-population(P(0))

while(terminal-condition 6=true) do P⁰(t) :=recombine-individuals(P(t)) P⁰⁰(t) :=mutate-individuals(P⁰(t))

P(t+ 1) :=select-individuals-for-survival(P(t),P⁰⁰(t)) t:=t+ 1;

end while