Többszintű genetikus algoritmus és táplálkozás- és életmód-tanácsadó

(1)

University of Pannonia

Faculty of Information Technology

Multi-Level Genetic Algorithms and Expert System for Health Promotion

Author:

Balázs Gaál

Supervisor:

Dr. György Kozmann

Dissertation presented to the Information Science & Technology PhD School of University of Pannonia in partial fulllment of the requirements for the degree of

Doctor of Philosophy December 1, 2009

(2)

(3)

Multi-Level Genetic Algorithms and Expert System for Health Promotion

(Többszint¶ genetikus algoritmus és táplálkozás- és életmód-tanácsadó szakért®i rendszer)

Értekezés doktori (PhD) fokozat elnyerése érdekében Írta:

Gaál Balázs

Készült a Pannon Egyetem Informatikai Tudományok Doktori Iskolája keretében Témavezet®: Dr. Kozmann György

Elfogadásra javaslom (igen / nem)

(aláírás) A jelölt a doktori szigorlaton ...%-ot ért el

Veszprém ...

a Szigorlati Bizottság elnöke Az értekezést bírálóként elfogadásra javaslom:

Bíráló neve: ... (igen / nem)

(aláírás) Bíráló neve: ... (igen / nem)

(aláírás) A jelölt az értekezés nyilvános vitáján ...%-ot ért el

Veszprém, ...

a Bíráló Bizottság elnöke A doktori (PhD) oklevél min®sítése ...

...

Az EDT elnöke

(4)

(5)

Tartalmi kivonat

A Többszint¶ genetikus algoritmus és táplálkozás- és életmód-tanácsadó szak- ért®i rendszer cím¶ disszertáció tartalmát képez® kutatási célok, vizsgálatok és ered- ményeik bemutatása - ahogy az a dolgozat címéb®l kiolvasható - kett® egyenl® fontosságú részre osztható. E két cím, két szakasz alatt bemutatott munka azonban szorosan össze- függ® és egymásba kapcsolódó.

A tézis els® jól körülhatárolható kutatási témájához a szerz® által kidolgozott új genetikus algoritmus (Genetic Algorithm (GA)) - amely a többszint¶ genetikus algoritmus nevet kapta (Multi-Level Genetic Algorithm (muleGA)) - és a hozzá kapcsolódó kísér- leti vizsgálatok bemutatása és eredményeinek értékelése tartozik. Ezen új algoritmus kifejlesztését a többszint¶, sokdimenziós és nagy keresési térrel rendelkez® optimalizá- lási feladatok megoldására jól alkalmazható módszerek iránti igény gerjesztette. (Ilyen optimalizálási feladatként kerül megfogalmazásra a táplálkozási- és életmód tanácsadás probléma is - lásd kés®bb). A többszint¶ optimalizálási problémának létezik egy ésszer¶

felbontása részproblémákra, és optimális megoldásának megtalálásához felhasználható a részproblémák, vagy akár azok továbbontása során létrejött rész-részproblémák (és így tovább) egyenkénti optimumának megkeresése. Az oszd-meg-és-uralkodj típusú dekom- pozíciós technológia tradicionális, azonban az evolúciós algoritmusok területén eddig nem került kell®en kiaknázásra. A muleGA kidolgozása egy lépés ebbe az irányba.

A dolgozat második jól körülhatárolható része egy olyan szakért®i rendszer mester- séges intelligencia moduljának kidolgozását mutatja be, amely személyhez szabott diet- etikai és életviteli tanácsokkal látja el a felhasználót: egyedi étrend- és zikai aktivitás tervet készít a páciens igényeit gyelembe véve. E tevékenység anamnézis és szintézis al- feladatokra bontható. Az el®bbi a felhasználói igények felmérését, a személyes optimum értékeinek feltárását, utóbbi ezen értékek alapján végzett javaslat el®állítását foglalja ma- gában. A disszertációban bemutatott munka a szintézis alfeladatra fókuszál. E feladat informatikai szemszögb®l megfogalmazva egy kombinatorikus optimalizálási probléma, amely nyomokban hasonlóságot mutat olyan ismert feladatokkal, mint az órarendterve- zés és ütemezés problémák (timetabling and scheduling) és a hátizsák probléma (knapsack problem), lényeges tulajdonsága, hogy a megoldás egészére és részmegoldásaira is vonatkoznak mennyiségi és összeférhet®ségi megkötések. El®bbiek a javaslat tartalmi szempontok szerinti értékeléséhez, utóbbiak a megoldást el®állító részkomponensek tet- sz®leges kombinációjára felírt, és azok harmóniájára vonatkozó követelmények megtartá- sához szükségesek.

(6)

(7)

Abstract

The aim of the thesis related research is twofold, with a connection existing between the two topics.

In the rst part of the dissertation, a novel Genetic Algorithm (GA), the Multi-Level Genetic Algorithm (muleGA) is presented, which has been developed by the author for the solution of high-dimensional, multi-objective and multi-level optimization problems.

Multi-Level Optimization Problems (MLOPs) are such problems, that have a reasonable decomposition into subproblems which may also have separate and rational subproblems (and so on). This divide-and-conquer type decomposition is traditional in solving large scaled design optimization problems, but its potential has not been exploited in the eld of Evolutionary Computation (EC), and there is a lack of truly competitive GA based methods for solving MLOPs. Furthermore, GAs lose their performance advantage over random search as the dimension of the search space increases. The introduction of muleGA is a step to overcome these limitations of GAs.

The second part of the thesis deals with the methods developed for a novel expert system, which attempts to generate an output that matches the quality of dietary menu plans and physical activity plans produced by human experts. A mathematical formalization of the goals of personalized planning is given, which reveals the task of personalized planning is a hierarchical, multi-objective MLOP whose subproblems have resemblaces to such traditional optimization problems like the Knapsack Problems (KPs) and the scheduling and timetabling problems. A muleGA based method is introduced for the solution of this MLOP problem, as well as a rule-based assessment technique, which is responsible for maintaining the harmony of the components of the plans. If congured properly, the novel methods succeed in creating quantitively and qualitatively adequate personalized dietary plans and workout timetables. Proof on the eectiveness of the newly developed methods is given through empirical testing and statistical analysis.

(8)

(9)

Zusammenfassung

Mehrstuger genetischer Algorithmus und fachkundiges Ernährungs- und Le- bensstilberatersystem. Wie aus dem Titel der Arbeit ersichtlich, hat der Autor den Inhalt der Dissertation in zwei gleichwertige Teile gegliedert. Diese zwei Teile sind jedoch eng miteinander verbunden.

Zum ersten Forschungsthema gehört der vom Autor ausgearbeitete genetische Algo- rithmus (GA), ein sogenannter mehrstuger genetischer Algorithmus (muleGA), und die Vorstellung der zu diesem Algorithmus gehörenden Untersuchungen und die Bewertung deren Ergebnisse. Die Anforderung auf eine gut verwendbare Methode für die Lösung der mehrstugen, mehrdimensionalen Optimierungsaufgaben für groÿe Suchräume, war die Grundlage für die Entwicklung dieses neuen Algorithmus. (Als solche Optimierungsauf- gabe wird auch die Nahrungs- und Lebensstilberatung formuliert - siehe weiter unten.) Unter einer mehrstugen Optimierungsaufgabe versteht sich ein solches Problem, bei dem eine sinnvolle Aufteilung in Teilprobleme existiert, und zum Finden der optimalen Lösung dieses Problems die Einzeloptima der Teilprobleme, oder sogar weitere Aufteilung in Teilteilprobleme usw. verwendet werden können. Die Technologie divide et impera (teile und herrsche) gilt als traditionell, aber bisher wurde diese Technologie auf dem Gebiet der evolutionären Algorithmen nicht gebührend ausgenutzt. Die Ausarbeitung von muleGA bedeutet einen Schritt in diese Richtung.

Der zweite Teil der Arbeit stellt die Ausarbeitung des Moduls einer künstlichen Intelli- genz eines fachkundigen Systems vor. Damit kann der Benutzer individuelle Ernährungs- und Lebensstillratschläge bekommen. Das Modul erstellt einen persönlichen Nahrungs- und Aktivitätsplan unter Berücksichtigung der Ansprüche des Patienten. Diese Tätigkeit kann in zwei Phasen - Anamnese und Synthese - geteilt werden. Erstere bedeutet die Ermittlung der Benutzeransprüche und die Erschlieÿung der persönlichen Optimalwerte, das Zweite die Erstellung eines Vorschlags, aufgrund der Ergebnisse der ersten Phase. In der vorliegenden Arbeit hat sich der Autor eher auf die Synthese konzentriert. Diese Auf- gabe gilt aus informationstechnischer Sicht als kombinatorisches Optimierungsproblem, welches Ähnlichkeiten mit bekannten Aufgaben, wie etwa die Probleme der Zeitplaner- stellung, Aufgabenplanung und der sog. Rucksackproblem hat. Eine wesentliche Eigen- schaft dieses Problems ist es, dass sich quantitative und Kompatibilitätsbedingungen sowohl auf die Gesamtlösung, als auch auf die Teillösungen beziehen. Die Ersteren sind zur Bewertung des Vorschlags vom inhaltlichen Standpunkt nötig, während die Letztere zur Einhaltung der Erfordernisse im Bezug auf die Harmonie der beliebigen Kombination der zur Lösung benötigten Teilkomponenten gebraucht werden.

(10)

it, why do you boast as though you did not?

1 Corinthians 4:7

I thank God for guiding me in life and towards the research of Evolutionary Com- putation through which I am profoundly edied by the imperfection and fragility of the phenomenon of convergence occurring in the simulated population of individuals via adaptation, natural selection, recombination and mutation. While inside the oce, working on improving the performance of Genetic Algorithms on problems with large and high-dimensional search spaces, I was becoming increasingly fascinated by the utmost perfection of the outside: Nature from the subatomic to the cosmic and Life evolved in incomprehensible dimensions. I am deeply grateful for my journey as a research student through which serendipitously I became a more perceptive observer of our world.

Hereby, I would like to express my gratitude to those who taught me, guided me and in many ways helped me in acquiring knowledge and experience I am honored to render into this thesis. I wish to convey my deepest thanks to the two people who were thoroughly supportive of my work, whose indispensable help and sincere assistance I warmly acknowledge and to whom I am greatly indebted. To my supervisor György Kozmann and to the supervisor of my Master's Thesis István Vassányi. What I have learnt from them is far more beyond the scope of the dissertation and research in general.

I am fortunate that after a few years of initial research and development and promising results, funds became available to create a pre-commercial level version of the expert system infrastructure, which would be built around the concepts and algorithms I made.

This initiated a development, which at the time of writing is still ongoing, and of which many exceptional software developers are or have been part of. Here, I would like to say many thanks to Ádám Endr®di and Balázs Pintér, the two most outstanding software developers I had the honor to work together with, for their multi-year contribution to the project, which is highly acknowledged. I would also like to thank Gábor Bata, Balázs Végs®, Attila Czigány, Zsuzsanna Szente for their eorts on establishing and improving the software infrastructure.

I express my thanks to the directors of the university organizations involved in the project, namely Katalin Hangos head of Department of Electrical Engineering and Infor- mation Systems, University of Pannonia, István Szabolcs head of Department of Dietetics and Nutrition Sciences, Semmelweis University, and Mária Figler head of Institute of Hu- man Nutrition and Dietetics, University of Pécs. I also thank each and every present and

(11)

former personnel of these organizations with whom I had great time working together.

I would like to give special thanks to Erzsébet Mák, Anna Medve, Márta Nemes, Szil- via Szabó, Gábor Balázs, Tibor Dulai, Csaba Fazekas, Kristóf Haraszti, Zoltán Juhász, Emil Mógor, Dániel Muhi, István Takács and Zsolt Tarjányi for their kind support and cooperation, and to every co-author of my papers and everyone who has worked on ad- ministering and maintaining the development infrastructure and on implementing tools for the project, especially to Kornél Fülöp, Viktor Hercinger, Attila Keszi and András Király. My thanks go to the Information Science & Technology PhD School and the Fa- culty of Information Technology of the University of Pannonia for supporting my research.

I would like to say thank you to my friends Brigitta Éva Farkas, Zsóa Kerekes, Eszter Városi, János Balogh, László Embersits, László Farkas, Tamás Harczos, Tamás Kiezer and András Kutrovics. I deeply and sincerely thank my friends Evelin Berta, Gábor Csullag, Péter Király, Attila Kutas, Tibor L®cze, Gábor Sárdi, Szabolcs Schmidt and Balázs Szabados for all of their generous and heartfelt help and encouragement. Here, I wish to express my love and gratitude towards my whole family for all of their support.

I thank my parents Edit and János and my brother Zoltán for everything they have done and given for me, and I gratefully and sincerely dedicate this thesis to them.

(12)

(13)

The order depicted by the black headed arrows on subgure 1.1(a) represents how the ongoing research work necessitated contributions. The order illustrated by the white headed arrows on subgure 1.1(b) shows how the results are used and built on top of the other. Contributions of the primary goal (expert system) are highlighted with dotted box, while those of the secondary goal (muleGA) with broken-line ellipse. . . . 7

2.1 The solutions (some of them marked with uppercase alphabetic characters) of a two-dimension minimization problem are shown in the gure. SolutionsA,B and D are non-dominated (with all those other ones in color red), thus belonging to the Pareto front (PF^∗), which is highlighted with a red line. SolutionsC and E (and all of those colored blue) are dominated. . . . . 13 2.2 The hierarchy of a three-level Multi-Level Optimization Problem (MLOP) P^(·,·)_(3,·)is

shown in the gure. The input problem mapping functions (f) are presented with arrows pointing towards the parent problems. Two special input problem mapping functions are depicted, one which maps decision variables to a lower level (dotted line), and one which maps to the top level from the bottom level (broken line). . 17 2.3 Classication of Global Optimization Methods . . . . 19 2.4 Loose estimate on the application of various global optimization methods for solv-

ing the Multi-Objective Knapsack Problem (MOKP). Results of the number of co-occurrences of multi-objective knapsack problem and the respective algorithm names indexed by Google. The percentage values show the share of the specic algorithm from the number the search for the single string: multi-objective knapsack problem results, which is20,300 at the22^thof October, 2009. . . . 20 2.5 A population with four individuals, with basic GA terminology highlighted. The

genotype (binary string encoding) and phenotype (an unsigned integer represented at locus 1-2, and a signed integer represented in two's complement form at locus 3-5) of the individuals are shown, as well as the calculated tness values ((signed integer)(unsigned integer)), for each individual. . . . 27 2.6 The most commonly used crossover operators for binary representations . . . . . 29

3.1 Mapping input problemsaandbto subproblemsAandB of solution2. . . . 41

iv

(17)

3.2 Outer and inner algorithmic cycles of Multi-Level Evolutionary Algorithms (MLEAs) with the inner cycle evolving populationP_(2,1)^(1,·) from the population scheduleP^(c). The outer cycle is at c = 3, with the rst population in P⁽³⁾ being P_(2,1)^(1,·) (also written interchangeably asP_[2]). In case input problem mapping occurs, the f_(l,u)^(2,1) functions map the information from the input problem's individual space to the subspace of the individual space of population P_(2,1)^(1,·). The dotted lines show optional events. . . . 46 3.3 An example for Multi-Level Evolutionary Algorithm (MLEA) populations and

their relationships. . . . 47 3.4 All possible individuals of the parent problem renderable through input problem

mappings are shown, considering each input problem population is made up of those two individuals highlighted on the gure. Note that the numbered circles and boxes denote the genotypes of the individuals and that the f-mappings are equal to the identical function, thus appearing the same genotypes in the parent problem's and subproblems' genotype spaces. . . . 51 3.5 An example of KPs arranged in a hierarchical structure, forming a 3-level KP. The

aim of this multi-level KP is to maximize the price vectors of the ve rst level KPs (and therefore the price vectors of all the upper level ones), while satisfying the weight constraints of each KP. The dotted blue lines represent the assignments from all those rst level knapsacks to the top-level one, which are assigned to a second-level knapsack. The broken red line highlights a special case, when a knapsack is assigned to more upper level knapsacks. From Denition 10, it follows that the content of this knapsack is duplicated, and half of this duplicated content is assigned to one of the second-level knapsacks, the other half is assigned to the other second-level knapsack. . . . 56 3.6 The notation of muleGA concepts - sols denoted with circles, attrs denoted with

squares standing on one vertex. Lines denote connections. . . . 60 3.7 The concepts of taxonomy and combination vals are shown in this gure. The

assignments of vals to sols are inferred from problem specic knowledge-bases and taxonomies encoding the non-quantiable information about the sols. . . . . 61 3.8 The results of the rst test conguration with number of boxes ranging from

2³·2ⁱ,(i= 1, . . . ,5) (x-coord), weight and prot dimensions ranging from2^j,(j= 1, . . . ,10)(y-coord) and dierence between the results of muleGA and GA given in percentage, calculated by subtracting the latter from the former (z-coord). Note that the coloring is only for separating the congurations and increasing visibility. 70 3.9 Percentage dierences in function of decision space dimensions (number of boxes)

and objective space dimensions (number of weight and prot dimensions) resulted from the rst test conguration. . . . 70 3.10 The results of the second test conguration with number of boxes ranging from

2³·2ⁱ,(i= 1, . . . ,5) (x-coord), weight and prot dimensions ranging from2^j,(j= 1, . . . ,10)(y-coord) and dierences between the results of muleGA and GA given in percentage, calculated by subtracting the latter from the former (z-coord). . . 71 3.11 The results of the third test conguration with number of boxes ranging from

2³·2ⁱ,(i= 1, . . . ,5) (x-coord), weight and prot dimensions ranging from2^j,(j= 1, . . . ,10)(y-coord) and dierence between the results of muleGA and GA given in percentage, calculated by subtracting the latter from the former (z-coord). . . 71 3.12 Enumerated Pareto fronts of the Binh1 and Poloni problems . . . . 75

(18)

(2 ,i= 1, . . . ,4). The dark-green bar represents the results of the random input problem mapping tests, the green bar of the semi-order input problem mapping tests, the light-green bar of the full-order input problem mapping tests, while the yellow bar of the single-level Non-dominated Sorting Genetic Algorithm II (NSGA-

II) tests. . . . 79

3.14 Results of the Poloni's problem based multi-level MOP related comparisons. The construction of the gure is equivalent to that of Figure 3.13. . . . 79

3.15 Distance ofPFknown to PFtrue . . . . 80

3.16 The minimal (best) tness . . . . 80

3.21 Distance ofPF_known to PF_true . . . . 82

3.23 Dierences in distances ofPF_knowntoPF_truein full-order input problem mapping muleGA to NSGA-II. . . . 82

3.24 Dierences of minimal (best) tnesses of full-order input problem mapping muleGA to NSGA-II. . . . 82

4.1 Contribution of diet and nutrition to the burden of disease in Europe. . . . 85

4.2 Diet as cause of reductions in disease. Subgure 4.2(a) shows the Hungarian mor- tality statistics for men with cardiovascular diseases giving 51.3% and neoplasia allocating 23.9% of the pie-chart. Subgure 4.2(b) highlights the preventive eect resides in diet and nutrition, through which 25.6% of the male population could extend the healthy and productive era of their lifetime. . . . 86

4.3 Schematic infrastructure of a comprehensive expert system, which integrates into daily life with its decision support logic readily accessible whenever and wherever needed. . . . 92

4.4 Nutritional Structure (NS) with Nutritional Levels (NLs), Dietary Menu Plans (DMPs) and Nutrition Hierarchy Objects (NHOs). . . . 95

4.5 Hierarchical Multi-Objective Dietary Menu Planning Problem with Harmony (HMO- DMPP-H) in Multi-Level GA structure . . . 109

4.6 The multi-level GA scheduling strategies (top-down, bottom-up, credit propa- gation, mutation based) in function of algorithmic levels. The numbers in the top-down and bottom-up columns show the order in which the various levels are evolved. . . 110

4.7 The crossover operator is shown in the gure in function of the nutritional level. Legend: M-Monday, Tu-Tuesday, W-Wednesday, Th-Thursday, F-Friday, Sa-Saturday, Su-Sunday BF-Breakfast, MS-Morning Snack, L-Lunch, AS-Afternoon Snack, D- Dinner S-Soup, G-Garnish, T-topping, Dr-drink, De-Dessert, cp-Crossover Point 110 4.8 A sample rule cache . . . 112 4.9 The occurrences (ordered by relative frequency, upper gure) of 120 of the 150

possible values of alleles in the best solutions of 15.000 runs. The lower gure shows the goodness of the best solution of which the respective allele was part of. 114

(19)

4.10 The relative occurrence of a particular solution (A) in function of the strictness

of two rules (penalizing alleles A and B). . . 115

4.11 Occurrences of the 15 possible alleles in a solution for a DMP in 1.000 runs in function of the strictness of the rule imposed on the15^thallele. . . 116

4.12 Statistical analysis of the distribution of the potential alleles in the best solutions and the mean occurrence of the alleles (A,B) on which the rules were imposed. . 116

A.1 Binary encoded chromosomes . . . 129

A.2 Real-value encoded chromosomes . . . 129

A.3 Permutation encoded chromosomes . . . 130

A.4 Value encoded chromosomes . . . 130

A.5 Tree encoded chromosomes . . . 131

A.6 The population transformationT deterministically maps the parent populationP (of sizeµ) to the ospring populationP⁰ (of sizeµ⁰). . . 133

A.7 The Random Population Transformation (RPT) R maps the random eventω to the population transformationT, which maps parent populations of sizeµ(which is independent of ω) to ospring populations of some xed size µ⁰ ∈Z⁺ (which may depend onω). . . 134

A.8 The evolutionary operator X maps the exogenous parameter(s) Θ to the RPT R. The underlying sample space of R is Ω. Each of the possible population transformations acts on populations of size µ. The ospring population sizeµ⁰∈ Z⁺ may depend onΘas well as on the random eventω∈Ω. . . 134

A.9 Non-dominated Sorting Genetic Algorithm (NSGA) ow chart . . . 136

A.10 Categorization of co-evolutionary algorithm properties . . . 147

B.1 Distance ofPFknown toPFtrue . . . 160

B.3 Distance ofP toPFtrue . . . 160

B.4 Distance ofP toPF_true . . . 160

B.5 Distance of best sol toPFtrue . . . 160

B.7 Number of pareto sols inP . . . 161

B.9 The minimal (best) tness values . . . 161

B.11 The maximal (worst) tness values . . . 161

B.13 The average tness values . . . 162

B.15 The diversity ofP . . . 162

B.17 The variance of the sols inP . . . 162

B.20 Distance ofP toPFtrue . . . 163

B.21 Distance of best sol toPF_true . . . 163

(20)

B.29 Distance ofP to PF_true . . . 164

B.30 Distance of best sol toPF_true . . . 165

B.37 Distance ofPFknown to PFtrue . . . 166

B.38 Distance ofP to PFtrue . . . 166

(21)

List of Tables

4.1 Real-world counterparts of Nutrition Hierarchy Objects (NHOs) from correspond-

ing NLs. . . . 93

4.2 Real-world counterparts of Activity Hierarchy Objects (AHOs) from corresponding Activity Levels (ALs). . . . 97

4.3 Domain of relation according to the type of Simple Dietary Menu Planning Problem (S-DMPP), whereOis the set of objects ([O] =m) andSis the set of slots ([S] =n). 99 4.4 Meaning of the variables used to formalize the Hierarchical Multi-Objective Di- etary Menu Planning Problem (HMO-DMPP). . . 100

4.5 Meaning of the expressions used to formalize the HMO-DMPP. . . 101

4.6 Example ontologies . . . 111

B.1 The parameters of the test problems . . . 158

ix

(22)

D Decoding function X Evolutionary operator Θ Exogenous parameter Ω Feasible region

T_s Fitness scaling function

f Input problem mapping operator I Individual space

f Objective function PF^∗ Pareto front P^∗ Pareto-optimal set

X Exogenous parameter space P Payload matrix

PFcurrent Current Pareto front PFknown Known Pareto front PF_true True Pareto front P Population µ Population size

T Population transformation operator ω Random event

R Random population transformation operator Ω Sample space

s Solution of a problem, Individual of a population Ω^s Search space

ι Termination condition

x

(23)

Acronyms

AHO Activity Hierarchy Object AI Articial Intelligence AL Activity Level AS Activity Structure ATP Activity Timetable Plan CPU Central Processing Unit EA Evolutionary Algorithm EC Evolutionary Computation EP Evolutionary Programming ES Evolution Strategy

FCDB Food Composition Database CO Collaborative Optimization DL Description Logic

DMP Dietary Menu Plan FOL First Order Logic GA Genetic Algorithm GP Genetic Programming

GPGPU General-purpose computing on graphics processing units GPU graphics processing unit

G-MOEA Guided Multi-Objective Evolutionary Algorithm HBGA Human-based Genetic Algorithm

IEC Interactive Evolutionary Computation IT Information Technology

KP Knapsack Problem LP Linear Programming

MDO Multi-Disciplinary Optimization

xi

(24)

MLOP Multi-Level Optimization Problem MOGA Multi-Objective Genetic Algorithm MOKP Multi-Objective Knapsack Problem

MOSCKP Multi-Objective Single-Constraint Knapsack Problem MOMCKP Multi-Objective Multiple-Constraint Knapsack Problem MOMGA Multi-Objective Messy Genetic Algorithm

MOMGA-II Multi-Objective Messy Genetic Algorithm MOP Multi-Objective Problem

MOPSO Multi-Objective Particle Swarm Optimization MOTS Multi-Objective Tabu Search

MPP Massive parallel processing

MOSA Multi-Objective Simulated Annealing muleGA Multi-Level Genetic Algorithm NBB Nutritional Building Block NFL No Free Lunch

NHO Nutrition Hierarchy Object NL Nutritional Level

NLP Nonlinear programming NS Nutritional Structure

NPGA Niched Pareto Genetic Algorithm NPGA2 Niched Pareto Genetic Algorithm 2 NSGA Non-dominated Sorting Genetic Algorithm NSGA-II Non-dominated Sorting Genetic Algorithm II PAES Pareto Archived Evolution Strategy

PESA Pareto Enveloped-based Selection Algorithm RPT Random Population Transformation

PSO Particle Swarm Optimization SA Simulated Annealing

SPEA Strength Pareto Evolutionary Algorithm SPEA2 Strength Pareto Evolutionary Algorithm 2

(25)

S-DMPP Simple Dietary Menu Planning Problem

MO-DMPP Multi-Objective Dietary Menu Planning Problem

HMO-DMPP Hierarchical Multi-Objective Dietary Menu Planning Problem

HMO-DMPP-H Hierarchical Multi-Objective Dietary Menu Planning Problem with Harmony TSP Travelling Salesman Problem

TS Tabu Search

VEGA Vector Evaluated Genetic Algorithm

(26)

(27)

Chapter 1

Introduction and Overview

In three sections, this chapter introduces the two main research topics of the dissertation and an overview is given on the contributions of the work and organization of the document. Only inner references to chapters and sections and no bibliography citations are used in this chapter, as it is intended to be thoroughly self-contained, introducing research questions and summarizing results in short, while ushering the reader into the concerned areas of the thesis.

1.1 Purpose and Scope

The title of the dissertation: Multi-Level Genetic Algorithms and Expert System for Health Promotion captures the main emphases of the contained twofold research work.

The rst research topic of the thesis focuses on the specication and evaluation of a new Genetic Algorithm (GA) named Multi-Level Genetic Algorithm (muleGA) (detailed in Chapter 3). The need for muleGA is induced by the demands for truly powerful and competitive algorithmical methods for solving Multi-Level Optimization Problems (MLOPs), either single-objective or multi-objective. As the usage of the epithet multi- level is ambiguous in the optimization literature, an exact denition of what is understood in this thesis to qualify as a Multi-Level Optimization Problem (MLOP) is given in Section 2.4.1. In a few words, to the class of MLOPs belong those problems, on which a reasonable decomposition to separate levels of subproblems can be carried out (top-down decomposition), and those problems, which are constructed by coupling subproblems to form a main optimization problem (bottom-up composition). The subproblems of the MLOPs form a hierarchical structure, and they are at least partially coupled. Optima of the MLOP are sought by solving these subproblems quasi independently, then using their results for conguring their parent problems. While the divide-and-conquer type of decomposition technique has its traditions in optimization, its potential has not been thoroughly exploited in the eld of Evolutionary Computation (EC). The introduction of the novel muleGA is a step in this direction. As EC methods tend to lose their strength as the complexity and dimension of the optimization problem increases, the exploitation of the knowledge residing in the hierarchy of the underlying structure, or

(28)

even the simple decomposition of the search space through dimension reduction can regain the performance of Evolutionary Algorithms (EAs). The eligibleness of muleGA is assessed through such hierarchical and Multi-Objective Problems (MOPs) and conclusion is drawn on how, when, and how much muleGA may improve optimization performance.

The second research topic of the thesis (detailed in Chapter 4) deals with the formalization, specication, methodology and design of a novel Expert System for Health Promotion. Denitions of health promotion are given in Section 4.1.1. In short, this um- brella term encompasses the concepts, practices, tools, science and art of enabling people to change their lifestyle and increase control over their health, and moving them toward a state of optimal health. The expert system presented in this thesis is developed to contribute to health promotion by providing advices and personalized plans for the end user regarding diet, nutrition and lifestyle. The goal of the expert system is to match the output of the human expert in quality, considering dietary menu plans, and as an optional feature, physical activity plans. These personalized consultancies in real-life practice are provided by allied health professionals (dietitians, nutritionists, athletic trainers, exercise physiologist and physiotherapists). The main focus is put on building the skeleton of the framework by mathematically formalizing the requirements of the personalized plans and architecting methods with which the encoding of expert knowledge is adequate and the desired level of output quality is attainable. There is connection between the rst and second research work, as muleGA forms the basis of the expert system. In particular, the output of the expert system is the result of an optimization process, which computes the optima of the mathematically formalized personalized planning problem (presented in Section 4.4). As the problem formalization highlights the diet and lifestyle planning problem is actually a MLOP, the need for muleGA is practically triggered by the expec- tations on the quality of the expert system generated personalized dietary and physical activity plans.

The goals and contributions of the thesis are presented in the following section and are introduced in the order in which the research topics induce one another, starting with the primal aim of the research and following with those it necessitates. This diers from how the document is organized (presented in Section 1.3), as it is constructed in accordance how the developed methods conduce one another. This bottom-up, built on top of the other, conducing approach, which starts with the generic method (muleGA), then follows with the expert system, has been used here as well to outline the purpose and scope of the dissertation.

1.2 Goals and Contributions

The primary and secondary goals of the thesis are denoted with G^pri and G^sec. Note that the two goals are considered to be of equal importance, the order only expresses that the solution of G^sec is a necessity of G^pri. The rst contribution to G^pri is denoted with C^pri₁ , thei^th with C^pri_i . Using this notation, the goals and contributions of the thesis and their relations are presented in the following.

(29)

1.2 Goals and Contributions 3

Primary Goal (G^pri) Design and create a novel expert system framework and algorithm for health promotion, which, if congured properly with valid and suce expert knowledge, can match the output of the domain experts in creating personalized dietary menu plans. By learning from the limitations of previous approaches, provide better algorithmic solutions to the problem, so that requirements on nutritional adequacy and harmony are satised in each component and subcomponent of the computer generated plan.

The research presented in this thesis is ignited by the aims G^pri represents. One may notice the optional support of physical activity planning is not included in the denition of the primary goal. Verily, it is not, because of the following reasons: the acts of personalized counseling, either with nutritional or physical concepts in mind, comprise very similar procedures and objectives. Although the acts and objectives are alike, which signals one algorithm may work for both tasks, the nutrition domain is larger, thus requiring a richer algorithmical toolbox. This comprehensive set of algorithmical tools however may well support personalized physical activity planning out of the box.

Focusing solely on the nutritional domain saves the repetitive discussion of concepts, notations and algorithms which are generally the same for both specialties. For this end, the domain of nutrition counseling and dietary menu planning, as one, if not the most important area of health promotion is addressed through the construction of the expert system. Nonetheless the analogousness of the concepts and objectives of physical activity planning with dietary menu planning is presented in the thesis (Section 4.3.4).

The ins and outs of the denition of G^pri are claried, so comes the G^pri related rst contribution, which formalizes the dietary menu planning task belonging to the nutrition domain of health promotion.

Formal denition of the Dietary Menu Planning Problem (C^pri₁ ) is given starting from the simplest representation of the problem, then iteratively including all necessary details to nally formulate the Hierarchical Multi-Objective Dietary Menu Planning Problem with Harmony (HMO-DMPP-H), which is an unprece- dentedly accurate mathematical rendition of the objectives of personalized dietary menu planning (detailed in Section 4.4.1.5).

Altogether four representations of the Dietary Menu Planning Problem are given, starting with the simplest Simple Dietary Menu Planning Problem (S-DMPP), through the intermediate Multi-Objective Dietary Menu Planning Problem (MO-DMPP) and Hierarchical Multi-Objective Dietary Menu Planning Problem (HMO-DMPP) until the most accurate Hierarchical Multi-Objective Dietary Menu Planning Problem with Harmony (HMO-DMPP-H) (all dened in Section 4.4). As the names of the representations reveal, a single objective version of the problem is formalized rst, then it is extended to handle multiple objectives, and after that, the hierarchical composition of the multi-level problem is given. The nal extension to the formalization is the concept of harmony, (see 4.4.1.4), whose function is to describe how harmonizing the components of the plan are. It is introduced in HMO-DMPP-H. The formalizations of C^pri₁ reveal HMO-DMPP

(30)

and HMO-DMPP-H are special Multi-Level Optimization Problems (MLOPs), mostly resembling to a hierarchical structure of interconnected Knapsack Problems (KPs). This makes algorithms for KPs primal candidates for solving the optimization tasks of dietary menu planning. Therefore overview of global optimization methods from the perspective of their viability in solving knapsack type optimization problems is given (Section 2). The summary of global optimization methods highlights GAs are the de facto techniques for solving KPs, but there are several other algorithms which provide good results. The main challenges for any particular algorithm are how it copes with the high-dimensionality, underlying hierarchical structure and harmony related requisites of the dietary menu planning problem. In general, all of the methods which provide satisfactory solutions for KPs with small search spaces, signicantly lose their potential as the size of the problem increases, thus some kind of decomposition and parallelization is necessary. A GA based technology is chosen as a base (causes behind algorithm choice presented in Section 4.5.1), around which a skeleton needs to be constructed that handles the high dimensionality and hierarchical nature of the problem. This unveils the secondary goal of the thesis, which is the following.

Secondary Goal (G^sec) Create a novel Genetic Algorithm (GA) which rst, overcomes the limitations of previous GAs regarding ineectiveness in solving high-dimensional MOPs, and second, supports the handling of large optimization problems, which are specially made up of a hierarchy of subcomponents also forming optimization problems.

The large optimization problems made up of a hierarchy of subcomponents are the Multi-Level Optimization Problems (MLOPs), which are dened in Section 2.4.1. The rst contribution of the thesis to G^sec is the creation of the Multi-Level Genetic Algorithm (muleGA) (detailed in Section 3.4) and is denoted with C^sec1 .

Novel Genetic Algorithm (C^sec₁ ) named muleGA has been developed to overcome the limitations of traditional GAs on problems with large search spaces and hierarchical inner structure.

The formalization and specication of muleGA requisites the denition of what does the algorithmic class named Multi-Level Evolutionary Algorithm (MLEA) stand for and how muleGA is related to this class of algorithms. There is no prior work formalizing MLEAs, presumably because these kind of algorithms are still fairly new. Those few belonging to this class are referred and cited in the thesis (Section 2.12). The second contribution of G^sec denoted by C^sec₂ is a framework for the unied formal description of multi-level type GAs.

Formalization (C^sec2 ) of the Multi-Level Evolutionary Algorithm (MLEA) has been given, which provides a generic abstract framework to dene and describe new type of GAs with multiple populations arranged in multi-level hierarchy.

(31)

1.2 Goals and Contributions 5

Utilizing the formalization, the operational sequence of muleGA and in general every MLEA is laid down and put on paper (Section 3.2 and 3.4). However, the proof of the pudding is in the eating, hence empirical studies are carried out examining the behavior of muleGA. These result in the third and last contribution (C^sec3 ) to G^sec.

Empirical analyses (C^sec₃ ) prove the viability and superiority of muleGA over traditional GA based methods in solving hierarchically structured Multi-Level Opti- mization Problems (MLOPs).

The contributions to G^sec are complete with C^sec1 , C^sec2 and C^sec3 making the secondary goal (G^sec) reached. However, G^sec is induced by the results of C^sec₁ belonging to the primary goal (G^pri), which still requires contributions besides C^sec₁ to be reached. Naturally, the contributions of G^sec are utilized to reach G^prithrough the application of muleGA for the optimization tasks of the expert system. The concepts of the HMO-DMPP are converted to the abstract representation required by muleGA (presented in Section 4.5.3), and through this association, a new method is fabricated for solving the dietary menu problem in such details previously unavailable.

New method (C^pri₂ ) has been produced for the solution of the dietary menu planning problem formalized in HMO-DMPP. The abstract concepts of muleGA have been utilized for solving the problem ensuring each subcomponent is optimized by a corresponding GA responsible for seeking nutritional adequate combinations.

The HMO-DMPP formalization is able to accurately describe the nutritional requirements (constraints and optima) of any Dietary Menu Plan (DMP). With the muleGA based method, a near-optimal solution can be sought. Regardless of the statistically ver- ied and conrmed performance of the muleGA (see Section 3.5), the optimality of the found solution is not guaranteed. Note that looking for the true optima of these multi- level, high-dimensional, complex combinatorial optimization problems is generally only available by the brute-force evaluation of the feasible region of the search space, which is computationally expensive. The muleGA based method provides solutions with good quality and a nicely approximated Pareto front more eectively. The fact muleGA only produces an approximate optima is not that bad news at all, as for the actual dietary eld, the dierence between a truly optimal solution and a near optimal one is smaller than those errors generated by the unavoidable inaccuracies of the Food Composition Databases (FCDBs) (detailed in 4.1.4). The numerical constraints of HMO-DMPP are satised, as good as they are, however a DMP is just as much about the harmony of its components. For this reason, rule-based assessment is introduced assessing each of the solutions of muleGA regardless of their status in the hierarchy. These are not production rules specifying symbol substitutions, but rather a compilation of attributes accompanied with a percentage value referring to the value of the list (see Section 4.5.5). The value over 100% favors, under 100% hinders the compilation. If the compilation matches a solution, then the rule belongs to the matching rules of the particular solution. How- ever, if there is more than one rule for a solution, only the more specic ones are red

(32)

considering they are commensurable. Non-commensurable rules assess dierent type of attributes, thus their eect is considered independently of each other. When a rule is red, it modies the tness of the solution, thus adjusting the goodness of the individual and altering the convergence of the population as whole.

Rule-based assessment (C^pri₃ ) has been introduced to the tness function of the muleGA based dietary menu planning method. With expert knowledge available in the form of simple rules, the dietary menu plans are assessable according to the harmony of their components. The eectiveness of the rule-based guide of the evolutionary search has been shown through empirical tests.

The rule-based assessment of the DMPs works as intended, but the great multitude of rules needed for the formalization of the expert knowledge renders the technique useful for only testing purposes. No human expert is able to rate each and every possible DMP according to the harmony of its components. Furthermore, the set of rules is not set in stone, as their values might change in dierent circumstances (like personal preferences or medical considerations). This calls for a more abstract representation of harmony related rules. There is a vast amount of tools for knowledge representation, so one is chosen for the task. As OWL-DL is seemingly becoming the de facto tool for the description of nutrition and dietary related knowledge, it is the pick for the job. The knowledge required by an expert system is more detailed and covers more aspects of the domain than for example presented in ontologies comprising semantic information for a simple semantic web application. Therefore, rather than creating a single ontology representing the concepts of the nutrition domain, a structure of a bunch of ontologies is created. The position of an ontology is assigned by its domain (what is described by the ontology) and the aspect of the description and categorization of the domain (detailed in Section 4.5.4).

Novel dietary ontology (C^pri4 ) architecture has been developed based on innovative organization and classication philosophy. It utilizes the possibilities of OWL-DL and provides a straightforward way for dietitians to record expert knowledge.

Of course, the ontology based knowledge is pre-processed and converted to the simple form used in the tness function of muleGA. Besides this application, the knowledge stored in the ontology can be used to categorize the computer generated DMP from pretty much any aspects, OWL-DL reasoners are available for this task. The application of OWL-DL also makes it possible to have taxonomy classes identied by OWL sentences, rather than by the manually constructed lists of their elements.

Through C^pri₁ , C^pri₂ , C^pri₃ and C^pri₄ the primary goal (G^pri) of the thesis is reached.

These contributions and the contributions (C^sec₁ , C^sec₂ , C^sec₃ ) of the secondary goal (G^sec) are discussed in detail in the thesis, although not in this inducing order (shown in Figure 1.1(a)). The structure in which this document is organized is presented in the following section.

(33)

1.3 Document Organization 7

(a) Inducing order (b) Conducing order

Figure 1.1: Goals and contributions of the thesis and their relations are shown in the gures.

The order depicted by the black headed arrows on subgure 1.1(a) represents how the ongoing research work necessitated contributions. The order illustrated by the white headed arrows on subgure 1.1(b) shows how the results are used and built on top of the other. Contributions of the primary goal (expert system) are highlighted with dotted box, while those of the secondary goal (muleGA) with broken-line ellipse.

1.3 Document Organization

The organization of the dissertation is presented here along with the short description of chapters. Also, tips are given on how to read the document considering reader interest.

The thesis encompasses research work from two relatively distinct area (evolutionary computation and lifestyle and nutrition counseling), and the reader may not regard both of the domains with the same importance.

The document is composed of fourteen main sections of which ve are initiatives, ve are numbered chapters, two are appendices and two are bibliographies.

• Contents

• List of Figures

• List of Tables

• Acronyms

• Acknowledgments

• Introduction and Overview (Chapter 1)

• Evolutionary Algorithms and Global Optimization (Chapter 2)

• Multi-Level Genetic Algorithm (muleGA) (Chapter 3)

• Expert System Design for Health Promotion (Chapter 4)

• Conclusions (Chapter 5)

(34)

• Multi-Objective Optimization (Appendix A)

• Test Congurations and Results (Appendix B)

• Thesis Related Publications

• Bibliography

The contents of the initials (Contents, List of Figures, List of Tables, Acronyms and Acknowledgments) are apparent and no further introduction is given on them. The short introduction of the chapters and appendices are given in the following.

Introduction and Overview (Chapter 1) introduces the two main research topics of the dissertation and an overview is given on the contributions of the work and organization of the document in three sections. Intended to be thoroughly self- contained, only inner references to chapters and sections and no bibliography citations are used.

Evolutionary Algorithms and Global Optimization (Chapter 2) aims to intro- duce the reader to the concepts of global optimization and Pareto terminology (through Sections 2.1, 2.2, 2.3). Utilizing these concepts, the domain of Multi- Level Optimization Problems (MLOPs) is dened (2.4.1), which is related to the ultimate optimization problem of the thesis formulated in HMO-DMPP-H (4.4.1.5).

Multi-Level Genetic Algorithm (muleGA) (Chapter 3) presents a novel GA (in Section 3.4), the muleGA for solving MLOPs. A novel Multi-Level Evolutionary Algorithm (MLEA) framework is also presented (in Section 3.2), which describes the structure of those EAs which evolve multiple populations arranged in a hierarchical structure. The muleGA (a new MLEA) is tested on multi-level KPs (Section 3.5.4) and numerical MOPs (Section 3.5.5), and proofs are given on the applicability of the new algorithm in solving MLOPs.

Expert System Design for Health Promotion (Chapter 4) deals with the research and development of a novel expert system. The personalized dietary menu planning (Section 4.3) and the physical activity timetable planning (Section 4.3.4) problems are discussed and a mathematical formalization is given which adequately models the goals of these problems 4.4. A new method utilizing muleGA is proposed for solving the health promotion related optimization problems which employs a rule- based assessment technique integrated into the tness function of muleGA (Section 4.5.5).

Conclusions (Chapter 5) summarizes the contributions and scientic results of the dissertation, and briey discusses the possible applications and future work.

Multi-Objective Optimization (Appendix A) contains complementary material for Chapter 2. Those global optimization and evolutionary algorithm related concepts are presented here, whose discussions would have broken the train of thought and