One of the earliest mixed-integer formulations **for** the subset selection **regression** problem (SSR k ) was given by Konno and Yamamoto (2009). Since then, interest in solving (SSR k )
via modern **discrete** **optimization** methods grew rapidly. Dong, Chen, and Linderoth (2015) applied the perspective reformulation to the subset selection **regression** enabling a much stronger relaxation and consequently allowing **for** the mixed-integer program to be solved faster. Bertsimas, King, and Mazumder (2016) present a mixed-integer quadratic formula- tion, a first-order warm start approach and an extensive study on the statistical quality of the subset selection **regression**. They argue that **discrete** **optimization** methods can play an important role in statistics. They provide evidence that the critique of **discrete** optimiza- tion being computationally impractical in fields like statistics is obsolete and that proper mixed-integer **optimization** can be highly valuable and worthwhile. Bertsimas resumed to work on several articles covering the subset selection **regression**: Bertsimas and King (2016) propose a framework which extends the subset selection **regression** to an automation process which promises to require minimal human interaction and understanding. Bertsimas and Van Parys (2017) reformulate the subset selection **regression** to a nonlinear binary program without any continuous variables and solve the problem with an outer approximation ap- proach. Due to high effectiveness of the formulation they apply it to polynomial **regression** with exponentially many variables (Bertsimas & Parys, 2017). Atamt¨urk and G´omez (2018) focus on the case when X T X is an M-matrix. They present a formulation which is inspired

Mehr anzeigen
152 Mehr lesen

158 Mehr lesen

Hydrodynamic stability plays a crucial role **for** many applications. Existing approaches focus on the dependence of the stability properties on control parameters such as the Reynolds or the Rayleigh number. In this paper we propose a numerical method which aims at solving shape **optimization** problems in the context of hydrodynamic stability. The considered approach allows to guarantee hydrodynamic stability by modifying parts of the underlying geometry within a certain flow regime. This leads to a formulation of a shape **optimization** problem with constraints on the eigenvalues related to the linearized Navier-Stokes equations. In that context the eigenvalue problem is generally non-symmetric and may involve complex eigenvalues. To validate the proposed numerical approach we consider the flow around a body in a channel. The shape of the body is parameterized and can be changed by means of a **discrete** number of design variables. It is our aim to find a design which minimizes the drag force and ensures at the same time hydrodynamic stability while keeping the volume of the body constant. The numerical results show that a transition from an unstable design to a stable one is attainable by considering an adequate change of the body shape. The resulting bodies are long and flat which corresponds to common intuition.

Mehr anzeigen
21 Mehr lesen

The motivation of using MOEAs mainly relies on their ability in producing a set of approximated pareto-optimal solution. Moreover, it is easy to implement and requires less physical knowledge of the problem. In addition, it should be mentioned that **for** most existing gradient-based optimal control solvers, it is assumed in the implementation that all the objectives and constraints have continuous first and second order derivatives. In some practical control problems, however, the nonlinearity and nonsmoothness of the objectives or path constraints can be high. That indicates it is hard to obtain the gradient information **for** constructing the Jacobian and Hessian matrix. This problem becomes more difficult when the dimension of the objective function increases. Therefore, in order to solve the multi-objective optimal control problem constructed in Section.II, an extended NSGA-III algorithm is proposed. Since the original NSGA-III algorithm has no capability in dealing with dynamic constraints (e.g. the vehicle dynamics), **discrete** techniques should be implemented such that the continuous-time problem can be transcribed to static NLPs. The discretization scheme applied in this paper is based on the multiple shooting technique. That is, the control variables are parameterized at temporal nodes [t0, t1, ..., tf ]. Then, the equations of motion are integrated with a numerical integration method, e.g. forth-order Runge-Kutta method. **For** convenience, let xk = [rk, θk, φk, Vk , γk, ψk] T denotes the approximation of states at tk time instant, and ξk stands **for** the step length **for** the kth time interval [tk , tk+1]. The discretized

Mehr anzeigen
14 Mehr lesen

1 Introduction
Structural estimation of economic models is an important approach to analyzing economic data. The main advantage of the structural approach is that it allows researchers to conduct counterfactual policy analysis, which cannot be undertaken using the reduced-form approach. However, the computational burden of estimating structural models can be a problem. It is commonly believed that these computational demands make it difficult to implement the most powerful and efficient statistical methods. **For** example, Rust (1987) proposed a computational strategy **for** the maximum-likelihood estimation of single-agent dynamic **discrete**-choice models, an approach referred to as the nested fixed-point (NFXP) algorithm. The NFXP algorithm is computationally demanding because it repeatedly takes a guess **for** structural parameters and then solves **for** the corresponding endogenous economic variables with high accuracy at each guess of the parameters.

Mehr anzeigen
22 Mehr lesen

fects (Ballot et al., 2014; Battiston et al., 2007; Feng et al., 2012). Usually, these approaches are based on **discrete** time frameworks and include a distinct sequence of events within each period.
Our modelling framework approaches these problems differently. While ABM strongly depart from the standard setup of economic models and reach a very high degree of complexity, general **constrained** dynamic models may still be formulated based on utility functions and constraints known from general equilibrium models. They thus allow to relax the restrictions about rationality and aggregation known from general equilibrium models, while remaining simpler and more easily accessible than most ABM. At the same time, GCD models may also be of use to economists working on such agent-based models. Usually, the aggregated results of ABM are compared to DSGE models (Fagiolo and Roventini, 2012), but as multiple equilibria and instabilities are not found in DSGE models, a dynamic model of **constrained** dynamics that is able to capture these may be more suitable **for** this type of meta modeling.

Mehr anzeigen
33 Mehr lesen

Σ { e X } = E{( e X − E{ e X })( e X − E{ e X }) T }. (4.11) However, as mentioned before, this second central moment would not contain the full stochastic information in the inequality **constrained** case because we have to deal with truncated PDFs. There- fore, it is more conducive to compute an m-dimensional histogram of the parameters. This histogram can be seen as a **discrete** approximation of the joint PDF of the parameters. Approximations of the marginal densities can be computed the same way, adding up the particular rows of the hyper matrix of the histogram. The quality of approximation of the continuous PDF depends directly on M (cf. Alkhatib and Schuh, 2007), which therefore has to be chosen in a way that allows a satisfactory approximation while keeping the computation time at an acceptable level. In each Monte Carlo iteration a new **optimization** problem has to be solved. However, as the solution of the original ICLS problem can be used as feasible initial value **for** the parameters, convergence of the active-set method is usually quite fast.

Mehr anzeigen
140 Mehr lesen

In recent years, model selection and regularization in **regression** models has been an area of intensive research. Often, penalized approaches are the method of choice. Examples are Ridge **regression** (Hoerl and Kennard, 1970), the least absolute shrinkage and selec- tion operator (Lasso; Tibshirani, 1996), the smoothly clipped absolute deviation penalty (SCAD; Fan and Li, 2001), the fused Lasso (Tibshirani et al., 2005), the elastic net (Zou and Hastie, 2005) and the (adaptive) group Lasso (Yuan and Lin, 2006; Wang and Leng, 2008), to mention only a few approaches. The number of applications is huge. In nonparametric **regression**, penalties smooth wiggly functions. Eilers and Marx (1996) work, **for** example, with Ridge penalties on higher order differences of B-spline coefficients. Meier et al. (2009) select splines with a group Lasso penalty. **For** wavelets and signals, L 0 penalties, or more general L q penalties, 0 ≤ q ≤ 1, are employed (Antoniadis and Fan, 2001; Rippe et al., 2012). Concerning categorical data, Bondell and Reich (2009) or Gertheiss and Tutz (2010) work with fused Lasso type penalties. Fahrmeir et al. (2010) offer a flexible framework **for** Bayesian regularization and variable selection, amongst others with spike and slab priors. Various efficient algorithms to solve the resulting **optimization** problems are available, be it in linear models, generalized linear models (GLMs), hazard rate models or others. Least angle **regression** (lars; Efron et al., 2004; Hastie and Efron, 2013) offers a conceptual frame- work to compute the entire regularization path of the Lasso by exploiting its piecewise lin- ear coefficient profiles. Osborne and Turlach (2011) propose a homotopy algorithm **for** the quantile **regression** Lasso and related piecewise linear problems. Meier et al. (2008) propose a coordinate-descent algorithm **for** the group Lasso in logistic **regression** problems. Goeman (2010) solves Lasso, fused Lasso and Ridge-type problems in high-dimensional models by a

Mehr anzeigen
196 Mehr lesen

the effect of correlated uncertain variables on plant design. Two approaches have been used to represent uncertain variables: **discrete** and continuous distribution. In the former, the bounded uncertain variables are discretized into multiple intervals such that each individual interval represents a scenario with an approximated **discrete** distribution (Halemane and Grossmann, 1993; Subrahmanyam et al., 1994; Pistikopoulos and Ierapetritou, 1995a; Rooney and Biegler, 1999). Thus, so-called multiperiod **optimization** problems are formulated. The second approach considers the continuous stochastic distribution of the uncertain variables, in which a multivariate numerical integration method will be chosen. This leads to a stochastic programming problem. An approximated integration through a sampling scheme (Diwekar and Kalagnannam, 1997) and a direct numerical integration (Bernado et al., 1999) have been used. Alternatively to sampled **optimization** algorithms, the stochastic problem can be relaxed to an equivalent NLP problem and then solved by using standard techniques. Thus, the **optimization** problem needs to be reformulated. If the uncertain variables have an impact on the objective function, it is usually formulated as the expected value of the objective function (Torvi and Herzberg, 1997; Acevedo and Pistikopoulos, 1998). Practically most of the previous cited works employed the two-stage programming method with the recourse formulation to deal with inequality constraints. In this approach the first-stage decision variables are predetermined before the realization of the uncertain variables, while the second- stage variables are decided after their realization. Moreover, in the recourse formulation, violation of the constraints is allowed, but penalized through penalty terms in the objective function. This leads to additional costs regarding the second-stage decisions. This approach is suitable when the objective function and constraint violations can be described by the same measurement, **for** example process planning problems under demand uncertainties (Clay and Grossmann, 1997; Gupta and Maranas, 2000). This compensation, however, requires a common measurement to describe the objective function and the constraint violations.

Mehr anzeigen
198 Mehr lesen

The chapter is structured as follows. Section 5.1 briefly sketches the main ideas behind projection. Section 5.2 gives an overview over polyhedral as- pects of dynamic programming. In particular, we review the fundamental work of Martin, Rardin, and Campbell [60] who have provided a framework **for** deriving linear characterizations of dynamic programs. In Section 5.3, we review the most important results from the literature related to the hop **constrained** path polytope. Moreover, we introduce two relaxations of this polytope mentioned above corresponding to the cases that the given length function d is nonnegative or has no negative cycles. Section 5.4 breaks down the dynamic programming paradigm to the hop **constrained** shortest path problem. Next, Section 5.5 relates the separation problems of the two re- laxations of the hop **constrained** path polytope to the multicommodity flow problem and length-bounded cut and flow problems. The results there imply that it will be quite hard to design combinatorial algorithms that solve these separation problems in polynomial time. Finally, Section 5.6 characterizes all 0/1-facet defining inequalities **for** the dominant of the hop **constrained** path polytope and all facet defining inequalities **for** the hop **constrained** walk polytope with coefficients in {−1, 0, 1} using the dynamic program paradigm. The attained results will be related to already known results.

Mehr anzeigen
229 Mehr lesen

In some context the density f is unknown but there is information like monotonicity, convexity or log-concavity about its shape. In general, the unconstrained estimator defined in (1) will not necessarily fulfill those shape constraints but if there is such information, it is desirable to have estimators which meet these constraints. In the literature, there are several proposals of such estimators in density as well as **regression** estimation but most methods rely on maximum likeli- hood estimation or **constrained** least squares where the estimator is obtained by minimizing the

27 Mehr lesen

Instead of adding every numerical constant to the terminal set beforehand, which yields the problems described in the previous section, numerical con- stants can be introduced by the use of ephemeral random constants ERCs [ Koza , 1992 ]. When ERCs are used, the special symbol R is added to the terminal set and every time the ERC symbol R is selected during tree cre- ation a new constant value is drawn from a predefined distribution. Common distributions the numeric values are drawn from are the uniform distribution or the Gaussian distribution. The properties of the distributions, such as the lower and upper limit or the average and mean, **for** the ERC symbol have to be adapted at the problem at hand to generate appropriate real-valued constants. Once the numeric values are generated, these values remain fixed and are not sampled again. In its initial definition, similarly to the constants added directly as symbols, ERCs once generated are not modified anymore and are moved between the individual solutions by the crossover operator. As a result the constant values can be combined in a way to generate inter- mediate values necessary **for** solving the symbolic **regression** problem. ERCs provide a greater flexibility, because it is possible to create real-values con- stants according to a predefined distribution, compared to adding constants directly to the terminal set.

Mehr anzeigen
157 Mehr lesen

These results caution against clustering standard errors by the running variable in em- pirical applications, in spite of its great popularity. 6 We therefore propose two alternative
methods **for** constructing CIs **for** the ATE in **discrete** RDDs that have guaranteed coverage properties under interpretable restrictions on the conditional expectation function. The first method makes the assumption that the magnitude of the approximation bias is no larger at the left limit of the threshold than at any point in the support of the running variable below the threshold, and similarly **for** the right limit. The second method relies on the assump- tion recently considered in Armstrong and Kolesár (2016b) that the second derivative of the conditional expectation function is bounded by a constant. Both CIs are “honest” in the sense of Li (1989) in that they achieve asymptotically correct coverage **for** all possible model parameters, that is, they are valid uniformly in the value of the conditional expectation function.

Mehr anzeigen
42 Mehr lesen

Received: 21 April 2016 / Accepted: 20 December 2016 / Published online: 9 January 2017 The Author(s) 2017. This article is published with open access at Springerlink.com
Abstract Integration of production planning and schedul- ing is a class of problems commonly found in manufac- turing industry. This class of problems associated with precedence constraint has been previously modeled and optimized by the authors, in which, it requires a multidi- mensional **optimization** at the same time: what to make, how many to make, where to make and the order to make. It is a combinatorial, NP-hard problem, **for** which no polynomial time algorithm is known to produce an optimal result on a random graph. In this paper, the further devel- opment of Genetic Algorithm (GA) **for** this integrated **optimization** is presented. Because of the dynamic nature of the problem, the size of its solution is variable. To deal with this variability and find an optimal solution to the problem, GA with new features in chromosome encoding, crossover, mutation, selection as well as algorithm struc- ture is developed herein. With the proposed structure, the proposed GA is able to ‘‘learn’’ from its experience. Robustness of the proposed GA is demonstrated by a complex numerical example in which performance of the proposed GA is compared with those of three commercial **optimization** solvers.

Mehr anzeigen
18 Mehr lesen

harder problems, more iterations and hence more time is needed. However, in every case, the optimal solution was found, which reflects that even **for** strongly nonlinear and almost singular problems, e.g. c = 10 3 , d = 10 −3 , the globalization mechanism and the preemptive adaptive
termination of the PPCG-iterations described in Section 5 lead to stable performance of the algorithm. Moreover, **for** time-dependent problems, the special block-diagonal structure of the differential operator A in the case of a discontinuous Galerkin method in time allows **for** a time- step wise factorization, which results in high saving of memory as opposed to factorization of the full matrix A. In the two settings c = d = 1 and c = 10, d = 0.1, no nonconvexities were encountered. This indicates, together with the low iteration numbers, that the starting value (y, u) = (0, 0) is close to a local solution. **For** the other two cases, nonconvexities and stronger nonlinearities led to higher computations times and to all mechanisms of the algorithm being used.

Mehr anzeigen
31 Mehr lesen

We have shown how to extend a terminological KR formalism by a construct that can express global restrictions on the **cardinality** of concepts. The usefulness of these **cardinality** restrictions on concepts was demonstrated by an example from a conguration application. Unlike role-value maps (which could be used to model similar situations), our new construct leaves all the important inference problems decidable. The consistency algorithm combines and simplies the ideas developed **for** the treatment of qualifying number restrictions and of terminological axioms.

24 Mehr lesen

In this section, we propose an efficient algorithm **for** the numerical computation of the singular system **for** a discretized (matrix) version of ΠK −1 B. The convergence
of the singular system of the discretized problem to the singular system of the continuous problem will be discussed elsewhere. In practice, it will be sufficient to compute only a partial SVD, starting with the largest singular value, down to a certain threshold, in order to collect the perturbation directions of greatest impact with respect to the observed quantity. The method we propose makes use of existing standard software which iteratively approximates the extreme eigenpairs of non-symmetric matrices, and it will be efficient in the following sense: It is unnecessary to assemble the (discretized) matrix ΠK −1 B, which is prohibitive **for**

Mehr anzeigen
21 Mehr lesen

started. In the simplest shape space case (landmark vectors), the distances between shapes can be measured by the Euclidean distance, but in general, the study of shapes and their similarities is a central problem. Now, we can ask how we can tackle natural questions like “How different are shapes?”, “Can we determine the measure of their difference?” or “Can we infer some information?”. In order to answer these questions mathematically, we have to put a metric on the shape space. There are various types of metrics on shape spaces, e.g., inner metrics [11, 12, 64], which can be seen as describing a deformable material that the shape itself is made of. In contrast to these inner metrics, there are also outer metrics [13, 17, 36, 48, 64]. Since the differential operator governing these metrics is defined even outside of the shape, they can be seen as describing some deformable material that the ambient space of the shape is made of. Furthermore, metamorphosis metrics [45, 101], the Wasserstein or Monge-Kantorovic metric on the shape space of probability measures [5, 14, 15], the Weil-Petersson metric [55, 91], current metrics [26, 27, 103] and metrics based on elastic deformations [32, 108] should be mentioned. However, it is a challenging task to model both, the shape space and the associated metric. There does not exist a common shape space or shape metric suitable **for** all applications. Different approaches lead to diverse models. The suitability of an approach depends on the requirements in a given situation. In this thesis, among all these shape space concepts, we pick the Riemannian shape manifold introduced by Peter W. Michor and David Mumford in which a two-dimensional shape is defined as a smooth embedding from the unit circle into the plane (cf. [63]). Moreover, we consider inner metrics. More precisely, we first work with so-called Sobolev metrics on this shape space. Of course, there are a lot of further metrics on it (cf. [64]), but the Sobolev metric is the most suitable choice **for** our applications. However, we consider also so-called Steklov-Poincaré metrics in order to enable the usage of volume integral formulations of shape derivatives in **optimization** strategies. Note that it is not possible to use these formulations by considering Sobolev metrics.

Mehr anzeigen
147 Mehr lesen

Heating
Fig. 2: State/control/constraint evolutions: Case 1 As can be observed from Figs.2-5, the pre-specified entry terminal boundary conditions and safety-related path constraints can be satisfied **for** all the mission cases, thereby confirming the validity of the investigated heuristic meth- ods. That is, both the penalty function-based and the multi- objective transformation-based constraint handling strategies are able to guide the searching direction toward the feasible region. In terms of the flight trajectories, same trend can be observed from the solutions generated by different heuristic algorithms **for** all the considered mission cases. Moreover, **for** the third mission case, the three evolutionary methods can produce almost identical solutions. Moreover, by viewing the system state and control profiles, it is clear that the obtained trajectories are relatively smooth. This can be attributed to the differential equation imposed on the actual bank angle variable 𝜎. This equation can also be understood as a first-order filter and it indirectly restricts the rate of the actual bank angle.

Mehr anzeigen
15 Mehr lesen

ABSTRACT:In this paper, we evaluate the application of Bayesian **Optimization** (BO) to **discrete** event simulation (DES) models. In a first step, we create a simple model, **for** which we know the optimal set of parameter values in advance. We implement the model in SimPy, a framework **for** DES written in Python. We then interpret the simulation model as a black box function subject to **optimization**. We show that it is possible to find the optimal set of parameter values using the open source library GPyOpt. To enhance our evaluation, we create a second and more complex model. To better handle the complexity of the model, and to add a visual component, we build the second model in Simio, a commercial off-the-shelf simulation modeling tool. To apply BO to a model in Simio, we use the Simio API to write an extension **for** **optimization** plug-ins. This extension encapsulates the logic of the BO algorithm, which we deployed as a web service in the cloud. The fact that simulation models are black box functions with regard to their behavior and the influence of their input parameters makes them an apparent candidate **for** Bayesian **Optimization** (BO). Simulation models are multivariable and stochastic, and their behavior is to a large extent unpredictable. In particular, we do not know **for** sure which input parameters to adjust to maximize (or minimize) the model’s outcome. In addition, the complex models can take a substantial amount of time to run. Bayesian **Optimization** is a sequential and self-learning algorithm to optimize black box functions similar to as we find them in simulation models: they contain a set of parameters **for** which we want to identify the optimal set, they are expensive to evaluate, and they exhibit stochastic noise. BO has proven to efficiently optimize black box functions from various disciplines. Among those, and most notably, it is successfully applied in machine learning algorithms to optimize hyperparameters.

Mehr anzeigen