# Nach oben pdf Cardinality-Constrained Discrete Optimization for Regression ### Cardinality-Constrained Discrete Optimization for Regression

One of the earliest mixed-integer formulations for the subset selection regression problem (SSR k ) was given by Konno and Yamamoto (2009). Since then, interest in solving (SSR k ) via modern discrete optimization methods grew rapidly. Dong, Chen, and Linderoth (2015) applied the perspective reformulation to the subset selection regression enabling a much stronger relaxation and consequently allowing for the mixed-integer program to be solved faster. Bertsimas, King, and Mazumder (2016) present a mixed-integer quadratic formula- tion, a first-order warm start approach and an extensive study on the statistical quality of the subset selection regression. They argue that discrete optimization methods can play an important role in statistics. They provide evidence that the critique of discrete optimiza- tion being computationally impractical in fields like statistics is obsolete and that proper mixed-integer optimization can be highly valuable and worthwhile. Bertsimas resumed to work on several articles covering the subset selection regression: Bertsimas and King (2016) propose a framework which extends the subset selection regression to an automation process which promises to require minimal human interaction and understanding. Bertsimas and Van Parys (2017) reformulate the subset selection regression to a nonlinear binary program without any continuous variables and solve the problem with an outer approximation ap- proach. Due to high effectiveness of the formulation they apply it to polynomial regression with exponentially many variables (Bertsimas & Parys, 2017). Atamt¨urk and G´omez (2018) focus on the case when X T X is an M-matrix. They present a formulation which is inspired
Mehr anzeigen

152 Mehr lesen ### Optimality Conditions and Numerical Methods for a Continuous Reformulation of Cardinality Constrained Optimization Problems

For all examples and all objective functions every method successfully computed a feasible point for (5.5). The best solutions, for all problem sizes, were either computed by the Kanzow- Schwartz regularisation or by the Scholtes-type regularisation, which yielded the best solution of circa 60% of the examples. Regarding the computation times there are distinct differences. Snopt is by far the fastest method. Among the regularisation methods the Scholtes-type regularisation is fastest while the Kanzow-Schwartz regularisation the slowest. However, as mentioned before, for the exponential regularisation the computation time strongly depends on the increase of the regularisation parameter in each iteration. The performance plots in Figures 5.3, 5.4 and 5.5 show that, while it did not compute better solutions than the Scholtes- type and Kanzow-Schwartz regularisations, Snopt still delivered comparatively good solutions. For the problem size n = 400 the Scholtes-type regularisation, the Kanzow-Schwartz regu- larisation and Snopt behaved almost identical. This is also the case for the starting vector (x 0 , y 0 ) = (0, e) for which plots are contained in Figure 5.5.
Mehr anzeigen

158 Mehr lesen ### Shape optimization towards stability in constrained hydrodynamic systems

Hydrodynamic stability plays a crucial role for many applications. Existing approaches focus on the dependence of the stability properties on control parameters such as the Reynolds or the Rayleigh number. In this paper we propose a numerical method which aims at solving shape optimization problems in the context of hydrodynamic stability. The considered approach allows to guarantee hydrodynamic stability by modifying parts of the underlying geometry within a certain flow regime. This leads to a formulation of a shape optimization problem with constraints on the eigenvalues related to the linearized Navier-Stokes equations. In that context the eigenvalue problem is generally non-symmetric and may involve complex eigenvalues. To validate the proposed numerical approach we consider the flow around a body in a channel. The shape of the body is parameterized and can be changed by means of a discrete number of design variables. It is our aim to find a design which minimizes the drag force and ensures at the same time hydrodynamic stability while keeping the volume of the body constant. The numerical results show that a transition from an unstable design to a stable one is attainable by considering an adequate change of the body shape. The resulting bodies are long and flat which corresponds to common intuition.
Mehr anzeigen ### Solving multiobjective constrained trajectory optimization problem by an extended evolutionary algorithm

The motivation of using MOEAs mainly relies on their ability in producing a set of approximated pareto-optimal solution. Moreover, it is easy to implement and requires less physical knowledge of the problem. In addition, it should be mentioned that for most existing gradient-based optimal control solvers, it is assumed in the implementation that all the objectives and constraints have continuous first and second order derivatives. In some practical control problems, however, the nonlinearity and nonsmoothness of the objectives or path constraints can be high. That indicates it is hard to obtain the gradient information for constructing the Jacobian and Hessian matrix. This problem becomes more difficult when the dimension of the objective function increases. Therefore, in order to solve the multi-objective optimal control problem constructed in Section.II, an extended NSGA-III algorithm is proposed. Since the original NSGA-III algorithm has no capability in dealing with dynamic constraints (e.g. the vehicle dynamics), discrete techniques should be implemented such that the continuous-time problem can be transcribed to static NLPs. The discretization scheme applied in this paper is based on the multiple shooting technique. That is, the control variables are parameterized at temporal nodes [t0, t1, ..., tf ]. Then, the equations of motion are integrated with a numerical integration method, e.g. forth-order Runge-Kutta method. For convenience, let xk = [rk, θk, φk, Vk , γk, ψk] T denotes the approximation of states at tk time instant, and ξk stands for the step length for the kth time interval [tk , tk+1]. The discretized
Mehr anzeigen ### Constrained optimization approaches to estimation of structural models

1 Introduction Structural estimation of economic models is an important approach to analyzing economic data. The main advantage of the structural approach is that it allows researchers to conduct counterfactual policy analysis, which cannot be undertaken using the reduced-form approach. However, the computational burden of estimating structural models can be a problem. It is commonly believed that these computational demands make it difficult to implement the most powerful and efficient statistical methods. For example, Rust (1987) proposed a computational strategy for the maximum-likelihood estimation of single-agent dynamic discrete-choice models, an approach referred to as the nested fixed-point (NFXP) algorithm. The NFXP algorithm is computationally demanding because it repeatedly takes a guess for structural parameters and then solves for the corresponding endogenous economic variables with high accuracy at each guess of the parameters.
Mehr anzeigen ### From constrained optimization to constrained dynamics: Extending analogies between economics and mechanics

fects (Ballot et al., 2014; Battiston et al., 2007; Feng et al., 2012). Usually, these approaches are based on discrete time frameworks and include a distinct sequence of events within each period. Our modelling framework approaches these problems differently. While ABM strongly depart from the standard setup of economic models and reach a very high degree of complexity, general constrained dynamic models may still be formulated based on utility functions and constraints known from general equilibrium models. They thus allow to relax the restrictions about rationality and aggregation known from general equilibrium models, while remaining simpler and more easily accessible than most ABM. At the same time, GCD models may also be of use to economists working on such agent-based models. Usually, the aggregated results of ABM are compared to DSGE models (Fagiolo and Roventini, 2012), but as multiple equilibria and instabilities are not found in DSGE models, a dynamic model of constrained dynamics that is able to capture these may be more suitable for this type of meta modeling.
Mehr anzeigen ### Convex Optimization for Inequality Constrained Adjustment Problems

Σ { e X } = E{( e X − E{ e X })( e X − E{ e X }) T }. (4.11) However, as mentioned before, this second central moment would not contain the full stochastic information in the inequality constrained case because we have to deal with truncated PDFs. There- fore, it is more conducive to compute an m-dimensional histogram of the parameters. This histogram can be seen as a discrete approximation of the joint PDF of the parameters. Approximations of the marginal densities can be computed the same way, adding up the particular rows of the hyper matrix of the histogram. The quality of approximation of the continuous PDF depends directly on M (cf. Alkhatib and Schuh, 2007), which therefore has to be chosen in a way that allows a satisfactory approximation while keeping the computation time at an acceptable level. In each Monte Carlo iteration a new optimization problem has to be solved. However, as the solution of the original ICLS problem can be used as feasible initial value for the parameters, convergence of the active-set method is usually quite fast.
Mehr anzeigen

140 Mehr lesen ### Penalized regression for discrete structures

In recent years, model selection and regularization in regression models has been an area of intensive research. Often, penalized approaches are the method of choice. Examples are Ridge regression (Hoerl and Kennard, 1970), the least absolute shrinkage and selec- tion operator (Lasso; Tibshirani, 1996), the smoothly clipped absolute deviation penalty (SCAD; Fan and Li, 2001), the fused Lasso (Tibshirani et al., 2005), the elastic net (Zou and Hastie, 2005) and the (adaptive) group Lasso (Yuan and Lin, 2006; Wang and Leng, 2008), to mention only a few approaches. The number of applications is huge. In nonparametric regression, penalties smooth wiggly functions. Eilers and Marx (1996) work, for example, with Ridge penalties on higher order differences of B-spline coefficients. Meier et al. (2009) select splines with a group Lasso penalty. For wavelets and signals, L 0 penalties, or more general L q penalties, 0 ≤ q ≤ 1, are employed (Antoniadis and Fan, 2001; Rippe et al., 2012). Concerning categorical data, Bondell and Reich (2009) or Gertheiss and Tutz (2010) work with fused Lasso type penalties. Fahrmeir et al. (2010) offer a flexible framework for Bayesian regularization and variable selection, amongst others with spike and slab priors. Various efficient algorithms to solve the resulting optimization problems are available, be it in linear models, generalized linear models (GLMs), hazard rate models or others. Least angle regression (lars; Efron et al., 2004; Hastie and Efron, 2013) offers a conceptual frame- work to compute the entire regularization path of the Lasso by exploiting its piecewise lin- ear coefficient profiles. Osborne and Turlach (2011) propose a homotopy algorithm for the quantile regression Lasso and related piecewise linear problems. Meier et al. (2008) propose a coordinate-descent algorithm for the group Lasso in logistic regression problems. Goeman (2010) solves Lasso, fused Lasso and Ridge-type problems in high-dimensional models by a
Mehr anzeigen

196 Mehr lesen ### Chance Constrained Optimization of Process Systems under Uncertainty

the effect of correlated uncertain variables on plant design. Two approaches have been used to represent uncertain variables: discrete and continuous distribution. In the former, the bounded uncertain variables are discretized into multiple intervals such that each individual interval represents a scenario with an approximated discrete distribution (Halemane and Grossmann, 1993; Subrahmanyam et al., 1994; Pistikopoulos and Ierapetritou, 1995a; Rooney and Biegler, 1999). Thus, so-called multiperiod optimization problems are formulated. The second approach considers the continuous stochastic distribution of the uncertain variables, in which a multivariate numerical integration method will be chosen. This leads to a stochastic programming problem. An approximated integration through a sampling scheme (Diwekar and Kalagnannam, 1997) and a direct numerical integration (Bernado et al., 1999) have been used. Alternatively to sampled optimization algorithms, the stochastic problem can be relaxed to an equivalent NLP problem and then solved by using standard techniques. Thus, the optimization problem needs to be reformulated. If the uncertain variables have an impact on the objective function, it is usually formulated as the expected value of the objective function (Torvi and Herzberg, 1997; Acevedo and Pistikopoulos, 1998). Practically most of the previous cited works employed the two-stage programming method with the recourse formulation to deal with inequality constraints. In this approach the first-stage decision variables are predetermined before the realization of the uncertain variables, while the second- stage variables are decided after their realization. Moreover, in the recourse formulation, violation of the constraints is allowed, but penalized through penalty terms in the objective function. This leads to additional costs regarding the second-stage decisions. This approach is suitable when the objective function and constraint violations can be described by the same measurement, for example process planning problems under demand uncertainties (Clay and Grossmann, 1997; Gupta and Maranas, 2000). This compensation, however, requires a common measurement to describe the objective function and the constraint violations.
Mehr anzeigen

198 Mehr lesen ### Polyhedral aspects of cardinality constrained combinatorial optimization problems

The chapter is structured as follows. Section 5.1 briefly sketches the main ideas behind projection. Section 5.2 gives an overview over polyhedral as- pects of dynamic programming. In particular, we review the fundamental work of Martin, Rardin, and Campbell  who have provided a framework for deriving linear characterizations of dynamic programs. In Section 5.3, we review the most important results from the literature related to the hop constrained path polytope. Moreover, we introduce two relaxations of this polytope mentioned above corresponding to the cases that the given length function d is nonnegative or has no negative cycles. Section 5.4 breaks down the dynamic programming paradigm to the hop constrained shortest path problem. Next, Section 5.5 relates the separation problems of the two re- laxations of the hop constrained path polytope to the multicommodity flow problem and length-bounded cut and flow problems. The results there imply that it will be quite hard to design combinatorial algorithms that solve these separation problems in polynomial time. Finally, Section 5.6 characterizes all 0/1-facet defining inequalities for the dominant of the hop constrained path polytope and all facet defining inequalities for the hop constrained walk polytope with coefficients in {−1, 0, 1} using the dynamic program paradigm. The attained results will be related to already known results.
Mehr anzeigen

229 Mehr lesen ### Shape constrained estimators in inverse regression models with convolution-type operator

In some context the density f is unknown but there is information like monotonicity, convexity or log-concavity about its shape. In general, the unconstrained estimator defined in (1) will not necessarily fulfill those shape constraints but if there is such information, it is desirable to have estimators which meet these constraints. In the literature, there are several proposals of such estimators in density as well as regression estimation but most methods rely on maximum likeli- hood estimation or constrained least squares where the estimator is obtained by minimizing the ### Local Optimization and Complexity Control for Symbolic Regression / eingereicht von Michael Kommenda

Instead of adding every numerical constant to the terminal set beforehand, which yields the problems described in the previous section, numerical con- stants can be introduced by the use of ephemeral random constants ERCs [ Koza , 1992 ]. When ERCs are used, the special symbol R is added to the terminal set and every time the ERC symbol R is selected during tree cre- ation a new constant value is drawn from a predefined distribution. Common distributions the numeric values are drawn from are the uniform distribution or the Gaussian distribution. The properties of the distributions, such as the lower and upper limit or the average and mean, for the ERC symbol have to be adapted at the problem at hand to generate appropriate real-valued constants. Once the numeric values are generated, these values remain fixed and are not sampled again. In its initial definition, similarly to the constants added directly as symbols, ERCs once generated are not modified anymore and are moved between the individual solutions by the crossover operator. As a result the constant values can be combined in a way to generate inter- mediate values necessary for solving the symbolic regression problem. ERCs provide a greater flexibility, because it is possible to create real-values con- stants according to a predefined distribution, compared to adding constants directly to the terminal set.
Mehr anzeigen

157 Mehr lesen ### Inference in Regression Discontinuity Designs with a Discrete Running Variable

These results caution against clustering standard errors by the running variable in em- pirical applications, in spite of its great popularity. 6 We therefore propose two alternative methods for constructing CIs for the ATE in discrete RDDs that have guaranteed coverage properties under interpretable restrictions on the conditional expectation function. The first method makes the assumption that the magnitude of the approximation bias is no larger at the left limit of the threshold than at any point in the support of the running variable below the threshold, and similarly for the right limit. The second method relies on the assump- tion recently considered in Armstrong and Kolesár (2016b) that the second derivative of the conditional expectation function is bounded by a constant. Both CIs are “honest” in the sense of Li (1989) in that they achieve asymptotically correct coverage for all possible model parameters, that is, they are valid uniformly in the value of the conditional expectation function.
Mehr anzeigen ### An improved genetic algorithm for multidimensional optimization of precedence-constrained production planning and scheduling

Received: 21 April 2016 / Accepted: 20 December 2016 / Published online: 9 January 2017  The Author(s) 2017. This article is published with open access at Springerlink.com Abstract Integration of production planning and schedul- ing is a class of problems commonly found in manufac- turing industry. This class of problems associated with precedence constraint has been previously modeled and optimized by the authors, in which, it requires a multidi- mensional optimization at the same time: what to make, how many to make, where to make and the order to make. It is a combinatorial, NP-hard problem, for which no polynomial time algorithm is known to produce an optimal result on a random graph. In this paper, the further devel- opment of Genetic Algorithm (GA) for this integrated optimization is presented. Because of the dynamic nature of the problem, the size of its solution is variable. To deal with this variability and find an optimal solution to the problem, GA with new features in chromosome encoding, crossover, mutation, selection as well as algorithm struc- ture is developed herein. With the proposed structure, the proposed GA is able to ‘‘learn’’ from its experience. Robustness of the proposed GA is demonstrated by a complex numerical example in which performance of the proposed GA is compared with those of three commercial optimization solvers.
Mehr anzeigen ### A Composite Step Method with Inexact Step Computations for PDE Constrained Optimization

harder problems, more iterations and hence more time is needed. However, in every case, the optimal solution was found, which reflects that even for strongly nonlinear and almost singular problems, e.g. c = 10 3 , d = 10 −3 , the globalization mechanism and the preemptive adaptive termination of the PPCG-iterations described in Section 5 lead to stable performance of the algorithm. Moreover, for time-dependent problems, the special block-diagonal structure of the differential operator A in the case of a discontinuous Galerkin method in time allows for a time- step wise factorization, which results in high saving of memory as opposed to factorization of the full matrix A. In the two settings c = d = 1 and c = 10, d = 0.1, no nonconvexities were encountered. This indicates, together with the low iteration numbers, that the starting value (y, u) = (0, 0) is close to a local solution. For the other two cases, nonconvexities and stronger nonlinearities led to higher computations times and to all mechanisms of the algorithm being used.
Mehr anzeigen ### Cardinality restrictions on concepts

We have shown how to extend a terminological KR formalism by a construct that can express global restrictions on the cardinality of concepts. The usefulness of these cardinality restrictions on concepts was demonstrated by an example from a con guration application. Unlike role-value maps (which could be used to model similar situations), our new construct leaves all the important inference problems decidable. The consistency algorithm combines and simpli es the ideas developed for the treatment of qualifying number restrictions and of terminological axioms. ### Quantitative stability analysis of optimal solutions in PDE-constrained optimization

In this section, we propose an efficient algorithm for the numerical computation of the singular system for a discretized (matrix) version of ΠK −1 B. The convergence of the singular system of the discretized problem to the singular system of the continuous problem will be discussed elsewhere. In practice, it will be sufficient to compute only a partial SVD, starting with the largest singular value, down to a certain threshold, in order to collect the perturbation directions of greatest impact with respect to the observed quantity. The method we propose makes use of existing standard software which iteratively approximates the extreme eigenpairs of non-symmetric matrices, and it will be efficient in the following sense: It is unnecessary to assemble the (discretized) matrix ΠK −1 B, which is prohibitive for
Mehr anzeigen ### OPUS 4 | Efficient PDE Constrained Shape Optimization in Shape Spaces

started. In the simplest shape space case (landmark vectors), the distances between shapes can be measured by the Euclidean distance, but in general, the study of shapes and their similarities is a central problem. Now, we can ask how we can tackle natural questions like “How different are shapes?”, “Can we determine the measure of their difference?” or “Can we infer some information?”. In order to answer these questions mathematically, we have to put a metric on the shape space. There are various types of metrics on shape spaces, e.g., inner metrics [11, 12, 64], which can be seen as describing a deformable material that the shape itself is made of. In contrast to these inner metrics, there are also outer metrics [13, 17, 36, 48, 64]. Since the differential operator governing these metrics is defined even outside of the shape, they can be seen as describing some deformable material that the ambient space of the shape is made of. Furthermore, metamorphosis metrics [45, 101], the Wasserstein or Monge-Kantorovic metric on the shape space of probability measures [5, 14, 15], the Weil-Petersson metric [55, 91], current metrics [26, 27, 103] and metrics based on elastic deformations [32, 108] should be mentioned. However, it is a challenging task to model both, the shape space and the associated metric. There does not exist a common shape space or shape metric suitable for all applications. Different approaches lead to diverse models. The suitability of an approach depends on the requirements in a given situation. In this thesis, among all these shape space concepts, we pick the Riemannian shape manifold introduced by Peter W. Michor and David Mumford in which a two-dimensional shape is defined as a smooth embedding from the unit circle into the plane (cf. ). Moreover, we consider inner metrics. More precisely, we first work with so-called Sobolev metrics on this shape space. Of course, there are a lot of further metrics on it (cf. ), but the Sobolev metric is the most suitable choice for our applications. However, we consider also so-called Steklov-Poincaré metrics in order to enable the usage of volume integral formulations of shape derivatives in optimization strategies. Note that it is not possible to use these formulations by considering Sobolev metrics.
Mehr anzeigen

147 Mehr lesen ### Solving constrained trajectory planning problems using biased particle swarm optimization

Heating Fig. 2: State/control/constraint evolutions: Case 1 As can be observed from Figs.2-5, the pre-specified entry terminal boundary conditions and safety-related path constraints can be satisfied for all the mission cases, thereby confirming the validity of the investigated heuristic meth- ods. That is, both the penalty function-based and the multi- objective transformation-based constraint handling strategies are able to guide the searching direction toward the feasible region. In terms of the flight trajectories, same trend can be observed from the solutions generated by different heuristic algorithms for all the considered mission cases. Moreover, for the third mission case, the three evolutionary methods can produce almost identical solutions. Moreover, by viewing the system state and control profiles, it is clear that the obtained trajectories are relatively smooth. This can be attributed to the differential equation imposed on the actual bank angle variable 𝜎. This equation can also be understood as a first-order filter and it indirectly restricts the rate of the actual bank angle.
Mehr anzeigen ### Evaluation of Bayesian Optimization applied to Discrete-Event Simulation

ABSTRACT:In this paper, we evaluate the application of Bayesian Optimization (BO) to discrete event simulation (DES) models. In a first step, we create a simple model, for which we know the optimal set of parameter values in advance. We implement the model in SimPy, a framework for DES written in Python. We then interpret the simulation model as a black box function subject to optimization. We show that it is possible to find the optimal set of parameter values using the open source library GPyOpt. To enhance our evaluation, we create a second and more complex model. To better handle the complexity of the model, and to add a visual component, we build the second model in Simio, a commercial off-the-shelf simulation modeling tool. To apply BO to a model in Simio, we use the Simio API to write an extension for optimization plug-ins. This extension encapsulates the logic of the BO algorithm, which we deployed as a web service in the cloud. The fact that simulation models are black box functions with regard to their behavior and the influence of their input parameters makes them an apparent candidate for Bayesian Optimization (BO). Simulation models are multivariable and stochastic, and their behavior is to a large extent unpredictable. In particular, we do not know for sure which input parameters to adjust to maximize (or minimize) the model’s outcome. In addition, the complex models can take a substantial amount of time to run. Bayesian Optimization is a sequential and self-learning algorithm to optimize black box functions similar to as we find them in simulation models: they contain a set of parameters for which we want to identify the optimal set, they are expensive to evaluate, and they exhibit stochastic noise. BO has proven to efficiently optimize black box functions from various disciplines. Among those, and most notably, it is successfully applied in machine learning algorithms to optimize hyperparameters.
Mehr anzeigen