Página 1 dos resultados de 2378 itens digitais encontrados em 0.012 segundos

Bilevel derivative-free optimization and its application to robust optimization

Conn, Andrew R.; Vicente, L. N.
Fonte: Centro de Matemática da Universidade de Coimbra Publicador: Centro de Matemática da Universidade de Coimbra
Tipo: Pré-impressão
ENG
Relevância na Pesquisa
66.33%
We address bilevel programming problems when the derivatives of both the upper and the lower level objective functions are unavailable. The core algorithms used for both levels are trust-region interpolation-based methods, using minimum Frobenius norm quadratic models when the number of points is smaller than the number of basis components. We take advantage of the problem structure to derive conditions (related to the global convergence theory of the underlying trust-region methods, as far as possible) under which the lower level can be solved inexactly and sample points can be reused for model building. In addition, we indicate numerically how effective these expedients can be. A number of other issues are also discussed, from the extension to linearly constrained problems to the use of surrogate models for the lower level response. One important application of our work appears in the robust optimization of simulation-based functions, which may arise due to implementation variables or uncertain parameters. The robust counterpart of an optimization problem without derivatives falls in the category of the bilevel problems under consideration here. We provide numerical illustrations of the application of our algorithmic framework to such robust optimization examples

On the robustness of structural risk optimization with respect to epistemic uncertainties.

Beck, André Teófilo; Gomes, Wellison José de Santana; Bazán, Felipe Alexander Vargas
Fonte: Redding Publicador: Redding
Tipo: Artigo de Revista Científica
ENG
Relevância na Pesquisa
56.39%
In the context of structural design, risk optimization allows one to find a proper point of balance between the concurrent goals of economy and safety. Risk optimization involves the minimization of total expected costs, which include expected costs of failure. Expected costs of failure are evaluated from nominal failure probabilities, which reflect the analyst�s degree of belief in the structure�s performance. Such failure probabilities are said to be nominal because they are evaluated from imperfect and/or incomplete mechanical, mathematical and probabilistic models. Hence, model uncertainty and other types of epistemic uncertainties are likely to compromise the results of risk optimization. In this paper, the concept of robustness is employed in order to find risk optimization solutions which are less sensitive to epistemic uncertainties. The investigation is based on a simple but illustrative problem, which is built from an elementary but fundamental structural (load-resistance) reliability problem. Intrinsic or aleatoric uncertainties, which can be quantified probabilistically and modeled as random variables or stochastic processes, are incorporated in the underlying structural reliability problem. Epistemic uncertainties that can only be quantified possibilistically are modeled as fuzzy variables...

Programação estocástica e otimização robusta no planejamento da produção de empresas moveleiras; Stochastic programming and robust optimization in the production planning of furniture industries

Alem Júnior, Douglas José
Fonte: Biblioteca Digitais de Teses e Dissertações da USP Publicador: Biblioteca Digitais de Teses e Dissertações da USP
Tipo: Tese de Doutorado Formato: application/pdf
Publicado em 08/04/2011 PT
Relevância na Pesquisa
66.34%
O planejamento da produção em indústrias moveleiras de pequeno porte é comumente constituído por decisões referentes ao volume de produção e à política de estoque, com o objetivo de minimizar o desperdício de material, os atrasos e as horas-extras utilizadas ao longo do horizonte de planejamento. Administrar tais decisões de uma maneira tratável e eficiente é, em geral, um desafio, especialmente considerando a natureza incerta dos dados. Nessa tese, são desenvolvidos modelos de otimização para apoiar tais decisões no contexto do problema combinado de dimensionamento de lotes e corte de estoque sob incertezas que surge em indústrias moveleiras. Para lidar com as incertezas dos dados, são investigadas duas metodologias: programação estocástica e otimização robusta. Dessa maneira, são propostos modelos de programação estocástica de dois estágios com recurso, assim como modelos estocásticos robustos que incorporam aversão ao risco. A motivação em também desenvolver modelos baseados em otimização robusta é considerar casos práticos em que não há uma descrição probabilística explícita dos dados de entrada, assim como evitar trabalhar com numerosos cenários, o que pode tornar o modelo estocástico computacionalmente intratável. Os experimentos numéricos baseados em exemplares reais de uma empresa moveleira de pequeno porte mostram que as soluções obtidas pelos modelos de programação estocástica fornecem planos de produção robustos e que o (a) decisor (a) pode designar suas preferências em relação ao risco aos modelos...

Otimização linear robusta multitemporal de uma carteira de ativos com parâmetros de média e dispersão incertos; Robust linear multistage portfolio optimization with location and dispersion parameters subject to uncertainty.

Godói, André Cadime de
Fonte: Biblioteca Digitais de Teses e Dissertações da USP Publicador: Biblioteca Digitais de Teses e Dissertações da USP
Tipo: Tese de Doutorado Formato: application/pdf
Publicado em 27/09/2011 PT
Relevância na Pesquisa
56.29%
Nos últimos anos, percebeu-se um avanço substancial das metodologias sistemáticas de seleção de ativos em portfólios financeiros, baseadas em técnicas de otimização. A maior pressão por desempenho sobre as gestoras de recursos e a evolução dos softwares e pacotes de otimização foram fatores que contribuíram para esse desenvolvimento. Dentre as técnicas mais reconhecidas utilizadas na gestão de portfólios está a de otimização robusta, cuja aplicação na solução de problemas com dados incertos iniciou-se na década de 1970 e, desde então, vem evoluindo em sofisticação. Partindo de uma extensão recente do método, propõe-se um novo modelo linear que resolve o problema de otimização de um portfólio para múltiplos estágios, com inovações no tratamento da incerteza das estimativas de dispersão dos retornos. Os resultados mostram que o método proposto desempenha muito bem em termos de rentabilidade e de métricas de risco-retorno em momentos de turbulência dos mercados. Por fim, demonstra-se empiricamente que o modelo alcança um desempenho ainda melhor em termos de rentabilidade com a adoção de um estimador eficiente para o valor esperado dos retornos e com a simultânea redução do nível de robustez do modelo.; It has been realized in the last years a remarkable development of the optimization techniques to solve the problem of financial portfolio selection. The pressure on asset management firms to maintain a more stable performance and the evolution of specialized software packages have enabled this positive trend. One of the most recognized approaches applied to the management of investments is the robust optimization...

Otimização robusta aplicada à operação de reservatórios para a geração de energia.; Robust optimization applied to reservoirs operation for hydropower generation.

Côrtes, Roberto Sarti
Fonte: Biblioteca Digitais de Teses e Dissertações da USP Publicador: Biblioteca Digitais de Teses e Dissertações da USP
Tipo: Dissertação de Mestrado Formato: application/pdf
Publicado em 02/07/2013 PT
Relevância na Pesquisa
66.25%
Este trabalho tem como objetivo avaliar a viabilidade da aplicação de técnicas de otimização robusta (OR) no planejamento da operação de reservatórios para geração de energia hidrelétrica. A OR é uma técnica de otimização que visa encontrar resultados que sejam menos sensíveis às incertezas nas variáveis do modelo através da minimização da variância da função objetivo para diferentes cenários. Desta forma foi desenvolvido um modelo de otimização robusta aplicado à operação de reservatórios para a geração de energia hidrelétrica, chamado HIDRO-OR, utilizando o software General Algebraic Modeling System (GAMS). Para estudo de caso foram utilizados os dados da UHE Sinop, a ser instalada no rio Teles Pires MT. Inicialmente foi realizada uma análise de sensibilidade utilizando diferentes combinações dos coeficientes de ponderação da função objetivo e três conjuntos de cenários. Nesta abordagem, o modelo resultou em vertimentos indesejados para realizar a diminuição do desvio padrão dos resultados entre os diferentes cenários. Uma solução encontrada para o problema foi realizar a otimização em duas etapas. Na primeira etapa ocorre a otimização robusta propriamente dita e são fixados os resultados para o primeiro mês de operação. Na segunda etapa...

A Robust Optimization Approach to Supply Chain Management

Bertsimas, Dimitris J.; Thiele, Aurélie
Fonte: MIT - Massachusetts Institute of Technology Publicador: MIT - Massachusetts Institute of Technology
Tipo: Artigo de Revista Científica Formato: 413796 bytes; application/pdf
EN_US
Relevância na Pesquisa
66.13%
We propose a general methodology based on robust optimization to address the problem of optimally controlling a supply chain subject to stochastic demand in discrete time. The attractive features of the proposed approach are: (a) It incorporates a wide variety of phenomena, including demands that are not identically distributed over time and capacity on the echelons and links; (b) it uses very little information on the demand distributions; (c) it leads to qualititatively similar optimal policies (basestock policies) as in dynamic programming; (d) it is numerically tractable for large scale supply chain problems even in networks, where dynamic programming methods face serious dimensionality problems; (e) in preliminary computation experiments, it often outperforms dynamic programming based solutions for a wide range of parameters.; Singapore-MIT Alliance (SMA)

Robust fluid control of multiclass queueing networks; Robust fluid control of multiclass queuing networks

Su, Hua, S.M. Massachusetts Institute of Technology
Fonte: Massachusetts Institute of Technology Publicador: Massachusetts Institute of Technology
Tipo: Tese de Doutorado Formato: 92 p.
ENG
Relevância na Pesquisa
56.34%
This thesis applies recent advances in the field of robust optimization to the optimal control of multiclass queueing networks. We develop models that take into account the uncertainty of interarrival and service time in multiclass queueing network problems without assuming a specific probability distribution, while remaining highly tractable and providing insight into the corresponding optimal control policy. Our approach also allows us to adjust the level of robustness of the solution to trade off performance and protection against uncertainty. We apply robust optimization to both open and closed queueing networks. For open queueing networks, we study control problems that involve sequencing, routing and input control decision, and optimize the total holding cost. For closed queueing networks, we focus on the sequencing problem and optimize the throughput. We compare the robust solutions to those derived by fluid control, dynamic programming and stochastic input control. We show that the robust control policy leads to better performance. Robust optimization emerges as a promising methodology to address a wide range of multiclass queueing networks subject to uncertainty, as it leads to representations of randomness that make few assumptions on the underlying probabilities. It also remains numerically tractable...

Empirical comparison of robust, data driven and stochastic optimization

Wang, Yanbo, S.M. Massachusetts Institute of Technology
Fonte: Massachusetts Institute of Technology Publicador: Massachusetts Institute of Technology
Tipo: Tese de Doutorado Formato: 49 leaves
ENG
Relevância na Pesquisa
56.26%
In this thesis, we compare computationally four methods for solving optimization problems under uncertainty: * Robust Optimization (RO) * Adaptive Robust Optimization (ARO) * Data Driven Optimization (DDO) * stochastic Programming (SP) We have implemented several computation experiments to demonstrate the different performance of these methods. We conclude that ARO outperform RO, which has a comparable performance with DDO. SP has a comparable performance with RO when the assumed distribution is the same as the true underlying distribution, but under performs RO when the assumed distribution is different from the true distribution.; by Wang, Yanbo.; Includes bibliographical references (leaf 49).; Thesis (S.M.)--Massachusetts Institute of Technology, Computation for Design and Optimization Program, 2008.

Likelihood Robust Optimization for Data-driven Problems

Wang, Zizhuo; Glynn, Peter; Ye, Yinyu
Fonte: Universidade Cornell Publicador: Universidade Cornell
Tipo: Artigo de Revista Científica
Relevância na Pesquisa
56.3%
We consider optimal decision-making problems in an uncertain environment. In particular, we consider the case in which the distribution of the input is unknown, yet there is abundant historical data drawn from the distribution. In this paper, we propose a new type of distributionally robust optimization model called the likelihood robust optimization (LRO) model for this class of problems. In contrast to previous work on distributionally robust optimization that focuses on certain parameters (e.g., mean, variance, etc.) of the input distribution, we exploit the historical data and define the accessible distribution set to contain only those distributions that make the observed data achieve a certain level of likelihood. Then we formulate the targeting problem as one of optimizing the expected value of the objective function under the worst-case distribution in that set. Our model avoids the over-conservativeness of some prior robust approaches by ruling out unrealistic distributions while maintaining robustness of the solution for any statistically likely outcomes. We present statistical analyses of our model using Bayesian statistics and empirical likelihood theory. Specifically, we prove the asymptotic behavior of our distribution set and establish the relationship between our model and other distributionally robust models. To test the performance of our model...

Algorithm Engineering in Robust Optimization

Goerigk, Marc; Schöbel, Anita
Fonte: Universidade Cornell Publicador: Universidade Cornell
Tipo: Artigo de Revista Científica
Relevância na Pesquisa
56.3%
Robust optimization is a young and emerging field of research having received a considerable increase of interest over the last decade. In this paper, we argue that the the algorithm engineering methodology fits very well to the field of robust optimization and yields a rewarding new perspective on both the current state of research and open research directions. To this end we go through the algorithm engineering cycle of design and analysis of concepts, development and implementation of algorithms, and theoretical and experimental evaluation. We show that many ideas of algorithm engineering have already been applied in publications on robust optimization. Most work on robust optimization is devoted to analysis of the concepts and the development of algorithms, some papers deal with the evaluation of a particular concept in case studies, and work on comparison of concepts just starts. What is still a drawback in many papers on robustness is the missing link to include the results of the experiments again in the design.

Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations

Esfahani, Peyman Mohajerin; Kuhn, Daniel
Fonte: Universidade Cornell Publicador: Universidade Cornell
Tipo: Artigo de Revista Científica
Publicado em 19/05/2015
Relevância na Pesquisa
56.23%
We consider stochastic programs where the distribution of the uncertain parameters is only observable through a finite training dataset. Using the Wasserstein metric, we construct a ball in the space of (multivariate and non-discrete) probability distributions centered at the uniform distribution on the training samples, and we seek decisions that perform best in view of the worst-case distribution within this Wasserstein ball. The state-of-the-art methods for solving the resulting distributionally robust optimization problems rely on global optimization techniques, which quickly become computationally excruciating. In this paper we demonstrate that, under mild assumptions, the distributionally robust optimization problems over Wasserstein balls can in fact be reformulated as finite convex programs---in many interesting cases even as tractable linear programs. Leveraging recent measure concentration results, we also show that their solutions enjoy powerful finite-sample performance guarantees. Our theoretical results are exemplified in mean-risk portfolio optimization as well as uncertainty quantification.; Comment: 32 pages, 6 figures

Commitment and Dispatch of Heat and Power Units via Affinely Adjustable Robust Optimization

Zugno, Marco; Morales, Juan M.; Madsen, Henrik
Fonte: Universidade Cornell Publicador: Universidade Cornell
Tipo: Artigo de Revista Científica
Publicado em 21/07/2015
Relevância na Pesquisa
56.3%
The joint management of heat and power systems is believed to be key to the integration of renewables into energy systems with a large penetration of district heating. Determining the day-ahead unit commitment and production schedules for these systems is an optimization problem subject to uncertainty stemming from the unpredictability of demand and prices for heat and electricity. Furthermore, owing to the dynamic features of production and heat storage units as well as to the length and granularity of the optimization horizon (e.g., one whole day with hourly resolution), this problem is in essence a multi-stage one. We propose a formulation based on robust optimization where recourse decisions are approximated as linear or piecewise-linear functions of the uncertain parameters. This approach allows for a rigorous modeling of the uncertainty in multi-stage decision-making without compromising computational tractability. We perform an extensive numerical study based on data from the Copenhagen area in Denmark, which highlights important features of the proposed model. Firstly, we illustrate commitment and dispatch choices that increase conservativeness in the robust optimization approach. Secondly, we appraise the gain obtained by switching from linear to piecewise-linear decision rules within robust optimization. Furthermore...

A cutting surface algorithm for semi-infinite convex programming with an application to moment robust optimization

Mehrotra, Sanjay; Papp, David
Fonte: Universidade Cornell Publicador: Universidade Cornell
Tipo: Artigo de Revista Científica
Relevância na Pesquisa
56.4%
We present and analyze a central cutting surface algorithm for general semi-infinite convex optimization problems, and use it to develop a novel algorithm for distributionally robust optimization problems in which the uncertainty set consists of probability distributions with given bounds on their moments. Moments of arbitrary order, as well as non-polynomial moments can be included in the formulation. We show that this gives rise to a hierarchy of optimization problems with decreasing levels of risk-aversion, with classic robust optimization at one end of the spectrum, and stochastic programming at the other. Although our primary motivation is to solve distributionally robust optimization problems with moment uncertainty, the cutting surface method for general semi-infinite convex programs is also of independent interest. The proposed method is applicable to problems with non-differentiable semi-infinite constraints indexed by an infinite-dimensional index set. Examples comparing the cutting surface algorithm to the central cutting plane algorithm of Kortanek and No demonstrate the potential of our algorithm even in the solution of traditional semi-infinite convex programming problems whose constraints are differentiable and are indexed by an index set of low dimension. After the rate of convergence analysis of the cutting surface algorithm...

Adaptive Robust Optimization with Dynamic Uncertainty Sets for Multi-Period Economic Dispatch under Significant Wind

Lorca, Alvaro; Sun, Andy
Fonte: Universidade Cornell Publicador: Universidade Cornell
Tipo: Artigo de Revista Científica
Relevância na Pesquisa
56.28%
The exceptional benefits of wind power as an environmentally responsible renewable energy resource have led to an increasing penetration of wind energy in today's power systems. This trend has started to reshape the paradigms of power system operations, as dealing with uncertainty caused by the highly intermittent and uncertain wind power becomes a significant issue. Motivated by this, we present a new framework using adaptive robust optimization for the economic dispatch of power systems with high level of wind penetration. In particular, we propose an adaptive robust optimization model for multi-period economic dispatch, and introduce the concept of dynamic uncertainty sets and methods to construct such sets to model temporal and spatial correlations of uncertainty. We also develop a simulation platform which combines the proposed robust economic dispatch model with statistical prediction tools in a rolling horizon framework. We have conducted extensive computational experiments on this platform using real wind data. The results are promising and demonstrate the benefits of our approach in terms of cost and reliability over existing robust optimization models as well as recent look-ahead dispatch models.; Comment: Accepted for publication at IEEE Transactions on Power Systems

Data-Driven Robust Optimization

Bertsimas, Dimitris; Gupta, Vishal; Kallus, Nathan
Fonte: Universidade Cornell Publicador: Universidade Cornell
Tipo: Artigo de Revista Científica
Relevância na Pesquisa
56.25%
The last decade witnessed an explosion in the availability of data for operations research applications. Motivated by this growing availability, we propose a novel schema for utilizing data to design uncertainty sets for robust optimization using statistical hypothesis tests. The approach is flexible and widely applicable, and robust optimization problems built from our new sets are computationally tractable, both theoretically and practically. Furthermore, optimal solutions to these problems enjoy a strong, finite-sample probabilistic guarantee. \edit{We describe concrete procedures for choosing an appropriate set for a given application and applying our approach to multiple uncertain constraints. Computational evidence in portfolio management and queuing confirm that our data-driven sets significantly outperform traditional robust optimization techniques whenever data is available.; Comment: 38 pages, 15 page appendix, 7 figures. This version updated as of Oct. 2014

Robust optimization with incremental recourse

Nasrabadi, Ebrahim; Orlin, James B.
Fonte: Universidade Cornell Publicador: Universidade Cornell
Tipo: Artigo de Revista Científica
Publicado em 14/12/2013
Relevância na Pesquisa
56.34%
In this paper, we consider an adaptive approach to address optimization problems with uncertain cost parameters. Here, the decision maker selects an initial decision, observes the realization of the uncertain cost parameters, and then is permitted to modify the initial decision. We treat the uncertainty using the framework of robust optimization in which uncertain parameters lie within a given set. The decision maker optimizes so as to develop the best cost guarantee in terms of the worst-case analysis. The recourse decision is ``incremental"; that is, the decision maker is permitted to change the initial solution by a small fixed amount. We refer to the resulting problem as the robust incremental problem. We study robust incremental variants of several optimization problems. We show that the robust incremental counterpart of a linear program is itself a linear program if the uncertainty set is polyhedral. Hence, it is solvable in polynomial time. We establish the NP-hardness for robust incremental linear programming for the case of a discrete uncertainty set. We show that the robust incremental shortest path problem is NP-complete when costs are chosen from a polyhedral uncertainty set, even in the case that only one new arc may be added to the initial path. We also address the complexity of several special cases of the robust incremental shortest path problem and the robust incremental minimum spanning tree problem.

Oracle-Based Robust Optimization via Online Learning

Ben-Tal, Aharon; Hazan, Elad; Koren, Tomer; Mannor, Shie
Fonte: Universidade Cornell Publicador: Universidade Cornell
Tipo: Artigo de Revista Científica
Publicado em 25/02/2014
Relevância na Pesquisa
56.42%
Robust optimization is a common framework in optimization under uncertainty when the problem parameters are not known, but it is rather known that the parameters belong to some given uncertainty set. In the robust optimization framework the problem solved is a min-max problem where a solution is judged according to its performance on the worst possible realization of the parameters. In many cases, a straightforward solution of the robust optimization problem of a certain type requires solving an optimization problem of a more complicated type, and in some cases even NP-hard. For example, solving a robust conic quadratic program, such as those arising in robust SVM, ellipsoidal uncertainty leads in general to a semidefinite program. In this paper we develop a method for approximately solving a robust optimization problem using tools from online convex optimization, where in every stage a standard (non-robust) optimization program is solved. Our algorithms find an approximate robust solution using a number of calls to an oracle that solves the original (non-robust) problem that is inversely proportional to the square of the target accuracy.

New results about multi-band uncertainty in Robust Optimization

Büsing, Christina; D'Andreagiovanni, Fabio
Fonte: Universidade Cornell Publicador: Universidade Cornell
Tipo: Artigo de Revista Científica
Publicado em 30/08/2012
Relevância na Pesquisa
56.27%
"The Price of Robustness" by Bertsimas and Sim represented a breakthrough in the development of a tractable robust counterpart of Linear Programming Problems. However, the central modeling assumption that the deviation band of each uncertain parameter is single may be too limitative in practice: experience indeed suggests that the deviations distribute also internally to the single band, so that getting a higher resolution by partitioning the band into multiple sub-bands seems advisable. The critical aim of our work is to close the knowledge gap about the adoption of a multi-band uncertainty set in Robust Optimization: a general definition and intensive theoretical study of a multi-band model are actually still missing. Our new developments have been also strongly inspired and encouraged by our industrial partners, which have been interested in getting a better modeling of arbitrary distributions, built on historical data of the uncertainty affecting the considered real-world problems. In this paper, we study the robust counterpart of a Linear Programming Problem with uncertain coefficient matrix, when a multi-band uncertainty set is considered. We first show that the robust counterpart corresponds to a compact LP formulation. Then we investigate the problem of separating cuts imposing robustness and we show that the separation can be efficiently operated by solving a min-cost flow problem. Finally...

A Practical Guide to Robust Optimization

Gorissen, Bram L.; Yanıkoğlu, Ihsan; Hertog, Dick den
Fonte: Universidade Cornell Publicador: Universidade Cornell
Tipo: Artigo de Revista Científica
Publicado em 12/01/2015
Relevância na Pesquisa
56.34%
Robust optimization is a young and active research field that has been mainly developed in the last 15 years. Robust optimization is very useful for practice, since it is tailored to the information at hand, and it leads to computationally tractable formulations. It is therefore remarkable that real-life applications of robust optimization are still lagging behind; there is much more potential for real-life applications than has been exploited hitherto. The aim of this paper is to help practitioners to understand robust optimization and to successfully apply it in practice. We provide a brief introduction to robust optimization, and also describe important do's and don'ts for using it in practice. We use many small examples to illustrate our discussions.

Multi-objective Robust Optimization using a Post-optimality Sensitivity Analysis Technique: Application to a Wind Turbine Design

Wang, Weijun; Caro, Stéphane; Bennis, Fouad; Soto, Ricardo; Crawford, Broderick
Fonte: Universidade Cornell Publicador: Universidade Cornell
Tipo: Artigo de Revista Científica
Publicado em 03/12/2014
Relevância na Pesquisa
56.25%
Toward a multi-objective optimization robust problem, the variations in design variables and design environment pa-rameters include the small variations and the large varia-tions. The former have small effect on the performance func-tions and/or the constraints, and the latter refer to the ones that have large effect on the performance functions and/or the constraints. The robustness of performance functions is discussed in this paper. A post-optimality sensitivity analysis technique for multi-objective robust optimization problems is discussed and two robustness indices are introduced. The first one considers the robustness of the performance func-tions to small variations in the design variables and the de-sign environment parameters. The second robustness index characterizes the robustness of the performance functions to large variations in the design environment parameters. It is based on the ability of a solution to maintain a good Pareto ranking for different design environment parameters due to large variations. The robustness of the solutions is treated as vectors in the robustness function space, which is defined by the two proposed robustness indices. As a result, the designer can compare the robustness of all Pareto optimal solutions and make a decision. Finally...