TY - JOUR
A1 - Rehbach, Frederik
A1 - Zaefferer, Martin
A1 - Naujoks, Boris
A1 - Bartz-Beielstein, Thomas
T1 - Expected Improvement versus Predicted Value in Surrogate-Based Optimization
N2 - Surrogate-based optimization relies on so-called infill criteria (acquisition functions) to decide which point to evaluate next. When Kriging is used as the surrogate model of choice (also called Bayesian optimization), one of the most frequently chosen criteria is expected improvement. We argue that the popularity of expected improvement largely relies on its theoretical properties rather than empirically validated performance. Few results from the literature show evidence, that under certain conditions, expected improvement may perform worse than something as simple as the predicted value of the surrogate model. We benchmark both infill criteria in an extensive empirical study on the ‘BBOB’ function set. This investigation includes a detailed study of the impact of problem dimensionality on algorithm performance. The results support the hypothesis that exploration loses importance with increasing problem dimensionality. A statistical analysis reveals that the purely exploitative search with the predicted value criterion performs better on most problems of five or higher dimensions. Possible reasons for these results are discussed. In addition, we give an in-depth guide for choosing the infill criteria based on prior knowledge about the problem at hand, its dimensionality, and the available budget.
T3 - CIplus - 4/2020
KW - Optimization
KW - Surrogate-based
Y1 -
U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:832-cos4-9055
ER -
TY - JOUR
A1 - Rehbach, Frederik
A1 - Gentile, Lorenzo
A1 - Bartz-Beielstein, Thomas
T1 - Variable Reduction for Surrogate-Based Optimization
N2 - Real-world problems such as computational fluid dynamics simulations and finite element analyses are computationally expensive. A standard approach to mitigating the high computational expense is Surrogate-Based Optimization (SBO). Yet, due to the high-dimensionality of many simulation problems, SBO is not directly applicable or not efficient. Reducing the dimensionality of the search space is one method to overcome this limitation. In addition to the applicability of SBO, dimensionality reduction enables easier data handling and improved data and model interpretability. Regularization is considered as one state-of-the-art technique for dimensionality reduction. We propose a hybridization approach called Regularized-Surrogate-Optimization (RSO) aimed at overcoming difficulties related to high-dimensionality. It couples standard Kriging-based SBO with regularization techniques. The employed regularization methods are based on three adaptations of the least absolute shrinkage and selection operator (LASSO). In addition, tree-based methods are analyzed as an alternative variable selection method. An extensive study is performed on a set of artificial test functions and two real-world applications: the electrostatic precipitator problem and a multilayered composite design problem. Experiments reveal that RSO requires significantly less time than standard SBO to obtain comparable results. The pros and cons of the RSO approach are discussed, and recommendations for practitioners are presented.
T3 - CIplus - 5/2020
KW - Optimization
KW - Variable reduction
KW - Surrogate-based
Y1 -
U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:832-cos4-9068
ER -
TY - JOUR
A1 - Rehbach, Frederik
A1 - Gentile, Lorenzo
A1 - Bartz-Beielstein, Thomas
T1 - Feature Selection for Surrogate Model-Based Optimization
N2 - We propose a hybridization approach called Regularized-Surrogate- Optimization (RSO) aimed at overcoming difficulties related to high- dimensionality. It combines standard Kriging-based SMBO with regularization techniques. The employed regularization methods use the least absolute shrinkage and selection operator (LASSO). An extensive study is performed on a set of artificial test functions and two real-world applications: the electrostatic precipitator problem and a multilayered composite design problem. Experiments reveal that RSO requires significantly less time than Kriging to obtain comparable results. The pros and cons of the RSO approach are discussed and recommendations for practitioners are presented.
T3 - CIplus - 3/2020
KW - Optimization
KW - Feature selection
KW - Surrogate model
Y1 -
U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:832-cos4-9044
ER -
TY - RPRT
A1 - Bartz-Beielstein, Thomas
A1 - Doerr, Carola
A1 - Bossek, Jakob
A1 - Chandrasekaran, Sowmya
A1 - Eftimov, Tome
A1 - Fischbach, Andreas
A1 - Kerschke, Pascal
A1 - Lopez-Ibanez, Manuel
A1 - Malan, Katherine M.
A1 - Moore, Jason H.
A1 - Naujoks, Boris
A1 - Orzechowski, Patryk
A1 - Volz, Vanessa
A1 - Wagner, Markus
A1 - Weise, Thomas
T1 - Benchmarking in Optimization: Best Practice and Open Issues
N2 - This survey compiles ideas and recommendations from more than a dozen researchers with different backgrounds and from different institutes around the world. Promoting best practice in benchmarking is its main goal. The article discusses eight essential topics in benchmarking: clearly stated goals, well- specified problems, suitable algorithms, adequate performance measures, thoughtful analysis, effective and efficient designs, comprehensible presentations, and guaranteed reproducibility. The final goal is to provide well-accepted guidelines (rules) that might be useful for authors and reviewers. As benchmarking in optimization is an active and evolving field of research this manuscript is meant to co-evolve over time by means of periodic updates.
T3 - CIplus - 2/2020
KW - Optimierung
KW - Benchmarking
KW - Performance
KW - Optimization
KW - Neural and Evolutionary Computing
KW - Benchmarking
Y1 -
U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:832-cos4-9022
ER -
TY - RPRT
A1 - Rehbach, Frederik
A1 - Zaefferer, Martin
A1 - Stork, Jörg
A1 - Bartz-Beielstein, Thomas
T1 - Comparison of Parallel Surrogate-Assisted Optimization Approaches
N2 - The availability of several CPU cores on current computers enables
parallelization and increases the computational power significantly.
Optimization algorithms have to be adapted to exploit these highly
parallelized systems and evaluate multiple candidate solutions in
each iteration. This issue is especially challenging for expensive
optimization problems, where surrogate models are employed to
reduce the load of objective function evaluations.
This paper compares different approaches for surrogate modelbased
optimization in parallel environments. Additionally, an easy
to use method, which was developed for an industrial project, is
proposed. All described algorithms are tested with a variety of
standard benchmark functions. Furthermore, they are applied to
a real-world engineering problem, the electrostatic precipitator
problem. Expensive computational fluid dynamics simulations are
required to estimate the performance of the precipitator. The task
is to optimize a gas-distribution system so that a desired velocity
distribution is achieved for the gas flow throughout the precipitator.
The vast amount of possible configurations leads to a complex
discrete valued optimization problem. The experiments indicate
that a hybrid approach works best, which proposes candidate solutions
based on different surrogate model-based infill criteria and
evolutionary operators.
T3 - CIplus - 7/2018
KW - Optimization
KW - Surrogates
KW - Modeling
KW - Parallelization
KW - Electrostatic Precipitator
Y1 - 2018
U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:832-cos4-7899
ER -
TY - INPR
A1 - Stork, Jörg
A1 - Eiben, A.E.
A1 - Bartz-Beielstein, Thomas
T1 - A new Taxonomy of Continuous Global Optimization Algorithms
N2 - Surrogate-based optimization and nature-inspired metaheuristics have become the state of the art in solving real-world optimization problems. Still, it is difficult for beginners and even experts to get an overview that explains their advantages in comparison to the large number of available methods in the scope of continuous optimization. Available taxonomies lack the integration of surrogate-based approaches and thus their embedding in the larger context of this broad field.
This article presents a taxonomy of the field, which further matches the idea of nature-inspired algorithms, as it is based on the human behavior in path finding. Intuitive analogies make it easy to conceive the most basic principles of the search algorithms, even for beginners and non-experts in this area of research. However, this scheme does not oversimplify the high complexity of the different algorithms, as the class identifier only defines a descriptive meta-level of the algorithm search strategies. The taxonomy was established by exploring and matching algorithm schemes, extracting similarities and differences, and creating a set of classification indicators to distinguish between five distinct classes. In practice, this taxonomy allows recommendations for the applicability of the corresponding algorithms and helps developers trying to create or improve their own algorithms.
T3 - CIplus - 4/2018
KW - Optimization
KW - Taxonomy
KW - Evolutionary Computation
KW - Metaheuristics
KW - Surrogate
KW - Globale Optimierung
KW - Taxonomie
KW - Evolutionärer Algorithmus
KW - Metaheuristik
Y1 - 2018
U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:832-cos4-7538
ER -
TY - RPRT
A1 - Stork, Jörg
A1 - Bartz-Beielstein, Thomas
T1 - Global Optimization Strategies: Analogies to Human Behavior
N2 - This short article presents a new taxonomy for modern global optimization heuristics based on analogies to human behavior.
T3 - CIplus - 2/2018
KW - Taxonomy
KW - Optimization
KW - Heuristics
KW - Surrogate
KW - Metaheuristik
KW - Taxonomie
KW - Optimierung
Y1 - 2018
U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:832-cos4-6967
ER -
TY - RPRT
A1 - Bartz-Beielstein, Thomas
A1 - Gentile, Lorenzo
A1 - Zaefferer, Martin
T1 - In a Nutshell: Sequential Parameter Optimization
N2 - The performance of optimization algorithms relies crucially on their parameterizations. Finding good parameter settings is called algorithm tuning. Using
a simple simulated annealing algorithm, we will demonstrate how optimization algorithms can be tuned using the Sequential Parameter Optimization Toolbox (SPOT). SPOT provides several tools for automated and interactive tuning. The underlying concepts of the SPOT approach are explained. This includes key techniques such as exploratory fitness landscape analysis and response surface methodology. Many examples illustrate
how SPOT can be used for understanding the performance of algorithms and gaining insight into algorithm behavior. Furthermore, we demonstrate how SPOT can be used as an optimizer and how a sophisticated ensemble approach is able to combine several meta models via stacking.
T3 - CIplus - 7/2017
KW - Algorithm Tuning
KW - Optimization
KW - Surrogate Models
KW - SPOT
KW - R
Y1 - 2017
U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:832-cos4-5928
ER -
TY - RPRT
A1 - Zaefferer, Martin
A1 - Fischbach, Andreas
A1 - Naujoks, Boris
A1 - Bartz-Beielstein, Thomas
T1 - Simulation-based Test Functions for Optimization Algorithms
N2 - When designing or developing optimization algorithms, test functions are crucial to evaluate
performance. Often, test functions are not sufficiently difficult, diverse, flexible or relevant to real-world
applications. Previously,
test functions with real-world relevance were generated by training a machine learning model based on
real-world data. The model estimation is used as a test function.
We propose a more principled approach using simulation instead of estimation.
Thus, relevant and varied test functions
are created which represent the behavior of real-world fitness landscapes.
Importantly, estimation can lead to excessively smooth test functions
while simulation may avoid this pitfall. Moreover, the simulation
can be conditioned by the data, so that the simulation reproduces the training data
but features diverse behavior in unobserved regions of the search space.
The proposed test function generator is illustrated with an intuitive, one-dimensional
example. To demonstrate the utility of this approach it
is applied to a protein sequence optimization problem.
This application demonstrates the advantages as well as practical limits of simulation-based
test functions.
T3 - CIplus - 3/2017
KW - Optimization
KW - Test function generator
KW - Simulation
KW - Modeling
Y1 - 2017
U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:832-cos4-4777
ER -
TY - RPRT
A1 - Fischbach, Andreas
A1 - Zaefferer, Martin
A1 - Stork, Jörg
A1 - Friese, Martina
A1 - Bartz-Beielstein, Thomas
T1 - From Real World Data to Test Functions
N2 - When researchers and practitioners in the field of
computational intelligence are confronted with real-world
problems, the question arises which method is the best to
apply. Nowadays, there are several, well established test
suites and well known artificial benchmark functions
available.
However, relevance and applicability of these methods to
real-world problems remains an open question in many
situations. Furthermore, the generalizability of these
methods cannot be taken for granted.
This paper describes a data-driven approach for the
generation of test instances, which is based on
real-world data. The test instance generation uses
data-preprocessing, feature extraction, modeling, and
parameterization. We apply this methodology on a classical
design of experiment real-world project and generate test
instances for benchmarking, e.g. design methods, surrogate
techniques, and optimization algorithms. While most
available results of methods applied on real-world
problems lack availability of the data for comparison,
our future goal is to create a toolbox covering multiple
data sets of real-world projects to provide a test
function generator to the research community.
T3 - CIplus - 6/2016
KW - Modeling
KW - Optimization
KW - Benchmarking
KW - Test Function
KW - Modelierung
KW - Optimierung
KW - Benchmarking
KW - Funktionstest
Y1 - 2016
U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:832-cos4-4326
ER -
TY - INPR
A1 - Zaefferer, Martin
A1 - Gaida, Daniel
A1 - Bartz-Beielstein, Thomas
T1 - Multi-fidelity Modeling and Optimization of Biogas Plants
N2 - An essential task for operation and planning of biogas plants is the optimization of substrate feed mixtures. Optimizing the monetary gain requires the determination of the exact amounts of maize, manure, grass silage, and other substrates. Accurate simulation models are mandatory for this optimization, because the underlying chemical processes are very slow. The simulation models themselves may be time-consuming to evaluate, hence we show how to use surrogate-model-based approaches to optimize biogas plants efficiently. In detail, a Kriging surrogate is employed. To improve model quality of this surrogate, we integrate cheaply available data into the optimization process. Doing so, Multi-fidelity modeling methods like Co-Kriging are employed. Furthermore, a two-layered modeling approach is employed to avoid deterioration of model quality due to discontinuities in the search space. At the same time, the cheaply available data is shown to be very useful for initialization of the employed optimization algorithms. Overall, we show how biogas plants can be efficiently modeled using data-driven methods, avoiding discontinuities as well as including cheaply available data. The application of the derived surrogate models to an optimization process is shown to be very difficult, yet successful for a lower problem dimension.
T3 - CIplus - 2/2014
KW - Biogas
KW - Simulation
KW - Modellierung
KW - Optimierung
KW - Kriging
KW - Biogas Plant
KW - Simulation
KW - Optimization
KW - Surrogate Models
KW - Multi-fidelity
KW - Co-Kriging
Y1 - 2014
U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:832-cos-644
ER -
TY - RPRT
A1 - Bartz-Beielstein, Thomas
T1 - Beyond Particular Problem Instances: How to Create Meaningful and Generalizable Results
N2 - Computational intelligence methods have gained importance in several real-world domains such as process optimization, system identification, data mining, or statistical quality control. Tools are missing, which determine the applicability of computational intelligence methods in these application domains in an objective manner. Statistics provide methods for comparing algorithms on certain data sets. In the past, several test suites were presented and considered as state of the art. However, there are several drawbacks of these test suites, namely: (i) problem instances are somehow artificial and have no direct link to real-world settings; (ii) since there is a fixed number of test instances, algorithms can be fitted or tuned to this specific and very limited set of test functions; (iii) statistical tools for comparisons of several algorithms on several test problem instances are relatively complex and not easily to analyze. We propose a methodology to overcome these difficulties. It is based on standard ideas from statistics: analysis of variance and its extension to mixed models. This paper combines essential ideas from two approaches: problem generation and statistical analysis of computer experiments.
T3 - CIplus - 3/2012
KW - Optimization
KW - Statistics
KW - Mixed Models
Y1 - 2012
U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hbz:832-cos-279
SN - 2194-2870
ER -