Building Ensembles of Surrogate Models by Optimal Convex Combination
- When using machine learning techniques for learning a function approximation from given data it is often a difficult task to select the right modeling technique. In many real-world settings is no preliminary knowledge about the objective function available. Then it might be beneficial if the algorithm could learn all models by itself and select the model that suits best to the problem. This approach is known as automated model selection. In this work we propose a generalization of this approach. It combines the predictions of several into one more accurate ensemble surrogate model. This approach is studied in a fundamental way, by first evaluating minimalistic ensembles of only two surrogate models in detail and then proceeding to ensembles with three and more surrogate models. The results show to what extent combinations of models can perform better than single surrogate models and provides insights into the scalability and robustness of the approach. The study focuses on multi-modal functions topologies, which are important in surrogate-assisted global optimization.
Author: | Martina Friese, Thomas Bartz-BeielsteinGND, Michael Emmerich |
---|---|
URN: | urn:nbn:de:hbz:832-cos4-3480 |
Series (Serial Number): | CIplus (4/2016) |
Document Type: | Report |
Language: | English |
Year of Completion: | 2016 |
Release Date: | 2016/04/01 |
Tag: | Automated Learning; Ensemble Methods; Function Approximation; Model Selection; Surrogate Models |
GND Keyword: | Globale Optimierung; Maschinelles Lernen |
Page Number: | 19 |
Institutes and Central Facilities: | Fakultät für Informatik und Ingenieurwissenschaften (F10) |
CCS-Classification: | J. Computer Applications / J.0 GENERAL |
Dewey Decimal Classification: | 000 Allgemeines, Informatik, Informationswissenschaft / 004 Informatik |
JEL-Classification: | C Mathematical and Quantitative Methods / C6 Mathematical Methods and Programming / C61 Optimization Techniques; Programming Models; Dynamic Analysis |
Open Access: | Open Access |
Licence (German): | Creative Commons - Namensnennung, Nicht kommerziell, Keine Bearbeitung |