Volltext-Downloads (blau) und Frontdoor-Views (grau)

Benchmarking in Optimization: Best Practice and Open Issues

  • This survey compiles ideas and recommendations from more than a dozen researchers with different backgrounds and from different institutes around the world. Promoting best practice in benchmarking is its main goal. The article discusses eight essential topics in benchmarking: clearly stated goals, well- specified problems, suitable algorithms, adequate performance measures, thoughtful analysis, effective and efficient designs, comprehensible presentations, and guaranteed reproducibility. The final goal is to provide well-accepted guidelines (rules) that might be useful for authors and reviewers. As benchmarking in optimization is an active and evolving field of research this manuscript is meant to co-evolve over time by means of periodic updates.

Download full text files

Export metadata

Additional Services

Search Google Scholar


Author:Thomas Bartz-Beielstein, Carola Doerr, Jakob Bossek, Sowmya Chandrasekaran, Tome Eftimov, Andreas Fischbach, Pascal Kerschke, Manuel Lopez-Ibanez, Katherine M. Malan, Jason H. Moore, Boris Naujoks, Patryk Orzechowski, Vanessa Volz, Markus Wagner, Thomas Weise
Series (Serial Number):CIplus (2/2020)
Document Type:Working Paper
Release Date:2020/07/16
Tag:Benchmarking; Neural and Evolutionary Computing; Optimization; Performance
GND Keyword:Optimierung; Benchmarking
Page Number:58
Institutes and Central Facilities:Fakultät für Informatik und Ingenieurwissenschaften (F10)
CCS-Classification:G. Mathematics of Computing / G.1 NUMERICAL ANALYSIS
I. Computing Methodologies / I.2 ARTIFICIAL INTELLIGENCE
Dewey Decimal Classification:500 Naturwissenschaften und Mathematik / 510 Mathematik
JEL-Classification:C Mathematical and Quantitative Methods / C6 Mathematical Methods and Programming
C Mathematical and Quantitative Methods / C9 Design of Experiments
Open Access:Open Access
Licence (German):License LogoCreative Commons - CC BY-NC-ND - Namensnennung - Nicht kommerziell - Keine Bearbeitungen 4.0 International