Kramer - A Brief Introduction to Continuous Evolutionary Optimization

The Author(s) 2014

Oliver Kramer 1

(1)

Abstract

Many optimization problems that have to be solved in practice are black box problems. Often, not much is known about an optimization problem except the information one can get via function evaluations. Neither derivatives nor constraints are known. In the worst case, nothing is even known about the characteristics of the fitness function, e.g., whether it is uni- or multimodal.

Many optimization problems that have to be solved in practice are black box problems. Often, not much is known about an optimization problem except the information one can get via function evaluations. Neither derivatives nor constraints are known. In the worst case, nothing is even known about the characteristics of the fitness function, e.g., whether it is uni- or multimodal. This scenario affords the application of specialized optimization strategies often called direct search methods. Evolutionary algorithms that mimic the biological notion of evolution and employ stochastic components to search in the solution space have grown to strong optimization methods. Evolutionary methods that are able to efficiently search in large optimization scenarios and learn from observed patterns in data mining scenarios have found broad acceptance in many disciplines, e.g., civil and electrical engineering. The methods have been influenced from various disciplines: robotics, statistics, computer science, engineering, and the cognitive sciences. This might be the reason for the large variety of techniques that have been developed in the last decades. The employment of computer simulations has become outstandingly successful in engineering within the last years. This development includes the application of optimization and learning techniques in the design and prototype process. Simulations allow the study of prototype characteristics before the product has actually been manufactured. Such a process allows an entirely computed-based optimization of the whole prototype or of its parts and can result in significant speedups and savings of material and money.

Learning and optimization are strongly related to each other. In optimization, one seeks for optimal parameters of a function or system w.r.t. a defined objective. In machine learning, one seeks for an optimal functional model that allows to describe relations between observations. Pattern recognition and machine learning problems also involve solving optimization problems. Many different optimization approaches are employed, from heuristics with stochastic components to exact convex methods.

The goal of this book is to give a brief introduction to latest heuristics in evolutionary optimization for continuous solution spaces. The beginning of the work gives a short introduction to the main problem classes of interest: optimization, super-, and unsupervised learning, see Fig.. Optimization is the problem of finding optimal parameters for arbitrary models and functions. Supervised learning is about finding functional models that best model observations with given label information. Unsupervised learning is about learning functional models only based on the structure of the data itself, i.e., without label information. The following three paragraphs give a short introduction to the three problem classes.