Industry is faced with solving complex optimization problems on a day to day basis in different domains including transportation, data mining, computer vision, computer security, robotics and scheduling amongst others. Machine learning and search algorithms play an important role in solving such problems. Given the growing complexity of optimization problems, the design of effective algorithms to solve complex optimization problems has become more challenging and time consuming. Hence, there is a demand, especially from industry and business, to automate the design process, thereby to remove the heavy reliance on human experts and to reduce the man hours involved in designing machine learning and search algorithms for solving these problems.
The aim is to develop out of the box tools that can be used by practitioners and researchers to make design decisions that are made by human experts when applying machine learning and search algorithms to complex optimization problems. Examples of design decisions include determining parameter and hyperparameter values, operator selection, choosing a good data representation, generating construction heuristics, creating new operators, identifying the algorithm or combination of algorithms to solve the problem at hand.
Machine learning and search algorithms require parameters, e.g. genetic operator probabilities for genetic algorithms, and hyperparameters, e.g. learning rate in deep learning, to be tuned, with the most appropriate parameter values being problem dependent. There are usually many options for hyper-parameters and automating the selection of these values reduces the time and human expertise required for this. Automating parameter selection allows for the parameter values to be configured and adapted dynamically during execution of the algorithm, resulting in parameter-less algorithms. For some algorithms, e.g. evolutionary algorithms, it is also necessary to select which operators to use, automating this process allows for different operators to be applied at different points in the algorithm. In machine learning, the way in which data are represented has a crucial impact on the algorithm's performance.
Automated design includes the generation of new constructs used by machine learning and search algorithms. For example, in solving optimization problems certain algorithms require an initial solution or set of solutions which the algorithm optimizes further. Construction heuristics, usually derived based on human intuition, are used to create these solutions. There has been a number of research initiatives into automating the derivation of these heuristics which have resulted in the creation of heuristics that perform better than human derived heuristics. The automated creation of operators used by search algorithms to improve initial solutions has also proven to be promising. Similarly, in the domain of machine learning there have been efforts to generate machine learning workplans and architectures, e.g. deep learning is capable of inferring rich data representations from raw sensor signals in so-called end-to-end learning, Bayesian modeling enables a structured inference along meta-parameters.
A further example of a design decision is identifying the appropriate algorithm to solve the problem at hand, e.g. which classification algorithm should be used, if a genetic algorithm or simulated annealing is more appropriate. In some instances it may also be necessary to decide which algorithms to combine and how these can be hybridized to solve the problem.
In some applications it may be sufficient for techniques used to automate these design decisions to operate as black-boxes, however in other instances users may require an explanation of how these decisions were arrived at before they can accept them as reliable. Thus, it is necessary to develop techniques that can explain the design decision reached.
Research into automating such design decisions has led to the development of areas such as self*-search and auto-ML, with various approaches including self-adaption, self-configuration, algorithm portfolios, adaptive metaheuristics, multilevel meta-heuristics and hyper-heuristics successfully being used for automation. A wide range of techniques, including for example those in machine learning, have been employed to effectively and automatically configure or design adaptive machine learning and search algorithms, utilizing either online or offline knowledge extracted. Explainable machine learning is a current hot topic, aimed at improving machine learning techniques to explain the decisions they have arrived at. The aim of this special issue is to collect and examine recent developments in the field, thus to promote and identify future directions and challenges, and demonstrate how these may be addressed.
The topics covered include, but are not limited to, the following:
Automated design of operators
Automated hybridization of algorithms
Automated operator selection
Automated parameter configuration and adaptation
Automated hyperparameter selection
Automated feature selection
Automated model selection
Automated heuristic generation
Automated operator creation
Automated machine learning workplan and architecture generation