Automated Design of Machine Learning and Search Algorithms

Foreword
Preface
Acknowledgements
Contents
Contributors
Acronyms
1 Recent Developments of Automated Machine Learning and Search Techniques
	1.1 Introduction
	1.2 Automated Algorithm Design and Machine Learning
	1.3 Challenges in Automated Design of Algorithms  and Machine Learning
	1.4 Conclusions
	References
2 Automated Machine Learning—A Brief Review at the End of the Early Years
	2.1 Introduction
	2.2 Fundamentals of AutoML
		2.2.1 Supervised Learning
		2.2.2 Notions of AutoML
		2.2.3 Disentangling AutoML Methods
	2.3 AutoML Methodologies
		2.3.1 First Wave: 2006–2010
		2.3.2 Second Wave: 2011–2016
		2.3.3 Third Wave: 2017 and On
	2.4 AutoML Challenges
	2.5 Open Issues and Research Opportunities
	2.6 Conclusions
	References
3 A General Model for Automated Algorithm Design
	3.1 Introduction
	3.2 Automated Algorithm Design
		3.2.1 Automated Algorithm Configuration
		3.2.2 Automated Algorithm Selection
		3.2.3 Automated Algorithm Composition
	3.3 The General Combinatorial Optimisation Problem
	3.4 Search Algorithms Defined with the GCOP Model
	3.5 Challenges in Automated Algorithm Design with the GCOP
	3.6 Conclusions
	References
4 Rigorous Performance Analysis  of Hyper-heuristics
	4.1 Introduction
	4.2 Selection Hyper-heuristics
		4.2.1 Heuristic Selection Methods
		4.2.2 Move Acceptance Operators
	4.3 Hyper-heuristics Are Necessary
	4.4 Learning is Necessary
	4.5 Hyper-heuristics Can Achieve Optimal Performance
		4.5.1 The RandomGradient Hyper-heuristic has Optimal Performance for LeadingOnes
		4.5.2 A Reinforcement Learning Hyper-heuristic has Optimal Performance for OneMax
	4.6 Automatically Adapting the Learning Period is Necessary
	4.7 Switching Between Move Acceptance Operators is Necessary
		4.7.1 Local Optima with Small Basins of Attraction
		4.7.2 Local Optima with Large Basins of Attraction
	4.8 Conclusion
	References
5 AutoMoDe: A Modular Approach  to the Automatic Off-Line Design  and Fine-Tuning of Control Software for Robot Swarms
	5.1 An Introduction to Swarm Robotics and Its Design Problem
	5.2 From Neuro-Evolutionary Robotics to AutoMoDe
	5.3 The Specializations of AutoMoDe
	5.4 Further Corroboration of Our Working Hypothesis
	5.5 Discussion on Future Research Directions
	References
6 A Cross-Domain Method for Generation of Constructive and Perturbative Heuristics
	6.1 Introduction
	6.2 Related Work
	6.3 Evaluation Domains and Their Representation as Graphs
	6.4 Using Grammatical Evolution to Evolve Low-Level Heuristics
		6.4.1 Evolving Constructive Heuristics
		6.4.2 Evolving Perturbative Heuristics
	6.5 Methodology
		6.5.1 Training Phase
		6.5.2 Test Phase
	6.6 Results
		6.6.1 Constructive Heuristics
	6.7 Results: Perturbative Heuristics
	6.8 Discussion
	6.9 Conclusion
	References
7 Hyper-heuristics: Autonomous Problem Solvers
	7.1 Introduction
	7.2 Motivation and Claims
	7.3 Background
		7.3.1 Selection Hyper-heuristics
		7.3.2 Generation Hyper-heuristics
	7.4 Challenges, Research Gaps, and Poor Practices
	7.5 Conclusion
	References
8 Toward Real-Time Federated Evolutionary Neural Architecture Search
	8.1 Introduction
	8.2 Computationally Efficient Neural Architecture Search
	8.3 Evolutionary Optimization of One-Shot Neural Architecture Search
		8.3.1 Training the Supernet
		8.3.2 Evolutionary Architecture Search
		8.3.3 One-Shot NAS in Federated Learning
	8.4 Real-Time Federated Evolutionary Neural Architecture Search
		8.4.1 Real-Time Evolutionary Neural Architecture Search
		8.4.2 Real-Time Federated Evolutionary Neural Architecture Search
	8.5 Steady State Evolutionary Federated Neural Architecture Search
	8.6 Illustrative Empirical Results
		8.6.1 Experimental Settings
		8.6.2 Dataset
		8.6.3 Real-Time Performances Between NSGA-II and Steady-State NSGA-II
		8.6.4 Real-Time Performances on Different Datasets
	8.7 Conclusion
	References
9 Knowledge Transfer in Genetic Programming Hyper-heuristics
	9.1 Introduction
	9.2 Background
		9.2.1 Genetic Programming Hyper-heuristic
		9.2.2 Transfer Learning and Optimisation
	9.3 Knowledge Transfer in GPHH for UCARP
		9.3.1 Existing GP Transfer Methods
		9.3.2 New GPHH with Biased Subtree Transfer
		9.3.3 New GPHH with Feature Importance Transfer
	9.4 Experimental Studies
		9.4.1 Transfer Scenarios
		9.4.2 Effectiveness of Existing GP Transfer Methods
		9.4.3 Effectiveness of GPHH with Biased Subtree Transfer
		9.4.4 Effectiveness of GPHH with Feature Importance Transfer
		9.4.5 Statistical Significance Test Results
	9.5 Conclusions and Future Work
	References
10 Automated Design of Classification Algorithms
	10.1 Introduction
	10.2 Design Decisions
	10.3 Genetic Algorithm (autoGA)
		10.3.1 Initial Population Generation
		10.3.2 Fitness Evaluation
		10.3.3 Selection
		10.3.4 Regeneration
		10.3.5 GA Parameter Values
	10.4 Grammatical Evolution (autoGE)
		10.4.1 Initial Population Generation
		10.4.2 Fitness Evaluation
		10.4.3 Selection
		10.4.4 Regeneration
		10.4.5 GE Parameter Values
	10.5 Experimental Setup
		10.5.1 Experiments
		10.5.2 Datasets
		10.5.3 Technical Specifications
	10.6 Results Discussion
	10.7 Conclusion
	References
11 Automated Design (AutoDes): Current Trends and Future Research Directions
	11.1 Introduction
		11.1.1 Reusability in AutoDes
		11.1.2 Explainable AutoDes
		11.1.3 Computational Cost of Algorithm Automation
		11.1.4 Theoretical Aspects
		11.1.5 AutoDes Standardization
		11.1.6 Semi-automated Design
	Reference