Shepherding UxVs for Human-Swarm Teaming: An Artificial Intelligence Approach to Unmanned X Vehicles (Unmanned System Technologies)

Foreword
Preface
Acknowledgements
Contents
Contributors
Generalised Shepherding Notations
1 Smart Shepherding: Towards Transparent Artificial Intelligence Enabled Human-Swarm Teams
	1.1 From Swarm Intelligence to Shepherding
	1.2 Shepherding
	1.3 The Practical Significance of Shepherding
	1.4 Reactive vs Cognitive Shepherds and Sheepdogs
	1.5 Swarm Ontology for Transparent Artificial Shepherding
	1.6 Artificial Intelligence Architecture for Shepherds and Sheepdogs
		1.6.1 Shepherds and Sheepdogs Autonomy Architecture
		1.6.2 Shepherds and Sheepdogs Contextual Awareness Architecture
		1.6.3 Smart Shepherds and Sheepdogs Overall Architecture
	1.7 Conclusion
	References
Part I Shepherding Simulation
	2 Shepherding Autonomous Goal-Focused Swarms in Unknown Environments Using Hilbert Space-Filling Paths
		2.1 Introduction
		2.2 Background Research
		2.3 Methodology
			2.3.1 Simulation Setup
			2.3.2 Force Modulation
			2.3.3 Path Planning
			2.3.4 Hilbert Space-Filling Curves
		2.4 Results and Discussion
			2.4.1 Force Weights
			2.4.2 Number of Goals
		2.5 Conclusion
		References
	3 Simulating Single and Multiple Sheepdogs Guidance of a Sheep Swarm
		3.1 Introduction
		3.2 Experimental and Computational Details
			3.2.1 Problem Formulation
			3.2.2 Sheep Agent Model
			3.2.3 Shepherd Agent Model
			3.2.4 Swarm Guidance Algorithm Design
			3.2.5 Experimental Design
		3.3 Simulation Results
			3.3.1 Herding with a Single Shepherd
			3.3.2 Herding with a Multi-Shepherd Swarm
			3.3.3 Herding with a Multi-Shepherd Swarm Plus Formation
			3.3.4 Analysis of the Shepherding Task as a Function of Guidance Scheme
		3.4 Conclusions
		References
	4 The Influence of Stall Distance on Effective Shepherdingof a Swarm
		4.1 Introduction
		4.2 Background
			4.2.1 Driving Interactions
			4.2.2 Collecting Interactions
		4.3 Methodology
		4.4 Experimental Design
			4.4.1 Genetic Algorithm Exploration of Stall Distance
			4.4.2 Systematic Analysis of Stall Distance
		4.5 Results
			4.5.1 Results of Genetic Algorithm Exploration of Stall Distance
			4.5.2 Systematic Analysis of Stall Distance
				4.5.2.1 Success Rates for Herding
				4.5.2.2 Herding Time Steps and Distances
		4.6 Conclusion
		References
Part II Learning and Optimisation for Shepherding
	5 Mission Planning for Shepherding a Swarm of Uninhabited Aerial Vehicles
		5.1 Introduction
		5.2 Overview of Mission Planning for Shepherding UAV Swarm
		5.3 Task Planning
			5.3.1 Task Decomposition
			5.3.2 Task Assignment
			5.3.3 Algorithms for Task Planning
		5.4 Motion Planning
			5.4.1 Path Planning
			5.4.2 Trajectory Planning
			5.4.3 Algorithms for Motion Planning
		5.5 Mission Planning
		5.6 Conclusion and Discussion
		References
	6 Towards Ontology-Guided Learning for Shepherding
		6.1 Introduction
		6.2 Learning Shepherding Systems
		6.3 Prior Knowledge in Learning Systems
		6.4 Hybrid Learning
			6.4.1 Guided Learning Systems
		6.5 Ontology Guided Shepherding
		6.6 Future Work
		6.7 Conclusion
		References
	7 Activity Recognition for Shepherding
		7.1 Introduction
			7.1.1 Problem Frame
			7.1.2 Motivation
		7.2 Activity Recognition
			7.2.1 Elements of Activity Recognition
				7.2.1.1 Agent
				7.2.1.2 Agent Types
				7.2.1.3 Action and Activity
				7.2.1.4 Defining Activity Recognition
			7.2.2 Problem Components
				7.2.2.1 Agent Design
			7.2.3 Approaches
				7.2.3.1 Data-Driven Approaches
				7.2.3.2 Knowledge-Driven Approaches
				7.2.3.3 Hybrid Approaches
		7.3 Shepherding
			7.3.1 Open Challenges
				7.3.1.1 Activity Verification
				7.3.1.2 Adversarial Activity Recognition
				7.3.1.3 Context-Aware Activity Recognition
				7.3.1.4 Cross-Domain (Multi-Modality) Activity Recognition
				7.3.1.5 Dynamic Activity Recognition
				7.3.1.6 Inter- and Intra-Activity Delay and Task Selection
			7.3.2 Solving the Activity Recognition for Shepherding Problem
				7.3.2.1 Shepherding Taxonomy
				7.3.2.2 Framework
				7.3.2.3 Central Challenge
		7.4 Formulating Activity Recognition for Shepherding
			7.4.1 Describing Shepherding Behaviours
			7.4.2 Classifying Behaviour Through Spatial Data
				7.4.2.1 Methodology
				7.4.2.2 Analysis
		7.5 Conclusion
		References
	8 Stable Belief Estimation in Shepherd-Assisted Swarm Collective Decision Making
		8.1 Introduction
		8.2 Related Work
		8.3 Problem Definition and Assumptions
		8.4 Shepherd-Assisted Algorithm
			8.4.1 Swarm Members' Behaviour
			8.4.2 Shepherd's Behaviour
		8.5 Experimental Results
		8.6 Discussion
		8.7 Conclusion and Future Directions
		References
Part III Sky Shepherding
	9 Sky Shepherds: A Tale of a UAV and Sheep
		9.1 Introduction
		9.2 Shepherding Models
		9.3 Flock Dynamics
		9.4 Autonomous Sky Shepherd Methodology
		9.5 Concluding Comments
		References
	10 Apprenticeship Bootstrapping Reinforcement Learning for Sky Shepherding of a Ground Swarm in Gazebo
		10.1 Introduction
		10.2 Aerial/Sky Shepherding of Ground Swarm
			10.2.1 Unmanned Air–Ground Vehicles Coordination
			10.2.2 A Brief Review of Coordination in Unmanned Air–Ground Vehicles
			10.2.3 Autonomous Aerial/Sky Shepherding in Air–Ground Coordination
		10.3 The Aerial/Sky Shepherding Task
			10.3.1 Description of the Aerial/Sky Shepherding Task
			10.3.2 The Aerial/Sky Shepherding Task as a Multi-Agent System
		10.4 Learning Approaches
			10.4.1 Reinforcement Learning
				10.4.1.1 Markov Decision Process
				10.4.1.2 Q-Learning
				10.4.1.3 Deep Q-Network
				10.4.1.4 Multi-Agent Reinforcement Learning
			10.4.2 Apprenticeship Learning
				10.4.2.1 Supervised Learning
				10.4.2.2 Inverse Reinforcement Learning
				10.4.2.3 Hybrid Methods
				10.4.2.4 Multi-Agent Apprenticeship Learning/Imitation Learning
			10.4.3 Apprenticeship Bootstrapping
				10.4.3.1 Clarifying Tasks
				10.4.3.2 Learning Tasks
				10.4.3.3 Apprenticeship Bootstrapping Approach
		10.5 Initial Results
			10.5.1 Proposed Methodology
				10.5.1.1 Evaluation Metrics
			10.5.2 Experimental Design
				10.5.2.1 Demonstration Interface
				10.5.2.2 Actions and States Space
				10.5.2.3 Experimental Setups
			10.5.3 Results and Discussion
				10.5.3.1 Training
				10.5.3.2 Testing
		10.6 Conclusions and Open Issues
		References
	11 Logical Shepherd Assisting Air Traffic Controllers for Swarm UAV Traffic Control Systems
		11.1 Introduction
		11.2 Background
			11.2.1 Advantages of Shepherding in Air Traffic Control
			11.2.2 Challenges for Shepherding in Air Traffic Control
		11.3 Asynchronous Shepherding
		11.4 The Digital Twin
			11.4.1 ATOMS
			11.4.2 UTC Interface
			11.4.3 Applying the Asynchronous Shepherding Rules
			11.4.4 Asynchronous Shepherding Algorithm
			11.4.5 Issues for Future Research
		11.5 Conclusion
		References
Part IV Human-Shepherding Integration
	12 Transparent Shepherding: A Rule-Based Learning Shepherd for Human Swarm Teaming
		12.1 Introduction
		12.2 Challenges for Efficient Human Swarm Teaming
		12.3 Fundamentals of Rule-Based Artificial Intelligence
			12.3.1 Structure
			12.3.2 Representation of Knowledge: Rules
			12.3.3 The Inference Mechanism
		12.4 Learning Classifier Systems
			12.4.1 Learning Classifier System Components
		12.5 Learning Classifier Systems for Human Swarm Teaming
			12.5.1 Transparency: Rules as Justifiers
			12.5.2 Flexibility
			12.5.3 Multi-Agent Coordination
		12.6 Learning Classifier System Model for Shepherding
			12.6.1 Sheep-Dog Herding Problem
			12.6.2 XCS Classifier Representation
			12.6.3 Experimental Setup
			12.6.4 Results
		12.7 Summary and Future Work
		References
	13 Human Performance Operating Picture for Shepherding a Swarm of Autonomous Vehicles
		13.1 Introduction
		13.2 Human Performance
		13.3 Performance Measures in Human-Swarm Teams
			13.3.1 Task- and System-Related Measures
			13.3.2 Human-Related Measures
			13.3.3 Workload
			13.3.4 Situation Awareness
			13.3.5 Trust
		13.4 Human Performance Operating Picture for Swarm Shepherding
			13.4.1 Design Considerations
			13.4.2 Elements of H-FOP Design
			13.4.3 Shepherding a Group of Robots for Effective Human-Swarm Teaming
		13.5 Conclusions
		References
Index