the BOXES METHODOLOGY : black box control of ill -defined systems.

Preface
	References
Acknowledgements
Donald Michie: A Personal Appreciation
Contents
1 Introduction
	1.1 Machine Intelligence
		1.1.1 Are Computers Intelligent?
		1.1.2 Can Computers Learn?
		1.1.3 The Robot and the Box
		1.1.4 Does the BOXES Algorithm Learn?
		1.1.5 Can Computers Think?
		1.1.6 Does the BOXES Algorithm Think?
	1.2 The Purpose of This Book
	1.3 A Road Map to Reading This Book
	1.4 Concluding Thoughts
	References
Part I Learning and Artificial Intelligence (AI)
2 The Game Metaphor
	2.1 Computers Can Be Programmed to Play Games
		2.1.1 Playing by Rules
	2.2 Reactionary Strategy Games
		2.2.1 Noughts and Crosses
		2.2.2 OXO Not Noughts and Crosses
	2.3 Incentives and Learning Velocity
	2.4 Design of a Noughts and Crosses Engine
		2.4.1 Overview
		2.4.2 Software Substructures
		2.4.3 Typical Results
	2.5 Chance and Trial and Error
		2.5.1 Chance and the Community Chest
		2.5.2 Learning with Guesswork
		2.5.3 Random Initialization
		2.5.4 Positional Move Strengths
	2.6 The Payoff Matrix
	2.7 The Signature Table
	2.8 Rewards and Penalties
	2.9 Failure-Driven Learning
	2.10 Concluding Thoughts
		2.10.1 Reversi (Othello®)
		2.10.2 The BOXES Method as a Game
	References
3 Introduction to BOXES
	3.1 Matchboxes
	3.2 Components of the BOXES Method
		3.2.1 Defining the Game Board
		3.2.2 Identifying Game Situations
		3.2.3 Selecting Game Piece Actions
		3.2.4 Real-Time Data Handling
		3.2.5 Detecting an End Game Situation
	3.3 Updating the Signature Table
		3.3.1 Overall Performance Data
		3.3.2 Desired Level of Achievement
		3.3.3 Individual Box Decision Data
	3.4 Overall Software Design
	3.5 Concluding Comments
	References
4 Dynamic Control as a Game
	4.1 Control of Dynamic Systems
		4.1.1 The Dynamic System Game Board
		4.1.2 State Variables and State Integers
		4.1.3 Creating a Unique System Integer
		4.1.4 Signature Table Control
		4.1.5 End of Game Action
	4.2 Actual Real-Time Data Collection
		4.2.1 Short Duration Mechanically Unstable Systems
		4.2.2 Continuous Systems with Sample Data
	4.3 Update Procedures
	4.4 Concluding Comments
	References
Part II The Trolley and Pole
5 Control of a Simulated Inverted Pendulum Using the BOXES Method
	5.1 Introduction
	5.2 The Trolley and Pole Model
		5.2.1 The Trolley and Pole Signature Table
		5.2.2 Systems Engineering
		5.2.3 An Overall Performance Metric
		5.2.4 The Importance of State Boundaries
		5.2.5 Computation of a Unique System Integer
	5.3 Simulation Software
		5.3.1 The Campaign Setup Phase
		5.3.2 The Individual Run Phase
	5.4 Simulation Results
		5.4.1 Typical Results
	5.5 Update of Statistical Databases
		5.5.1 Determination of Decision Strength
		5.5.2 Near-Neighbor Advisor Cells
	5.6 Conclusions
	References
6 The Liverpool Experiment
	6.1 Introduction to Reality
	6.2 The Liverpool Trolley and Pole Rig
		6.2.1 Practical Aspects of the Liverpool System
		6.2.2 Instrumentation and the State Variables
		6.2.3 Manual Auto-start
		6.2.4 Driving the Trolley
		6.2.5 The Microprocessor
		6.2.6 The BOXES Algorithm and the Real-Time Monitor
	6.3 Systems Engineering
		6.3.1 How Boundaries on Each State Variable Were Imposed
	6.4 Results from the Liverpool Rig
	6.5 Conclusions
	References
7 Solving the Auto-Start Dilemma
	7.1 Introduction to the Auto-Start Dilemma
	7.2 Random Restart Simulation Software
		7.2.1 Restart Software
		7.2.2 Random Initial State Integers
	7.3 Automated Catch-Up Restart Method
		7.3.1 Catch-Up Restart in Simulated Systems
		7.3.2 Catch-Up Restart in a Real Trolley and Pole Rig
		7.3.3 Catch-Up Method Conclusion
	7.4 Systems Engineering
	7.5 Manual Experiments
	7.6 Conclusions
	References
Part III Other BOXES Applications
8 Continuous System Control
	8.1 Continuous Control
	8.2 A Different Perspective on Failure
		8.2.1 Constructive Failure
		8.2.2 Limited Failure
		8.2.3 Creating a Learning Interlude
	8.3 Outcome-Based Performance Evaluation
		8.3.1 An Example Outcome-Based Approach
		8.3.2 Outcome-Based Assessment
	8.4 Training Continuous Automata
	8.5 BOXES Control of a Continuous System
		8.5.1 PID Control
		8.5.2 State Variable Control
		8.5.3 BOXES for Continuous Systems
	8.6 A BOXES Augmented Controller Results
		8.6.1 Outcome-Based Reward and Penalty
		8.6.2 Results Obtained for the Example
	8.7 Conclusions
	References
9 Other On/Off Control Case Studies
	9.1 On/Off Control
		9.1.1 Learning Algorithms
		9.1.2 Run Time Data
		9.1.3 Signature Table Update
		9.1.4 Application Summary
	9.2 Fedbatch Fermentation
		9.2.1 The Fedbatch Fermentation Process
		9.2.2 A Fedbatch Fermentation Model
		9.2.3 BOXES Control of a Fedbatch Fermentor
	9.3 A Municipal Incinerator with BOXES Control
		9.3.1 A Model of a Municipal Incinerator
		9.3.2 BOXES Control of the Municipal Incinerator
	9.4 Reversing a Tractor Trailer
		9.4.1 Model of the Tractor Trailer
		9.4.2 BOXES Control of Tractor Trailer Reversal
	9.5 Conclusions
	References
10 Two Nonlinear Applications
	10.1 Introduction
	10.2 Database Access Forecasting
		10.2.1 Application of BOXES to Disk Accessing
		10.2.2 Learning in the Adapted BOXES Algorithm
		10.2.3 Forecasting
		10.2.4 Simulation Results
		10.2.5 Conclusions Disk Accessing
	10.3 Stabilizing Lorentz Chaos
		10.3.1 Introduction
		10.3.2 BOXES and the Fractal Dimension
		10.3.3 Results for Lorenz Chaos Under BOXES Control
		10.3.4 Conclusions Lorentz Equations
	10.4 Conclusions
	References
Part IV Extending the Algorithm
11 Accelerated Learning
	11.1 Introduction
	11.2 Preset Fixed Signature Table Values
	11.3 Reduction of the Importance of Short Runs
	11.4 Collaboration Among Cells
		11.4.1 Playing Bridge
		11.4.2 Swarm and Ant Colony Systems
		11.4.3 Advisors in the BOXES Algorithm
		11.4.4 Locating Peer Advisor States
	11.5 Relocation of Boundaries
	11.6 Conclusions
	References
12 Two Advising Paradigms
	12.1 Introduction
		12.1.1 Decision Strengths of Cells
		12.1.2 Identification and Ranking of Advisor Cells
		12.1.3 Advisor Strength Criteria
	12.2 Advising Schemas
		12.2.1 Advising by Voting
		12.2.2 Advising Using Cell Strengths
		12.2.3 Delayed Advising
	12.3 Advisor Accountability
	12.4 Conclusions
	References
13 Evolutionary Studies Research
	13.1 Introduction
	13.2 State Boundary Experiments
		13.2.1 Variation in the Number and Size of State Zones
		13.2.2 Preliminary Conclusions
	13.3 An Evolutionary Paradigm
		13.3.1 Types of Zones
		13.3.2 An Evolutionary Method
		13.3.3 Interpreting Signature Table Statistics
		13.3.4 An Evolutionary Algorithm
		13.3.5 Example Results
	13.4 Conclusions
	References
Part V Further Thoughts
14 A Priori Knowledge
	14.1 What are Black Box Systems?
		14.1.1 An Intelligent Pacemaker
		14.1.2 Controlling a Steel Rolling Mill
		14.1.3 Why Use the Cart-and-Pole Exemplar?
	14.2 What does the BOXES Paradigm Need to Know?
		14.2.1 The Physical System to be Controlled
		14.2.2 The Division of the State Variables into Zones
	14.3 Other BOXES Configuration Data Items
		14.3.1 The Signature Table
		14.3.2 Evaluating System Merit
		14.3.3 In-Run Data Collection
		14.3.4 The Cell Usage Database
		14.3.5 Learning Parameters
	14.4 Fixed Cell Strategies
		14.4.1 Create Non-changeable States
		14.4.2 Strong Cell Freezing
		14.4.3 Can Fixed Cells Participate in Evolutionary Studies?
	14.5 Conclusions
	References
15 Detecting and Handling Jitter
	15.1 Introduction
	15.2 Why Does Jittering Occur?
	15.3 How Jitter Affects Merit
		15.3.1 A Numerical Illustration
	15.4 How Jittering Corrupts Individual Cell Data
	15.5 Detection of Jittering in a BOXES System
	15.6 Possible Strategies for Jitter Remediation
		15.6.1 Executive Internal Action
		15.6.2 Jitter Proof Software
	15.7 Conclusions
	References
16 Handling Untrained Data
	16.1 Glossary of Computational Terms
	16.2 A Mathematical BOXES Test Engine
		16.2.1 Scenario 1: Linear Increase or Decrease
		16.2.2 Scenario 2: Linear Saw Tooth
		16.2.3 Scenario 3: Complex or Random Pattern
	16.3 Forgetfulness Observations Using the BOXES Test Engine
		16.3.1 Aging Global Use (GU)
		16.3.2 Aging Global Life (GL)
		16.3.3 System Merit just Aging the Global Life (GL)
		16.3.4 System Merit for a More Complex Scenario
		16.3.5 Effect of Differing Values of Δk
		16.3.6 Summary of δk as a Learning Agent
	16.4 An Alternate Aging Strategy Using a FIFO Stack Architecture
		16.4.1 The FIFO Data Structure
		16.4.2 Application of the FIFO Stack to BOXES Merit
		16.4.3 Connection of the Test Engine to the FIFO Stack
		16.4.4 Merit Values Using the FIFO Stack
	16.5 Conclusions
	References
Part VI Conclusions
17 Summary and Conclusions
	17.1 Some Philosophical Commentary
	17.2 Bloom’s Taxonomy
	17.3 Introduction and Part I: Learning and Artificial Intelligence
		17.3.1 Chapter 1: Introduction
		17.3.2 Chapter 2: The Game Metaphor
		17.3.3 Chapter 3: Introduction to BOXES
		17.3.4 Chapter 4: Dynamic Control as a Game
	17.4 Part II: The Trolley and Pole
		17.4.1 Chapter 5: Control of an Inverted Pendulum Using BOXES
		17.4.2 Chapter 6: The Liverpool Experiment
		17.4.3 Chapter 7: Solving the Auto-start Dilemma
	17.5 Part III: Other BOXES Applications
		17.5.1 Chapter 8: Continuous System Control
		17.5.2 Chapter 9: Other On/Off Control Case Studies
		17.5.3 Chapter 10: Two Nonlinear Applications
	17.6 Part IV: Improving the Algorithm
		17.6.1 Chapter 11: Accelerated Learning
		17.6.2 Chapter 12: Two Advising Paradigms
		17.6.3 Chapter 13: Evolutionary Studies Research
	17.7 Part V: Further Thoughts
		17.7.1 Chapter 14: A Priori Knowledge
		17.7.2 Chapter 15: Detecting and Handling Jitter
		17.7.3 Chapter 16: Handling Untrained Data
	17.8 Modifications to Standard BOXES Software
	17.9 Research Questions for Future Study
		17.9.1 Is There an Optimal Number of States?
		17.9.2 Is There a Generic Merit Formula?
		17.9.3 Is There a Generic Cell Strength Formula?
	17.10 Conclusions
		17.10.1 Some More Final Thoughts
	References
Appendix A Glossary of Terms and Abbreviations
Appendix B BOXES Software Notes
B.1 QB64: A Better Quickbasic
B.2 Essentials of Simulation Software
B.2.1 Housekeeping
B.2.2 Auto-start
B.2.3 Reset Inner Loop Variables
B.2.4 Inner Loop Control
B.2.5 Normalization of State Variables
B.2.6 Calculation of System Integers
B.2.7 Accessing the Signature Table
B.2.8 Saving In-run Data
B.2.9 Attaching the BOXES Controller
B.2.10 State Equations of the System Model
B.2.11 Numerical Integration
B.2.12 Saving Global Data at the End of a Run
B.2.13 Detecting and Avoiding Jitter
B.3 Real-Time Software
B.4 Conclusions
Appendix C BOXES Publications, Lectures, and Presentations by the Author
Author Biography
Index