Artificial Intelligence Principles and Practice

Preface
	1.1 Using This Book
	1.2 Programming
	1.3 About the Author
Contents
I: Introducing Artificial Intelligence
	1: The Pre-History of Artificial Intelligence
		1.1 Mary Shelley, Frankenstein, and Prometheus
		1.2 The Age of Rationalism
		1.3 The Empiricist Tradition
		1.4 Immanuel Kant: Bridging the Rationalist/Empiricist Viewpoints
		1.5 The Reverend Thomas Bayes
		1.6 The Mathematical Foundations for Artificial Intelligence
		1.7 American Pragmatism
		1.8 The Turing Test: Can a Machine Be “Intelligent”?
		1.9 The 1956 Dartmouth Summer Workshop
		1.10 Summary
		1.11 Exercises
	2: Computing, Representations, and Definitions of Artificial Intelligence
		2.1 Artificial Intelligence: Attempting a Definition
		2.2 Computer-Based Representations of the World
		2.3 The General Themes of Current AI Practice
			2.3.1 Symbol-Based AI
			2.3.2 Neural Networks
			2.3.3 Genetic and Emergent AI
			2.3.4 Probabilistic Models
		2.4 Summary of Part I and an Introduction to Part II
		2.5 Exercises
II: Symbol-Based AI: Foundations
	3: The State Space, Finite State Machines, and Artificial Life
		3.1 Graph Theory: The Origins of the State Space Representation
		3.2 The State Space Representation
		3.3 The Finite State Machine
		3.4 Artificial Life: The Emergence of Complexity
			3.4.1 Artificial Life: Examples
			3.4.2 Extensions of A-Life
		3.5 Summary
		3.6 Exercises
	4: Searching the State Space
		4.1 Data-Driven and Goal-Driven Search
		4.2 Implementing Graph Search with the Backtrack Algorithm
		4.3 Breadth-First and Depth-First Search
			4.3.1 The Breadth-First Algorithm
			4.3.2 The Depth-First Search Algorithm
			4.3.3 Depth-First Search with Iterative Deepening
		4.4 Extending Search Strategies to and/or Graphs
		4.5 Summary
		4.6 Exercises
	5: Heuristic Search
		5.1 An Introduction to Heuristics
		5.2 Hill-Climbing
		5.3 Best-First Search
			5.3.1 The Best_First_Search Algorithm
			5.3.2 Adding a Depth Measure to Best-First Search
		5.4 Genetic Algorithms: Evaluating Multiple “Best” States
		5.5 In Summary
		5.6 Exercises
	6: Heuristics: 2-Person Games and Theoretical Constraints
		6.1 The Minimax Procedure on Exhaustively Searchable Graphs
		6.2 Using Minimax to a Fixed Ply Depth
		6.3 The Alpha-Beta Procedure
		6.4 Multi-Person Games, Admissibility, Monotonicity, and Informedness
			6.4.1 Using Heuristics, the Zero-Sum Hypothesis, and Multi-Person Games
			6.4.2 Admissibility Measures
			6.4.3 Monotonicity
			6.4.4 When One Heuristic Is “Better” Than Another: Informedness
		6.5 Heuristics and Complexity
		6.6 Summary
		6.7 Exercises
III: The Propositional and Predicate Calculi and Resolution-Based Reasoning
	7: Introduction to the Propositional and Predicate Calculi
		7.1 The Propositional Calculus
			7.1.1 Symbols and Sentences
			7.1.2 The Semantics of Propositional Calculus
		7.2 The Predicate Calculus
			7.2.1 The Syntax of Predicates and Sentences
			7.2.2 A Semantics of Predicate Calculus
			7.2.3 A Blocks World Example of Semantic Meaning
		7.3 Summary
		7.4 Exercises
	8: The Predicate Calculus and Unification
		8.1 Using Reasoning Rules to Produce New Predicate Expressions
		8.2 The Unification Algorithm
			8.2.1 Unification: An Example
			8.2.2 Unification Application: A Logic-Based Financial Advisor
			8.2.3 The Financial Advisor: Using Heuristics
		8.3 Summary
		8.4 Exercises
	9: Resolution: Reasoning with the Propositional and Predicate Calculi
		9.1 Introduction to Resolution
		9.2 Resolution Refutation Systems
			9.2.1 Producing the Clause Form for Resolution Refutations
			9.2.2 The Binary Resolution Proof Procedure
		9.3 Strategies and Simplification Techniques for Resolution
			9.3.1 The Breadth-First Strategy
			9.3.2 The Set of Support Strategy
			9.3.3 The Unit Preference Strategy
			9.3.4 The Linear Input Form Strategy
			9.3.5 Other Strategies and Techniques
		9.4 Extracting Answers from Resolution Refutations
		9.5 Logic Programming and Prolog
			9.5.1 Introduction
			9.5.2 Logic Foundations for Prolog
			9.5.3 Prolog and Automated Reasoning
		9.6 Summary
		9.7 Exercises
IV: Advanced Applications of Symbol-Based AI
	10: The Production System Representation and Search Engine
		10.1 The Production System: An Architecture for Organizing Search
		10.2 More Examples of Production System Problem-Solving
			10.2.1 The 8-Puzzle Revisited
			10.2.2 The Knight’s Tour Problem
		10.3 The Expert System
		10.4 The Physical Symbol System Hypothesis and the Birth of Cognitive Science
		10.5 Summary
		10.6 Exercises
	11: Advanced Applications of Symbol-Based AI: Planning and Learning
		11.1 Introduction to Planning and Robotics
		11.2 Using Planning Macros: STRIPS
		11.3 Model-Based Planning: A NASA Example (Williams and Nayak 1996, 1997)
		11.4 Symbol-Based Learning
			11.4.1 Decision Trees
			11.4.2 Reinforcement Learning
		11.5 Bacon: Modeling the Celestial Environment
		11.6 Expertise Wherever It Is Needed
		11.7 Summary
		11.8 Exercises
	12: Uncertain Reasoning: Symbol Based
		12.1 Logic-Based Reasoning in Uncertain Situations
			12.1.1 Nonmonotonic Reasoning
			12.1.2 Truth Maintenance Systems
			12.1.3 Logics Based on Minimum Models
		12.2 Uncertain Reasoning: Alternatives to Logic
			12.2.1 The Stanford Certainty Factor Algebra
			12.2.2 Reasoning with Fuzzy Sets
		12.3 Summary and Pointers to Parts VI and VII
		12.4 Exercises
V: Symbol-Based Associational Models for AI
	13: Introduction to Association-Based Knowledge Representations
		13.1 The Behaviorist Tradition and Semantic Networks
			13.1.1 The Foundation for Graphical Representations of Meaning
			13.1.2 Semantic Networks
		13.2 Conceptual Dependencies
		13.3 Scripts
		13.4 Summary
		13.5 Exercises
	14: Association-Based Representations: Frames, Conceptual Graphs, WordNet, and FrameNet
		14.1 Frames
		14.2 Conceptual Graphs
			14.2.1 An Introduction to Conceptual Graphs
			14.2.2 Types, Individuals, and Names
			14.2.3 The Type Hierarchy
			14.2.4 Generalization and Specialization
			14.2.5 Propositional Nodes and Transformations to Logic
		14.3 Current Uses of Association-Based Representations
		14.4 Summary
		14.5 Exercises
VI: Neural or Connectionist Networks
	15: An Introduction to Neural Networks
		15.1 An Artificial Neuron and Applications
		15.2 Early Research: McCulloch, Pitts, and Hebb
		15.3 Perceptrons
			15.3.1 An Example: Perceptron Classification
			15.3.2 Linear Separability and the Exclusive-or Problem
		15.4 Summary
		15.5 Exercises
	16: The Delta Rule, Backpropagation, and Matrix Representations
		16.1 The Generalized Delta Rule
		16.2 The Backpropagation Algorithm
			16.2.1 Example: Backpropagation Solving the Exclusive-Or Problem
		16.3 Matrix Representations for Network Processing
			16.3.1 A Review of Matrix Algebra
			16.3.2 A Review of Vector Algebra and the Cosine Distance Calculation
		16.4 Matrix Representations and Neural Network Solutions
			16.4.1 Representing Perceptron Networks
			16.4.2 Representing Feed-Forward Backpropagation Networks
			16.4.3 Meta-parameters for Small Networks
		16.5 Summary
		16.6 Exercises
	17: Deep Learning: Introduction and Representations
		17.1 Toward Deep Learning
			17.1.1 From Backpropagation to Deep Learning
			17.1.2 AlphaGo Zero and Alpha Zero
			17.1.3 Robot Navigation: PRM-RL
			17.1.4 Deep Learning and Video Games
			17.1.5 Deep Learning and Natural Language Understanding
		17.2 Meta-parameters for Very Large Networks
			17.2.1 Softmax: Representing Network Output as a Probability Distribution
			17.2.2 Softmax Activation and Cross-Entropy Loss
			17.2.3 Data Handling in Network Learning
		17.3 Alternative Architectures: Convolutional and Recurrent Networks
			17.3.1 Convolutional Neural Networks
			17.3.2 Recurrent Neural Networks and Long Short-Term Memory
		17.4 Autoencoders and Transfer Learning
			17.4.1 Autoencoders
			17.4.2 Transfer Learning
				17.4.2.1	 Algorithms, and Related Data Structures
				17.4.2.2	 Transfer Learning with Latent Semantic Analysis and Transformers
		17.5 Summary
		17.6 Exercises
	18: Building Language Models and Transformers
		18.1 Latent Semantic Analysis: Distributed Semantic Representations
		18.2 Building Large Language Models
			18.2.1 Preparing the Text
			18.2.2 Creating Word/Token Embeddings
			18.2.3 Bigrams and Trigrams
			18.2.4 Training the Language Model
		18.3 Toward Transformer-Based Large Language Models
			18.3.1 Transformer Models
			18.3.2 Attention
		18.4 The Transformer in Practice
			18.4.1 Pretraining: The Corpora
			18.4.2 Fine-Tuning the LLM
			18.4.3 Prompt Engineering
			18.4.4 Sample Applications of Generative AI
				18.4.4.1	 Image- to-Image Conversion (url: 18.1)
				18.4.4.2	 Text to Images and Images to Text (url: 18.4 and url: 18.10)
				18.4.4.3	 Text to Video
				18.4.4.4	 Music Production (url: 18.6)
				18.4.4.5	 Code Generation (url: 18.2 and url: 18.12)
				18.4.4.6	 Protein Design and Generation (url: 18.13)
				18.4.4.7	 Continued Extensions of Google’s DeepMind Problem Solvers (url: 18.14)
		18.5 Summary
		18.6 Exercises
	19: Alternative Network Architectures: Prototypes and Classifiers
		19.1 A Kohonen Network: Winner-Takes-All Classification
		19.2 A Kohonen Network: Learning Prototypes
		19.3 Outstar Networks and Counterpropagation
		19.4 Supervised Hebbian Learning
		19.5 Associative Memories and the Linear Associator
		19.6 Summary
		19.7 Exercises
	20: Alternative Network Architectures: Attractor Networks and Memories
		20.1 Introduction to Associative Memories
		20.2 BAM, the Bidirectional Associative Memory
			20.2.1 The BAM Network Architecture
			20.2.2 Examples of BAM Processing
		20.3 Autoassociative Memories and Hopfield Networks
		20.4 Summary
		20.5 Exercises
VII: Probabilistic Artificial Intelligence
	21: Counting, the Foundation for Probabilities
		21.1 Introduction to Probabilistic Reasoning
		21.2 The Elements of Counting
			21.2.1 The Addition and Multiplication Rules
			21.2.2 Permutations and Combinations
		21.3 Elements of Probability Theory
			21.3.1 The Sample Space, Probabilities, and Independence
			21.3.2 Probabilistic Inference: An Example
		21.4 Summary
		21.5 Exercises
	22: Bayes’ Theorem
		22.1 Random Variables
		22.2 Conditional Probability and an Introduction to Bayesian Reasoning
		22.3 Bayes’ Theorem
		22.4 Two Examples of Bayesian Reasoning
			22.4.1 Example: Purchasing an Automobile
			22.4.2 Example: Extending the Traffic Slowdown Problem
		22.5 Summary
		22.6 Exercises
	23: Bayesian Belief Networks and Observable Markov Models
		23.1 Introduction to Stochastic Models
		23.2 A Directed Graphical Model: The Bayesian Belief Network
			23.2.1 Directed Graphical Models: d-separation
			23.2.2 Graphical Models: An Inference Algorithm
		23.3 Dynamic Bayesian Networks
		23.4 Observable Markov Models
		23.5 Summary
		23.6 Exercises
	24: Hidden Markov and Alternative Probabilistic Models
		24.1 Hidden Markov Models
		24.2 Important Variants of Hidden Markov Models
			24.2.1 Auto-Regressive HMMs, AR-HMMs
			24.2.2 Factorial HMMs
			24.2.3 N-Gram HMMs
		24.3 A Short Survey of Alternative Markov Models
			24.3.1 Hierarchical HMMs, HHMMs
			24.3.2 Semi-Markov Models
			24.3.3 Markov Decision Processes
		24.4 First-Order Alternatives to BBNs and HMMs
			24.4.1 Bayesian Network Construction from Knowledge Bases
			24.4.2 Bayesian Logic Programs, BLPs
			24.4.3 Probabilistic Relational Models, PRMs
			24.4.4 Markov Logic Networks, MLNs
			24.4.5 Loopy Logic
		24.5 Two Stochastic Engineering Examples: An MDP and a DBN
			24.5.1 Using a Markov Decision Process to Model a Reward-Driven Robot
			24.5.2 A Dynamic Bayesian Network Model of Nuclear Power Generation
		24.6 Summary
		24.7 Exercises
VIII: AI: Ethical Issues, Fundamental Limitations, and Future Promise
	25: Artificial Intelligence: User’s Ethical Issues
		25.1 Artificial and Human Intelligence
			25.1.1 The Foundations of Artificial Intelligence
			25.1.2 Artificial and Human Intelligence: A Category Difference with Ethical Consequences
		25.2 Ethical Issues from the Perspective of the AI User
			25.2.1 Artificial Intelligence vs Human Intelligence
			25.2.2 AI Products Are Human Created and Require Human Responsibility
			25.2.3 AI Computation Has an Environmental Cost
		25.3 We Are Already Embedded in an AI World
			25.3.1 How to I Perform Responsible Internet Research?
			25.3.2 Who Owns AI-Generated Intellectual Property?
			25.3.3 Who Is the Merchandise in an Internet Environment?
			25.3.4 Who Owns Your Persona?
			25.3.5 An Ethical Stance Requires Being Aware of AI’s Limitations
		25.4 Summary
		25.5 Exercises
	26: AI Ethical Issues: From a Social Perspective
		26.1 Building Society-Oriented AI Projects
		26.2 If We Do Build These AI Projects, What Levels of Protection Are Needed?
			26.2.1 Ethical Issues for AI Designers and Programmers
			26.2.2 Ethical Issues for AI Sales and Management
			26.2.3 Ethical Issues for the Purchaser of AI Products
		26.3 What Is Data?
		26.4 Data and Algorithm Transparency and Explainability
		26.5 Design Honesty, Transparency, Explanations, Data Bias, and Privacy
			26.5.1 Design Knowledge and Honest Technical Presentation
			26.5.2 Transparency and Explanations
			26.5.3 Medical Diagnostics, Explanations, and Privacy
			26.5.4 Natural Language and Sentiment Analysis
			26.5.5 Dealing with Dynamic Systems
			26.5.6 Privacy: The User as Merchandise
			26.5.7 Protection of Intellectual Property
		26.6 Safeguarding AI Research and Practice
		26.7 Summary
		26.8 Exercises
	27: AI: Philosophical Perspectives, Current Limitations, and Future Promise
		27.1 Modern AI: A Psychological, Mathematical, and Philosophical Perspective
		27.2 Several Fundamental Limitations of Current AI Technology
			27.2.1 Limitations of Current AI Practice
			27.2.2 What Is the Role of Embodiment and Culture in Intelligence?
			27.2.3 Meaning: The Grounding Problem
		27.3 Artificial Intelligence: Where Are We Going?
Bibliography
URL
Index