Artificial Intelligence and Machine Learning Fundamentals

Preface......Page 30
Objectives......Page 32
Minimum Hardware Requirements......Page 34
Software Requirements......Page 35
Installation and Setup......Page 36
Starting Anaconda......Page 38
Additional Resources......Page 41
Principles of Artificial Intelligence......Page 43
Introduction......Page 45
How does AI Solve Real World Problems?......Page 46
Diversity of Disciplines......Page 48
Fields and Applications of Artificial Intelligence......Page 50
Simulating Intelligence – The Turing Test......Page 55
AI Tools and Learning Models......Page 56
Classification and Prediction......Page 57
Learning Models......Page 58
The Role of Python in Artificial Intelligence......Page 59
Why is Python Dominant in Machine Learning, Data Science, and AI?......Page 60
Anaconda in Python......Page 61
Python Libraries for Artificial Intelligence......Page 63
A Brief Introduction to the NumPy Library......Page 65
Exercise 1: Matrix Operations Using NumPy......Page 70
Intelligent Agents in Games......Page 75
Breadth First Search and Depth First Search......Page 77
Exploring the State Space of a Game......Page 85
Exercise 2: Estimating the Number of Possible States in Tic-Tac-Toe Game......Page 92
Exercise 3: Creating an AI Randomly......Page 94
Activity 1: Generating All Possible Sequences of Steps in a Tic-Tac-Toe Game......Page 100
Summary......Page 103
AI with Search Techniques and Games......Page 104
Exercise 4: Teaching the Agent to Win......Page 106
Activity 2: Teaching the Agent to Realize Situations When It Defends Against Losses......Page 109
Activity 3: Fixing the First and Second Moves of the AI to Make it Invincible......Page 111
Creating Heuristics......Page 115
Admissible and Non-Admissible Heuristics......Page 117
Heuristic Evaluation......Page 118
Exercise 5: Tic-Tac-Toe Static Evaluation with a Heuristic Function......Page 126
Types of Heuristics......Page 130
Pathfinding with the A* Algorithm......Page 132
Exercise 6: Finding the Shortest Path to Reach a Goal......Page 138
Exercise 7: Finding the Shortest Path Using BFS......Page 139
Introducing the A* Algorithm......Page 143
A* Search in Practice Using the simpleai Library......Page 167
Game AI with the Minmax Algorithm and Alpha-Beta Pruning......Page 173
Search Algorithms for Turn-Based Multiplayer Games......Page 174
The Minmax Algorithm......Page 179
Optimizing the Minmax Algorithm with Alpha-Beta Pruning......Page 196
DRYing up the Minmax Algorithm – The NegaMax Algorithm......Page 205
Using the EasyAI Library......Page 207
Activity 4: Connect Four......Page 211
Summary......Page 214
Regression......Page 215
Introduction......Page 217
Linear Regression with One Variable......Page 218
What Is Regression?......Page 219
Features and Labels......Page 229
Feature Scaling......Page 231
Cross-Validation with Training and Test Data......Page 234
Fitting a Model on Data with scikit-learn......Page 236
Linear Regression Using NumPy Arrays......Page 239
Fitting a Model Using NumPy Polyfit......Page 253
Predicting Values with Linear Regression......Page 270
Activity 5: Predicting Population......Page 273
Multiple Linear Regression......Page 277
The Process of Linear Regression......Page 279
Loading Stock Prices with Yahoo Finance......Page 280
Loading Stock Prices with Quandl......Page 284
Exercise 8: Using Quandl to Load Stock Prices......Page 285
Preparing Data for Prediction......Page 287
Performing and Validating Linear Regression......Page 296
Predicting the Future......Page 301
Polynomial and Support Vector Regression......Page 311
Polynomial Regression with One Variable......Page 312
Exercise 9: 1st, 2nd, and 3rd Degree Polynomial Regression......Page 314
Polynomial Regression with Multiple Variables......Page 322
Support Vector Regression......Page 325
Support Vector Machines with a 3 Degree Polynomial Kernel......Page 331
Activity 6: Stock Price Prediction with Quadratic and Cubic Linear Polynomial Regression with Multiple Variables......Page 334
Summary......Page 336
Classification......Page 337
Introduction......Page 339
The Fundamentals of Classification......Page 340
Exercise 10: Loading Datasets......Page 342
Data Preprocessing......Page 349
Exercise 11: Pre-Processing Data......Page 355
Minmax Scaling of the Goal Column......Page 359
Identifying Features and Labels......Page 361
Activity 7: Preparing Credit Data for Classification......Page 362
The k-nearest neighbor Classifier......Page 363
Introducing the K-Nearest Neighbor Algorithm......Page 365
Distance Functions......Page 366
Exercise 12: Illustrating the K-nearest Neighbor Classifier Algorithm......Page 372
Exercise 13: k-nearest Neighbor Classification in scikit-learn......Page 379
Exercise 14: Prediction with the k-nearest neighbors classifier......Page 381
Parameterization of the k-nearest neighbor Classifier in scikit-learn......Page 382
Activity 8: Increasing the Accuracy of Credit Scoring......Page 383
Classification with Support Vector Machines......Page 385
What are Support Vector Machine Classifiers?......Page 386
Understanding Support Vector Machines......Page 388
Support Vector Machines in scikit-learn......Page 397
Activity 9: Support Vector Machine Optimization in scikit-learn......Page 399
Summary......Page 402
Using Trees for Predictive Analysis......Page 403
Introduction to Decision Trees......Page 405
Entropy......Page 416
Exercise 15: Calculating the Entropy......Page 421
Information Gain......Page 425
Gini Impurity......Page 427
Exit Condition......Page 433
Building Decision Tree Classifiers using scikit-learn......Page 434
Evaluating the Performance of Classifiers......Page 436
Exercise 16: Precision and Recall......Page 440
Exercise 17: Calculating the F1 Score......Page 442
Exercise 18: Confusion Matrix......Page 448
Activity 10: Car Data Classification......Page 452
Random Forest Classifier......Page 455
Constructing a Random Forest......Page 457
Random Forest Classification Using scikit-learn......Page 459
Feature Importance......Page 461
Extremely Randomized Trees......Page 463
Activity 11: Random Forest Classification for Your Car Rental Company......Page 464
Summary......Page 466
Clustering......Page 467
Introduction to Clustering......Page 469
Defining the Clustering Problem......Page 470
Clustering Approaches......Page 477
Clustering Algorithms Supported by scikit-learn......Page 478
The k-means Algorithm......Page 480
Exercise 19: k-means in scikit-learn......Page 482
Exercise 20: Retrieving the Center Points and the Labels......Page 494
Activity 12: k-means Clustering of Sales Data......Page 496
Exercise 21: Illustrating Mean Shift in 2D......Page 499
Mean Shift Algorithm in scikit-learn......Page 511
Image Processing in Python......Page 517
Activity 13: Shape Recognition with the Mean Shift Algorithm......Page 521
Summary......Page 523
Deep Learning with Neural Networks......Page 524
Introduction......Page 526
TensorFlow for Python......Page 527
Installing TensorFlow in the Anaconda Navigator......Page 528
Exercise 22: Using Basic Operations and TensorFlow constants......Page 530
Placeholders and Variables......Page 533
Global Variables Initializer......Page 534
Introduction to Neural Networks......Page 536
Biases......Page 549
Use Cases for Artificial Neural Networks......Page 552
Activation Functions......Page 554
Exercise 23: Activation Functions......Page 563
Forward and Backward Propagation......Page 567
Importing the TensorFlow Digit Dataset......Page 569
Modeling Features and Labels......Page 571
TensorFlow Modeling for Multiple Labels......Page 575
Optimizing the Variables......Page 577
Training the TensorFlow Model......Page 581
Testing the Model......Page 582
Randomizing the Sample Size......Page 584
Activity 14: Written Digit Detection......Page 585
Adding Layers......Page 588
Convolutional Neural Networks......Page 591
Activity 15: Written Digit Detection with Deep Learning......Page 592
Summary......Page 595
Appendix......Page 597
Activity 1: Generating All Possible Sequences of Steps in the tic-tac-toe Game......Page 599
Activity 2: Teach the agent realize situations when it defends against losses......Page 606
Activity 3: Fix the first and second moves of the AI to make it invincible......Page 610
Activity 4: Connect Four......Page 614
Activity 5: Predicting Population......Page 622
Activity 6: Stock Price Prediction with Quadratic and Cubic Linear Polynomial Regression with Multiple Variables......Page 627
Activity 7: Preparing Credit Data for Classification......Page 638
Activity 8: Increase the accuracy of credit scoring......Page 645
Activity 9: Support Vector Machine Optimization in scikit-learn......Page 646
Activity 10: Car Data Classification......Page 649
Activity 11: Random Forest Classification for your Car Rental Company......Page 655
Activity 12: k-means Clustering of Sales Data......Page 668
Activity 13: Shape Recognition with the Mean Shift algorithm......Page 671
Activity 14: Written digit detection......Page 699
Activity 15 : Written Digit Detection with Deep Learning......Page 706