Neural Networks from Scratch in Python: Building Neural Networks in Raw Python

Acknowledgements
Copyright
License for Code
Readme
Introducing Neural Networks
	A Brief History
	What is a Neural Network?
Coding Our First Neurons
	A Single Neuron
	A Layer of Neurons
	Tensors, Arrays and Vectors
	Dot Product and Vector Addition
	A Single Neuron with NumPy
	A Batch of Data
	Matrix Product
	Transposition for the Matrix Product
	A Layer of Neurons & Batch of Data w/ NumPy
Adding Layers
	Training Data
	Dense Layer Class
	Full code up to this point:
Activation Functions
	The Step Activation Function
	The Linear Activation Function
	The Sigmoid Activation Function
	The Rectified Linear Activation Function
	Why Use Activation Functions?
	Linear Activation in the Hidden Layers
	ReLU Activation in a Pair of Neurons
	ReLU Activation in the Hidden Layers
	ReLU Activation Function Code
	The Softmax Activation Function
	Full code up to this point:
Calculating Network Error with Loss
	Categorical Cross-Entropy Loss
	The Categorical Cross-Entropy Loss Class
	Combining everything up to this point:
	Accuracy Calculation
Introducing Optimization
	Full code up to this point:
Derivatives
	The Impact of a Parameter on the Output
	The Slope
	The Numerical Derivative
	The Analytical Derivative
	Summary
Gradients, Partial Derivatives, and the Chain Rule
	The Partial Derivative
	The Partial Derivative of a Sum
	The Partial Derivative of Multiplication
	The Partial Derivative of Max
	The Gradient
	The Chain Rule
	Summary
Backpropagation
	Categorical Cross-Entropy loss derivative
	Categorical Cross-Entropy loss derivative code implementation
	Softmax activation derivative
	Softmax activation derivative code implementation
	Common Categorical Cross-Entropy loss and Softmax activation derivative
	Common Categorical Cross-Entropy loss and Softmax activation derivative - code implementation
	Full code up to this point:
Optimizers
	Stochastic Gradient Descent (SGD)
	Learning Rate
	Learning Rate Decay
	Stochastic Gradient Descent with Momentum
	AdaGrad
	RMSProp
	Adam
	Full code up to this point:
Testing with Out-of-Sample Data
Validation Data
Training Dataset
L1 and L2 Regularization
	Forward Pass
	Backward pass
Dropout
	Forward Pass
	Backward Pass
	The Code
Binary Logistic Regression
	Sigmoid Activation Function
	Sigmoid Function Derivative
	Sigmoid Function Code
	Binary Cross-Entropy Loss
	Binary Cross-Entropy Loss Derivative
	Binary Cross-Entropy Code
	Implementing Binary Logistic Regression and Binary Cross-Entropy Loss
	Full code up to this point:
Regression
	Linear Activation
	Mean Squared Error Loss
	Mean Squared Error Loss Derivative
	Mean Squared Error (MSE) Loss Code
	Mean Absolute Error Loss
	Mean Absolute Error Loss Derivative
	Mean Absolute Error Loss Code
	Accuracy in Regression
	Regression Model Training
	Full code up to this point:
Model Object
	Full code up to this point:
A Real Dataset
	Data preparation
	Data loading
	Data preprocessing
	Data Shuffling
	Batches
	Training
	Full code up to now:
Model Evaluation
Saving and Loading Models and Their Parameters
	Retrieving Parameters
	Setting Parameters
	Saving Parameters
	Loading Parameters
	Saving the Model
	Loading the Model
Prediction / Inference
	Full code:
Closing