Before Machine Learning

Contents
Contents 3
1 Why Calculus? 8
2 Pointing Fingers and Crossing Lines: The last breath
of just linearity. 10
2.1 Swipe Right for co-domains: Functions. . . . . . . . 18
3 Changing Times and Tangent Lines: The Derivative. 26
3.1 The Tale of the Eight with Wobbly Knees: Infinity. 44
3.1.1 Forever Close, Never Met: The Untouchables
Asymptotes. . . . . . . . . . . . . . . . . . . 52
3.1.2 Better Sharpen Our Pencils: Continuity, The
No-Lift Zone! . . . . . . . . . . . . . . . . . . 58
3.1.2.1 Boomerang Bonanza: The Inverse
Function’s Return Trip. . . . . . . 58
3.1.2.2 The Mood Swings of the Graph World:
Piecewise Functions. . . . . . . . 61
3.2 Claustrophobia in the Calculus Corridor: The Limit. 63
3.3 Deciphering the Undefined - L’Hopital’s Rule. . . . 77
3.3.1 After You, Please: The Cordial Conduct of
Function Composition. . . . . . . . . . . . 86
3.3.1.1 The Key to the Calculus Handcuffs:
Chain Rule. . . . . . . . . . . . . . 86
3.4 Get Me Higher and I’ll Give You Better Curves: Taylor Series. . . . . . . . . . . . . . . . . . . . . . . . 94
3.5 Valleys and Hills: A Critical Situation for Points
and Their Roots. . . . . . . . . . . . . . . . . . . . 105
3.5.1 Zeroing in: Newton-Raphson Method. . . 114
3.5.2 The Original GPS: Gradient Descent. . . . 134
3.5.3 Dare You Cross Me, Line? Convexity. . . . . 139
3.5.4 Really?! Dishing Out More Dough? The
Cost Function. . . . . . . . . . . . . . . . . 143
4 Cleaning Up The Derivatives Debris: The Integral. 157
4.1 That Riemann Guy Is a Bit Square, Isn’t He? . . . 162
4.2 This seems important: The Fundamental Theorem of Calculus. . . . . . . . . . . . . . . . . . . . 167
4.3 Hold On, Logs Aren’t Just for Fireplaces? Scaling
Data. . . . . . . . . . . . . . . . . . . . . . . . . . . 194
5 A Free Upgrade: More Dimensions. 200
5.1 Numerous Entries But Single Exit: Scalar Functions.201
5.1.1 Grab Your 3D Glasses for these Multidimensional Integrals. . . . . . . . . . . . . . . . . 202
5.1.2 I Am Quite Partial to These Derivatives. . 205
5.1.3 Just Follow My Lead: The Gradient. . . . . 208
5.1.4 GPS 2.0: Multidimensional Gradient Descent. . . . . . . . . . . . . . . . . . . . . . . 211
5.1.5 Bending Realities, Unveiling Curvatures: The
Hessian. . . . . . . . . . . . . . . . . . . . . 212
5.1.6 I Thought We Were Past Lines: Linear Regression. . . . . . . . . . . . . . . . . . . . . 219
5.1.7 Newton, Raphson, and Hessian Walk Into
a Bar: No Joke, They Found a Root! . . . . . 226
5.1.8 A Mathematical Marriage: Scalar Functions
and Neural Networks. . . . . . . . . . . . . 232
5.1.8.1 Wake Up Call for Neurons: Curve
Alert with Activation Functions. . 243
5.1.8.2 Wait, Which Way? The Min-Max
Scaling. . . . . . . . . . . . . . . . . 249
5.1.8.3 The Price of Disorder: Cross Entropy as a Cost Function. . . . . . 257
5.1.8.4 A Detour Through Randomness: Stochastic Gradient Descent. . . . . . . . 260
5.1.8.5 Upgrading Our Locksmith Skills: The
Multivariate Chain Rule. . . . . 262
3
Contents
5.1.8.6 One Step Back to Give Two Steps
Forward: Backpropagation. . . . . 269
5.1.8.7 Customized Routes to Minima: RMSProp. . . . . . . . . . . . . . . . . 278
5.2 A Crowd Comes In, A Parade Goes Out: Vector
Functions. . . . . . . . . . . . . . . . . . . . . . . . 286
5.2.1 A Clandestine Mathematical Liaison: Vector
Functions and Neural Networks. . . . . . 294
5.2.1.1 Gathering Steam: The Momentum
Algorithm. . . . . . . . . . . . . . . 303
5.2.2 Smooth Sailing with a Customized Map: Embarking on the Adaptive Moment Estimation. . . . . . . . . . . . . . . . . . . . . . . . 310