Machine Learning Reference Material Guide

Math Theory

Linear Argebra:

In ML, Linear Algebra comes up everywhere. Topics such as Principal Component Analysis (PCA), Singular Value Decomposition (SVD), Eigendecomposition of a matrix, LU Decomposition, QR Decomposition/Factorization, Symmetric Matrices, Orthogonalization & Orthonormalization, Matrix Operations, Projections, Eigenvalues & Eigenvectors, Vector Spaces and Norms are needed for understanding the optimization methods used for machine learning.

• Deep Learning Book, Chapter 2: Linear Algebra. A quick review of the linear algebra concepts relevant to machine learning. https://goo.gl/O5vgpm
• A First Course in Linear Model Theory by Nalini Ravishanker and Dipak Dey. Textbook introducing linear algebra in a statistical context. https://goo.gl/2A4Wi5

Probability Theory and Statistics:

Because Machine Learning and Statistics aren’t very different fields. Some of the fundamental Statistical and Probability Theory needed for ML are Combinatorics, Probability Rules & Axioms, Bayes’ Theorem, Random Variables, Variance and Expectation, Conditional and Joint Distributions, Standard Distributions (Bernoulli, Binomial, Multinomial, Uniform and Gaussian), Moment Generating Functions, Maximum Likelihood Estimation (MLE), Prior and Posterior, Maximum a Posteriori Estimation (MAP) and Sampling Methods.

• All of Statistics: A Concise Course in Statistical Inference, by Larry Wasserman. Introductory text on statistics: https://goo.gl/lSSXYU

Calculus:

Some of the necessary topics include Differential and Integral Calculus, Partial Derivatives, Vector-Values Functions, Directional Gradient, Hessian, Jacobian, Laplacian and Lagragian Distribution.

• Calculus: A Complete Introduction by Hugh Neill: https://goo.gl/WsMJYG

Algorithms and Complex Optimizations:

This is important for understanding the computational efficiency and scalability of our Machine Learning Algorithm and for exploiting sparsity in our datasets. Knowledge of data structures (Binary Trees, Hashing, Heap, Stack etc), Dynamic Programming, Randomized & Sublinear Algorithm, Graphs, Gradient/Stochastic Descents and Primal-Dual methods are needed.

• Data Structures and Algorithms in Java. By Robert Lafore: https://goo.gl/CJXDq2 Although it has Java in the title, the content of this book applies to any other language.

Machine Learning:

• Deep Learning (Adaptive Computation and Machine Learning series) by Ian Goodfellow: https://goo.gl/O5vgpm
• Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems: https://goo.gl/mDxPaU