Math Foundations

For ownersThe math under every model. You can skip it; the engineers you hire should know it.

The intuition behind every AI algorithm, through code, not textbooks.

22 lessons

Start with lesson 1

Lesson 1 of 22, ~60 minutesLinear Algebra Intuition

The lessons

  1. 01Linear Algebra IntuitionLinear algebra is the math of lists of numbers (vectors) and grids of numbers (matrices). AI models store what they know and do their work as this kind of math.Learn · ~60 minutes
  2. 02Vectors, Matrices & OperationsThe basic moves with vectors and matrices: adding them, scaling them, the dot product that measures how much two vectors point the same way, and matrix multiplication.Build · ~60 minutes
  3. 03Matrix TransformationsA matrix can be seen as a machine that moves every point in space: it can stretch, rotate, flip or squash shapes. Eigenvectors are the directions it only stretches.Build · ~75 minutes
  4. 04Calculus for Machine LearningDerivatives measure how much an output changes when you nudge an input. In machine learning they show which way to adjust each setting so the error gets smaller.Learn · ~60 minutes
  5. 05Chain Rule & Automatic DifferentiationThe chain rule finds the derivative of functions stacked inside each other. Automatic differentiation applies it by computer, which is how neural networks learn which weights to change.Build · ~90 minutes
  6. 06Probability and DistributionsProbability puts a number on uncertainty. A distribution describes how likely each possible outcome is, such as the bell curve for heights or the odds of each next word.Learn · ~75 minutes
  7. 07Bayes' TheoremBayes' theorem updates a belief when new evidence arrives. It combines how likely something was before with how well the evidence fits it, giving a revised probability.Build · ~75 minutes
  8. 08OptimizationOptimization is the search for settings that make an error as small as possible. Gradient descent and its variants like momentum and Adam take repeated downhill steps.Build · ~75 minutes
  9. 09Information TheoryInformation theory measures surprise and uncertainty in bits. Entropy, cross-entropy and KL divergence come from it, and they are the scores most AI models learn to lower.Learn · ~60 minutes
  10. 10Dimensionality ReductionDimensionality reduction squeezes data with many columns into a few that keep most of its structure. PCA, t-SNE and UMAP are common tools, often used to make charts of complex data.Build · ~90 minutes
  11. 11Singular Value DecompositionSVD breaks any matrix into three simpler pieces: rotations and a list of strengths. Keeping only the strongest pieces gives a smaller version that keeps most of the information.Build · ~120 minutes
  12. 12Tensor OperationsA tensor is a grid of numbers with any number of axes, such as a batch of images or a batch of sentences. Reshaping, broadcasting and combining tensors is everyday AI work.Build · ~90 minutes
  13. 13Numerical StabilityComputers store numbers with limited precision, so tiny rounding errors, overflow and underflow can wreck results. Tricks like log-sum-exp and careful precision choices keep the math sane.Build · ~120 minutes
  14. 14Norms and DistancesNorms measure the size of a vector and distances measure how far apart two things are. Euclidean, Manhattan and cosine are common choices, and each defines similar in its own way.Build · ~90 minutes
  15. 15Statistics for Machine LearningStatistics tells you whether a result is real or luck. Means, variance, confidence intervals and hypothesis tests show how much to trust a model's score or an A/B test.Build · ~120 minutes
  16. 16Sampling MethodsSampling means drawing examples from a distribution. It covers random draws, Monte Carlo estimates, importance sampling, and how language models pick the next word using temperature or top-p.Build · ~120 minutes
  17. 17Linear SystemsA linear system is a set of equations like Ax = b, solved for the unknowns x. Methods like Gaussian elimination, LU and least squares solve them, and they sit under regression and many models.Build · ~120 minutes
  18. 18Convex OptimizationA convex problem has a single bowl-shaped valley, so any downhill path reaches the best answer. Neural networks are not convex, which is why their training is harder to predict.Build · ~90 minutes
  19. 19Complex Numbers for AIComplex numbers add an imaginary part to normal numbers. They make rotations and waves easy to handle, which matters for signal processing, Fourier transforms and some position tricks in language models.Learn · ~60 minutes
  20. 20The Fourier TransformThe Fourier transform splits a signal into the sine waves that make it up, showing which frequencies are present and how strong each is. The FFT computes it very fast.Build · ~90 minutes
  21. 21Graph Theory for Machine LearningA graph is a set of things (nodes) joined by connections (edges). Graph theory gives tools to walk, search and measure these networks, and graph neural networks learn from them.Build · ~90 minutes
  22. 22Stochastic ProcessesA stochastic process is randomness that unfolds over time with some structure. Random walks, Markov chains, Brownian motion and Langevin dynamics are examples, and diffusion image models rely on them.Learn · ~75 minutes

Lessons from AI Engineering from Scratch by Rohit Ghumare and contributors, MIT license. Notes for owners by Lozo. License

Next step

Rather have it built for you?

Reading is optional. Tell me what's stuck and I'll build it into the business you already run.

Tell me what's stuck
  • A real conversation, not a pitch
  • You keep your accounts
  • Nothing sends without your yes