Deep Learning Core

For ownersHow a neural network learns from examples. Background for everything after it; skim the notes.

Neural networks from first principles. No frameworks until you build one yourself.

13 lessons

Start with lesson 1

Lesson 1 of 13, ~60 minutesThe Perceptron

The lessons

  1. 01The PerceptronA perceptron is the simplest building block of a neural network. It multiplies each input by a weight, adds them up with a bias, and makes a yes or no call.Build · ~60 minutes
  2. 02Multi-Layer Networks and Forward PassA multi-layer network stacks groups of simple units so the output of one layer feeds the next. The forward pass is the trip data takes from input to final prediction.Build · ~90 minutes
  3. 03Backpropagation from ScratchBackpropagation is how a network learns from mistakes. It measures how much each weight contributed to the error, then sends that blame backward through the layers so every weight can be adjusted.Build · ~120 minutes
  4. 04Activation FunctionsAn activation function is a small curve applied after each layer. It lets the network bend its decisions instead of only drawing straight lines, which is what makes deep networks useful.Build · ~75 minutes
  5. 05Loss FunctionsA loss function is the score that tells a model how wrong its prediction was. Training tries to make this number as small as possible, so the loss decides what the model learns to care about.Build · ~75 minutes
  6. 06OptimizersAn optimizer is the rule that decides how far and in which way to change the weights after each step of training. Options like SGD and Adam differ in speed and stability.Build · ~75 minutes
  7. 07RegularizationRegularization is a set of tricks that stop a model from memorizing its training data. It pushes the model to learn general patterns that still work on new, unseen examples.Build · ~75 minutes
  8. 08Weight Initialization and Training StabilityWeight initialization is how a network sets its starting numbers before training. Good starting values keep signals from fading out or blowing up as they pass through many layers.Build · ~90 minutes
  9. 09Learning Rate Schedules and WarmupThe learning rate sets how big each training step is. A schedule changes it over time, often starting small, rising during warmup, then shrinking so the model settles on a good answer.Build · ~90 minutes
  10. 10Build Your Own Mini FrameworkThis lesson wires the earlier pieces, like layers, losses and optimizers, into one small library. It shows how tools like PyTorch are organized under the hood.Build · ~120 minutes
  11. 11Introduction to PyTorchPyTorch is the most widely used library for building and training neural networks in Python. It handles tensors, gradients and GPU use so developers can focus on the model.Build · ~75 minutes
  12. 12Introduction to JAXJAX is a Python library from Google that treats models as pure math functions. It can compile them to run fast on GPUs and TPUs and compute gradients automatically.Build · ~90 minutes
  13. 13Debugging Neural NetworksDebugging a neural network means finding why it runs without errors yet gives bad results. It relies on checks like overfitting a tiny batch, watching loss curves and inspecting gradients.Build · ~90 minutes

Lessons from AI Engineering from Scratch by Rohit Ghumare and contributors, MIT license. Notes for owners by Lozo. License

Next step

Rather have it built for you?

Reading is optional. Tell me what's stuck and I'll build it into the business you already run.

Tell me what's stuck
  • A real conversation, not a pitch
  • You keep your accounts
  • Nothing sends without your yes