Generative AI

For ownersMaking images, audio and video. Handy for drafts, with a person checking each one before it goes out.

Create images, video, audio, 3D, and more.

15 lessons

Start with lesson 1

Lesson 1 of 15, ~45 minutesGenerative Models — Taxonomy & History

The lessons

  1. 01Generative Models — Taxonomy & HistoryA map of the main families of models that create new content, such as autoencoders, GANs, diffusion and autoregressive models, and how the field moved from one to the next.Learn · ~45 minutes
  2. 02Autoencoders & Variational Autoencoders (VAE)An autoencoder squeezes data into a small code and rebuilds it. A variational autoencoder shapes that code so you can sample new points from it and get new, realistic outputs.Build · ~75 minutes
  3. 03GANs — Generator vs DiscriminatorA GAN pairs two networks: one makes fake samples and the other tries to spot them. Each improves by competing until the fakes look real.Build · ~75 minutes
  4. 04Conditional GANs & Pix2PixA conditional GAN takes an extra input, such as a label or an image, so you control what it makes. Pix2Pix uses this to turn one kind of image into another.Build · ~75 minutes
  5. 05StyleGANStyleGAN is a GAN design that feeds a style signal into each layer separately, which gives cleaner control over coarse features like pose and fine ones like texture.Build · ~45 minutes
  6. 06Diffusion Models — DDPM from ScratchA diffusion model learns to remove noise. Training adds noise to real data in small steps, and the model learns to undo each step, so it can start from pure noise and build a new sample.Build · ~75 minutes
  7. 07Latent Diffusion & Stable DiffusionLatent diffusion runs the noise-removal process on a compressed version of the image instead of every pixel, which makes text-to-image generation cheap enough to run on ordinary hardware.Build · ~75 minutes
  8. 08ControlNet, LoRA & ConditioningControlNet steers an image model with a guide like a pose, edge map or depth map. LoRA fine-tunes a big model cheaply by training a small add-on instead of all its weights.Build · ~75 minutes
  9. 09Inpainting, Outpainting & Image EditingInpainting regenerates a masked part of an image, outpainting extends the canvas beyond its edges, and editing changes an image while keeping the rest intact.Build · ~75 minutes
  10. 10Video GenerationVideo generation extends image models to frames over time, so the model must keep objects, motion and lighting consistent from frame to frame. It is much more compute heavy than images.Build · ~45 minutes
  11. 11Audio GenerationAudio models turn sound into compact tokens with a neural codec, generate those tokens with a language-style or diffusion model, then decode them back into speech, music or effects.Build · ~45 minutes
  12. 123D Generation3D generation makes objects or scenes from text, images or photos, often by generating several 2D views and reconstructing a 3D shape. Gaussian Splatting is a fast way to represent and render scenes.Learn · ~45 minutes
  13. 13Flow Matching & Rectified FlowsFlow matching trains a model to move noise to data along straight paths. Straighter paths need fewer steps, so images and video come out faster with similar quality.Build · ~45 minutes
  14. 14Evaluation — FID, CLIP Score, Human PreferenceHow image models get scored: FID compares generated and real image statistics, CLIP score checks how well an image matches its prompt, and human preference tests ask people to pick the better one.Build · ~45 minutes
  15. 15Visual Autoregressive Modeling (VAR): Next-Scale PredictionVAR generates an image coarse to fine: it predicts a tiny version first, then each larger scale based on the one before, using a language-model style transformer instead of diffusion.Build · ~90 minutes

Lessons from AI Engineering from Scratch by Rohit Ghumare and contributors, MIT license. Notes for owners by Lozo. License

Next step

Rather have it built for you?

Reading is optional. Tell me what's stuck and I'll build it into the business you already run.

Tell me what's stuck
  • A real conversation, not a pitch
  • You keep your accounts
  • Nothing sends without your yes