Computer Vision
For ownersSoftware that reads images: receipts, shelf photos, damage photos, scanned forms.
From pixels to understanding across image, video, and 3D.
The lessons
- 01Image Fundamentals — Pixels, Channels, Color SpacesA digital image is a grid of numbers. Each pixel holds brightness values for color channels like red, green and blue, and color spaces are different ways of organizing those numbers.Build · ~45 minutes
- 02Convolutions from ScratchA convolution slides a small grid of weights across an image and scores each patch. It is how vision models detect edges, textures and shapes no matter where they appear.Build · ~75 minutes
- 03CNNs — LeNet to ResNetConvolutional neural networks stack convolution, activation and shrinking steps to understand images. Famous designs from LeNet to ResNet each added one idea, such as deeper stacks or skip connections.Learn + Build · ~75 minutes
- 04Image ClassificationImage classification means giving a picture one label from a fixed list, such as damaged or not damaged. The model outputs a probability for each label and picks the highest.Build · ~75 minutes
- 05Transfer Learning & Fine-TuningTransfer learning starts from a model already trained on millions of images and adapts it to your task. Fine-tuning adjusts some or all of its weights using your own smaller dataset.Build · ~75 minutes
- 06Object Detection — YOLO from ScratchObject detection finds every object in an image and draws a box around each one with a label. YOLO is a fast family of detectors that does this in a single pass.Build · ~75 minutes
- 07Semantic Segmentation — U-NetSemantic segmentation labels every pixel in an image, such as road, sky or crack. U-Net does this by shrinking the image to understand it, then growing it back to full detail.Build · ~75 minutes
- 08Instance Segmentation — Mask R-CNNInstance segmentation outlines each separate object, so two overlapping boxes on a shelf get two different masks. Mask R-CNN adds a mask step on top of a box detector.Build + Learn · ~75 minutes
- 09Image Generation — GANsA GAN is a pair of networks trained against each other. One makes fake images and the other tries to tell fake from real, and both improve until the fakes look convincing.Build · ~75 minutes
- 10Image Generation — Diffusion ModelsA diffusion model learns to remove a little noise from an image. Starting from pure noise and repeating that cleanup many times, it produces a brand new image.Build · ~75 minutes
- 11Stable Diffusion — Architecture & Fine-TuningStable Diffusion runs the diffusion process on a compressed version of the image to save compute, and uses text to steer what it draws. It can be fine-tuned on your own images.Learn + Use · ~75 minutes
- 12Video Understanding — Temporal ModelingVideo understanding models look at a sequence of frames and reason about motion and change over time. They can recognize actions, events or steps in a process.Learn + Build · ~45 minutes
- 133D Vision — Point Clouds & NeRFs3D vision works out where things are in space. Point clouds are dots measured by depth sensors, while NeRFs are learned models that can render a scene from new viewpoints.Learn + Build · ~45 minutes
- 14Vision Transformers (ViT)A vision transformer cuts an image into small square patches and treats each patch like a word in a sentence. A standard transformer then learns how the patches relate.Build · ~45 minutes
- 15Real-Time Vision — Edge DeploymentEdge deployment means running a vision model on a small local device, like a camera or phone, fast enough for live video. It involves shrinking and speeding up the model.Learn + Build · ~75 minutes
- 16Build a Complete Vision Pipeline — CapstoneA real vision system is a chain of steps: get the image, detect, crop, classify or read text, then apply business rules. This capstone connects those parts end to end.Build · ~120 minutes
- 17Self-Supervised Vision — SimCLR, DINO, MAESelf-supervised learning teaches a model useful image features from huge piles of unlabeled photos, using tasks like matching two views of the same image or filling in hidden patches.Learn + Build · ~75 minutes
- 18Open-Vocabulary Vision — CLIPCLIP learns from image and caption pairs so pictures and text land in the same space. That lets it match images to any text description, even labels it never trained on.Build + Use · ~45 minutes
- 19OCR & Document UnderstandingOCR turns pictures of text into editable text. Document understanding goes further, finding where each field sits and what it means, such as an invoice total or a due date.Learn + Use · ~45 minutes
- 20Image Retrieval & Metric LearningImage retrieval finds pictures similar to a query image by comparing number fingerprints called embeddings. Metric learning trains those fingerprints so similar items sit close together.Build · ~45 minutes
- 21Keypoint Detection & Pose EstimationKeypoint detection finds specific points on an object, like elbows and knees on a person. Pose estimation connects those points to describe how the body or object is positioned.Build · ~45 minutes
- 223D Gaussian Splatting from ScratchGaussian splatting represents a scene as millions of small, soft colored blobs in 3D. Learned from photos, it renders realistic views of the scene very quickly from any angle.Build · ~90 minutes
- 23Diffusion Transformers & Rectified FlowNewer image generators replace the older U-Net inside diffusion with a transformer, and use rectified flow, which learns a straight path from noise to image so fewer steps are needed.Learn + Build · ~75 minutes
- 24SAM 3 & Open-Vocabulary SegmentationOpen-vocabulary segmentation lets you type a phrase like yellow forklift and get exact outlines of every match in an image. SAM 3 from Meta does this in one model.Use + Build · ~60 minutes
- 25Vision-Language Models — The ViT-MLP-LLM PatternA vision-language model lets a chat model see. An image encoder turns the picture into tokens, a small adapter maps them into the language model, and the model answers in text.Learn + Use · ~75 minutes
- 26Monocular Depth & Geometry EstimationMonocular depth estimation predicts how far away each pixel is using a single ordinary photo, with no special depth sensor. The output is a depth map of the scene.Build + Use · ~60 minutes
- 27Multi-Object Tracking & Video MemoryMulti-object tracking follows many objects across video frames and keeps a steady ID for each one. It detects objects in every frame, then matches them to the ones seen before.Build · ~60 minutes
- 28World Models & Video DiffusionA world model predicts what happens next in a scene. Video diffusion models that generate future frames, especially when given actions as input, act like learned simulators.Learn + Build · ~75 minutes
Lessons from AI Engineering from Scratch by Rohit Ghumare and contributors, MIT license. Notes for owners by Lozo. License
Next step
Rather have it built for you?
Reading is optional. Tell me what's stuck and I'll build it into the business you already run.