NLP, Foundations to Advanced
For ownersWorking with text: sorting email, pulling details out of documents, reading reviews at scale.
Language is the interface to intelligence. Master every layer.
The lessons
- 01Text Processing — Tokenization, Stemming, LemmatizationBefore a computer can work with text, the text has to be cut into small pieces and tidied up. This lesson covers splitting text into words, trimming words to their root, and mapping word forms to a base form.Build · ~45 minutes
- 02Bag of Words, TF-IDF, and Text RepresentationBag of words turns a document into word counts. TF-IDF adjusts those counts so common words like "the" matter less and words that are rare across your files matter more.Build · ~75 minutes
- 03Word Embeddings — Word2Vec from ScratchWord2Vec learns a list of numbers for each word by looking at the words around it. Words used in similar ways end up with similar numbers, so the computer can measure how related two words are.Build · ~75 minutes
- 04GloVe, FastText, and Subword EmbeddingsGloVe and FastText are other ways to build word vectors. GloVe learns from how often words appear together across a whole body of text. FastText builds vectors from word pieces, so it can handle words it has never seen.Build · ~45 minutes
- 05Sentiment AnalysisSentiment analysis labels text as positive, negative or neutral. The lesson uses it to teach the basics of sorting text into categories, from simple word lists to trained models.Build · ~75 minutes
- 06Named Entity RecognitionNamed entity recognition finds names of people, companies, places, dates and amounts inside text and labels each one. It turns loose sentences into fields a system can use.Build · ~75 minutes
- 07POS Tagging and Syntactic ParsingPart-of-speech tagging marks each word as a noun, verb, adjective and so on. Parsing maps how the words in a sentence connect, such as which noun a verb acts on.Build · ~45 minutes
- 08CNNs and RNNs for TextBefore transformers, two kinds of neural networks handled text. Convolutional networks spot short word patterns. Recurrent networks read word by word and carry a memory forward.Build · ~75 minutes
- 09Sequence-to-Sequence ModelsSequence-to-sequence models take one piece of text in and produce another out, such as a sentence in one language to the same sentence in another. One network reads, a second one writes.Build · ~75 minutes
- 10Attention Mechanism — The BreakthroughAttention lets a model look back at every part of its input and decide which parts matter most for the word it is producing right now, instead of relying on one compressed summary.Build · ~45 minutes
- 11Machine TranslationMachine translation turns text from one language into another automatically. The lesson walks from rule-based systems to neural models and how translation quality gets measured.Build · ~75 minutes
- 12Text SummarizationText summarization shortens a document. Extractive methods pick the most important original sentences. Abstractive methods write new sentences that capture the main points.Build · ~75 minutes
- 13Question Answering SystemsQuestion answering systems take a question and return an answer. Some highlight the answer inside a given text, some first search a document set, and some write the answer from what the model learned.Build · ~75 minutes
- 14Information Retrieval and SearchInformation retrieval is how a system finds the most relevant documents for a query. Keyword methods like BM25 match exact words, dense methods match meaning, and hybrid search uses both.Build · ~75 minutes
- 15Topic Modeling — LDA and BERTopicTopic modeling groups a pile of documents by the themes they share, without anyone labeling them first. LDA does it with word statistics. BERTopic clusters documents by meaning.Learn · ~45 minutes
- 16Text Generation Before Transformers — N-gram Language ModelsBefore modern AI, text was generated with n-gram models that guess the next word from the few words before it, using counts from real text. Perplexity measures how surprised such a model is by new text.Build · ~45 minutes
- 17Chatbots — Rule-Based to Neural to LLM AgentsChatbots moved from scripted pattern matching, to systems that detect intent and fill in details, to language models that write replies, to agents that use tools and check their work.Learn · ~75 minutes
- 18Multilingual NLPMultilingual models learn many languages in one model. Skills learned with lots of data in one language can carry over to languages with little or no training data.Learn · ~45 minutes
- 19Subword Tokenization — BPE, WordPiece, Unigram, SentencePieceSubword tokenization splits text into common chunks smaller than words but bigger than letters. Every modern language model reads and writes using one of these chunk vocabularies.Learn · ~60 minutes
- 20Structured Outputs & Constrained DecodingStructured outputs make a language model return data in an exact format, such as valid JSON matching a schema. Constrained decoding blocks any next token that would break the format.Build · ~60 minutes
- 21Natural Language Inference — Textual EntailmentNatural language inference decides whether one statement follows from another, contradicts it, or neither. It is a simple test of whether a model understands meaning.Learn · ~60 minutes
- 22Embedding Models — The 2026 Deep DiveEmbedding models turn a passage of text into a list of numbers that captures its meaning. Similar passages get similar numbers, which is what makes meaning-based search work.Learn · ~60 minutes
- 23Chunking Strategies for RAGChunking is how long documents get cut into smaller passages before they are stored for AI search. The size and split points decide what the system can find later.Build · ~60 minutes
- 24Coreference ResolutionCoreference resolution works out which words refer to the same person or thing, such as linking "she", "the manager" and a name back to one person across a passage.Learn · ~60 minutes
- 25Entity Linking & DisambiguationEntity linking takes a name found in text and connects it to the exact real-world thing it means in a reference list, so two things with the same name are not mixed up.Build · ~60 minutes
- 26Relation Extraction & Knowledge Graph ConstructionRelation extraction finds how named things in text are connected, such as who works for which company. A knowledge graph stores those people, things and links, with a note on where each fact came from.Build · ~60 minutes
- 27LLM Evaluation — RAGAS, DeepEval, G-EvalLLM evaluation frameworks like RAGAS, DeepEval and G-Eval score AI answers for things like accuracy, relevance and faithfulness to sources, often using another model as the judge.Build · ~75 minutes
- 28Long-Context Evaluation — NIAH, RULER, LongBench, MRCRLong-context evaluation tests whether a model can actually use very long inputs. A model may accept a huge document but still miss or mix up details buried in the middle.Learn · ~60 minutes
- 29Dialogue State TrackingDialogue state tracking keeps a running record of what a user wants during a conversation, such as date, time and party size, and updates it as they change their mind.Build · ~75 minutes
Lessons from AI Engineering from Scratch by Rohit Ghumare and contributors, MIT license. Notes for owners by Lozo. License
Next step
Rather have it built for you?
Reading is optional. Tell me what's stuck and I'll build it into the business you already run.