AI Lab / AI Learning Tracks
LLM From Scratch, No Incense Required
Tokenizer, embeddings, attention, training loop, mini-GPT, finetuning, and why scale changes behavior.
Learning track
Build ladder.
Train a tiny text model, inspect its failures, and connect the toy version to modern LLM systems.
Prerequisites
- Python
- basic calculus comfort
- basic neural network idea
- 01 Text to numbers
Understand character, byte, and subword tokenization.
Write a tokenizer visualizer for your own bio. - 02 Embeddings
See how IDs become learned vectors.
Plot nearest neighbors in a toy corpus. - 03 Attention
Explain query, key, value, masking, and context windows.
Animate one attention head on a short sentence. - 04 Training loop
Run loss, batches, gradient steps, and sampling.
Train a tiny quote generator. - 05 Failure diary
Track repetition, hallucination, memorization, and brittleness.
Publish the model's wrong outputs with explanations.
Checkpoints
- Can define token
- Can trace attention
- Can train a toy model
- Can sample outputs
- Can explain why tiny models fail