Choose this when
You want to go beneath API usage into neural nets, transformers, training, and fine-tuning.
All learning paths
AI learning path
Model internals
Understand enough of the machinery behind LLMs to reason about behavior, limits, and training tradeoffs.
- Best for
- Engineers who want deeper understanding
- Level
- Intermediate to advanced
- Time
- 20-40 hours
You should be able to
You can read model architecture explanations and implement small training loops with less mystery.
Checkpoint
Move on when transformer diagrams map to code you can roughly follow.
Do
Learning sequence
This is the route through the topic. Watch and open the material inside the step where it is used.
Step 1
Build intuition
Learn gradient descent, embeddings, attention, and transformer blocks visually and in code.
- Neural nets
- Attention
- Embeddings
Watch here
Intro to Large Language Models
Andrej Karpathy
High-level architecture and behavior before building pieces yourself.
Open here
Neural Networks: Zero to Hero
Video course · Andrej Karpathy · Intermediate
You want to understand neural networks and language models from code.
Open resourceStep 2
Implement small pieces
Write toy versions of tokenization, training loops, and generation.
- Tokenization
- Training loop
- Sampling
Watch here
Neural Networks: Zero to Hero
Andrej Karpathy
Use this when implementing the fundamentals in code.
Let's build GPT from scratch
Andrej Karpathy
Hands-on transformer and training-loop implementation.
Open here
Build a Large Language Model From Scratch
GitHub repo / book · Sebastian Raschka · Intermediate
You want to build an LLM step by step.
Open resourceStep 3
Connect to modern models
Learn fine-tuning, inference, quantization, and open-model tradeoffs.
- Fine-tuning
- Inference
- Open models
Open here
The Illustrated Transformer
Visual guide · Jay Alammar · Beginner to intermediate
Transformer architecture still feels fuzzy.
Open resourceHugging Face LLM Course
Free course · Hugging Face · Beginner to intermediate
You need the Transformer, LLM, and inference basics behind many AI apps.
Open resourcePractice task
Train or inspect a tiny language model and explain each major step in plain English.
Reference
All resources in this path
Step 1
Neural Networks: Zero to Hero
Video course · Andrej Karpathy · Intermediate
You want to understand neural networks and language models from code.
Step 2
Build a Large Language Model From Scratch
GitHub repo / book · Sebastian Raschka · Intermediate
You want to build an LLM step by step.
Step 3
The Illustrated Transformer
Visual guide · Jay Alammar · Beginner to intermediate
Transformer architecture still feels fuzzy.
Step 3
Hugging Face LLM Course
Free course · Hugging Face · Beginner to intermediate
You need the Transformer, LLM, and inference basics behind many AI apps.
Educators to follow
Andrej Karpathy
Intermediate
Watch micrograd, then makemore, then the GPT video.
View educator
Sebastian Raschka
Intermediate
Clone the repo and work through the notebooks with the book.
View educator
Jay Alammar
Beginner to intermediate
Read The Illustrated Transformer, then The Illustrated GPT-2.
View educator
Jeremy Howard
Intermediate
Practical Deep Learning for Coders, lesson 1.
View educator
Swyx
Intermediate to advanced
Watch AI Engineer talks for production patterns and tool choices.
View educator
Andrew Ng
Beginner to advanced
Start with ChatGPT Prompt Engineering for Developers, then pick a RAG or agents course.
View educator