Before you start
This page covers what classical RL doesn't teach you about how frontier LLMs are built. It builds directly on the pages below — start there if anything here is unfamiliar.
Start here if new to RL
Classical Reinforcement Learning
Agent, environment, Q-functions, DQN, policy gradients — the bicycle analogy explained interactively.
Explore →
LLM fundamentals
Limitations & Frontiers
Hallucinations, emergent abilities, cross-entropy, adversarial examples — where LLMs break.
Explore →
Fine-tuning basics
Custom LLMs Guide
SFT, data preparation, algorithm selection, evaluation — the practitioner's playbook.
Read guide →
What classical RL doesn't cover
Classical RL gives you the vocabulary: agent, environment, reward, policy. But training a frontier language model adds seven ideas that classical RL never had to solve. Each one below has an interactive lesson you can run yourself.
The 4-Phase Pipeline
SFT → RLVR → SWE-RLVR → RLHF — and why that exact order matters
SFT as RL Bootstrap
RL can't start from zero reward — the math teacher analogy
RLVR — Verifiable Rewards
Only train on tasks where "correct" is computable — tic-tac-toe vs poetry
Reward Hacking
Models learn the shortcut, not the goal — the cheese vs banana problem
Multi-Environment RL
70+ simultaneous environments — the anti-hacking fix
GRPO
Generate 8, keep the above-average — the floor keeps rising
RLHF vs RLVR — Order Matters
Capability first, pleasantness last — and why reversing it fails
Interactive Sandbox
Seven lessons, each building on the last. Work through them in order — or jump to any tab.
The 4-Phase Post-Training Pipeline
Click Next Phase → to walk through how Nemotron goes from raw pre-trained weights to a reasoning system. Watch capability fill as each phase completes.
Pre-trained
Raw token predictor. Smart autocomplete, nothing more.
① SFT
Learns the shape of a good answer from curated examples.
② RLVR
Practices on tasks where correctness is computable.
② SWE-RLVR
Parallel code track — slow verification needs its own loop.
③ RLHF
Human preferences polish tone, safety, and style.
Public Sources & Further Reading
NVIDIA Nemotron-4 Blog
developer.nvidia.com — Nemotron-4 340B technical overview
Nemotron-4 340B Paper
arXiv:2402.19379 — post-training alignment and synthetic data
NVIDIA NeMo Framework
github.com/NVIDIA/NeMo — open-source LLM training framework
My workflow: A GenAcademy session introduced me to Chris Alexiuk's work on NVIDIA's post-training stack. From there I explored NVIDIA's public research papers, developer blog, and the open-source NeMo framework, then built this guide to understand the SFT → alignment → RLHF pipeline by building it out myself. This is my personal synthesis. A live session has far more depth — for the full curriculum, reach out to GenAcademy directly.