After Training: Two Ways to Deploy a LoRA Adapter
You've trained your LoRA adapter — a compact set of weights that specialize your base model for a task. Before you can call it in production, you face a decision that shapes your entire serving architecture.
My field notes
Building the VocabForge pipeline end-to-end taught me that the initial training run is the smaller half of the work. Everything after — packaging it so it doesn't become a black box, choosing a deployment path that doesn't overextend you on day one, building feedback loops so the model improves rather than just repeating itself — that's where most of the decisions live. The sections below are the ones I had to figure out in practice.
Merge Into Base Model
Compute the combined weights mathematically (base + adapter = new model). Deploy this merged artifact. Every inference call hits the single combined model — no PEFT stack, no extra steps.
Keep Adapters Separate
One base model, many task-specific adapters stored independently. At inference time, load the base model and the chosen adapter together. You share one large base model across all your specializations.
Model Cards — Your Model's Passport
Every fine-tuned model is a new artifact with its own identity. A model card documents that identity completely — so anyone (including future you) can understand exactly what was trained, how it was trained, and what it replaced.
Adapter Weights
The actual trained LoRA adapter weights or full fine-tuned checkpoint. The core artifact of everything you built.
Tokenizer Config
Critical. Using the wrong tokenizer at inference breaks the model silently. Document the exact tokenizer version used during training.
Architecture Details
Which base model, which version, LoRA rank, target modules, and all hyperparameters used. Reproducibility depends on this.
Eval Results
Base model performance vs fine-tuned performance on your golden dataset. The delta is why you trained.
Training Metadata
Dataset used, training date, framework version, GPU type. Needed for debugging and for regulatory audit trails.
Stack Dependencies
Every piece of infrastructure this model depends on — vector stores, retrieval pipelines, API endpoints, memory layers.
Where to store it: Google Cloud has a Model Card Kit that feeds into a Model Registry. HuggingFace model cards are the standard for open-source releases. AWS and GCP both have model registries. The platform matters less than the habit — document before you deploy, not after something breaks.
Model Cards in the Wild
Every major lab publishes them. They are the canonical reference point for what a model is and what it was designed to do:
HuggingFace (the standard)
Every model on HuggingFace has a model card. The format — created by Google Brain researchers — is now the de facto industry standard. Meta's Llama 3, Mistral 7B, Qwen all use it. The card lives in the repo, versioned alongside the weights.
Meta — Llama 3 Model Card
One of the most detailed public examples. Covers intended use, out-of-scope uses, benchmark evals, safety red-teaming results, carbon footprint of training, and hardware specs. Often cited as the template for responsible release.
Anthropic — Claude Model Card
Published for each Claude version. Details training approach, safety evaluations, alignment properties, known limitations, and intended deployment contexts. Available at anthropic.com alongside the model release.
OpenAI — System Cards
OpenAI calls theirs "system cards" for major releases (GPT-4, DALL-E 3, Sora). The scope is broader — covers the full deployed system, not just the model weights. Same concept, different name.
Google — Gemini Model Cards
Google publishes model cards for Gemini models and maintains the open-source Model Card Toolkit on GitHub. They pioneered the formal model card concept in a 2019 paper (Mitchell et al.) that the whole industry now references.
Internal registries (enterprise)
At companies like LinkedIn, Pinterest, and Airbnb, internal ML platforms host private model registries — same idea, proprietary tooling. AWS SageMaker Model Registry and Vertex AI Model Registry are the managed equivalents teams use without building from scratch.
Canary Deployments & Stack Rollback
Never replace a running model with a new version in one shot. The same principle that governs software deploys applies to model updates — and the cost of a bad ML deployment can be harder to diagnose than a bad code deploy.
-
1%
Canary phase — 1% of traffic
Route a tiny slice of real production traffic to the new model version. Watch for unexpected outputs, error rates, latency changes. Compare directly against the current version side by side.
-
5%
Ramp phase — 5% and growing
If 1% shows no regressions, expand the rollout. Sample across demographics, user types, and edge-case query patterns. No model improvement should degrade another part of the behavior.
-
→
Full rollout — confirmed better
Only after the new model demonstrably outperforms the old one on your eval metrics — and shows no regressions — do you complete the cutover.
-
↺
Rollback — the whole stack, not just the model
If something goes wrong post-rollout, reverting the model version is not enough. All dependent services — retrieval layers, routing logic, prompt templates, A/B configs — must be rolled back together. This is exactly why the model card exists: it documents what all those dependencies were pointing to.
Reinforcement Learning — The Alignment Engine
Reinforcement Learning was built for games — environments where an agent can take actions, observe outcomes, and receive rewards. The maze is the classic example: go left, right, forward, or back; get a reward for moving toward the goal; take a penalty for wrong turns. Collect all those (action, state, reward) tuples and use them to update the agent's weights.
The same loop maps directly to LLM fine-tuning. The "maze" becomes a task. The "actions" become response choices. The "rewards" come from a grader.
Because LLMs are non-deterministic, you generate multiple responses for each prompt. The grader scores each one. Responses that score high push the model's weights in that direction; low-scoring responses push it away. Repeat across thousands of prompts and iterations. The model learns not just what you showed it, but how to generate the kind of output your grader rewards.
Why multiple responses? A single attempt doesn't tell you much. Generating five responses to the same prompt reveals the model's probability distribution — some approaches work better than others, and the reward signal gives the model contrast to learn from.
How Rewards Work: RLVR vs RLHF
The grader is the heart of RL fine-tuning. Who builds it — and how — defines two distinct approaches. Choose based on whether your task is quantifiable.
Reinforcement Learning with Verifiable Rewards
You write a deterministic Python function that scores every model output. No humans in the loop at all. The function is the judge: it takes the model's response, checks it against verifiable criteria, and returns a number.
Works best when the task has a right answer — math problems, code generation, SQL queries, classification tasks. The score can be binary (correct/wrong = 1/0) or nuanced:
By rewarding correct intermediate reasoning steps — not just the final answer — you train the model to think through problems, not just guess the output. This is how reasoning models are built. The more nuanced the grader, the faster and better the model learns.
Reinforcement Learning from Human Feedback
When the task has no deterministic right answer — writing, summarization, creative content, tone matching — you can't write a function that scores it. You need humans.
How RLHF works: Humans rank or rate multiple model responses. That ranking data trains a separate model — called a reward model — to approximate human preferences. You then use that reward model as the grader in the RL loop.
Key distinction from DPO: DPO (Direct Preference Optimization) also uses good/bad labels — but there is no separate reward model. You label responses as preferred/rejected and directly fine-tune the main LLM on those labels. It's a supervised approach. RLHF builds an intermediary model first; DPO skips that step.
DPO is cheaper and simpler. RLHF is more expensive but gives a reusable reward model that can be continuously updated with new human ratings without retraining from scratch.
The Method Ladder — Always SFT Before RL
Prompt Engineering
System prompts, few-shot examples, format rules. Free and reversible. Always the first move.
Context Engineering
RAG, memory layers, tool descriptions, structured injection. Still no weight changes.
Supervised Fine-Tuning
Input→output pairs. Improves response quality on your eval metric. More control, visible progress.
RL Fine-Tuning
Alignment and behavior improvements on top of SFT. Cutting-edge, complex, high-expertise.
What each step delivers: SFT improves quality metrics — your model gets better at producing the kind of output your dataset shows. RL gives alignment and behavioral improvements — the model learns to reason, follow complex constraints, and generalize the goal rather than just imitate examples.
Who actually uses RL? As of today, RL fine-tuning is only running at companies with cutting-edge, niche product needs and specialized ML teams — Cursor for code generation, Notion for structured editing, similar products. It requires deep expertise, significant compute, and a well-maintained SFT base to build on top of. Most teams stop at step 3.
Synthetic Data — Teaching Without Enough Real Examples
Fine-tuning requires data. But real-world, cleanly labeled data is often scarce. Synthetic data — generated by AI — fills the gap. It's not a perfect substitute, but it's a standard part of the practitioner toolkit.
Expand the Dataset
You have 100 real examples but need 10,000. Use AI to generate variations: rephrase the prompt, add synonyms, change domain context, simulate edge cases. This is exactly what image augmentation (flip, rotate, brightness) did for vision models — applied to text.
Clean Messy Data
Real-world data is noisy — inconsistent formatting, typos, missing structure. Use a strong model to clean and standardize it for fine-tuning. The cleaned version is synthetic; the underlying meaning came from real data.
Simulate Missing Groups
Your product runs in North America. You have no data for how 35+ year-olds in other markets use it. Synthesize plausible examples based on assumptions. Imperfect, but better than training with a systematic blind spot.
Two Pitfalls That Corrupt Everything
Distillation — The Teacher-Student Protocol
When DeepSeek was accused of "stealing" from Anthropic's models, the actual methodology being described was knowledge distillation — a standard, well-documented technique that every major lab uses internally. Understanding it clarifies both the controversy and the strategy.
The process: pass a large set of domain-specific queries to a powerful teacher model (GPT-4, Claude, Gemini). Collect its responses. Now you have a synthetic dataset of (input, output) pairs. Fine-tune a much smaller model on those pairs. The student learns to approximate the teacher's behavior on your specific domain — at a fraction of the inference cost.
Why do this? A 10B parameter distilled model handling your specific task can be cheaper to run, lower latency, and privately owned — compared to paying per-token to a frontier API forever. You get the intelligence of the large model, distilled into something you control.
My field notes — the DeepSeek reframe
When the DeepSeek headlines dropped, the framing in most coverage was competitive threat. The reframe I landed on: a large model can effectively train a smaller, faster one to replicate the job — which is exactly the pattern of an expert whose work a junior learns from. That's not a controversy, that's a workflow. Once you see it that way, distillation stops being an advanced technique and starts being the obvious next step after your first successful fine-tuning run.
I built VocabForge — a fine-tuned Qwen3 model for transforming raw performance descriptions into structured professional narrative — to test this end-to-end. The hard parts weren't the AI. They were the debugging, the environment, the formats, the evaluation logic. The model was almost the easy part. Fireworks.ai's API wasn't OpenAI-compatible for fine-tuning; Colab lost my training run three times. Every section of this guide is something I hit in practice, not just in theory.
GPU Cost Architecture — Three Tiers
How you access GPUs for training determines your cost structure, your flexibility, and how much capacity risk you carry. Most individuals start at tier one; most enterprises migrate toward tier three as volume grows.
Sign up, add credits, and run. The platform allocates GPU capacity from a shared pool when available. No upfront commitment. Variable wait times when the pool is busy.
Commit to a certain number of GPU hours with a cloud provider. Your training jobs are prioritized over serverless users. Predictable scheduling, consistent throughput.
Own the physical hardware. Build a private GPU cluster (or negotiate a private cloud partition). Maximum optimization potential — LinkedIn runs this way for complete stack control. Maximum capital outlay.
Training vs Inference Cost — The Break-Even
A common misunderstanding: "fine-tuning is expensive, so using an API is cheaper." This is only true at low volume. At scale, it inverts — and understanding the break-even is what determines whether your fine-tuning project makes financial sense.
The economics: APIs charge per token with no upfront cost. Fine-tuning has significant upfront cost — compute, data, engineering time — but a smaller, task-specific model has lower per-inference cost than a large general API call. At sufficient volume, the cumulative API spend exceeds the fine-tuning investment, and the crossover point becomes your break-even.
Three cost components to compare:
| Approach | Setup Cost | Per-Inference Cost | Performance |
|---|---|---|---|
| API (no fine-tuning) | ✓ None | High | General |
| RAG + Prompting | ✓ Low | Medium | Good for knowledge |
| Fine-Tuned Model | High | Medium | Task-optimized |
| LoRA Adapter | Low–Medium | ✓ Low | High + efficient |
Ollama & Local Inference — AI That Runs Anywhere
Under the hood: Ollama downloads models in GGUF format — a single file that bundles the model weights, tokenizer configuration, prompt template, and metadata. It quantizes to 4-bit (trading a small amount of precision for a large reduction in memory and compute). It runs on CPU if no GPU is available — making it usable on a standard laptop.
Once running, Ollama starts a local HTTP server at localhost:11434. You make REST requests to it — and optionally configure it to behave like the OpenAI API format, which means existing LangChain code, RAG pipelines, and agent frameworks plug in with minimal changes. Build a local RAG app, run lightweight agents, classify intent from a fine-tuned model — all on your machine, all offline-capable.
- Privacy-sensitive industries where data cannot leave your environment (HIPAA, GDPR, financial compliance)
- Regulated workflows needing data residency guarantees — even signed contracts with cloud providers aren't always enough
- Edge deployments — factory cameras, on-device inference, manufacturing defect detection where there's no reliable network
- Fast prototyping without API keys, billing, or rate limits slowing you down
- High token volume where the cost math crosses the break-even against a hosted API
- Hardware-constrained: you're limited to RAM and CPU of your laptop (or however many GPUs you own)
- Open-source models only — the quality gap between frontier closed-source models and open-source alternatives is real for some tasks
- Smaller context windows compared to hosted frontier models
- Not production-grade for concurrency: Ollama is low-concurrency by design — serving thousands of simultaneous users requires different tooling
- MLOps overhead at scale: redundancy, monitoring, and throughput optimization require engineering investment
The practical flow: Install Ollama → ollama run llama3.2 → model downloads in GGUF format → local HTTP server starts → call it from Python, LangChain, or your agent framework exactly as you would a cloud API. Your fine-tuned model can be converted to GGUF and loaded into Ollama the same way — so you can run your own trained model locally with a single command.
The Production Checklist
- Model card written before deployment — weights, tokenizer, eval results, dependencies
- Full eval suite re-run on the fine-tuned model (not just fine-tuning loss)
- Canary deployment: 1% → 5% → full rollout with comparison at each stage
- Stack rollback plan documented — not just the model, but every dependent service
- GPU tier chosen to match your current volume and cost math
- RL considered only after SFT is working and the use case is quantifiable or human-reviewable
- Deploying without a model card — you have no rollback reference when something breaks
- Using the wrong tokenizer at inference — breaks the model silently, hard to debug
- Jumping straight to RL from a base model without SFT foundation
- Generating synthetic data without separating train/test splits first — contaminates evals
- Assuming low diversity in synthetic data is fine — models overfit to narrow patterns
- Rolling back only the model version instead of the entire dependent stack
Public Sources & Further Reading
HuggingFace PEFT / LoRA
huggingface.co/docs/peft — LoRA & QLoRA documentation
QLORA Paper — Dettmers et al.
arXiv:2305.11206 — 4-bit quantization for fine-tuning
Ollama
ollama.com — run LLMs locally, zero infra
RLHF — OpenAI Research
Reinforcement learning from human feedback — original paper
My workflow: A GenAcademy session introduced me to the post-training landscape — LoRA paths, RLHF alignment, synthetic data, and distillation. From there I explored public research papers, HuggingFace documentation, and the Ollama/Unsloth open-source ecosystems, then built this guide to understand by doing. This is my personal learning synthesis — a live session has far more depth and takeaways than any set of notes captures. For the full curriculum, reach out to GenAcademy directly.