My field notes
"Some sessions you attend. Some you can't stop building from."
This was one of the second kind. After the RAG session I went deep into the external references — RAGAS, LlamaIndex eval guide, Pinecone and Weaviate production guides, MTEB leaderboard. Each added a layer I hadn't seen before. I built this toolkit to hold all of it in one place: open it before you design a pipeline, open it when something breaks mid-build.
Before you start
This toolkit is a decision companion — it helps you configure and troubleshoot, not learn from scratch. If any term below is new to you, read the RAG guide first, then come back here to build.
Prerequisite read
Ultimate RAG Guide
Architecture, chunking strategies, vector databases, evaluation metrics, and limitations — the complete reference.
Read the guide →
Visualise your chunks
ChunkViz
Paste any text, choose a splitter, and see exactly how your document gets broken apart before it hits the embedding model.
Open tool ↗
Pick your model
MTEB Leaderboard
Filter by task, language, and model size. The authoritative benchmark for choosing an embedding model for your domain.
Open leaderboard ↗
The 7 design levers
A RAG pipeline has exactly seven parameters you make conscious decisions about. Get the defaults right and you have a working system. Tune them with evidence and you have a production one. Ignore them and you get confident-sounding nonsense.
Chunk Size Default: 512 tokens
How many tokens each stored passage contains — smaller chunks give sharper matches, larger chunks give richer context per fetch.
512 tokens ≈ two solid paragraphs from a policy document.
Chunk Overlap Default: 10–15%
Tokens shared between adjacent chunks so meaning doesn't vanish at a hard boundary cut.
64-token overlap on a 512-token chunk: the closing sentence of chunk N opens chunk N+1.
Embedding Model Must match ingestion ↔ query
The neural network that converts text to a dense vector — every vector in your database must live in the same coordinate space.
Switching models mid-project means re-indexing everything. Lock it before you ingest a single document.
Top-K Default: 5
How many chunks are fetched from the vector database and placed into the LLM's context window on each query.
K=5 narrows 100,000 stored chunks down to 5 paragraphs the model actually reads.
Retrieval Strategy Dense / Sparse / Hybrid
Whether you search by meaning (embedding similarity), by keyword overlap (BM25), or both fused together.
"Refund policy" → dense finds semantic matches; BM25 catches exact phrase hits. Hybrid captures both.
Re-ranker Strongly recommended
A second-pass model that re-scores retrieved chunks by contextual relevance before the LLM sees them — eliminating near-miss noise.
Vector search finds the neighbourhood. Re-ranker finds the right house on the right street.
System Prompt Constraint Non-negotiable
The instruction that tells the LLM to answer only from retrieved context and say "I don't know" when the answer isn't there.
Without this, the model will fill gaps with training knowledge — plausibly, confidently, wrongly.
Interactive Toolkit
Configure your pipeline top to bottom across seven tabs, then use the debug signals and pre-launch checklist before shipping.
The Full RAG Pipeline — Offline vs Online
RAG splits into two distinct phases. The offline phase runs once when you ingest data (and again when data changes). The online phase runs on every user query. Click any step to see what it does and where it breaks.
Offline — runs once
Online — every query
Click any pipeline step above to see what it does, where it breaks, and the rule of thumb for that stage.
Public Sources & Further Reading
LangChain RAG Docs
python.langchain.com — loaders, splitters, embeddings, retrievers
LlamaIndex Documentation
docs.llamaindex.ai — indexing strategies and query engines
Pinecone Docs
docs.pinecone.io — vector search, hybrid retrieval, namespaces
My workflow: A GenAcademy session introduced me to RAG as a production discipline — not just a demo trick. From there I went deep into LangChain, LlamaIndex, Pinecone, and Weaviate public docs, then built this toolkit by working through the failure modes myself. This is my personal synthesis. A live session has far more depth and takeaways than any set of notes captures — for the full curriculum, reach out to GenAcademy directly.