RAGCheatsheet

RAG Patterns Cheatsheet📋

Every essential RAG pattern for production systems in one place. Covers document processing, embeddings, vector stores, retrieval strategies, generation patterns, advanced architectures like agentic RAG and self-RAG, plus evaluation with RAGAS. Copy-paste these Python examples and build reliable retrieval-augmented generation pipelines.

Quick Reference

Name	Syntax	Description
Basic RAG Pipeline	`load -> chunk -> embed -> store -> retrieve -> generate`	The core RAG loop: ingest documents, split into chunks, embed, store in a vector database, retrieve relevant chunks at query time, and generate an answer with context.
Chunking	`RecursiveCharacterTextSplitter(chunk_size, chunk_overlap)`	Split documents into smaller pieces for embedding. Chunk size and overlap control the granularity and continuity of retrieval units.
Embedding	`model.encode(texts) -> List[List[float]]`	Convert text into dense vector representations. Use the same model for both indexing documents and encoding queries.
Vector Search	`collection.query(query_embedding, n_results=5)`	Find the most similar document chunks to a query by comparing embedding vectors using cosine similarity or L2 distance.
Re-Ranking	`CrossEncoder(model).predict([(query, doc) for doc in docs])`	Score and re-order retrieved documents using a cross-encoder model for higher precision than vector similarity alone.
Hybrid Search	`alpha * vector_score + (1 - alpha) * bm25_score`	Combine dense vector search with sparse keyword search (BM25) for better recall on both semantic and exact-match queries.
Multi-Query RAG	`generate_queries(question) -> [q1, q2, q3] -> union(results)`	Generate multiple query variations with an LLM, retrieve for each, and merge results to improve recall on ambiguous questions.
Agentic RAG	`agent.run(query, tools=[retriever, web_search, calculator])`	An LLM agent that decides when and how to retrieve, can reformulate queries, and chains multiple retrieval steps together.
Evaluation Metrics	`evaluate(dataset, metrics=[faithfulness, relevancy, precision])`	Measure RAG quality with metrics like faithfulness (grounded in context), answer relevancy, and context precision using frameworks like RAGAS.
Semantic Caching	`cache.lookup(query_embedding, threshold=0.95)`	Cache RAG responses keyed by query embedding similarity to avoid redundant retrieval and generation for semantically identical questions.

Document Processing

Fixed-Size Chunking

CharacterTextSplitter(chunk_size=1000, chunk_overlap=200)

The simplest chunking strategy: split text into fixed-size windows with overlap to preserve context across chunk boundaries.

python

Tips

Overlap of 10-20% of chunk_size prevents losing context at boundaries
Good baseline for unstructured text like articles and documentation
Does not respect document structure like headings or paragraphs

Recursive Character Splitting

RecursiveCharacterTextSplitter(chunk_size, separators=[...])

Recursively splits using a hierarchy of separators (paragraphs first, then sentences, then words). Respects document structure better than fixed-size splitting.

python

Tips

The default separator hierarchy handles most text well: paragraphs -> lines -> sentences -> words
Use from_language() for code to split at function/class boundaries
This is the most widely used splitter and a good default for most RAG pipelines

Semantic Chunking

SemanticChunker(embeddings, breakpoint_threshold_type)

Groups consecutive sentences into chunks based on embedding similarity. Creates semantically coherent chunks rather than arbitrary size-based splits.

python

Tips

More expensive than rule-based splitting since it requires embedding each sentence
Best for documents where topic boundaries matter (research papers, legal docs)
Use 'percentile' threshold type as a starting point; tune based on your data

Metadata Extraction and Document Parsing

Document(page_content, metadata={source, page, title, ...})

Parse documents from various formats and attach metadata for filtering during retrieval. Metadata like source, page number, and custom tags enable precise scoped search.

python

Tips

Always preserve the source file and page number in metadata for citations
Add timestamps to metadata so you can filter out stale documents at query time
Custom metadata fields enable scoped retrieval (e.g., filter by department or doc_type)

Embeddings & Indexing

Embedding Models

SentenceTransformer(model_name).encode(texts)

Generate embeddings with open-source models from sentence-transformers. Normalize embeddings for cosine similarity and batch-encode for efficiency.

python

Tips

Always use the same model for indexing and querying or results will be meaningless
Normalize embeddings (normalize_embeddings=True) when using cosine similarity
BGE, E5, and GTE model families are strong open-source choices as of 2025

ChromaDB Vector Store

chroma_client.get_or_create_collection(name)

Use ChromaDB as a lightweight vector store with built-in embedding support and metadata filtering. Great for prototyping and small-to-medium datasets.

python

Tips

PersistentClient saves to disk; use Client() for in-memory during development
Metadata filtering with $in, $eq, $gt, etc. narrows results before vector similarity
ChromaDB handles embedding automatically if you provide an embedding_function

pgvector with PostgreSQL

CREATE INDEX ON items USING ivfflat (embedding vector_cosine_ops)

Use pgvector to add vector similarity search to PostgreSQL. Ideal when you already use PostgreSQL and want to avoid a separate vector database.

python

Tips

The <=> operator computes cosine distance; use <#> for inner product, <-> for L2
HNSW indexes are faster than IVFFlat for most workloads; prefer HNSW
Combine vector search with standard SQL WHERE clauses for metadata filtering

Qdrant Vector Store

qdrant_client.search(collection_name, query_vector, limit)

Qdrant is a production-grade vector database with rich filtering, payload indexing, and horizontal scaling. Use it for large-scale RAG deployments.

python

Tips

Qdrant supports complex filters with must, should, must_not conditions
Use payload indexes for frequently filtered fields to speed up filtered search
Qdrant Cloud offers managed hosting; local Docker works for development

Retrieval

Similarity Search with Score Threshold

vectorstore.similarity_search_with_score(query, k, score_threshold)

Retrieve documents by vector similarity with optional score thresholds to filter out irrelevant results. Always set a threshold in production to avoid returning noise.

python

Tips

Score threshold prevents returning irrelevant documents when the knowledge base has no good match
k=5 to k=10 is a good starting range; increase if you use re-ranking downstream
Use similarity_search_with_relevance_scores to inspect confidence for debugging

Hybrid Search (Vector + BM25)

EnsembleRetriever(retrievers=[bm25, vector], weights=[0.4, 0.6])

Combine BM25 keyword search with vector similarity for hybrid retrieval. BM25 catches exact term matches that vector search may miss, and vice versa.

python

Tips

Start with weights [0.4, 0.6] (BM25, vector) and tune based on evaluation results
BM25 excels at exact term matching (product names, error codes, acronyms)
Vector search excels at semantic similarity (paraphrases, related concepts)

Cross-Encoder Re-Ranking

CrossEncoder(model).predict([(query, doc) for doc in candidates])

Re-rank retrieved candidates with a cross-encoder for higher precision. Cross-encoders jointly attend to the query and document, producing more accurate relevance scores than bi-encoder similarity.

python

Tips

Cross-encoders are slow: retrieve broadly (top 20-50), then re-rank to top 3-5
ms-marco-MiniLM is a good default re-ranker; bge-reranker-v2 is also strong
Re-ranking typically improves precision by 10-20% over vector search alone

Maximal Marginal Relevance (MMR)

vectorstore.max_marginal_relevance_search(query, k, fetch_k, lambda_mult)

MMR selects documents that are both relevant to the query and diverse from each other. Prevents returning multiple chunks that say the same thing.

python

Tips

Set lambda_mult=0.7 as a starting point (0.5-0.8 range works well)
Use fetch_k=3x to 5x your desired k to give MMR enough candidates to diversify
MMR is especially useful when your documents have overlapping or redundant content

Generation

RAG Prompt Template

ChatPromptTemplate.from_template(template_with_context)

Structure RAG prompts with explicit instructions to ground answers in retrieved context. Include citation formatting for traceability.

python

Tips

Always instruct the model to say 'I don't know' when context is insufficient
Number your sources so the model can cite them inline with [1], [2] notation
Place the context before the question so the model reads context first

Context Window Management

trim_context(docs, max_tokens, tokenizer)

Manage context window limits by trimming retrieved documents to a token budget. Prevents exceeding model limits and controls cost.

python

Tips

Reserve at least 1000-2000 tokens for the model's answer in your budget calculation
Prioritize higher-ranked documents: they appear first and are kept if trimming occurs
Track truncated documents via metadata so you know when context was cut

Citation and Attribution

extract_citations(answer, sources) -> List[Citation]

Parse and verify citations from RAG-generated answers. Extract source references for traceability and enable users to verify claims against original documents.

python

Tips

Instruct the LLM to cite using [1], [2] format in the system prompt
Verify that cited source numbers actually exist in the retrieved documents
Display citations as clickable links in the UI for user verification

Advanced Patterns

Agentic RAG

agent = create_react_agent(llm, tools=[retriever_tool, ...])

An LLM agent that decides when to retrieve, which tool to use, and can chain multiple retrieval steps. The agent reasons about when vector search alone is insufficient and takes corrective actions.

python

Tips

Give tools clear descriptions so the agent knows when to use each one
Set max_iterations to prevent infinite loops on ambiguous queries
Agentic RAG adds latency: each reasoning step is an LLM call

Multi-Hop Retrieval

iterative retrieve -> extract entities -> retrieve again -> answer

Iteratively retrieve and refine context for complex questions that require connecting information from multiple documents. Each hop identifies missing information and generates targeted follow-up queries.

python

Tips

Limit max hops to 3-5 to control latency and cost
Deduplicate accumulated context across hops to avoid redundancy
Multi-hop is essential for questions that bridge multiple topics or documents

Corrective RAG (CRAG)

retrieve -> evaluate relevance -> web search fallback -> generate

Corrective RAG evaluates retrieval quality and takes corrective action (query rewriting, additional retrieval) when initial results are irrelevant. Improves robustness for out-of-distribution queries.

python

Tips

Grade documents with an LLM to detect retrieval failures early
Query rewriting is the cheapest corrective action; try it before web search fallback
Corrective RAG adds 1-2 LLM calls but significantly improves answer quality on hard queries

Query Expansion and Decomposition

decompose(question) -> [sub_q1, sub_q2] -> retrieve each -> merge

Expand a single query into multiple variations or decompose complex questions into sub-questions. Each variation retrieves independently, then results are merged and deduplicated for broader recall.

python

Tips

Query expansion typically improves recall by 15-30% over single-query retrieval
Deduplicate results by content hash to avoid returning the same chunk multiple times
Decomposition works best for multi-part questions that span different topics

Evaluation & Monitoring

RAGAS Evaluation Framework

evaluate(dataset, metrics=[faithfulness, answer_relevancy, ...])

Use the RAGAS framework to evaluate RAG pipeline quality across four dimensions: faithfulness (grounding), relevancy (answer quality), context precision, and context recall.

python

Tips

Faithfulness measures hallucination: does the answer only contain claims from the context?
Context precision measures retrieval quality: are the top-ranked documents relevant?
Aim for faithfulness > 0.9 and context_precision > 0.8 in production systems

Custom Evaluation Pipeline

evaluate_rag(pipeline, test_cases) -> EvalResults

Build a custom evaluation pipeline with LLM-as-a-judge for faithfulness and relevance scoring. Tracks latency and sources for each test case.

python

Tips

LLM-as-a-judge scales better than human evaluation for regression testing
Track latency alongside quality metrics: a 10-second response is unusable regardless of accuracy
Build a test suite of 50+ question-answer pairs for reliable evaluation

Retrieval Quality Metrics

hit_rate, mrr, ndcg = compute_retrieval_metrics(results, ground_truth)

Compute standard information retrieval metrics (Hit Rate, MRR, nDCG) to evaluate your retrieval component in isolation from generation.

python

Tips

Hit Rate@5 above 0.85 is good; below 0.7 means your retrieval needs improvement
MRR penalizes relevant results that appear at lower ranks
Evaluate retrieval separately from generation to pinpoint where failures occur

Common Patterns

Basic RAG Pipeline End-to-End

python

A complete RAG pipeline from document ingestion to answer generation. Loads a PDF, chunks it with recursive splitting, embeds with a sentence-transformer model, stores in ChromaDB, and queries using a LangChain chain that retrieves context and generates grounded answers. This is the foundational pattern that all other RAG architectures build upon.

Hybrid Search with Re-Ranking

python

Combines BM25 keyword search with dense vector retrieval for broad recall, then applies a cross-encoder re-ranker for precision. The hybrid stage retrieves 10-20 candidates from each method, merges and deduplicates them, and the cross-encoder scores each (query, document) pair to produce a final top-k ranking. This two-stage approach (retrieve broadly, re-rank precisely) is the standard production pattern for high-quality RAG retrieval.

Agentic RAG with Tool Use

python

An agentic RAG system where the LLM autonomously decides which retrieval tools to call and when. Unlike basic RAG that always retrieves, the agent reasons about whether it needs documentation, code examples, or both. It can call multiple tools in sequence, reformulate queries based on initial results, and synthesize information from different sources into a coherent answer with citations. This pattern is ideal for complex questions that span multiple knowledge domains.

Watch Out For

Chunk size too large returns unfocused context; too small loses surrounding meaning and creates fragments that lack sufficient context for the LLM

Start with 500-1000 tokens per chunk and 10-20% overlap. Evaluate with your actual queries: if answers miss details, reduce chunk size; if context lacks coherence, increase it. Tune chunk size as a hyperparameter using retrieval metrics like Hit Rate and MRR.

Using different embedding models for indexing and querying produces meaningless similarity scores because the vector spaces are incompatible

Always use the exact same embedding model (and version) for both indexing and querying. Store the model name in your vector store metadata. When upgrading the embedding model, re-embed your entire document corpus before serving queries.

Document updates and deletions are not reflected in the vector store, causing the RAG system to return stale or deleted information

Track document versions with metadata (e.g., indexed_at timestamp, doc_version). Implement an incremental indexing pipeline that detects changes, deletes old vectors by document ID, and re-indexes updated content. Schedule periodic full re-indexes as a safety net.

Ignoring metadata filtering leads to retrieving irrelevant documents from unrelated sections, departments, or document types

Attach rich metadata (source, section, date, category) during indexing. Apply metadata filters at query time to scope retrieval. For example, filter by doc_type='api_reference' for API questions. Metadata filtering is essentially free and dramatically improves precision.

Relying solely on vector similarity without re-ranking returns plausible-looking but subtly wrong documents, especially for nuanced or specific queries

Add a cross-encoder re-ranking stage after initial retrieval. Retrieve broadly (top 20-50 candidates) with vector search, then re-rank to the final top 3-5 using a cross-encoder like ms-marco-MiniLM. This two-stage approach typically improves precision by 10-20% with minimal latency increase.

Dive Deeper

Rag Retrieval Augmented Generation Vector Embeddings Vector Databases Document Chunking Agentic Rag Hybrid Search