Advanced RAG patterns

Embedding models trained on (question, passage) pairs (BGE, E5, Cohere v3) work — but the user's question and the answer passage live in different language registers. "Why does my SaaS bill jump in March?" and "Annual renewals on customer contracts re-bill on the anniversary date..." share almost no vocabulary. Query transforms close that gap before retrieval.

Ask an LLM to write the answer it expects, then embed that hypothetical answer instead of the question. The hallucinated text shares vocabulary with real source passages, so the embedding lands much closer to them in vector space. Originally proposed by Gao et al. (2022); often beats the raw-question baseline by 5–15% recall on domain-shifted corpora.

HYDE_PROMPT = """Write a concise, factual paragraph that would answer the
following question. Don't say 'I don't know' — make a plausible attempt.

Question: {question}
Answer:"""

def hyde_retrieve(question, k=10):
    hypothetical = llm.complete(HYDE_PROMPT.format(question=question))
    q_vec = embed(hypothetical)            # embed the fake answer
    return vector_db.search(q_vec, k=k)

Ask the LLM to rewrite the question 3–5 ways, retrieve top-k for each variant, then deduplicate. Coverage improves because different phrasings hit different neighborhoods of the vector space — a one-shot fix for the "the user used a synonym we don't index" problem.

PARAPHRASE = """Rewrite the question below in 4 different ways that preserve
its meaning. Output as a JSON array of strings.

Question: {q}"""

def multi_query(q, k=10):
    variants = json.loads(llm.complete(PARAPHRASE.format(q=q)))
    seen = {}
    for v in [q, *variants]:
        for hit in vector_db.search(embed(v), k=k):
            seen[hit.id] = hit          # dedup by ID, keep best score
    return list(seen.values())

Multi-query gets you N candidate lists; Reciprocal Rank Fusion merges them without needing comparable scores. Each document gets a score of Σ 1 / (k + rank_i) across the lists it appears in (k=60 is the canonical constant). Documents ranked high by multiple paraphrases bubble to the top.

from collections import defaultdict

def rrf(ranked_lists, k=60):
    scores = defaultdict(float)
    for lst in ranked_lists:
        for rank, doc in enumerate(lst):
            scores[doc.id] += 1.0 / (k + rank)
    return sorted(scores.items(), key=lambda x: -x[1])

For reasoning-heavy questions, ask the LLM to abstract one level up first ("What general principle does this question rely on?"), then retrieve for both the original and the abstract question. Helps with questions like "Why does this code throw a NullPointerException on line 42?" where the line-42 detail is a red herring and the real retrieval target is "how does Java null dereference work".

Not every question wants the vector store. A small classifier (or structured LLM call) picks the tool: vector DB for semantic lookup, SQL for "how many", web search for fresh facts, calculator for math. This is the gateway drug to agentic RAG (next page).

ROUTER = """Classify the question into exactly one of: VECTOR, SQL, WEB, MATH.

Q: {q}
Class:"""

def route(q):
    return llm.complete(ROUTER.format(q=q)).strip()