Advanced RAG Architectures: Hybrid Search (BM25+Dense), Cross-Encoder Re-Ranking, HyDE & The RAG Triad

Advanced RAG Architectures & Evaluation

Basic vector retrieval often fails when users query exact model numbers, acronyms, or complex questions. Production RAG relies on hybrid search, re-ranking, and formal evaluation frameworks.

#1. Hybrid Search & Reciprocal Rank Fusion (RRF)

Combines sparse BM25 keyword matching with dense vector search:

where rₘ(d) is document d's rank in retriever m, and k ≈ 60 is a smoothing constant.

User Query ──┬──> [ Dense Vector Retrieval (HNSW) ] ──> Rank List A ──┐
             │                                                        ├──> [ RRF Fusion ] ──> Top-K
             └──> [ Sparse Keyword Retrieval (BM25) ] ──> Rank List B ──┘

#2. Cross-Encoder Re-Ranking

Bi-encoders generate query and document vectors independently. A Cross-Encoder processes the query and candidate chunk jointly through full self-attention, re-ranking top 50 candidates down to top 5.

#3. HyDE (Hypothetical Document Embeddings)

Prompts an LLM to generate a hypothetical answer first, embeds the answer, and searches the vector store for matching document chunks.

#4. The RAG Triad Evaluation Framework

from sentence_transformers import CrossEncoder
reranker = CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
scores = reranker.predict([[query, chunk] for chunk in candidate_chunks])