RLM + GEPA for Retrieval
x

u/adnanqidwai/projects/rlm-gepa-retrieval.md

RLM + GEPA for Retrieval

RLM + GEPA for Retrieval is a trace-grounded retrieval-policy harness. A DSPy RLM answers questions over a local Markdown corpus by calling host-backed search and document tools, while evaluation scores citation support, answer terms, and budget use from the recorded tool trace.

What It Builds

  • Split-disjoint Markdown QA task bank with required answer terms and gold support passage IDs.
  • Host-backed tools for lexical search, document opening, and targeted in-document search with recorded execution traces.
  • Deterministic controls: a lexical heuristic and a single-shot RAG baseline.
  • Trace-aware scoring for answer correctness, evidence recall, citation precision, and budget efficiency.
  • GEPA optimization path for improving the textual retrieval policy from deterministic failure feedback.

Why It Matters

  • Keeps retrieval behavior auditable through execution traces rather than model self-reporting.
  • Compares agentic retrieval against retrieve-once RAG before claiming policy improvements.
  • Turns citation and budget errors into concrete feedback for policy optimization.
  • Provides a small local setup for studying agentic retrieval without requiring a vector database or LLM judge.
  • retrieval
  • RAG
  • citations
  • budget scoring
  • DSPy
  • GEPA
  • single-shot RAG
  • tool traces