PatchGym
x

u/adnanqidwai/projects/patchgym.md

PatchGym

PatchGym is a deterministic coding-agent evaluation harness and RL-style environment. It generates small multi-file Python repositories with seeded bugs, exposes read/edit/test/submit episode actions, and verifies repairs with public tests, hidden tests, API-contract checks, and edit-budget checks.

What It Builds

  • Task generators for parser, CLI, and SQLite mini-repository templates.
  • Verifier stack with public tests, hidden tests, API-contract checks, and patch-budget checks.
  • RL-style reset/step API with read_file, write_file, run_public_tests, and submit_patch actions.
  • Agent runner that stores trace logs and final repository snapshots.
  • Optional Prime Intellect verifiers adapter and DSPy RLM + GEPA patch-planning path.

Why It Matters

  • Separates visible-test success from hidden-test and API-contract failures.
  • Makes coding-agent repairs measurable beyond a single pass/fail test run.
  • Uses deterministic feedback so prompt optimization can be evaluated on held-out tasks.
  • coding agents
  • RL environment
  • verifiers
  • hidden tests
  • Prime Intellect
  • GEPA
  • DSPy RLM