u/adnanqidwai/projects/patchgym.md
PatchGym
PatchGym is a deterministic coding-agent evaluation harness and RL-style environment. It generates small multi-file Python repositories with seeded bugs, exposes read/edit/test/submit episode actions, and verifies repairs with public tests, hidden tests, API-contract checks, and edit-budget checks.
What It Builds
- Task generators for parser, CLI, and SQLite mini-repository templates.
- Verifier stack with public tests, hidden tests, API-contract checks, and patch-budget checks.
- RL-style reset/step API with read_file, write_file, run_public_tests, and submit_patch actions.
- Agent runner that stores trace logs and final repository snapshots.
- Optional Prime Intellect verifiers adapter and DSPy RLM + GEPA patch-planning path.
Why It Matters
- Separates visible-test success from hidden-test and API-contract failures.
- Makes coding-agent repairs measurable beyond a single pass/fail test run.
- Uses deterministic feedback so prompt optimization can be evaluated on held-out tasks.
- coding agents
- RL environment
- verifiers
- hidden tests
- Prime Intellect
- GEPA
- DSPy RLM