Data types
Why issue-to-fix histories are useful for coding agents
Linked issues, patches, and tests capture the full loop from problem to verified solution — the loop coding agents need to learn.
Much publicly available code shows the end state of a program. Coding agents, however, are asked to move a codebase from a broken state to a working one. Data that captures that transition is structurally different from a snapshot of source files.
The anatomy of a good record
- A problem statement written by a person, in the language real users and engineers use.
- The repository state before the change, so the task is reproducible.
- The change set that resolved the problem, ideally with intermediate commits.
- Tests that fail before the change and pass after it, providing an executable check.
Why private repositories matter
Commercial codebases often have long-lived services, internal conventions, and complex test suites that look different from small open-source projects. When the owner has the right to license them, they can broaden the range of engineering situations represented in training and evaluation data.
Quality questions to ask
- Are issues linked to fixes by explicit references, or by heuristics that may be wrong?
- Can the tests actually be run, and were they re-executed during preparation?
- Were secrets, credentials, and personal data removed — and how was that verified?
- Is there overlap with public benchmarks that could contaminate evaluation?