Solution

Code model pre-training data

Source licensed, rights-reviewed software repositories and engineering history for code LLM pre-training, with provenance records, de-duplication, and secret scrubbing.

The problem

Why this is hard

Public code corpora are saturated, license-ambiguous, and heavily contaminated with benchmark solutions. Teams that need defensible provenance for pre-training data struggle to prove who owned the code and what uses were permitted.

What we source

Data types

  • Private and first-party repositories licensed by their legal owners
  • Commit histories with messages, diffs, and authorship metadata
  • Documentation, READMEs, and architecture decision records linked to code
  • Language- and framework-specific slices (for example Python services, TypeScript front ends, Go infrastructure)

What you receive

Deliverables

  • Read-only access to a private copy of the reviewed repository, with a documented schema and dataset card
  • Per-source rights record: owner, inbound licenses, restrictions, permitted uses
  • A secret scan of the repository copy before anyone is given access
  • De-duplication and benchmark-contamination screening on request, agreed in the proposal

Quality

Checks on every delivery

  • Ownership and authority review
  • License and restriction review
  • Secret and PII scanning
  • Near-duplicate and contamination screening on request

FAQ

Code model pre-training data: common questions

Can you license open-source code for pre-training?
We never assume public code can be relicensed for any purpose. Each source is reviewed against its license, contributor agreements, and your intended use, and the review scope is recorded.
What volume can you supply?
It depends on language, domain, and rights position. Share your target and we will tell you honestly what is achievable and over what timeline.

Tell us what your models need to learn.

Share what you need or what you own. We reply with next steps — or tell you honestly if we can’t help.