Data preparation
From raw engineering history to training-ready data.
Preparation is often the difference between data that can be licensed and data that can’t. We work with suppliers getting data ready, and with buyers who need it in a specific shape.
Pipeline
From raw history to a documented dataset
- 01IntakeScope, schema, and permitted uses agreed in writing
- 02ScrubSecrets and personal data found and removed
- 03Normalise & filterOne schema; licence filtering; de-duplication
- 04ValidateLinks and tests checked; contamination screened on request
- 05PackageSplits and dataset card in a private, read-only repository
ServicesFor
- Secret & credential scanning
- Detect keys, tokens, and passwords in code and history; report findings so owners can rotate them. Suppliers
- PII removal
- Remove or pseudonymise names, emails, and identifiers in commits, issues, and reviews. Suppliers
- Normalisation
- Map repositories, trackers, and review systems to one documented schema. Both
- Licence filtering
- Identify file-level licences and exclude material outside the permitted scope. Both
- De-duplication
- Exact and near-duplicate removal across sources and against public corpora where requested. Buyers
- Linking & validation
- Link issues to fixes and tests; re-run tests to confirm before/after behaviour where feasible. Both
- Contamination screening
- On request: screen evaluation items against known public benchmarks. Buyers
- Packaging & documentation
- Train/eval splits and dataset cards, delivered as a private read-only repository. Both
How we work
Scoped, documented, and secure
Scoped in writing
Each engagement has a written scope, deliverables, and price before work starts.
Documented
You receive a record of every transformation and check applied, and what it did not cover.
Access-controlled
Data is handled only in access-controlled environments under an agreed retention period.
Discuss a preparation project.
Share what you need or what you own. We reply with next steps — or tell you honestly if we can’t help.