Expert-authored data, off the shelf or built to your spec.
Start from a pre-built collection, or co-design data and environments with our applied AI research team for specific domains and failure surfaces.
Pre-built datasets
Datasets and RL environments authored by engineers from our network, with tasks, verifiers, and trajectories, in formats you already run.
- Request samplesTerminal-Bench 4.0 Core
- Request samplesSWE-Bench Pro Hard
- Request samplesSWE-Lancer Hard
More collections on request. Tell us what you’re training for.
Custom data
Our applied AI research team co-designs data with your researchers for specific domains and failure surfaces. We deliver in the formats you run, on your platform or ours. Custom data is built for your lab only, unless we agree otherwise.
Task specification and rubric design
A researcher works through your training objective with your team, then sets the domain, the task format, and the rubrics each task is graded against.
Dataset authoring
SFT demonstrations, agent trajectories, and preference data, authored or improved by engineers who specialize in that domain.
RL environment development
Containerized tasks with verifiers and proof of solvability, built for the tools and repositories your agent runs.
Benchmark and eval expansion
New tasks in the formats of public benchmarks you already run, task collections calibrated to a difficulty you set, and evals scored against your rubric.
Expert review and provenance
We select and manage the experts. Every task is graded by tests or rubrics, reviewed by someone who does that work, and delivered with a record of who authored and reviewed it.
Case studies
Custom data programs we ran for AI labs.
Full-trace data for a coding agent
Read the full storyThe problem
The lab needed full traces of engineers resolving real GitHub issues, reasoning included. No collection method existed yet, and the tasks needed senior Python engineers with experience in specific repositories, from Pandas to Qiskit.
What we built
We co-designed the workflow with the lab’s researchers. Each task started from a Dockerfile that reproduced the repository state and failing tests that reproduced the issue. A VS Code extension recorded the engineer’s actions and reasoning at key moments. Engineers were matched by repository experience and paid by the quality of each submission.
daily trace collections, from 10 to 100 in two months
average handle time per task, down from 330
average quality rating, up from 70%
follow-on projects the lab started with us
More case studies
approval rate, up from 54%
Interface generation
A design-first workflow for a lab evaluating front-end generation. Designs were approved before any code was written.
Read the storyto 1,000+ expert-evaluated tasks, across 25 parallel queues
Code evals across models
Task-specific rubrics that turned clarity, scalability, and design into criteria a reviewer can verify, for benchmarking across models.
Read the storyWho does the work
For more than ten years, Revelo has hired, tested, and placed senior engineers in real jobs. Today, those same engineers author the datasets, RL environments, and evals that leading AI labs use to train and evaluate their models.
Hired and tested
Our tasks are authored by engineers who passed our hiring process and hold real jobs.
Matched by specialty
Each task goes to engineers who do that kind of work in their own jobs.
Graded and reviewed
Every task is graded by tests or rubrics and reviewed by someone who does that work.
Your collections share names with public benchmarks. Are they the same tasks?
No. They follow the same formats, but our engineers author new tasks, many from merged pull requests that fixed real issues.
What do I receive?
Each RL environment ships as a container with the task, a verifier, proof that it can be solved, and its pass rate. SFT data includes chain-of-thought, diffs, and trajectories where you need them.
How is task difficulty calibrated?
A solvability review first removes tasks that are broken or ambiguous. We then run frontier models against the collection before it ships and report the pass rate, with the model, the attempt count, and the grading method attached.
What formats do datasets ship in?
SWE-bench style task sets, Harbor-format terminal tasks, agent trajectories, and DPO-ready preference pairs. If you need a format that isn’t on that list, we’ll match it.
How fast can a custom project start?
Most projects launch in two days, and first samples are ready in under 72 hours, depending on the complexity. We work with your current data formats and processes, so you don’t need to change anything.
Can we pilot with a small batch first?
Yes. Most clients start with a pilot, usually a few hundred evaluations across different problem types. It lets us calibrate to your needs and show our quality before scaling up.
How involved do we need to be?
As much or as little as you want. Some labs are hands-off and want the data; others collaborate closely on evaluation criteria. We recommend initial calibration sessions to align on quality standards.
Can you work inside our stack and QA process?
Yes. Our engineers can work in your tools and follow your QA steps, or deliver finished data from our platform with our own review. You can also combine the two.
Can you build custom benchmarks and evals for our use case?
Yes. We build task collections calibrated to a difficulty you set, execution-graded environments, and evals scored against your rubric.
Who are your experts, and how are they vetted?
Engineers from our network of 400,000+. Each one passes technical assessments, manual reviews, and fraud checks, the same vetting we’ve used for more than ten years of hiring. We then match each task to engineers who do that kind of work in their own jobs.
Put our engineering expertise to work.
A researcher will help you plan the data, environments, and evals your model needs.