Mercor
SWE-Bench Task Auditor
Review repository-level software engineering benchmarks used to train and evaluate a frontier AI lab’s models. You’ll examine task quality, reference patches, test harnesses, Docker isolation, and grading integrity, then write clear feedback against a rubric. The role calls for professional software engineering experience and open-source contribution or maintainer work. The listing is limited to the United States.
- Location and eligibility
- Remote · United States only
- Engagement type
- Hourly
- Compensation
- $70–$90/hr
- python
- java
- go

