AIWorkJobs
Mercor logoMercorPlatform listing

AI Developer Trace Task Auditor

Review coding sessions created with AI developer tools for a frontier AI lab’s model training and evaluation work. You’ll judge whether each trace reaches a correct result, follows a sound development process, and…

Remote · United States onlyHourly
Work arrangement
Remote
Eligibility
United States only
Engagement
Hourly
Compensation
$70–$90/hr
Specialty
AI Training
Weekly commitment
40 hours per week

About the role

Review coding sessions created with AI developer tools for a frontier AI lab’s model training and evaluation work. You’ll judge whether each trace reaches a correct result, follows a sound development process, and shows sound reasoning. Your written feedback should explain your assessment and follow the provided rubric.

What you would do

  • Assess end-to-end software development traces produced with AI-assisted developer tools.
  • Check code and outcomes for correctness across multi-step coding sessions.
  • Evaluate workflow choices and reasoning against sound development practices.
  • Write clear, rubric-based feedback on each trace.

Requirements

  • At least 3 years of professional software development experience.
  • Hands-on experience with AI coding tools such as Cursor, GitHub Copilot, or Claude Code, or similar tools.
  • Experience with agentic or spec-driven development workflows.
  • Strong code-reading and debugging skills across full-stack or backend systems.
  • Ability to assess multi-step coding trajectories for correctness and development best practices.

Preferred qualifications

The source marks these as preferred, not required.

  • Experience with Kiro or Amazon CodeCatalyst.
  • Previous experience evaluating or grading AI-generated code.
  • Contributions to developer tooling.

Who should apply

Software developers with at least three years of professional experience and hands-on familiarity with AI coding tools can bring the code judgment this work needs. Experience reviewing multi-step workflows, including agentic or spec-driven development, will help you assess both results and process.

How applying works on Mercor

Create a profile, complete the role checklist and AI interview, then verify your identity before a project offer.

  1. Create an account and confirm your résumé.
  2. Complete the role checklist, written assessment, and AI interview where the listing requires them.
  3. Submit the application and wait for review or matching.
  4. Complete a work trial and identity verification before onboarding.

Typical timeline: The AI interview is about 20 minutes; work trials are often 4–6 hours; identity checks are usually minutes.

Paid steps: The work trial is paid.

Assessment and interview requirements vary by role. Check the checklist inside the specific listing.

Process checked Nov 19, 2026. AIWorkJobs does not submit applications or charge a fee. Read the full Mercor process and its official source on the Mercor page.

Skills mentioned by the source

  • software development
  • ai-assisted coding
  • cursor
  • github copilot
  • claude code
  • agentic workflows
  • spec-driven development
  • code review
  • debugging
  • full-stack development
  • backend development
  • ai-generated code evaluation

Specialty areas

Specialties come from the source platform’s category taxonomy and are used to filter the job index.

Listing reference: ai-developer-trace-task-auditor-NKaJl4. Source: Mercor. If something here contradicts the source, the source controls. Report a correction.

Related roles

Other work on the index.

See more in this category

Turing · via Braintrust

Senior Software Engineer - AI Code Quality

Use production engineering expertise to assess AI-generated code, trace defects to their causes, and create clear reference implementations for AI training. The work centers on software engineering and code evaluation, not manual testing. The project runs for three months and requires at least 40 hours per week, including four hours of daily Pacific Time overlap.

Location and eligibility
Remote · Eligible in 9 countries
Engagement type
Contract
Compensation
$23/hr
  • software engineering
  • code review
  • python
Quality Assurance & TestingView job

Meridial by Invisible

Technical Task Auditor

Audit technical tasks used to train and evaluate AI systems as an independent contractor. You will focus on one specialty that matches your experience, such as GPU kernels, Kubernetes, machine learning, or application security. Your reviews will check task quality, reproducibility, tests, and evaluation criteria, and you will provide actionable feedback on any problems.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$60/hr
  • gpu kernels
  • aws trainium
  • nki
Quality Assurance & TestingView job

SME Careers

C++ Team Lead

Lead quality checks for C++ AI training work by reviewing generated code, debugging answers, algorithms, tests, and contributor feedback against project rubrics. You will guide remote trainers and reviewers, flag safety or correctness risks, and maintain onboarding and QA materials. This contract role has no immediate project; qualified candidates may hear about future assignments. The selection process includes an AI interview, a domain task, and a recruiter interview.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $75/hr
  • c++
  • ai training
  • llm evaluation
Quality Assurance & TestingView job