AIWorkJobs
Braintrust logoGoogle DeepMindListed on Braintrust

Code Quality Specialist / Research Engineer

Review AI coding attempts and determine whether failures stem from the code, benchmark tests, task instructions, or execution environment. You’ll examine Python and other code, test behavior, and run logs, then…

Remote · United States onlyContract
Work arrangement
Remote
Eligibility
United States only
Engagement
Contract
Compensation
$75–$105/hr
Specialty
Quality Assurance & Testing
Experience
Under 3 years

About the role

Review AI coding attempts and determine whether failures stem from the code, benchmark tests, task instructions, or execution environment. You’ll examine Python and other code, test behavior, and run logs, then document evidence-based conclusions. The contract requires 35 hours per week and is open to candidates in the United States. Work starts immediately.

What you would do

  • Examine code, tests, execution logs, and the steps an AI coding agent took to complete a task.
  • Investigate benchmark concerns such as missing definitions, undocumented expectations, and restrictive test assertions.
  • Classify reported problems as genuine flaws, issues that still allow reasonable solutions to pass, or mistakes in the AI’s implementation.
  • Assess whether an identified issue changed the task’s pass or fail result.
  • Explain conclusions with relevant excerpts from task instructions, tests, and run logs.
  • Diagnose test harness and environment problems by reviewing Docker configuration, dependencies, and execution logs.
  • Compare findings with research scientists in daily calibration discussions and align evaluation decisions.
  • Use review tools and technical resources to investigate individual claims, which take about 30 to 60 minutes each.

Requirements

  • Have either 2 or more years of professional experience writing, reviewing, and debugging software, or a completed STEM master’s degree or PhD with intensive programming and computational research experience.
  • Show strong Python code-reading, debugging, and comprehension skills, including familiarity with complex frameworks and test assertions.
  • Read at least one of JavaScript or TypeScript, Go, C++, or Rust.
  • Have practical experience with Git and Docker.
  • Know unit-testing frameworks such as pytest, unittest, Jest, or Mocha, and software build environments.
  • Assess edge cases, ambiguous requirements, and inconsistencies in test logic.
  • Communicate technical findings in fluent professional English, both in writing and conversation.
  • Be located in the United States and able to commit 35 hours per week.

Preferred qualifications

The source marks these as preferred, not required.

  • Experience with LLM benchmarking, AI evaluation, prompt engineering, or coding-agent workflows.
  • Familiarity with SWE-bench or CodeContests.
  • Experience in AI red teaming, safety, or reliability auditing.
  • Previous collaboration with AI or machine-learning research scientists.

Who should apply

Candidates with professional software debugging experience or a STEM graduate degree and substantial computational research experience may suit this work. Strong Python analysis, familiarity with another programming language, and clear technical communication are key.

How applying works on Braintrust

Create a Braintrust profile, complete the AI skills interview, then browse and accept tasks from the dashboard.

  1. Create and complete a Braintrust profile.
  2. Complete the AI skills interview used for expertise and pay-level matching.
  3. Browse the dashboard for available tasks.
  4. Accept a task and complete the work.

Typical timeline: A new member profile takes about 20 minutes.

Paid steps: Not stated by the source

Individual client projects can add their own interview, assessment, or identity check.

Process checked Nov 19, 2026. AIWorkJobs does not submit applications or charge a fee. Read the full Braintrust process and its official source on the Braintrust page.

Skills mentioned by the source

  • python
  • git
  • docker
  • javascript
  • typescript
  • go
  • c++
  • rust
  • pytest
  • unittest
  • jest
  • mocha
  • unit testing
  • software debugging
  • code review
  • test assertions

Specialty areas

Specialties come from the source platform’s category taxonomy and are used to filter the job index.

Listing reference: code-quality-specialist-research-engineer-18169. Source: Braintrust. If something here contradicts the source, the source controls. Report a correction.

Related roles

Other work on the index.

See more in this category

Turing · via Braintrust

Senior Software Engineer - AI Code Quality

Use production engineering expertise to assess AI-generated code, trace defects to their causes, and create clear reference implementations for AI training. The work centers on software engineering and code evaluation, not manual testing. The project runs for three months and requires at least 40 hours per week, including four hours of daily Pacific Time overlap.

Location and eligibility
Remote · Eligible in 9 countries
Engagement type
Contract
Compensation
$23/hr
  • software engineering
  • code review
  • python
Quality Assurance & TestingView job

Handshake AI

Senior Software Engineer - AI Data (US)

Work with coding agents on real software engineering problems to pinpoint errors in frontier AI models and assess their output. The contract offers 40 hours of work each week, with flexible scheduling and weekly renewal tied to quality standards. The role requires US work authorization, 4+ years of professional software engineering experience, and a computer science or related degree. No AI background is needed.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
$150/hr
  • software engineering
  • coding agents
  • code review
Software EngineeringView job

Turing

Software Engineer – AI Research & Evaluation (US-based)

Create and evaluate datasets to enhance AI-generated coding solutions.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Pay not listed
  • python
  • javascript
  • java
Machine Learning & AIView job