AIWorkJobs
Turing logoTuringPlatform listing

Software Engineer – AI Code Evaluation & Benchmarking (US candidates only)

Evaluate and improve AI-generated code for accuracy and quality.

Remote · United States onlyContract
Work arrangement
Remote
Eligibility
United States only
Engagement
Contract
Compensation
Pay not listed
Specialty
AI Training

About the role

Review and test AI-generated code against real software engineering tasks, then help build the datasets, benchmarks, and grading rubrics used to assess coding models. The work calls for sound judgment in code review, debugging, and technical evaluation. This is a one-month contractor assignment for US-based candidates, with a minimum weekly commitment and PST overlap.

What you would do

  • Review AI-generated code for correctness, efficiency, maintainability, and fit with stated requirements.
  • Check software engineering solutions against expected results and task criteria.
  • Reproduce bugs, debug code, and confirm fixes across programming environments.
  • Evaluate the technical accuracy of model-generated explanations, reasoning, and implementation plans.
  • Build, update, and maintain coding evaluation datasets, benchmarks, and grading rubrics.
  • Find edge cases, common failures, and areas where AI models struggle with software engineering tasks.
  • Record findings and give structured feedback to support consistent evaluation.
  • Work with project teams to set quality standards and evaluation methods.

Requirements

  • Bachelor's or Master's degree in Computer Science, Software Engineering, or a related technical field.
  • At least 3 years of professional software engineering experience.
  • Strong proficiency in one or more of these languages: Python, Java, C/C++, Go, Swift, Objective-C, PHP, or SQL.
  • Solid knowledge of data structures, algorithms, software design principles, and debugging methods.
  • Experience reviewing code and judging quality in production or large-scale codebases.
  • Ability to assess complex technical problems and solution correctness with minimal supervision.
  • Familiarity with version control such as Git and modern software development workflows.
  • Strong written communication and attention to detail.
  • Commit to at least 4 hours per day and 20 hours per week, including 4 hours of overlap with PST.
  • Complete an online automated coding challenge that tests Python and Docker (RHLF).

Preferred qualifications

The source marks these as preferred, not required.

  • Experience with AI/ML data annotation, NLP, prompt engineering, model evaluation, or LLM projects.
  • Experience evaluating AI-generated code, creating benchmarks, or assessing software quality is highly preferred.

Who should apply

Software engineers with at least 3 years of professional experience and a background in code review, debugging, and production-quality software will fit the work. Applicants must be based in the US, meet the stated weekly commitment and PST overlap, and complete the Python and Docker coding challenge.

How applying works on Turing

Create a profile, finish any required proctored assessments, then complete screening or a role interview.

  1. Apply through work.turing.com and complete your profile.
  2. Finish the required proctored assessments for the role.
  3. Complete shortlisting, screening, or a technical interview.
  4. Confirm rate, availability, and eligibility before an offer.

Typical timeline: Review takes 48 hours to six weeks after all required steps; consideration replies are usually within 12 hours.

Paid steps: Not stated by the source

Not every role requires tests. Turing does not review an application until required assessments are complete.

Process checked Nov 19, 2026. AIWorkJobs does not submit applications or charge a fee. Read the full Turing process and its official source on the Turing page.

Skills mentioned by the source

  • python
  • git
  • docker
  • java
  • c
  • c++
  • go
  • swift
  • objective-c
  • php
  • sql
  • data structures
  • algorithms
  • code review
  • debugging
  • software design

Specialty areas

Specialties come from the source platform’s category taxonomy and are used to filter the job index.

Listing reference: software-engineer-ai-code-evaluation-benchmarking-us-candidates-only-91017. Source: Turing. If something here contradicts the source, the source controls. Report a correction.

Related roles

Other work on the index.

See more in this category

Turing

Software Engineer – AI Research & Evaluation (US-based)

Create and evaluate datasets to enhance AI-generated coding solutions.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Pay not listed
  • python
  • javascript
  • java
Machine Learning & AIView job

Turing · via Braintrust

Senior Software Engineer - AI Code Quality

Use production engineering expertise to assess AI-generated code, trace defects to their causes, and create clear reference implementations for AI training. The work centers on software engineering and code evaluation, not manual testing. The project runs for three months and requires at least 40 hours per week, including four hours of daily Pacific Time overlap.

Location and eligibility
Remote · Eligible in 9 countries
Engagement type
Contract
Compensation
$23/hr
  • software engineering
  • code review
  • python
Quality Assurance & TestingView job

Handshake AI

Senior Software Engineer - AI Data (US)

Work with coding agents on real software engineering problems to pinpoint errors in frontier AI models and assess their output. The contract offers 40 hours of work each week, with flexible scheduling and weekly renewal tied to quality standards. The role requires US work authorization, 4+ years of professional software engineering experience, and a computer science or related degree. No AI background is needed.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
$150/hr
  • software engineering
  • coding agents
  • code review
Software EngineeringView job

Turing

Software Engineer – AI Research & Evaluation (US-based)

Create and evaluate datasets for training large language models in Python.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Pay not listed
  • python
  • javascript
  • reactjs
Machine Learning & AIView job