AIWorkJobs
Turing logoTuringPlatform listing

Senior Software Engineer - AI Evaluation / Coding Agents

Evaluate and improve AI coding models by reviewing generated code.

Remote · Eligibility not specifiedContract
Work arrangement
Remote
Eligibility
Eligibility not specified
Engagement
Contract
Compensation
$100–$150/hr
Specialty
AI Training

About the role

Assess AI coding agents against real-world repositories rather than build production applications. You’ll judge whether generated code solves the task, identify technical errors and agent failure modes, and create evaluation criteria and feedback that support model improvement. The freelance engagement runs for about three months and calls for 20–40 hours per week.

What you would do

  • Create and refine rubrics and evaluation criteria for preference data.
  • Review and label evaluation data, catching subtle quality issues.
  • Build and maintain pipelines and infrastructure for data generation, collection, and evaluation.
  • Turn evaluation findings into clear updates, write-ups, and recommendations.
  • Work with researchers and engineers to turn technical judgment into repeatable processes.
  • Assess whether coding agents understand tasks, choose suitable approaches, and produce correct, maintainable code.

Requirements

  • At least 5 years of hands-on software engineering experience.
  • Strong skills in Python, TypeScript, JavaScript, Go, or another major production language.
  • Experience with substantial real-world codebases.
  • Strong code review skills and technical judgment.
  • Ability to explain why an implementation works, fails, or needs improvement.
  • Strong written communication.
  • Experience with modern LLMs or AI coding tools.
  • Availability for 20–40 hours per week, including at least 6 hours of Pacific Time overlap per day.
  • Ability to complete an AI interview, a practical code and AI evaluation exercise, and a hiring manager interview. The practical exercise tests review of AI-generated code, not competitive programming or algorithm puzzles.

Preferred qualifications

The source marks these as preferred, not required.

  • Experience with LLM evaluation, coding agents, RLHF, preference data, rubric design, or post-training.

Who should apply

Software engineers with at least five years of production experience, strong code review judgment, and hands-on familiarity with modern LLMs or AI coding tools may suit this work. Candidates must be based in North America and able to meet the stated weekly availability and Pacific Time overlap.

How applying works on Turing

Create a profile, finish any required proctored assessments, then complete screening or a role interview.

  1. Apply through work.turing.com and complete your profile.
  2. Finish the required proctored assessments for the role.
  3. Complete shortlisting, screening, or a technical interview.
  4. Confirm rate, availability, and eligibility before an offer.

Typical timeline: Review takes 48 hours to six weeks after all required steps; consideration replies are usually within 12 hours.

Paid steps: Not stated by the source

Not every role requires tests. Turing does not review an application until required assessments are complete.

Process checked Nov 19, 2026. AIWorkJobs does not submit applications or charge a fee. Read the full Turing process and its official source on the Turing page.

Skills mentioned by the source

  • python
  • typescript
  • javascript
  • go
  • java
  • github
  • ci/cd
  • open source
  • llm
  • code review
  • code analysis
  • llm evaluation
  • coding agents
  • rlhf
  • preference data
  • rubric design

Specialty areas

Specialties come from the source platform’s category taxonomy and are used to filter the job index.

Listing reference: senior-software-engineer-ai-evaluation-coding-agents-104644. Source: Turing. If something here contradicts the source, the source controls. Report a correction.

Related roles

Other work on the index.

See more in this category

Turing

Senior Software Engineer – LLM Evaluation (US-based)

Create and evaluate datasets to enhance large language model performance.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Pay not listed
  • python
  • javascript
  • react
Machine Learning & AIView job

Turing · via Braintrust

Senior Software Engineer - AI Code Quality

Use production engineering expertise to assess AI-generated code, trace defects to their causes, and create clear reference implementations for AI training. The work centers on software engineering and code evaluation, not manual testing. The project runs for three months and requires at least 40 hours per week, including four hours of daily Pacific Time overlap.

Location and eligibility
Remote · Eligible in 9 countries
Engagement type
Contract
Compensation
$23/hr
  • software engineering
  • code review
  • python
Quality Assurance & TestingView job

Handshake AI

Senior Software Engineer - AI Data (CA)

Review coding agents on real engineering tickets, run their code, and assess the results as you would a teammate’s pull request. You’ll also work through a problem in a codebase you know, noting where the model falters. The contract provides 40 hours of work each week, with flexible scheduling, and requires at least four years of professional software engineering experience.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$120/hr
  • software engineering
  • code review
  • coding agents
Software EngineeringView job

Turing

Senior Python Engineer – LLM Evaluation (US-based)

Create and evaluate datasets to enhance large language model performance.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Pay not listed
  • python
  • c++
  • c
Machine Learning & AIView job