AIWorkJobs
Turing logoTuringPlatform listing

Senior Software Engineer – C++ (LLM Evaluation & Repository Validation)

Analyze GitHub issues and evaluate test quality for LLM projects.

Remote · Eligibility not specifiedContract
Work arrangement
Remote
Eligibility
Eligibility not specified
Engagement
Contract
Compensation
Pay not listed
Specialty
Software Engineering

About the role

Build and assess LLM training and evaluation datasets from public software repositories. You will triage GitHub issues, prepare repository environments, review unit test quality, and run code to assess how well models handle bug fixes. The role calls for at least 3 years of experience and offers 20, 30, or 40 hours of work per week, with PST overlap.

What you would do

  • Review and triage issues across active open-source libraries.
  • Configure code repositories and prepare development environments, including Docker setup.
  • Assess unit test coverage and the quality of tests in target repositories.
  • Run and modify codebases on your machine to evaluate LLM performance on bug-fixing tasks.
  • Work with researchers to select repositories and issues that challenge LLMs.
  • Help lead junior engineers on project work when needed.

Requirements

  • At least 3 years of overall experience as a software engineer, at tech lead level.
  • Strong experience with C++.
  • Proficiency with Git and Docker.
  • Experience setting up basic software pipelines.
  • Ability to navigate complex codebases and run, modify, and test real-world projects on your machine.
  • Commit to at least 20 hours per week, with at least 4 hours of overlap with PST each day.

Preferred qualifications

The source marks these as preferred, not required.

  • Experience contributing to or evaluating open-source projects.
  • Prior involvement in LLM research or evaluation.
  • Experience building or testing developer tools or automation agents.

Who should apply

Software engineers with 3 or more years of experience, strong C++ skills, and confidence working in complex public repositories suit this assignment. Candidates must be available for at least 20 hours each week and provide 4 hours of daily overlap with PST.

How applying works on Turing

Create a profile, finish any required proctored assessments, then complete screening or a role interview.

  1. Apply through work.turing.com and complete your profile.
  2. Finish the required proctored assessments for the role.
  3. Complete shortlisting, screening, or a technical interview.
  4. Confirm rate, availability, and eligibility before an offer.

Typical timeline: Review takes 48 hours to six weeks after all required steps; consideration replies are usually within 12 hours.

Paid steps: Not stated by the source

Not every role requires tests. Turing does not review an application until required assessments are complete.

Process checked Nov 19, 2026. AIWorkJobs does not submit applications or charge a fee. Read the full Turing process and its official source on the Turing page.

Skills mentioned by the source

  • c++
  • git
  • docker
  • software pipelines
  • github
  • unit testing
  • test coverage
  • open-source software
  • llm evaluation
  • repository analysis

Specialty areas

Specialties come from the source platform’s category taxonomy and are used to filter the job index.

Listing reference: senior-software-engineer-c-llm-evaluation-repository-validation-37598. Source: Turing. If something here contradicts the source, the source controls. Report a correction.

Related roles

Other work on the index.

See more in this category