AIWorkJobs
Turing logoTuringPlatform listing

Senior Software Engineer – Python (LLM Evaluation & Repository Validation)

Analyze and triage GitHub issues while evaluating software test quality.

Remote · Eligibility not specifiedContract
Work arrangement
Remote
Eligibility
Eligibility not specified
Engagement
Contract
Compensation
Pay not listed
Specialty
Software Engineering

About the role

Build and assess realistic software engineering tasks for LLM evaluation and training datasets. You will work with public GitHub repositories, triage issues, configure development environments, and judge unit test quality. The role also involves testing how LLMs handle bug fixes and working with researchers to select challenging repositories and issues.

What you would do

  • Analyze and triage GitHub issues in trending open-source libraries.
  • Set up repositories and configure development environments, including Dockerization.
  • Assess unit test coverage and quality.
  • Run and modify codebases to evaluate LLM performance on bug-fixing tasks.
  • Work with researchers to identify repositories and issues that challenge LLMs.
  • Lead junior engineers on project work when needed.

Requirements

  • At least 3 years of overall experience.
  • Strong experience with Python.
  • Proficiency with Git, Docker, and basic software pipeline setup.
  • Ability to understand and navigate complex codebases.
  • Comfort running, modifying, and testing real-world projects on a local machine.
  • Availability for at least 20 hours per week, with 4 hours of overlap with PST each day.

Preferred qualifications

The source marks these as preferred, not required.

  • Experience contributing to or evaluating open-source projects.
  • Previous participation in LLM research or evaluation projects.
  • Experience building or testing developer tools or automation agents.

Who should apply

Software engineers with at least 3 years of experience and strong Python skills will fit the role. Candidates should be comfortable working with complex repositories, Git, Docker, and software tests, and able to meet the stated weekly hours and PST overlap.

How applying works on Turing

Create a profile, finish any required proctored assessments, then complete screening or a role interview.

  1. Apply through work.turing.com and complete your profile.
  2. Finish the required proctored assessments for the role.
  3. Complete shortlisting, screening, or a technical interview.
  4. Confirm rate, availability, and eligibility before an offer.

Typical timeline: Review takes 48 hours to six weeks after all required steps; consideration replies are usually within 12 hours.

Paid steps: Not stated by the source

Not every role requires tests. Turing does not review an application until required assessments are complete.

Process checked Nov 19, 2026. AIWorkJobs does not submit applications or charge a fee. Read the full Turing process and its official source on the Turing page.

Skills mentioned by the source

  • python
  • git
  • docker
  • software testing
  • unit testing
  • test coverage
  • github
  • open-source software
  • software pipelines
  • llm evaluation
  • repository validation
  • development environment setup

Specialty areas

Specialties come from the source platform’s category taxonomy and are used to filter the job index.

Listing reference: senior-software-engineer-python-llm-evaluation-repository-validation-37595. Source: Turing. If something here contradicts the source, the source controls. Report a correction.

Related roles

Other work on the index.

See more in this category