AIWorkJobs
Turing logoTuringPlatform listing

Senior Software Engineer – Ruby (LLM Evaluation & Repository Validation)

Analyze GitHub issues and evaluate LLM performance in software projects.

Remote · Eligibility not specifiedContract
Work arrangement
Remote
Eligibility
Eligibility not specified
Engagement
Contract
Compensation
Pay not listed
Specialty
Software Engineering

About the role

Build and assess datasets that help language models solve realistic software engineering tasks. You will work with public GitHub repositories, triage issues, configure development environments, and judge test quality while checking how models handle bug fixes. The role calls for a senior-level engineer with at least 3 years of experience and strong Ruby skills.

What you would do

  • Review and triage issues across popular open-source GitHub repositories.
  • Set up repositories and configure development environments, including Docker.
  • Assess unit test coverage and test quality.
  • Run and modify codebases to evaluate LLM performance on bug-fixing tasks.
  • Work with researchers to select repositories and issues that challenge language models.
  • Lead junior engineers on project work when needed.

Requirements

  • At least 3 years of overall professional experience.
  • Strong experience with Ruby.
  • Proficiency with Git and Docker.
  • Knowledge of basic software pipeline setup.
  • Ability to understand and navigate complex codebases.
  • Comfort running, modifying, and testing real-world software projects.

Preferred qualifications

The source marks these as preferred, not required.

  • Experience contributing to or assessing open-source projects.
  • Prior participation in LLM research or evaluation.
  • Experience building or testing developer tools or automation agents.

Who should apply

Senior software engineers with strong Ruby experience and at least 3 years in the field will suit this work. Candidates who know Git and Docker and can assess complex repositories, tests, and bug-fixing results are a strong fit.

How applying works on Turing

Create a profile, finish any required proctored assessments, then complete screening or a role interview.

  1. Apply through work.turing.com and complete your profile.
  2. Finish the required proctored assessments for the role.
  3. Complete shortlisting, screening, or a technical interview.
  4. Confirm rate, availability, and eligibility before an offer.

Typical timeline: Review takes 48 hours to six weeks after all required steps; consideration replies are usually within 12 hours.

Paid steps: Not stated by the source

Not every role requires tests. Turing does not review an application until required assessments are complete.

Process checked Nov 19, 2026. AIWorkJobs does not submit applications or charge a fee. Read the full Turing process and its official source on the Turing page.

Skills mentioned by the source

  • ruby
  • git
  • docker
  • software pipelines
  • github
  • unit testing
  • test coverage
  • llm evaluation
  • repository validation
  • open-source software

Specialty areas

Specialties come from the source platform’s category taxonomy and are used to filter the job index.

Listing reference: senior-software-engineer-ruby-llm-evaluation-repository-validation-37600. Source: Turing. If something here contradicts the source, the source controls. Report a correction.

Related roles

Other work on the index.

See more in this category

SME Careers

Ruby Engineer

Review AI-generated Ruby and Rails code, write model solutions, and explain whether each answer follows sound reasoning and idiomatic practices. You will assess application structure, readability, and correctness, then compare responses for quality. The role is remote and contract-based, with no immediate project; qualified engineers may be contacted when relevant work becomes available.

Location and eligibility
Remote · Eligible in 17 countries
Engagement type
Contract
Compensation
Up to $55/hr
  • ruby
  • ruby on rails
  • mvc
Machine Learning & AIView job