AIWorkJobs

Job index

AI / LLM Evaluation jobs

Current roles from specialist AI work platforms. Pay and eligibility are shown as each source publishes them.

Filters2ShowHide
Pay
Reset

SME Careers

Data Scientist Team Lead

As a Data Scientist Quality Assurance Lead, you will review AI-generated data science work and contributor submissions for statistical soundness, code accuracy, reproducibility, and rubric fit. You will guide trainers and QAs, resolve technical questions, and maintain calibration and onboarding resources. This remote contract role has no immediate project; qualified experts may be contacted when relevant work becomes available.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $110/hr
  • data science
  • data science quality assurance
  • llm evaluation
Data Science & AnalyticsView job

Mercor

Responsible AI / Security Specialist

Work on AI training data and evaluations for security and responsible AI, creating adversarial tasks, reviewing solutions, and assessing model responses. Cincinnatus LLC employs specialists as W-2 staff and places them with a leading AI lab. The arrangement includes payroll, benefits, and compliance support, with a weekday commitment of at least 20 hours per week.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
$90–$140/hr
  • cybersecurity
  • responsible ai
  • red teaming
CybersecurityView job

micro1

Teacher (US/CAN based)

Use your teaching expertise to assess educational materials and help develop training data for AI systems. You’ll review AI-generated lessons, quizzes, rubrics, student feedback, and parent communications, then flag accuracy, age-appropriateness, and teaching-practice concerns. No AI background is needed, but you must work full time in education and have at least two years of hands-on teaching experience.

Location and eligibility
Remote · Canada, United States
Engagement type
Contract
Compensation
$20–$80/hr
  • lesson plan evaluation
  • curriculum design
  • assessment writing
Education & TrainingView job

Turing · via Braintrust

Senior Software Engineer - Python, TypeScript, JavaScript or Go

Help an AI research team build dependable evaluation workflows by pairing software engineering with preference-data review and rubric development. The remote contract runs for about three months, pays $100 per hour, and is open to candidates in the United States or Canada. The selection process starts with an information form and AI interview; selected candidates may then complete a coding challenge, with scores of 75% or higher advancing to team review and a possible live interview.

Location and eligibility
Remote · Canada, United States
Engagement type
Contract
Compensation
$100/hr
  • python
  • typescript
  • javascript
Software EngineeringView job

AfterQuery

Applied Statistician (Survey, Actuarial, Quality)

Bring your experience with real operational, survey, or insurance data to the creation of research-level problems that test and train AI models. You will use Python or a similar language and explain technical work in your field. The work follows a flexible, task-based contract with no set weekly schedule. AfterQuery requires standard background and identity checks for accepted experts.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$50/hr
  • statistics
  • biostatistics
  • actuarial science
Data Science & AnalyticsView job

Turing

Strategic Project Lead – Software Engineering

Operate and manage software engineering data programs for AI models.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
Pay not listed
  • project management
  • software engineering
  • python
Software EngineeringView job

Handshake AI

Quantitative Analyst

Use your quant experience on an ongoing, project-based effort to improve AI research. You’ll create realistic quantitative challenges, assess model answers for accuracy and practical value, and write feedback that reflects how working quants approach research, modeling, and strategy. Set your own pace with no fixed schedule or minimum hours, alongside your current role.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
$150/hr
  • quantitative analysis
  • quant research
  • alpha generation
Data Science & AnalyticsView job

Terac

Industrial Engineers: Equipment Diagnostics and Sensor Data Analysis

Terac is conducting a paid study on industrial equipment diagnostics and sensor data analysis. You’ll review equipment data and diagnostic outputs, explain how you assess equipment health, and judge whether automated workflows produce accurate, reliable evaluations. The study is remote and open to candidates in the listed eligible countries.

Location and eligibility
Remote · Eligible in 146 countries
Engagement type
Contract
Compensation
Pay not listed
  • industrial engineering
  • equipment maintenance
  • reliability engineering
EngineeringView job

Ethos

Software Engineer

Work with a foundational AI lab to train a language model on software development. You will create, assess, and refine AI-generated work across core engineering workflows. The role is remote with a flexible schedule. Work depends on client task volume, and most experts contribute fewer than 10 hours per week; hours and earnings have no weekly cap but are not guaranteed.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Full-time
Compensation
$100/hr
  • software engineering
  • system design
  • algorithms
Software EngineeringView job

Meridial by Invisible

Application-Level AI Audio Testing Expert

As an independent contractor, you’ll install a specified application version and assess its voice and audio behavior against detailed project instructions. The work covers instruction adherence, model logic, dynamic test scenarios, and the naturalness of recorded audio outputs. A compatible device and operating system are required; the project team will confirm setup during application. Referrals who finish all 10 tasks earn you a $25 USD bonus when you invite them with your unique tracked link.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$9/hr
  • audio evaluation
  • voice testing
  • application testing
Quality Assurance & TestingView job

SME Careers

Turkish Team Lead

Lead Turkish quality checks across AI training projects by reviewing generated content and contributor work against project rubrics. You will give actionable feedback, guide trainers and QAs through language questions, and keep Turkish style guides and quality records current. The role also covers onboarding and contributor follow-up, with team communication through tools such as Discord and Google Sheets.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $25/hr
  • turkish
  • english
  • llm evaluation
Language & LinguisticsView job

Mercor

Mechanical Engineer Reviewer

Reviewers help assess AI evaluation material for enterprise product design and manufacturing. You will check technical scenarios, reference answers, and grading rubrics against either the US ASME track, the international ISO track, or both. The work reflects the design complexity, safety demands, and production scale found at Fortune 500 companies and large industrial manufacturers.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Hourly
Compensation
$90/hr
  • mechanical engineering
  • asme
  • iso
EngineeringView job

micro1

AI Safety & Evaluation Specialist

Evaluate how personal AI agents handle sensitive information through privacy-focused tests and response reviews. The work centers on spotting improper disclosures and data-protection risks, then recording evidence-based feedback. This is a United States-based, part-time contractor project lasting six weeks. Applicants complete an AI interview; no other assessments are required.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
$30–$50/hr
  • privacy risk assessment
  • data protection
  • ai evaluation
Data Science & AnalyticsView job

Outlier · via Braintrust

Senior Systems & Hardware Experts - AI Benchmarking

Use your systems or hardware engineering experience to create difficult, clearly specified challenges for AI agents to solve in a Linux terminal. The project is task-based and expected to last a few weeks. Tasks average six to seven hours, with early submissions potentially taking longer. Earn up to $1,350 per task, with higher rates possible based on task quality and submission volume. Onboarding requires two courses, a graded assessment of about 90 minutes, and a live webinar.

Location and eligibility
Remote · Eligible in 25 countries
Engagement type
Contract
Compensation
$1,350 fixed
  • systems engineering
  • hardware engineering
  • linux
Hardware & Embedded SystemsView job

AfterQuery

Power Systems & Protection Engineer

Use hands-on power engineering experience to create research-level problems that test and train AI models. The work draws on real relay records and test data from substations, generators, and drives. AfterQuery offers remote, flexible contract tasks with no fixed weekly schedule. Accepted experts complete standard background and identity verification.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$50/hr
  • power systems
  • power system protection
  • python
Energy & Power GenerationView job

Turing

AI Quality Analyst (Personalization) - Turkish

Evaluate Turkish Gemini responses using personal context for relevance.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
$15/hr
  • turkish
  • gemini
  • prompt engineering
Quality Assurance & TestingView job

Handshake AI

Software Engineer

Help assess and improve coding agents by giving them real engineering tickets, running their work, and tracing mistakes through a codebase you know. The contract offers 40 hours each week at $6,000, paid each week, with hours you set. It renews each week when your work meets quality standards. Selection involves a proctored coding assessment and a paid trial task.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
$150/hr
  • software engineering
  • code review
  • coding agents
Software EngineeringView job

SME Careers

Indonesian Quality Assurance Lead (QAL)

Lead language quality across Indonesian AI training projects by reviewing model content and trainer or QA work against project rubrics. You will assess grammar, meaning, tone, cultural fit, and instruction-following, then give contributors clear feedback and maintain Indonesian-specific guidance. The role also covers onboarding, quality documentation, and contributor follow-up using tools such as Discord and Google Sheets.

Location and eligibility
Remote · Indonesia only
Engagement type
Contract
Compensation
Up to $25/hr
  • bahasa indonesia
  • indonesian
  • english
Quality Assurance & TestingView job

Mercor

SWE-Bench Task Auditor

Review repository-level software engineering benchmarks used to train and evaluate a frontier AI lab’s models. You’ll examine task quality, reference patches, test harnesses, Docker isolation, and grading integrity, then write clear feedback against a rubric. The role calls for professional software engineering experience and open-source contribution or maintainer work. The listing is limited to the United States.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$70–$90/hr
  • python
  • java
  • go
Quality Assurance & TestingView job

micro1

Member of Technical Staff, Coding Agent Research & Evaluations

Shape how frontier coding agents are tested and improved. You will create evaluation methods, datasets, and tools, then study model performance across software engineering tasks. The work combines Python and software engineering, LLM and agent systems, benchmark design, and model failure analysis. You will also share research findings with technical and client audiences.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Full-time
Compensation
$8–$9/hr
  • Coding Agent Evaluation & Benchmark Design
  • Software Engineering & Python
  • LLM & Agentic Systems
Machine Learning & AIView job

Showing 20 active roles of 113. Dates reflect what the source published, not when AIWorkJobs last synced.

Sources: ten specialist AI work platforms. AIWorkJobs does not verify every claim made by a source. Report a listing that looks wrong through the contact page.