AIWorkJobs

Job category

AI Evaluation Jobs

AI evaluation roles ask you to judge how well a model performed. That can mean scoring responses against a rubric, identifying factual errors, comparing two answers, or testing a model against professional standards.

Active roles
772
With listed pay
688
Countries represented
147

Current roles

Browse all 772

SME Careers

Talent Assessment Specialist - Bangladesh

Review candidate CVs and recorded AI interviews to support hiring decisions across technical and non-technical roles. You will assess qualifications, identify relevant experience, and write structured recommendations that help maintain fair, consistent hiring evaluations. The role requires fluent English, a bachelor’s degree or higher, and a personal computer suitable for reviewing candidate materials and completing evaluation work.

Location and eligibility
On-site · Bangladesh only
Engagement type
Contract
Compensation
Up to $4/hr
  • candidate assessment
  • cv screening
  • english proficiency
Talent Acquisition & RecruitingView job

SME Careers

Public Health Expert

Use your epidemiology and public health expertise to assess AI-generated answers or write reference prompts and model responses. You will check population-health reasoning, evidence interpretation, surveillance logic, and recommendations for accuracy, clarity, and safety. This remote, hourly contract requires at least five years of professional public health experience and C1 English proficiency; qualified experts may be contacted when relevant projects arise.

Location and eligibility
Remote · Eligible in 17 countries
Engagement type
Contract
Compensation
Up to $70/hr
  • public health
  • epidemiology
  • biostatistics
Health Informatics & Medical RecordsView job

micro1

Teacher (US/CAN based)

Use your teaching expertise to assess educational materials and help develop training data for AI systems. You’ll review AI-generated lessons, quizzes, rubrics, student feedback, and parent communications, then flag accuracy, age-appropriateness, and teaching-practice concerns. No AI background is needed, but you must work full time in education and have at least two years of hands-on teaching experience.

Location and eligibility
Remote · Canada, United States
Engagement type
Contract
Compensation
$20–$80/hr
  • lesson plan evaluation
  • curriculum design
  • assessment writing
Education & TrainingView job

Google DeepMind · via Braintrust

Trade Surveillance & Markets Compliance Expert

Bring hands-on capital markets compliance expertise to the evaluation of AI systems that detect suspicious trading and potential market abuse. You’ll review AI-generated alerts and investigations, assess trading patterns and electronic communications, and provide structured feedback on the accuracy of investigative findings and surveillance logic. Applicants must be based in the United States and have at least four years of relevant professional experience.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
$100–$145/hr
  • trade surveillance
  • markets compliance
  • electronic trading
Market Research & InsightsView job

micro1

Counterintelligence Technology & Program SME — AI Evaluation

Contribute counterintelligence expertise to the development and evaluation of AI systems for security applications. You’ll assess case studies and model responses against real-world practice, legal authorities, and oversight rules, with particular attention to the line between valid threat analysis and prohibited surveillance or targeting. Prior AI experience is not required; counterintelligence knowledge is the focus.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
$90–$140/hr
  • CI Threat Judgement
  • Legitimate/prohibited discrimination
  • Program and technology sense
CybersecurityView job

Terac

Government and Public Policy Professionals: AI Training Tasks

Use your public-sector expertise to create and assess tasks that help train AI models on real government workflows. You will write, review, and grade policy prompts covering public procurement, budget planning, fiscal analysis, and program evaluation through a desktop platform. The project is remote and requires at least 20 hours per week through December.

Location and eligibility
Remote · Eligible in 14 countries
Engagement type
Contract
Compensation
Pay not listed
  • ai training
  • government administration
  • public policy
Machine Learning & AIView job

Flash · via Mercor

AI Evaluation Rubric Auditor

Audit AI agents’ work on realistic shopping and delivery tasks by reviewing the prompt, grading rubric, and full agent transcript. You’ll judge whether the rubric treats failures fairly, follows the guidelines, and matches the prompt, then give reviewers specific feedback on prompt and rubric issues. Applicants need substantial experience writing or reviewing rubrics for LLM evaluations and must list those projects.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Hourly
Compensation
$60–$85/hr
  • llm evaluation
  • evaluation rubrics
  • rubric auditing
AI evaluationView job

Ethos

Expert Call: AI Model Post-Training Landscape

Take part in a paid, video-based research interview of about 45 minutes on how organizations select post-training platforms and services. Discuss your use case, provider choices, evaluation process, data and evaluation work, costs, results, and business value. Interviews can be scheduled at your convenience, and the client will not request confidential code or proprietary data.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
Pay not listed
  • ai model post-training
  • supervised fine-tuning
  • reinforcement learning fine-tuning
Machine Learning & AIView job

Turing

DataGen & Evals

Review Japanese patient histories and evaluate AI-generated clinical plans.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
Pay not listed
  • japanese
  • english
  • internal medicine
AI evaluationView job

AfterQuery

Applied Statistician (Survey, Actuarial, Quality)

Use your applied statistics, biostatistics, actuarial science, or related expertise to create research-level problems that test and train AI models. AfterQuery works with leading AI labs and seeks contributors who know real operational, survey, or insurance data. The remote contract pays $300 per approved task, with each task taking about six hours. You can choose your workload, with no fixed weekly schedule or cap on the number of tasks.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$50/hr
  • statistics
  • biostatistics
  • actuarial science
Data Science & AnalyticsView job

Handshake AI

Pricing Analyst

Use your retail pricing experience on an ongoing, project-based AI research opportunity. You’ll create training scenarios, assess AI responses against real pricing and markdown practices, and write feedback to guide model improvement. The part-time work is remote and self-scheduled, with no set hours or minimum hour requirement, so you can take it on alongside your current work.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
$80/hr
  • retail pricing
  • price management
  • pricing strategy
Data Science & AnalyticsView job

Meridial by Invisible

Dutch speakers specialist

As an independent contractor on an audio and language model evaluation project, you’ll assess Dutch samples for accuracy, fluency, naturalness, and overall quality. Your reviews may require close attention to grammar, pronunciation, tone, phrasing, and word choice, including whether spoken Dutch fits its context. Each task takes about 30 seconds to 4 minutes.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$1.3 fixed
  • dutch
  • dutch grammar
  • dutch spelling
Language & LinguisticsView job

What evaluators do

Evaluators review model outputs for correctness, relevance, completeness, and safety. Many projects use structured rubrics, so the work rewards careful reading and consistent judgment more than speed alone.

Subject knowledge matters. A finance evaluation project may require professional finance experience; a medical evaluation project may require a clinical background.

Skills that help

Clear written feedback is central to evaluation work. You should be able to explain why an answer is wrong, not only mark it as wrong.

Familiarity with the domain, attention to detail, and comfort working through structured guidelines are usually more relevant than prior AI experience.