AIWorkJobs

Job index

AI / LLM Evaluation jobs

Current roles from specialist AI work platforms. Pay and eligibility are shown as each source publishes them.

Filters1ShowHide
Pay
Reset

Mercor

Infectious Disease Physician

Evaluate health AI models with a leading AI lab by reviewing de-identified patient charts and judging clinical answers in infectious disease. The remote project offers flexible, hourly part-time work and an immediate start. Applicants complete a 25-minute interview and credential checks covering licensure, board certification, training, and current practice.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$200/hr
  • infectious disease
  • clinical evaluation
  • medical chart review
Health Informatics & Medical RecordsView job

SME Careers

Malay Language Expert

Review Bahasa Melayu AI responses or create reference answers across general knowledge, customer support, education, and conversation. You will judge grammar, meaning, tone, factual claims, and cultural fit, then explain issues in written English. This remote contract has no immediate project; qualified experts may hear about relevant work through the expert network.

Location and eligibility
Remote · Eligible in 6 countries
Engagement type
Contract
Compensation
Up to $20/hr
  • bahasa melayu
  • malay
  • english
Language & LinguisticsView job

Mercor

Endocrinologist

Assess health AI models with your endocrinology expertise by reviewing de-identified patient charts and evaluating model answers to clinical questions. You will compare two responses, judge their accuracy and safety against current standards of care, and document your reasoning. The project offers remote, flexible part-time hours and requires current US clinical practice and an active, unrestricted US medical license.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$200/hr
  • endocrinology
  • diabetes
  • clinical chart review
Health Informatics & Medical RecordsView job

SME Careers

Anthropology Quality Assurance Lead (QAL)

As an Anthropology Quality Assurance Lead, you will review AI-generated anthropology content and contributor work against project rubrics, then give clear feedback to trainers and QAs. You will also maintain review materials and help contributors apply consistent standards across remote projects. This is a contract role with no immediate project; qualified experts may be contacted for future work. Selection includes an AI interview, a domain task, and a recruiter interview.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $65/hr
  • anthropology
  • cultural anthropology
  • archaeology
Quality Assurance & TestingView job

Mercor

Ophthalmologist

Assess health AI responses to ophthalmology questions using de-identified patient charts. You will compare two answers, judge their clinical accuracy, safety, and fit with current standards of care, then document your reasoning and flag risks to patient safety. Applicants need board certification, an active US medical license, and current patient care practice in the US.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$175/hr
  • ophthalmology
  • clinical chart review
  • clinical answer evaluation
Healthcare & MedicalView job

SME Careers

Filipino / Tagalog Team Lead

Guide quality across remote Filipino/Tagalog AI training projects by reviewing language data and trainer work against project rubrics. You will coach trainers and QA reviewers, answer language questions, and maintain resources such as style guides and quality trackers. The role calls for native or near-native Filipino/Tagalog, strong English, and at least three years of relevant professional experience.

Location and eligibility
Remote · Philippines, Saudi Arabia, United States
Engagement type
Contract
Compensation
Up to $25/hr
  • filipino
  • tagalog
  • english
Language & LinguisticsView job

Mercor

Otolaryngologist (ENT)

Evaluate health AI models with a leading AI lab by reviewing de-identified patient charts and assessing answers to otolaryngology questions. You will compare clinical responses, judge their safety and accuracy, and explain your decisions. The work is remote, hourly, and part-time with flexible hours, and the project has an immediate start.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$150/hr
  • otolaryngology
  • head and neck surgery
  • clinical evaluation
Healthcare & MedicalView job

SME Careers

Tamil Expert with Mathematical Background

Use your Tamil language expertise and mathematics degree to shape AI training content and assess model responses. You will write prompts and reference answers, compare outputs for accuracy, fluency, tone, and cultural fit, and explain language or reasoning issues in English. The role requires native or near-native written Tamil, C1 English, and at least three years of relevant professional experience.

Location and eligibility
Remote · Eligible in 4 countries
Engagement type
Contract
Compensation
Up to $15/hr
  • tamil
  • english
  • mathematics
Language & LinguisticsView job

Mercor

Geriatrician

Assess health AI responses to clinical questions in geriatric medicine by reviewing de-identified patient charts and comparing answers. The work calls for current US clinical practice and careful judgments about accuracy, safety, and standards of care. The project is remote, hourly, part-time, and set to start immediately. Applicants submit a resume, complete a 25-minute conversational interview, and undergo credential checks covering licensure, board certification, training, and practice setting.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$150/hr
  • geriatric medicine
  • clinical evaluation
  • medical chart review
Healthcare & MedicalView job

SME Careers

Healthcare Specialist – Rubric Design

Apply your healthcare specialty to build questions that test advanced AI systems and scoring rubrics that separate sound clinical reasoning from unsafe or incomplete answers. You will assess model responses, review colleagues’ rubrics, and create reference material for AI evaluation and training workflows. Applicants need a recognized healthcare qualification, at least three years of experience, and English proficiency at C1 or higher.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $100/hr
  • healthcare
  • clinical pharmacology
  • clinical reasoning
Health Informatics & Medical RecordsView job

Mercor

Psychiatrist

Assess health AI responses to psychiatry questions by comparing them with de-identified patient charts. You will judge clinical accuracy, safety, and alignment with current standards of care, then explain your decisions in structured rationales. The project offers remote, hourly work with flexible part-time hours and an immediate start. Applicants complete a 25-minute interview and credential verification.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$150/hr
  • psychiatry
  • clinical evaluation
  • medical chart review
Healthcare & MedicalView job

SME Careers

Telugu Team Lead with Mathematical Background

Lead Telugu quality across AI training projects by reviewing language data and contributor work against project rubrics. You will give trainers and reviewers specific feedback, resolve questions on Telugu usage, and maintain guidance and tracking materials in Google Docs, Google Sheets, and Discord. The role calls for a mathematics degree, strong Telugu and English, and at least three years of relevant professional experience. Selection includes an AI interview, a domain task, and a recruiter interview.

Location and eligibility
Remote · India, United Arab Emirates, United States
Engagement type
Contract
Compensation
Up to $25/hr
  • telugu
  • english
  • mathematics
Language & LinguisticsView job

Mercor

Generalist Expert

Mercor keeps this listing open to build a pool of generalists for future AI training and evaluation projects; it does not represent a specific opening. After you submit a resume, complete an assessment, and confirm your work location, Mercor verifies your background. The team may contact you when a project fits your experience, from within a week to several months later.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$40–$70/hr
  • ai training
  • ai evaluation
  • prompt writing
Machine Learning & AIView job

SME Careers

Croatian Language Expert

Review and create Croatian content used to train and assess AI models. You will compare model responses, write reference answers, and explain language issues in English, from case and agreement errors to awkward translations and mismatched tone. The role requires native or near-native Croatian, English at C1 or higher, and at least three years of language-focused professional experience.

Location and eligibility
Remote · Croatia only
Engagement type
Contract
Compensation
Up to $25/hr
  • croatian
  • english
  • copyediting
Language & LinguisticsView job

Mercor

Materials Science Domain Expert

Cincinnatus LLC employs this full-time W-2 materials science expert and places them with a leading AI lab. You will assess model responses, create gold-standard materials problems, and help researchers build domain-specific evaluations. The hybrid role requires Bay Area residence and on-site work with the client several days each week when needed. The client provides work accounts and equipment.

Location and eligibility
Hybrid · United States only
Engagement type
Full-time
Compensation
$70–$110/hr
  • materials science
  • materials engineering
  • chemistry
Science & ResearchView job

SME Careers

Neuroscience Quality Assurance Lead (QAL)

Guide scientific quality across neuroscience and cognitive science AI training work. You will review model-generated explanations and contributor work against project rubrics, coach trainers and QAs, and maintain the documentation that keeps reviews consistent. This remote contract has no immediate project; qualified experts may hear about future work. Selection includes an AI interview, a domain task, and a recruiter interview. Key areas include LLM evaluation and scientific review.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $90/hr
  • neuroscience
  • cognitive science
  • cognitive psychology
Quality Assurance & TestingView job

Mercor

General Physician

Assess health AI responses across diagnosis, clinical reasoning, and care workflows in a six-week project with a leading AI lab. You’ll judge clinical accuracy and safety, document clear feedback, and flag risks to patient care. Applicants must hold an active, unrestricted US medical license, have completed residency, and see patients in practice. The commitment is part-time, at least 20 hours per week.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$150/hr
  • clinical evaluation
  • clinical reasoning
  • diagnosis
Healthcare & MedicalView job

SME Careers

Portuguese Language Expert

Review and compare AI-generated answers, check their reasoning and factual accuracy, and create Portuguese-language examples that show correct methods. Your Portuguese localization expertise will guide decisions on terminology, tone, and cultural context, while MQM and LQA methods help you assess errors consistently. This remote contract role is open only to candidates based in Brazil; there is no immediate project, but qualified experts may be contacted for future work.

Location and eligibility
Remote · Brazil, Portugal
Engagement type
Contract
Compensation
Up to $30/hr
  • portuguese
  • english
  • localization
Language & LinguisticsView job

Mercor

Audiobook QA Expert — Spanish (US)

Listen to AI-narrated audiobooks in US and North American Spanish, then assess their accuracy, naturalness, and appeal for listeners. You’ll mark speech errors and audio defects in review tools and share where the narration holds up or disrupts the listening experience. The role offers remote, part-time work at about 20 hours per week, or roughly four hours per day.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Hourly
Compensation
$15–$20/hr
  • spanish
  • audiobook quality assurance
  • audio annotation
Language & LinguisticsView job

SME Careers

Odia (Oriya) Language Expert

Review AI-generated Odia responses and create reference answers for tasks such as customer support, education, and general knowledge. You will judge grammar, meaning, tone, cultural fit, and factual reliability, then explain issues or provide improved Odia wording in English. The role calls for native or near-native written Odia, strong English, and at least three years of relevant language-focused experience.

Location and eligibility
Remote · Eligible in 4 countries
Engagement type
Contract
Compensation
Up to $15/hr
  • odia
  • english
  • odia grammar
Language & LinguisticsView job

Showing 20 active roles of 782. Dates reflect what the source published, not when AIWorkJobs last synced.

Sources: ten specialist AI work platforms. AIWorkJobs does not verify every claim made by a source. Report a listing that looks wrong through the contact page.