AIWorkJobs

Job index

AI / LLM Evaluation jobs

Current roles from specialist AI work platforms. Pay and eligibility are shown as each source publishes them.

Filters1ShowHide
Pay
Reset

Mercor

Cardiologist

Review de-identified patient charts and compare AI-generated answers to cardiology questions. You will assess clinical accuracy, safety, and consistency with current standards of care, then explain your judgments in structured rationales. The project offers remote, hourly work with flexible part-time hours and an immediate start.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$350/hr
  • cardiology
  • abim cardiovascular disease
  • clinical chart review
Health Informatics & Medical RecordsView job

SME Careers

Romanian Team Lead

Lead Romanian-language quality work across AI training projects by reviewing generated content and contributor submissions against project rubrics. You will give actionable feedback, guide trainers and QAs through changing requirements, and maintain resources such as Romanian style guides and quality trackers. The role calls for native or near-native Romanian, strong English, and experience with language-review workflows.

Location and eligibility
Remote · Romania only
Engagement type
Contract
Compensation
Up to $25/hr
  • romanian
  • english
  • linguistic quality assurance
Language & LinguisticsView job

Mercor

Anesthesiologist

Review de-identified patient charts and assess AI responses to anesthesiology questions for clinical accuracy, patient safety, and consistency with current care standards. You will compare two answers, select the stronger response, and explain your judgment. Applicants need an active, unrestricted US medical license, current US clinical practice, and board certification in ABA Anesthesiology. The project offers remote, flexible part-time hours and an immediate start.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$350/hr
  • anesthesiology
  • aba anesthesiology
  • clinical evaluation
Healthcare & MedicalView job

SME Careers

Urdu Language Expert

Review AI-generated Urdu and build reference answers that show strong language use across customer support, education, translation, localization, and other domains. You will compare model responses and give specific feedback on Urdu grammar, meaning, tone, cultural fit, and Nastaliq and right-to-left formatting. The role requires native or near-native Urdu, C1 English, and at least three years of relevant language-focused experience.

Location and eligibility
Remote · Eligible in 4 countries
Engagement type
Contract
Compensation
Up to $15/hr
  • urdu
  • english
  • urdu grammar
Language & LinguisticsView job

Mercor

Orthopedic Surgeon

Use your orthopedic surgery expertise to assess health AI models developed with a leading AI lab. You will review de-identified patient charts, evaluate clinical answers for accuracy and safety, compare two responses, and explain your judgments in structured rationales. Applicants need ABOS Orthopaedic Surgery board certification, an active unrestricted US medical license, and current patient care practice in the US.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$350/hr
  • orthopedic surgery
  • orthopaedic surgery
  • abos certification
Healthcare & MedicalView job

SME Careers

Dutch Language Expert

Review AI-generated Dutch and create reference answers that show strong grammar, accurate meaning, and an appropriate tone. You will compare model responses, check claims and cultural fit, and explain edits in written English. The role calls for native or near-native Dutch, English at C1 or higher, and at least three years of relevant language-focused experience.

Location and eligibility
Remote · Belgium, Netherlands, Suriname
Engagement type
Contract
Compensation
Up to $45/hr
  • dutch
  • english
  • dutch grammar
Language & LinguisticsView job

Mercor

Dermatologist

Use your dermatology expertise to assess health AI responses against de-identified patient charts. You will compare two answers to clinical questions, judge their accuracy and safety, and explain your decisions. Applicants need an active, unrestricted US medical license, board certification in ABD Dermatology, and active clinical practice in the US after completing residency and any required fellowship.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$275/hr
  • dermatology
  • clinical evaluation
  • medical chart review
Healthcare & MedicalView job

SME Careers

Civil Engineer Quality Assurance Lead (QAL)

Guide quality across civil engineering AI training projects by reviewing model outputs and contributor work against technical rubrics. You will check calculations, assumptions, standards references, units, and safety implications, then coach trainers and reviewers through specific feedback and project updates. The role also calls for maintaining review documentation and supporting remote teams through Discord, shared trackers, and onboarding sessions.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $105/hr
  • civil engineering
  • engineering quality assurance
  • technical review
Quality Assurance & TestingView job

Mercor

General Surgeon

Assess health AI models by reviewing de-identified patient charts and judging answers to general surgery questions. You will compare two responses, assess their accuracy and safety against the clinical record and current standards of care, and explain your judgment. The project offers remote, flexible part-time hours on an hourly basis, with an immediate start.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$200/hr
  • general surgery
  • clinical evaluation
  • medical chart review
Healthcare & MedicalView job

SME Careers

Chinese (Simplified) Quality Assurance Lead (QAL)

Lead quality reviews for Mainland China Simplified Chinese work across AI training projects. You will assess contributor and model output against project rubrics, give actionable feedback, guide onboarding, and maintain language resources such as style guides and FAQs. This is a contract role with no immediate project; qualified candidates may be contacted first when relevant work becomes available.

Location and eligibility
Remote · China only
Engagement type
Contract
Compensation
Up to $35/hr
  • simplified chinese
  • mandarin
  • english
Quality Assurance & TestingView job

Mercor

Critical Care Physician

Review de-identified patient charts and assess AI responses to critical care questions for accuracy, safety, and consistency with current standards of care. You will compare two answers, explain your clinical judgment in structured rationales, and flag mistakes that could put patients at risk. The project offers remote, hourly work with flexible part-time hours and an immediate start.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$200/hr
  • critical care medicine
  • clinical evaluation
  • patient chart review
Healthcare & MedicalView job

SME Careers

Computer Science & IT Team Lead

Lead quality checks for computer science and IT AI training work, reviewing technical answers, code, and troubleshooting guidance against project rubrics. You will coach trainers and QA contributors, document standards, and flag inaccurate or unsafe recommendations. The role calls for at least three years of relevant technical experience, strong English communication, and native Punjabi fluency.

Location and eligibility
Remote · India only
Engagement type
Contract
Compensation
Up to $20/hr
  • computer science
  • information technology
  • software engineering
Quality Assurance & TestingView job

Mercor

Obstetrician-Gynecologist (OB/GYN)

Review de-identified patient charts and assess AI responses to obstetrics and gynecology questions for accuracy, safety, and consistency with current care standards. The project offers remote, flexible part-time hours on an hourly basis, with an immediate start. Applicants submit a resume, complete a 25-minute conversational interview, and undergo verification of their license, certification, training, and practice setting. Onboarding details follow within a few days.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$200/hr
  • obstetrics
  • gynecology
  • ob/gyn
Healthcare & MedicalView job

SME Careers

Gujarati Team Lead with Mathematical Background

Lead quality checks for Gujarati AI training projects by reviewing generated content and contributor work against project rubrics. You will guide trainers and QA reviewers, resolve language questions, and maintain Gujarati-specific resources such as style guides and onboarding materials. The role requires a Mathematics degree, advanced Gujarati and English skills, and at least three years of relevant experience. Selection includes an AI interview, a domain task, and a recruiter interview.

Location and eligibility
Remote · India, United Kingdom, United States
Engagement type
Contract
Compensation
Up to $30/hr
  • gujarati
  • english
  • mathematics
Language & LinguisticsView job

Mercor

Neurologist

Assess health AI models by reviewing de-identified patient charts and comparing their answers to neurology questions. You will judge clinical accuracy, safety, and consistency with current standards of care, then explain your decisions in structured rationales. Applicants must hold an active, unrestricted US medical license and practice in the US.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$200/hr
  • neurology
  • clinical evaluation
  • medical chart review
Healthcare & MedicalView job

SME Careers

Radiologist

Review AI-generated imaging responses or create expert content on modality choice, protocols, differential diagnoses, and radiology reports. You will assess clinical reasoning, accuracy, clarity, and safety, including CT, MRI, contrast, and radiation considerations. This remote, hourly contract has no immediate project; qualified experts may receive invitations when relevant work becomes available.

Location and eligibility
Remote · Eligible in 17 countries
Engagement type
Contract
Compensation
Up to $120/hr
  • diagnostic radiology
  • clinical reasoning
  • computed tomography
Health Informatics & Medical RecordsView job

Mercor

Rheumatologist

Assess AI-generated rheumatology answers against de-identified patient charts, judging clinical accuracy, safety, and fit with current standards of care. The remote project offers flexible, part-time hourly work. Applicants need board certification, an active unrestricted US medical license, and at least one year of independent practice after training while seeing patients in the US.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$200/hr
  • rheumatology
  • abim rheumatology
  • clinical evaluation
Health Informatics & Medical RecordsView job

SME Careers

English Quality Assurance Lead (QAL)

Guide English quality across AI training projects by reviewing contributor work and generated content against project rubrics. You will spot-check submissions for accuracy, clarity, tone, grammar, formatting, and instruction-following, then share actionable feedback through Discord and maintain guidance in Google Docs and Google Sheets. The role also covers onboarding, contributor follow-ups, and consistent application of changing guidelines across remote teams.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $35/hr
  • english grammar
  • english quality assurance
  • content review
Quality Assurance & TestingView job

Mercor

Nephrologist

Use your nephrology expertise to assess health AI models developed with a leading AI lab. You will review de-identified patient charts, compare model responses to clinical questions, and judge their accuracy, safety, and consistency with current standards of care. The project offers remote, flexible part-time hours on an hourly basis, with an immediate start.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$200/hr
  • nephrology
  • abim nephrology
  • clinical evaluation
AI evaluationView job

SME Careers

Lithuanian Language Expert

Review AI-generated Lithuanian and write reference answers that show accurate, natural language use. You will compare model responses, assess grammar, meaning, tone, and cultural fit, then explain issues and suggest corrections in English. The role requires native or near-native Lithuanian, C1 English, and at least three years of language-focused experience; the work follows an hourly remote contractor arrangement.

Location and eligibility
Remote · Lithuania only
Engagement type
Contract
Compensation
Up to $25/hr
  • lithuanian
  • english
  • copyediting
Language & LinguisticsView job

Showing 20 active roles of 782. Dates reflect what the source published, not when AIWorkJobs last synced.

Sources: ten specialist AI work platforms. AIWorkJobs does not verify every claim made by a source. Report a listing that looks wrong through the contact page.