AIWorkJobs

Job index

AI / LLM Evaluation jobs

Current roles from specialist AI work platforms. Pay and eligibility are shown as each source publishes them.

Filters1ShowHide
Pay
Reset

Mercor

Behavioral Health Expert, AI Safety and Model Evaluation

Review AI conversations about relationships, family, emotional wellbeing, personal beliefs and difficult decisions. You’ll judge whether responses stay balanced, safe and within the model’s role, then explain your reasoning and help researchers set evaluation standards. The part-time role requires at least 20 weekday hours per week, with the option to work up to 40. Cincinnatus LLC employs the expert on a W-2 basis and places them with a leading AI lab.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
$45–$70/hr
  • behavioral health
  • mental health counseling
  • social work
Machine Learning & AIView job

SME Careers

Vietnamese Language Expert

Review and create Vietnamese-language material for AI training, assessing response accuracy, reasoning, and compliance with prompts. You will apply senior-level localization judgment, MQM/LQA methods, and CAT tools to check terminology, tone, cultural nuance, and factual details. The role is remote and hourly under a contract arrangement; no immediate project is available, but qualified experts may be contacted for future work.

Location and eligibility
Remote · Vietnam only
Engagement type
Contract
Compensation
Up to $20/hr
  • vietnamese
  • english
  • translation
Language & LinguisticsView job

Mercor

Drug Discovery Specialist, Pharmacology & ADME/Tox

Bring hands-on drug discovery expertise to the design and evaluation of AI models for science. You will help research and engineering teams improve model performance across pharmacology, ADME/Tox, and PK/PD. The part-time role runs through Cincinnatus LLC or an appropriate international entity, with placement at a leading AI lab. The remote position lists London, UK, and the US East Coast.

Location and eligibility
Remote · United Kingdom, United States
Engagement type
Part-time
Compensation
$70–$100/hr
  • pharmacology
  • drug discovery
  • dmpk
AI evaluationView job

SME Careers

Catalan Team Lead

Guide quality across Catalan AI training projects by reviewing language data and contributor work against project rubrics. You will coach trainers and QA reviewers, resolve questions on Catalan usage and regional standards, and keep style guides and onboarding resources current. Strong Catalan writing skills and clear English communication are central to the role.

Location and eligibility
Remote · France, Spain
Engagement type
Full-time
Compensation
Up to $36/hr
  • catalan
  • english
  • ai training
Language & LinguisticsView job

Mercor

Atomic Layer Deposition (ALD) Experts

Bring hands-on expertise in atomic layer deposition and thin-film processes to scientific data work for a frontier AI research lab focused on semiconductors and physical sciences. You’ll create and assess training and evaluation material, test AI-generated reasoning, and help models handle questions about deposition, materials, and semiconductor processes. The engagement is remote for US-based experts, runs long term, and offers 10 to 40 hours per week.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$84/hr
  • atomic layer deposition
  • thin-film deposition
  • semiconductor processing
Machine Learning & AIView job

SME Careers

Finance & Accounting Specialist – Rubric Design

Use your finance or accounting expertise to create realistic professional problems and assess AI-generated analyses. You will build scoring rubrics that capture sound calculations, assumptions, methods, and judgment, then review outputs and peer criteria for technical accuracy and completeness across areas such as valuation, reporting, audit, and investment analysis.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $100/hr
  • accounting
  • finance
  • financial analysis
Finance & AccountingView job

Mercor

VFX & Compositing Supervisor

Help assess generative video models designed for feature-quality visual effects and preproduction work. You’ll set evaluation standards, build challenging benchmarks, and review generated footage for issues such as temporal instability, compositing seams, physics, and fine detail. The full-time role calls for about 40 hours per week on weekdays. Cincinnatus LLC employs you as a W-2 employee and may place you with a leading AI lab.

Location and eligibility
Remote · United States only
Engagement type
Full-time
Compensation
$70–$110/hr
  • visual effects
  • compositing
  • nuke
Design & CreativeView job

SME Careers

Thai Language Expert

Review AI-generated Thai responses and create reference answers that demonstrate accurate, natural writing. You will judge grammar, meaning, tone, cultural fit, and prompt compliance, then explain issues in English and compare response quality. The hourly remote contract has no immediate project; qualified experts may hear about relevant work through the expert network.

Location and eligibility
Remote · Cambodia, Malaysia, Thailand
Engagement type
Contract
Compensation
Up to $20/hr
  • thai
  • english
  • copyediting
Language & LinguisticsView job

Mercor

Cinematographer

Help a leading AI lab assess generative video models for cinematic camera and lighting quality. You’ll set evaluation standards, curate challenging visual examples, and review footage for details such as lens behavior, camera movement, motivated lighting, exposure, and color. The role is full-time at about 40 hours per week, with W-2 employment through Cincinnatus LLC and placement within the lab’s extended workforce.

Location and eligibility
Remote · United States only
Engagement type
Full-time
Compensation
$70–$110/hr
  • cinematography
  • lens optics
  • camera movement
Design & CreativeView job

SME Careers

Catalan Language Expert

Review AI-generated Catalan and create reference answers that model accurate, natural writing. You will judge grammar, meaning, tone, regional fit, and cultural context, then explain errors and compare response quality. Strong Catalan writing and C1 English are essential for following evaluation guidance and delivering detailed feedback in a remote contract workflow.

Location and eligibility
Remote · France, Spain
Engagement type
Contract
Compensation
Up to $25/hr
  • catalan
  • english
  • catalan grammar
Language & LinguisticsView job

Mercor

Compliance & Risk Specialist Talent Network

Apply to Mercor’s network for future contract projects with AI labs and companies. After you complete your profile and pass an AI interview, Mercor may match you to projects as they arise. Experts train and evaluate AI models, build work from real-world compliance and risk scenarios, and give domain feedback. Project scope and duration vary; typical commitments run 15–30 hours per week.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$60–$80/hr
  • regulatory compliance
  • risk assessment
  • risk mitigation
Machine Learning & AIView job

SME Careers

Java Team Lead

Guide quality across Java AI training projects by reviewing generated code and contributor work against project rubrics. You will assess correctness, runtime behavior, design, security, and explanations, then give precise feedback and maintain resources for remote contributors. This contract role has no immediate project; qualified experts may be contacted when relevant work opens. Selection includes an AI interview, a domain task, and a recruiter interview.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $70/hr
  • java
  • jvm
  • spring boot
Quality Assurance & TestingView job

Mercor

Engineering & Software Domain Expert

A senior engineering specialist will help a leading AI lab assess and improve how its models handle real software and engineering work. You will review model outputs, write task instructions and reference solutions, and create evaluation benchmarks with the research team. The position is full-time W-2 employment through Cincinnatus LLC, based in the Bay Area with on-site work multiple days each week when required.

Location and eligibility
Hybrid · United States only
Engagement type
Full-time
Compensation
$65–$105/hr
  • software engineering
  • computer science
  • distributed systems
Software EngineeringView job

SME Careers

Red-Teaming Quality Assurance Lead (QAL)

Review adversarial prompts, model responses, risk analyses, and safety evaluations against project rubrics. You will identify weak scenarios, policy errors, unsafe outputs, and gaps in vulnerability testing, then guide trainers and QAs through feedback, onboarding, and documentation. The work uses Discord, Google Sheets, and Google Docs. No project is available immediately; qualified experts may be contacted for future work.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $100/hr
  • ai red-teaming
  • ai safety
  • llm evaluation
Quality Assurance & TestingView job

Mercor

Office & Productivity Software Expert

Mercor maintains a pool of office software experts for AI training and evaluation projects. You may create realistic tasks with reference answers, assess AI-produced documents, spreadsheets, and presentations, or identify issues that could arise with file conversion, version compatibility, and later editing. Projects open at short notice and may arise within a week or several months; each project sets its own rate, hours, and client.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$60–$70/hr
  • word processing
  • document editing
  • spreadsheets
Machine Learning & AIView job

SME Careers

Czech Team Lead

Lead quality workflows for Czech AI training projects as a remote contractor. You will review Czech-language content and contributor work against project rubrics, coach trainers and QA reviewers, and maintain resources such as style guides, trackers, and onboarding materials. The selection process includes an AI interview, a domain-specific task, and a recruiter interview; interviews take place in English.

Location and eligibility
Remote · Czechia only
Engagement type
Contract
Compensation
Up to $30/hr
  • czech
  • english
  • ai training
AI evaluationView job

Mercor

Epidemiologist — Patient Population Sizing

Assess patient populations for commercial drugs and development programs in a paid expert-evaluation pilot with a biotech and pharma research team. You’ll build and review analysis rubrics, test whether population estimates hold up to their sources, and identify undercounts or overcounts across several drugs and indications. The pilot covers about 10–20 hours over one to two weeks, with recurring work possible as the project grows.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
$150–$175/hr
  • epidemiology
  • patient population sizing
  • prevalence
Health Informatics & Medical RecordsView job

SME Careers

Geology Quality Assurance Lead (QAL)

As a remote, hourly contractor, you will review geology and earth science content for scientific accuracy, sound reasoning, and alignment with project rubrics. You will coach trainers and QAs, maintain review materials, and track contributor activity across AI training projects. No project is active now; qualified experts may be contacted for future work. Selection includes an AI interview, a domain task, and a recruiter interview.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $70/hr
  • geology
  • earth sciences
  • geoscience
Quality Assurance & TestingView job

Mercor

Data Scientist Talent Network

Mercor is assembling a pool of experienced data scientists for future projects with AI research organizations. Assignments may assess how well AI systems handle practical data science work, from analysis and modeling to technical reports. No project is open now; Mercor may contact qualified applicants when a suitable opportunity becomes available.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Hourly
Compensation
$100–$150/hr
  • data science
  • python
  • sql
Data Science & AnalyticsView job

SME Careers

C++ Team Lead

Lead quality checks for C++ AI training work by reviewing generated code, debugging answers, algorithms, tests, and contributor feedback against project rubrics. You will guide remote trainers and reviewers, flag safety or correctness risks, and maintain onboarding and QA materials. This contract role has no immediate project; qualified candidates may hear about future assignments. The selection process includes an AI interview, a domain task, and a recruiter interview.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $75/hr
  • c++
  • ai training
  • llm evaluation
Quality Assurance & TestingView job

Showing 20 active roles of 782. Dates reflect what the source published, not when AIWorkJobs last synced.

Sources: ten specialist AI work platforms. AIWorkJobs does not verify every claim made by a source. Report a listing that looks wrong through the contact page.