AIWorkJobs

Job index

AI / LLM Evaluation jobs

Current roles from specialist AI work platforms. Pay and eligibility are shown as each source publishes them.

Filters1ShowHide
Pay
Reset

AfterQuery

Materials Science Expert (AI Safety)

Help assess how AI models handle safety-sensitive questions in materials science. You will create evaluation prompts, review work from other experts, and explain the technical basis for each safe or unsafe classification. The role requires a graduate degree in a relevant discipline and citizenship and residence in a Five Eyes country while you complete the work.

Location and eligibility
Remote · Eligible in 5 countries
Engagement type
Not specified
Compensation
$150/hr
  • materials science
  • materials engineering
  • metallurgy
CybersecurityView job

Outlier · via Braintrust

Senior Science & Clinical Experts - AI Benchmarking

Use scientific or clinical expertise to assess how AI systems handle domain-specific work. You will create Linux-terminal benchmarks, set standards for verifiable results, and review the accuracy and reasoning behind proposed solutions. The project expects 20 hours per week for 1–6 months and pays $1,350 per task.

Location and eligibility
Remote · Eligible in 25 countries
Engagement type
Contract
Compensation
$1,350 fixed
  • linux
  • ai evaluation
  • scientific benchmarking
AI evaluationView job

General Translation · via Mercor

Australian English Linguistic Expert

Assess how well a large language model handles Australian English, then help improve its language data and outputs. You’ll review text for translation, summarization, dialogue, and grammar correction; refine content for Australian usage; and share linguistic analysis with AI researchers. The work calls for deep knowledge of Australian English and professional experience in language or NLP data work.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$40/hr
  • australian english
  • linguistic annotation
  • translation
Language & LinguisticsView job

Handshake AI

Mandarin AI Evaluation Specialist (Singaporean Dialect)

Use your fluency in Singaporean Mandarin to compare AI-generated answers and explain which responses better fit a user. You’ll share feedback through an annotation platform, drawing on your experience with AI tools. The work is ongoing and project-based, with no set schedule or minimum hours; prior AI training experience is not required.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$30/hr
  • mandarin
  • singaporean mandarin
  • english
AI evaluationView job

SME Careers

Romanian Language Expert

Review AI-generated Romanian and create reference answers that demonstrate accurate, natural writing. You will assess meaning, grammar, spelling, tone, cultural fit, and prompt compliance, then explain issues and provide clear corrections in English. The role requires native or near-native Romanian, at least C1 English, and three or more years of language-focused professional experience.

Location and eligibility
Remote · Romania only
Engagement type
Contract
Compensation
Up to $20/hr
  • romanian
  • english
  • copyediting
Language & LinguisticsView job

micro1

Cardiac Electrophysiologist

Review AI-generated clinical material and scenarios in cardiac electrophysiology, with a focus on arrhythmias, ECGs, and rhythm management. The work also covers clinical reasoning about cardiac devices and ablation. Prior AI experience is not required. Applicants proceed through an application, AI interview, specialty screening, and onboarding call.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
$80–$285/hr
  • cardiac electrophysiology
  • ecg interpretation
  • arrhythmia diagnosis and management
Health Informatics & Medical RecordsView job

Ethos

Nonprofit / Philanthropy / Community Programs

Help train a foundational AI lab’s language model by creating, reviewing, and refining professional nonprofit documents, spreadsheets, and slide decks. You’ll draw on experience with grants, donor reporting, program models, and community programs. Work remotely on your own schedule. Client task volume varies, hours are not guaranteed, and most experts work fewer than 10 hours per week, with no weekly cap.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$70/hr
  • nonprofit program management
  • philanthropy
  • grant proposal writing
Nonprofit & Community DevelopmentView job

Turing

Top GitHub Contributors – Agentic Review Project

Design and analyze challenging benchmark tasks to enhance AI reasoning.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
Pay not listed
  • python
  • rust
  • github
AI evaluationView job

AfterQuery

Advanced Coding Expert

AfterQuery needs experienced software engineers to create realistic, multi-step coding challenges for advanced AI coding agents. You will draw on a substantial private codebase you own, frame problems that require investigation across modules, and create Docker-based environments with tests that assess solutions and catch regressions.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$85/hr
  • javascript
  • typescript
  • java
Software EngineeringView job

Outlier · via Braintrust

Senior Finance & Operations Experts - AI Benchmarking

Create realistic finance and operations challenges that help benchmark AI systems. You’ll build scenarios from professional workflows, set clear criteria for correct results, and assess proposed answers for accuracy and sound reasoning. The project expects 20 hours of work per week for 1–6 months and pays $1,350 per task.

Location and eligibility
Remote · Eligible in 25 countries
Engagement type
Contract
Compensation
$1,350 fixed
  • finance
  • operations
  • compliance
Machine Learning & AIView job

General Translation · via Mercor

British English Linguistic Expert

Assess how well a large language model handles British English in translation, summarization, dialogue, and grammar correction. You will refine and annotate language data, flag errors and gaps, and share detailed analysis with AI researchers to support model training. The project focuses on British usage, including regional variation, register, and cultural context.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$40/hr
  • british english
  • linguistics
  • translation
Language & LinguisticsView job

Handshake AI

Inventory Control Manager

Use your retail warehouse and inventory expertise on project-based AI research work. You’ll write training questions and scenarios drawn from real inventory workflows, assess AI responses, and give structured feedback. The work is part-time, ongoing, and flexible, with no set schedule or minimum hours, so you can fit tasks around your current work.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
$80/hr
  • inventory management
  • warehouse operations
  • materials handling
AI evaluationView job

SME Careers

Maltese Team Lead

Lead quality reviews for Maltese AI training projects by checking generated content and contributor work against project rubrics. You will coach trainers and QA reviewers, resolve language questions, maintain Maltese-specific guidance, and track contributor activity using tools such as Discord, Google Docs, and Google Sheets. The role calls for native or near-native Maltese and strong English.

Location and eligibility
Remote · Malta only
Engagement type
Contract
Compensation
Up to $45/hr
  • maltese
  • english
  • maltese grammar
Language & LinguisticsView job

micro1

Cardiac Imaging Specialist

A cardiologist with cardiovascular imaging expertise will review AI interpretations and clinical explanations, checking their diagnostic accuracy and reasoning. The work covers echocardiography, cardiac MRI, cardiac CT, and related imaging scenarios, with expert feedback on findings, terminology, and recommendations. The role is remote and hourly for an independent contractor. English proficiency at B2 level or above is required.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
$80–$285/hr
  • cardiac imaging
  • echocardiography
  • cardiac mri
Health Informatics & Medical RecordsView job

Ethos

Sales Engineer / Solutions Consultant

Help train a leading foundational AI lab’s language model on professional presentation work. You’ll create, assess, and refine AI-generated slide decks for technical sales workflows, drawing on your experience building presentations as a sales engineer or solutions consultant. The role is fully remote, offers schedule flexibility, and has variable client-driven hours; most experts work fewer than 10 hours per week, with no weekly cap.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$80/hr
  • sales engineering
  • solutions consulting
  • powerpoint
Sales Engineering & Solutions ConsultingView job

Turing

AI Quality Analyst (Personalization) - Dutch

Evaluate the quality of AI's personalized responses using your experiences.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
$20/hr
  • dutch
  • ai quality evaluation
  • personalization evaluation
AI evaluationView job

AfterQuery

Computational Physics & Astrophysics Researcher (Task Author)

Turn computational work from your physics research into evaluation tasks that test AI models on expert-level scientific problems. You will build each task around work you have run and debugged, such as a simulation, numerical solver, data-reduction pipeline, or model fit, then provide a Python reference solution and tests. The role covers several physics fields and follows a flexible, task-based schedule.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$100/hr
  • computational physics
  • astrophysics
  • applied physics
AI evaluationView job

Turing · via Braintrust

Staff Research Engineer - Frontier AI and Reinforcement Learning - International

Advance frontier AI research across synthetic and agentic data, reinforcement learning, post-training, model understanding, benchmarks, and evaluation. Build research-grade prototypes and help translate validated findings into scalable systems. The project runs for three months and starts immediately. It requires at least 40 hours per week, eight hours per day, with four hours of daily Pacific Time overlap. Selection includes a 20-minute client AI interview and a 45-60-minute technical delivery interview. Include your LinkedIn profile URL with your resume.

Location and eligibility
Remote · Eligible in 9 countries
Engagement type
Contract
Compensation
$26/hr
  • python
  • machine learning
  • reinforcement learning
Machine Learning & AIView job

General Translation · via Mercor

Turkish Linguistic Expert

Assess Turkish-language output from a large language model and help shape the data used to improve its understanding and generation. You’ll review tasks such as translation, summaries, dialogue, and grammar correction, then annotate and refine Turkish text for accuracy, natural phrasing, and cultural fit. Your findings will inform AI researchers as they address model errors and language gaps.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$40/hr
  • turkish
  • english
  • linguistic annotation
Language & LinguisticsView job

Handshake AI

Category Manager

Use your retail category management experience to help improve AI research. You’ll write training questions and scenarios grounded in real workflows, assess AI responses, and give structured feedback. The ongoing, project-based work focuses on pricing strategy, markdown planning, and competitive analysis, with no fixed schedule or minimum hour requirement.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
$80/hr
  • retail category management
  • buying
  • pricing strategy
Product ManagementView job

Showing 20 active roles of 782. Dates reflect what the source published, not when AIWorkJobs last synced.

Sources: ten specialist AI work platforms. AIWorkJobs does not verify every claim made by a source. Report a listing that looks wrong through the contact page.