AIWorkJobs

Job index

AI / LLM Evaluation jobs

Current roles from specialist AI work platforms. Pay and eligibility are shown as each source publishes them.

Filters1ShowHide
Pay
Reset

micro1

Senior Forward Deployed Engineer, U.S. Government

Lead applied AI engineering from research through production for high-assurance, mission-critical U.S. Government work. You’ll shape architecture, build agentic systems, and turn mission needs into reliable deployments with government partners. The role is based in the Greater Washington, DC area and requires onsite work 3-4 days each week.

Location and eligibility
Hybrid · United States only
Engagement type
Full-time
Compensation
$40–$60/hr
  • Agentic AI & LLM Systems Engineering
  • Python & Production Software Engineering
  • ML Infrastructure & Model Evaluation
Machine Learning & AIView job

Turing

Project Lead - AI Training

Lead large-scale AI annotation projects while managing clients and teams.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Freelance
Compensation
Pay not listed
  • ai annotation
  • multimodal annotation
  • project management
Machine Learning & AIView job

AfterQuery

Veterinarian Expert

AfterQuery seeks licensed veterinarians to create clinical cases and assess how AI models handle veterinary reasoning. You’ll write scenarios and reference answers across species, then review model responses for diagnostic accuracy, toxicity risks, and safety. The work runs remotely on an async schedule as ongoing contractor work, with a minimum commitment of 10 hours per week.

Location and eligibility
Remote · United States only
Engagement type
Not specified
Compensation
$120/hr
  • veterinary medicine
  • clinical reasoning
  • small animal medicine
AI evaluationView job

Mercor

Open Source Maintainer / Core Contributor – Software Engineer

Bring deep ownership of a mature open-source project to AI model training work focused on software engineering, code review, and maintenance. You’ll create repository-based tasks, assess AI-generated code and tests, and explain your technical judgments. The commitment is 30–40 hours each week as your primary work focus, with flexible scheduling, mostly asynchronous work, and occasional researcher meetings. Cincinnatus LLC engages you and may place you with a leading AI lab.

Location and eligibility
Remote · Canada, United States
Engagement type
Hourly
Compensation
$150/hr
  • open-source maintenance
  • code review
  • software engineering
Machine Learning & AIView job

Handshake AI

Physics and Engineering Expert

Use your advanced physics training to assess AI-generated material and help improve its treatment of scientific topics. You’ll create field-specific questions and review answers for accuracy, sound reasoning, and scientific rigor. The work runs on an hourly, project basis and can fit alongside research, teaching, postdoctoral work, or industry employment. Projects are remote, with flexible scheduling and no minimum hours.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
Up to $100/hr
  • physics
  • engineering
  • experimental methodology
EngineeringView job

SME Careers

R Team Lead

Lead quality checks for R programming and data-analysis work used in AI training projects. You will assess generated code, statistical explanations, charts, and trainer reviews against project rubrics, then give specific feedback and maintain guidance for contributors. The role is a remote contract with no immediate project; qualified experts may be contacted when relevant work becomes available. Selection includes an AI interview, a domain task, and a recruiter interview.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $65/hr
  • r
  • tidyverse
  • base r
Quality Assurance & TestingView job

micro1

Federal Civil Litigation Attorney

A practicing attorney with federal civil litigation experience will assess AI-generated motions and legal arguments against real-world court standards. The work combines comparative review, editing to a ready-to-file standard, and the creation of realistic motion-writing assignments and evaluation criteria.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
$150–$250/hr
  • federal civil litigation
  • motion drafting
  • legal research
LegalView job

Turing

Python + Full-Stack (JS) Developer

Develop and maintain high-quality code for AI model training and optimization.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
Pay not listed
  • python
  • javascript
  • typescript
Machine Learning & AIView job

AfterQuery

Computational Biology Researcher (Research Coding Tasks)

AfterQuery hires computational biology researchers and research engineers for contract work that supports AI model testing. You will create self-contained research coding tasks and assess model outputs in areas such as read alignment, genome assembly, indexing, and variant calling. The project lead provides the task scope and a worked example after onboarding. Work has no fixed weekly schedule.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$50/hr
  • computational biology
  • research coding
  • read alignment
Science & ResearchView job

Mercor

Disease-Area Clinician — Trial Endpoints & Prescribing

A biotech and pharma research team needs a practising physician to assess commercial drugs and development programs in a paid pilot. You’ll build and review evaluation rubrics, interpret clinical trial endpoints, and judge whether trial results could change prescribing or affect real-world uptake in your therapeutic area. The pilot runs remotely for about 10–20 hours over one to two weeks, with potential for recurring work as the project grows.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
$150–$230/hr
  • clinical trial endpoints
  • prescribing
  • drug development
Healthcare & MedicalView job

Handshake AI

Senior Software Engineer - AI Data (MX)

Review coding agents as they tackle real engineering tickets, then work through problems in codebases you know to pinpoint and correct model mistakes. The contract offers 40 hours of work each week, with flexible scheduling and weekly renewal when your work meets quality standards. The selection process includes a proctored coding assessment and a paid trial task.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$60/hr
  • software engineering
  • code review
  • coding agents
Machine Learning & AIView job

SME Careers

Polish Team Lead

Lead quality workflows for Polish AI training projects by reviewing Polish-language content and contributor work against project rubrics. You will give trainers and QAs specific feedback, address language and localization questions, and keep style guides and quality records current. The role calls for strong Polish and English, plus experience in language review; selection includes an AI interview, a domain-specific task, and a recruiter interview.

Location and eligibility
Remote · Poland only
Engagement type
Contract
Compensation
Up to $35/hr
  • polish
  • english
  • polish grammar
Language & LinguisticsView job

micro1

Civilian Intelligence Agency Expert – U.S. & Allied Government

Current and former civilian intelligence professionals can bring firsthand agency knowledge to a project with a frontier AI lab. You will assess AI-generated intelligence content and unclassified scenarios, then give evidence-based feedback to improve how AI systems represent agency operations, decision-making, and national security work.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
$80–$170/hr
  • intelligence analysis
  • program management
  • organizational decision-making
CybersecurityView job

Turing

Clinical & Health Specialist

Develop and evaluate complex clinical tasks and scenarios for accuracy.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
Pay not listed
  • clinical reasoning
  • chart review
  • clinical documentation
Healthcare & MedicalView job

AfterQuery

Certified Financial Planner Expert

Create and assess financial planning material that AI labs use to evaluate model judgment. You will build client scenarios, explain the reasoning behind recommendations, and spot advice that could put a client at risk. The work is remote and fully async, with no fixed hours or weekly maximum. Expect ongoing independent-contractor work, with a 10-hour weekly minimum.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$108/hr
  • financial planning
  • retirement planning
  • tax planning
Finance & AccountingView job

Mercor

Partner & General Counsel - Legal Domain Expert

Cincinnatus LLC employs a senior legal specialist for a full-time, 40-hour-per-week role supporting a leading AI lab’s research and program management teams. You will assess legal work and model responses, write task instructions and reference solutions, and develop evaluations for legal reasoning. The initial engagement lasts six months. The role requires regular on-site work in the Bay Area.

Location and eligibility
Hybrid · United States only
Engagement type
Full-time
Compensation
$100–$150/hr
  • legal reasoning
  • legal research
  • legal writing
LegalView job

Handshake AI

Senior Software Engineer - AI Data (IN)

Use your software engineering experience to assess coding agents on real engineering problems. You’ll give an agent a ticket and review its work, then work through a problem in a codebase you know and mark where the model fails. The contract offers 40 hours of work each week, with flexible scheduling and weekly renewal based on quality standards.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$60/hr
  • software engineering
  • code review
  • pull requests
Machine Learning & AIView job

SME Careers

Slovak Team Lead

Guide Slovak quality across AI training projects by reviewing model outputs and trainer work against project rubrics. You will give precise feedback, resolve language questions, and keep Slovak style guides and quality records current. The role also includes onboarding trainers and QAs, sharing project updates through Discord, and tracking contributor activity.

Location and eligibility
Remote · Slovakia only
Engagement type
Contract
Compensation
Up to $25/hr
  • slovak
  • english
  • slovak grammar
Language & LinguisticsView job

micro1

Tax Practitioner, Federal Individual Income Tax (CPA/EA/Tax Attorney)

Apply federal individual income tax expertise to help assess and improve AI systems. You’ll interpret tax authorities, create answer-scoring criteria, and review AI-generated analyses and calculations. The work also calls for realistic test cases and clear written feedback. Applicants may work from the US or India; prior AI experience is not required.

Location and eligibility
Remote · India, United States
Engagement type
Contract
Compensation
$60–$90/hr
  • federal individual income tax law interpretation
  • internal revenue code analysis
  • tax research and compliance
Finance & AccountingView job

Turing

Portfolio Management Expert

Evaluate AI models for portfolio management and performance analysis.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Freelance
Compensation
$100/hr
  • portfolio management
  • asset allocation
  • portfolio construction
Machine Learning & AIView job

Showing 20 active roles of 782. Dates reflect what the source published, not when AIWorkJobs last synced.

Sources: ten specialist AI work platforms. AIWorkJobs does not verify every claim made by a source. Report a listing that looks wrong through the contact page.