AIWorkJobs

Job index

AI / LLM Evaluation jobs

Current roles from specialist AI work platforms. Pay and eligibility are shown as each source publishes them.

Filters1ShowHide
Pay
Reset

Handshake AI

Japanese AI Evaluation Specialist (US / Japan)

Use your Japanese language skills and real-world experience with AI tools to compare and rank AI-generated responses. You’ll explain your preferences in clear written feedback that can help future models tailor answers to individual users. The ongoing, project-based work has no set schedule or minimum hours, so you can complete tasks at your own pace alongside your current role. No prior AI training experience is required.

Location and eligibility
Remote · Japan, United States
Engagement type
Part-time
Compensation
$30/hr
  • japanese
  • english
  • ai response evaluation
AI evaluationView job

SME Careers

Estonian Team Lead

Lead quality checks for Estonian AI training projects by reviewing generated content and contributor work against project rubrics. You will coach trainers and QA reviewers, resolve language questions, and keep Estonian style guides and quality records current. The role calls for strong written Estonian and English, plus experience with language-review workflows.

Location and eligibility
Remote · Estonia only
Engagement type
Contract
Compensation
Up to $40/hr
  • estonian
  • english
  • estonian grammar
Language & LinguisticsView job

micro1

Military Training and Readiness Advisor

As a contractor on flexible, project- or task-based work, you’ll apply military expertise to AI training and evaluation projects. You’ll assess unclassified scenarios, training content, and readiness plans, with a focus on doctrine, preparedness, and operational capability. Project length depends on customer needs. Applicants complete an AI interview as part of the application process.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
$90–$170/hr
  • military training assessment
  • operational readiness evaluation
  • doctrine development
Business Analysis & StrategyView job

Turing

Senior Software Engineer – LLM Evaluation (US-based)

Create and evaluate datasets to enhance large language model performance.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Pay not listed
  • python
  • javascript
  • react
Machine Learning & AIView job

AfterQuery

Occupational Therapist Expert

Create occupational therapy scenarios that help AI labs assess model judgment about daily function and independence. You will write clinical reference answers, assess model responses, and identify advice that would not work in a real home or clinic. The role is remote, asynchronous contractor work with no fixed schedule. Set your weekly hours, with a commitment of at least 10 hours.

Location and eligibility
Remote · United States only
Engagement type
Not specified
Compensation
$88/hr
  • occupational therapy
  • nbcot certification
  • ot assessment
AI evaluationView job

Confidential employer · via Braintrust

Principal Coding Annotator / LLM Evaluation Engineer

Bring hands-on software engineering judgment to the evaluation of large language models. You’ll create coding benchmarks, assess model responses, and investigate security-related tasks and reasoning failures. The contract starts at six months, with potential for a longer engagement. Paris or London candidates are preferred; strong candidates elsewhere in Europe may work remotely.

Location and eligibility
On-site · Eligible in 10 countries
Engagement type
Contract
Compensation
$75–$90/hr
  • software engineering
  • python
  • data annotation
Machine Learning & AIView job

Mercor

Brand & Advertising Art Director (AI Evaluation)

Shape design-focused AI training by creating realistic brand and advertising assignments, reviewing model outputs, and setting clear standards for quality. The role calls for an experienced art director or senior designer with agency, studio, or in-house expertise across identity, packaging, out-of-home, and campaign work. Cincinnatus LLC employs this full-time W-2 role at 40 hours per week and places the employee with a leading AI lab.

Location and eligibility
Remote · United States only
Engagement type
Full-time
Compensation
$70–$120/hr
  • brand identity
  • advertising design
  • graphic design
Design & CreativeView job

Handshake AI

Computer Science Expert

Handshake AI's year-round fellowship connects Computer Science doctoral students and graduates with projects that help improve large language models. You may develop subject-specific prompts, assess model responses, and research topics of interest with AI. Work is remote and asynchronous, with flexible hours and no minimum time commitment. Project placement depends on availability in your domain.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
Up to $75/hr
  • computer science
  • large language models
  • prompt development
Machine Learning & AIView job

SME Careers

Kannada Expert with Mathematical Background

Use your Kannada expertise and mathematics degree to assess AI-generated language and create reference answers across general knowledge, customer support, education, and conversation. You will judge grammar, meaning, tone, cultural fit, and prompt adherence, then explain issues in English. The role is remote and hourly, with a contractor workflow; selection includes an AI interview, a domain task, and a recruiter interview.

Location and eligibility
Remote · Eligible in 5 countries
Engagement type
Contract
Compensation
Up to $20/hr
  • kannada
  • mathematics
  • english
Language & LinguisticsView job

micro1

NSA / Signals Intelligence (SIGINT) Expert

Apply your government or military SIGINT experience to projects that test AI systems. You will build unclassified scenarios, review generated intelligence assessments, and help set standards for sound analysis. The work is flexible and project- or task-based. Candidates complete an AI interview during the application process.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
$70–$140/hr
  • signals intelligence (sigint) analysis
  • intelligence scenario development
  • analytical reasoning
CybersecurityView job

Turing

Ph.D. / Postdoctoral / Master’s Expert

Design and solve complex STEM problems to enhance language model performance.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Freelance
Compensation
Pay not listed
  • physics
  • mathematics
  • biology
Science & ResearchView job

AfterQuery

Media and Communications Expert

Create and assess media and communications challenges for AI training modules. You will build scenarios around messaging, content strategy, and public relations, then review model answers against communications best practices. The role suits someone with relevant education or equivalent experience and a background in media production, communications strategy, public relations, or journalism. The work is remote and flexible.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$55/hr
  • media production
  • communications strategy
  • public relations
Content, Communications & PRView job

Scaler AI Labs · via Braintrust

Consumer App Data Contributor - AI Training Project

Contribute authentic data from 6–8 mobile or web applications you use to support AI research. You’ll export your own account data, submit it through PeakTalent, and follow its redaction and quality review steps. The remote project is open to US residents age 18 or older, with no technical or professional experience required. Payment ranges from $350 to $450, based on approved submissions, and comes after final approval.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
$350 fixed
  • data collection
  • data annotation
  • data export
AI evaluationView job

Mercor

Senior Motion Designer & Creative Technologist (AI Evaluation)

Create motion design tasks and assess AI-generated video for a leading AI lab. The work calls for deep experience with motion graphics, code-driven animation, and generative image and video tools. Cincinnatus LLC employs this full-time W-2 role and places the designer within the client’s team. The schedule requires 40 hours each week on weekdays.

Location and eligibility
Remote · United States only
Engagement type
Full-time
Compensation
$70–$120/hr
  • motion design
  • after effects
  • kinetic typography
Design & CreativeView job

Handshake AI

Energy Auditor

Use your building energy expertise on ongoing, project-based AI research work. You’ll write training questions grounded in real auditing and modeling workflows, assess AI responses, and give written feedback. The work is part-time and remote, with no set schedule or minimum hour requirement, so you can complete tasks at your own pace alongside other work.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
Up to $80/hr
  • building energy management
  • energy auditing
  • building energy modeling
Energy & Power GenerationView job

SME Careers

Swedish Team Lead

Guide Swedish quality work across AI training projects by reviewing generated content and contributor submissions against project rubrics. You will give trainers and QA reviewers specific feedback, resolve language questions, and keep Swedish guidance current in Discord and project documentation. The role calls for native or near-native Swedish, strong English, and at least three years of relevant language or quality experience.

Location and eligibility
Remote · Sweden only
Engagement type
Contract
Compensation
Up to $60/hr
  • swedish
  • english
  • ai training
Language & LinguisticsView job

micro1

Federal Management & Program Analyst

Bring federal program and policy expertise to an AI evaluation project. You will write realistic agency scenarios, prepare reference materials, and assess work from other contributors against federal planning, evaluation, and reporting practices. The role draws on skills such as policy analysis, decision memo writing, and executive briefing preparation. No previous AI experience is required, but applicants must be based in the United States.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
$60–$80/hr
  • federal program management
  • policy analysis
  • decision memo writing
Business Analysis & StrategyView job

Turing

AI Quality Analyst (Personalization) - Indonesian

Evaluate personalized AI responses in Indonesian for relevance and accuracy.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
$15/hr
  • indonesian
  • ai evaluation
  • personalization
AI evaluationView job

AfterQuery

Medical MD Expert

Create medical cases and decision challenges that help train AI systems on complex healthcare questions. You will assess model responses for sound clinical reasoning and alignment with current guidelines, then work with a cross-functional team to shape cases around real practice. The role is remote and asynchronous, and requires an MD degree and current licensure.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$160/hr
  • clinical medicine
  • medical case studies
  • clinical decision-making
Healthcare & MedicalView job

Outlier · via Braintrust

Senior U.S. Attorney & Legal Expert - AI Training

Create challenging legal prompts and evaluations for a short-term AI training project, using multiple input files as source material. You’ll assess model responses for legal accuracy, realism, and faithful use of those materials. The project is open to U.S.-based professionals and pays $105 per hour.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
$105/hr
  • legal writing
  • contract law
  • landlord-tenant law
LegalView job

Showing 20 active roles of 782. Dates reflect what the source published, not when AIWorkJobs last synced.

Sources: ten specialist AI work platforms. AIWorkJobs does not verify every claim made by a source. Report a listing that looks wrong through the contact page.