AIWorkJobs

Job index

AI / LLM Evaluation jobs

Current roles from specialist AI work platforms. Pay and eligibility are shown as each source publishes them.

Filters1ShowHide
Pay
Reset

Turing

Full Stack Engineer

Create and evaluate code examples for training AI language models.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Pay not listed
  • python
  • c++
  • rust
Software EngineeringView job

AfterQuery

Product Manager Expert

Shape how AI systems handle real product decisions. You’ll write scenarios drawn from ambiguous product problems, explain the reasoning behind a strong decision, and assess model responses to specs, roadmaps, and strategy. The work is remote and async, with a minimum commitment of 10 hours per week and no weekly maximum. AfterQuery hires contributors as independent contractors for ongoing project work.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$85/hr
  • product management
  • product strategy
  • product roadmaps
Product ManagementView job

Mercor

Chinese Linguistic Experts

Assess how well a large language model handles Chinese across translation, summaries, dialogue, and grammar correction. You’ll refine and annotate Chinese text, flag language issues, and share detailed feedback with AI researchers to support model training. The work calls for strong Chinese language expertise and professional experience with translation, editing, linguistic annotation, or NLP data.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$50/hr
  • chinese
  • linguistic annotation
  • translation
Language & LinguisticsView job

Handshake AI

Process Safety Engineer

Use your energy industry experience to help improve AI understanding of EHS workflows, process safety, and regulatory compliance. You’ll create expert training material, assess AI-generated responses, and share feedback based on the tools and practices you use at work. The project is remote, ongoing, and flexible, with no minimum hours; contributors typically work about 5–20 hours per week during active projects. No AI experience is required.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
Up to $85/hr
  • environmental health and safety
  • process safety
  • energy regulatory compliance
Data EngineeringView job

SME Careers

Filipino (Tagalog) Language Expert

Review Filipino/Tagalog AI responses or create reference answers that model accurate, natural writing. You will assess grammar, meaning, register, Taglish use, cultural fit, factual claims, and prompt compliance, then explain issues in English. This remote contractor role pays hourly; no project is active now, but qualified experts may be contacted when relevant work opens.

Location and eligibility
Remote · Philippines only
Engagement type
Contract
Compensation
Up to $20/hr
  • filipino
  • tagalog
  • english
Language & LinguisticsView job

Turing

Music and Audio experts

Evaluate AI-generated audio by identifying instruments in audio clips.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
$160/yr
  • instrument recognition
  • audio evaluation
  • music performance
Data annotationView job

AfterQuery

Literature Expert

AfterQuery engages literary scholars as independent contractors to create and assess close-reading scenarios for AI model evaluations. You will ground interpretations in textual evidence and judge model responses for fidelity, depth, and craft. Work is remote and asynchronous, with a minimum commitment of 10 hours each week, no fixed schedule, and ongoing projects. The selection process may include a written assessment.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$75/hr
  • close reading
  • literary analysis
  • comparative literature
Language & LinguisticsView job

Mercor

Telugu Linguistic Experts

Assess how well an LLM handles Telugu across translation, summarization, dialogue, and grammar correction. You’ll refine Telugu text, annotate linguistic features, and share detailed findings with AI researchers. The project also involves curating language data to support the model’s training and improve its Telugu comprehension and generation.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$30/hr
  • telugu
  • linguistic annotation
  • translation
Language & LinguisticsView job

Handshake AI

Visual Design Professional

As a Handshake AI Fellowship participant, you’ll help assess and develop how large language models handle visual storytelling, brand design, and professional communication. Work remotely on flexible, self-scheduled project assignments with no minimum hour commitment. Projects may involve writing design-focused prompts, reviewing model responses, and researching topics with AI. Placement depends on project availability, and the fellowship runs year-round with openings that vary by design specialty.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
$100–$125/hr
  • visual design
  • brand design
  • product design
Design & CreativeView job

SME Careers

Python Expert

Review AI-generated Python 3 answers and Chinese-language explanations for correctness, clarity, and adherence to prompts. You will compare responses, write reference solutions, and create Mandarin-language coding and debugging scenarios, with Traditional Chinese preferred. The remote contract work requires fluent Mandarin, C1-level English, and consistent application of evaluation rubrics.

Location and eligibility
Remote · Eligible in 7 countries
Engagement type
Full-time
Compensation
Up to $50/hr
  • python
  • python 3
  • code review
AI evaluationView job

Turing

MLE Bench – Data Analyst

Analyze ML datasets to evaluate and improve system performance.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
Pay not listed
  • machine learning
  • python
  • sql
Machine Learning & AIView job

AfterQuery

IT Support Specialist Expert

AfterQuery hires IT support specialists as independent contractors to create and evaluate troubleshooting material for frontier AI labs. You’ll write scenarios that cover the path from an initial user report to a fix, explain the diagnostic logic behind reference answers, and assess model responses for technical accuracy and user safety. The project offers ongoing work, a remote and asynchronous setup, and a minimum commitment of 10 hours each week with no maximum.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$63/hr
  • it support
  • help desk
  • system administration
IT & Systems AdministrationView job

Mercor

Hindi Linguistic Experts

Assess Hindi-language output from a large language model and prepare linguistic data that supports its training and improvement. You will review text across translation, summarization, dialogue, and grammar correction, then help an AI research team address language errors and gaps. The work calls for strong Hindi expertise and sound judgment on grammar, style, cultural context, and regional variation.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$30/hr
  • hindi
  • grammar
  • syntax
Language & LinguisticsView job

Handshake AI

Career and Technical Education Professors

College vocational instructors and technical education teachers can apply their workplace expertise to an hourly, temporary AI research project. You’ll review AI-generated material in your field and give clear, structured feedback about the tasks and language used in your work. No AI background is needed. Project openings appear throughout the year across different areas of expertise.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
Up to $75/hr
  • career and technical education
  • vocational instruction
  • technical education
Machine Learning & AIView job

SME Careers

Icelandic Language Expert

Review AI-generated Icelandic and create reference answers that model strong writing across general knowledge, customer support, education, and conversation. You will assess grammar, meaning, tone, cultural fit, factual claims, and prompt compliance, then explain your judgments in English. This hourly remote contract has no immediate project; qualified experts may receive invitations to relevant work through the network.

Location and eligibility
Remote · Iceland only
Engagement type
Contract
Compensation
Up to $55/hr
  • icelandic
  • english
  • copyediting
Language & LinguisticsView job

Turing

LLM Java Developer

Develop and maintain Java code for training and optimizing AI models.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
Pay not listed
  • java
  • large language models
  • llm training
Machine Learning & AIView job

Mercor

Arabic Linguistic Experts

Assess how well a large language model handles Arabic tasks such as translation, summarization, dialogue, and grammar correction. You will refine Arabic text, annotate linguistic features, and share findings with AI researchers to support model training. The role calls for native or near-native Arabic fluency and professional experience with Arabic language work.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$50/hr
  • arabic
  • arabic grammar
  • arabic syntax
Language & LinguisticsView job

Handshake AI

Music Professional

Use your expertise in instrumental performance, composition, or music production to create training data and assess AI-generated music content. You’ll give feedback on accuracy and relevance across musical styles, traditions, and cultures. The remote work runs on a flexible, asynchronous schedule with no minimum hours; contributors often work about 5–20 hours per week during active projects. Project placement depends on current needs, with future projects also possible.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
Up to $100/hr
  • instrumental performance
  • composition
  • music production
Machine Learning & AIView job

SME Careers

Hebrew Language Expert

Review and compare Hebrew and English AI responses, checking factual accuracy, reasoning, clarity, and fit with each prompt. You may also write bilingual training examples and model explanations. This remote, hourly contract role has no immediate project; qualified experts may receive invitations when relevant work becomes available.

Location and eligibility
Remote · Israel only
Engagement type
Contract
Compensation
Up to $50/hr
  • hebrew
  • english
  • bilingual evaluation
Language & LinguisticsView job

Turing

AI Quality Analyst (Personalization) - Korean

Evaluate personalized AI responses in Korean for relevance and grounding.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
$15/hr
  • korean
  • prompt engineering
  • ai evaluation
AI evaluationView job

Showing 20 active roles of 782. Dates reflect what the source published, not when AIWorkJobs last synced.

Sources: ten specialist AI work platforms. AIWorkJobs does not verify every claim made by a source. Report a listing that looks wrong through the contact page.