AIWorkJobs

Job index

AI / LLM Evaluation jobs

Current roles from specialist AI work platforms. Pay and eligibility are shown as each source publishes them.

Filters2ShowHide
Pay
Reset

SME Careers

Customer Service (Call Center) Expert

Use your call-center experience to enact realistic customer interactions for a speech-to-speech AI dataset. You will follow project scenarios and quality guidance, then assess AI responses for accuracy, fluency, and fit with the conversation. The work draws on your knowledge of customer inquiries, complaints, escalation, and resolution.

Location and eligibility
Remote · Canada only
Engagement type
Contract
Compensation
Up to $30/hr
  • customer service
  • call center
  • ai training
Customer SupportView job

General Translation · via Mercor

Hebrew Linguistic Expert

Assess Hebrew output from a large language model across translation, summaries, dialogue, and grammar correction. You’ll also refine and annotate Hebrew text, then share linguistic findings with AI researchers to support model training. The project centers on Hebrew language quality, cultural fit, and the model’s ability to understand and generate natural text.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$40/hr
  • hebrew
  • english
  • translation
Language & LinguisticsView job

Outlier · via Braintrust

Senior Media, Audio & Linguistics Experts - AI Benchmarking

Create and assess benchmarks that test AI systems against real creative and technical workflows in music, audio, speech, linguistics, visual media, or design. The work includes Linux-based tasks and domain-focused review of AI results. The project pays $1,350 per task and expects a commitment of 20 hours per week for 1–6 months.

Location and eligibility
Remote · Eligible in 25 countries
Engagement type
Contract
Compensation
$1,350 fixed
  • linux
  • ai evaluation
  • audio production
Language & LinguisticsView job

AfterQuery

Speech Language Pathologist Expert

AfterQuery needs practicing speech-language pathologists to create and assess clinical scenarios that help AI labs evaluate model judgment in communication and swallowing care. You’ll write cases and reference answers, then review model responses for accuracy, safety, and sound clinical reasoning. The independent contractor work runs remotely and asynchronously, with a minimum commitment of 10 hours per week.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$88/hr
  • speech-language pathology
  • clinical evaluation
  • diagnosis
Language & LinguisticsView job

Handshake AI

AI Evaluation Specialist

Contribute to short-term AI research projects with leading AI labs by applying your academic knowledge to improve large language models in specific fields and topics. Work remotely with no minimum hour commitment. Handshake provides brief training, and you must pass one assessment before starting. Successful project work may qualify you for future projects.

Location and eligibility
Remote · United States only
Engagement type
Part-time
Compensation
$17/hr
  • large language models
  • ai evaluation
  • ai training
Machine Learning & AIView job

Turing

Business Analyst (German Language)

Analyze content and provide feedback to improve AI language models.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Freelance
Compensation
Pay not listed
  • business analysis
  • german
  • english
Business Analysis & StrategyView job

Meridial by Invisible

Danish speakers specialist

Work on an audio and language model evaluation project by assessing Danish samples for accuracy, fluency, naturalness, and overall quality. You will spot fine-grained differences in grammar, pronunciation, tone, phrasing, and word choice, including how spoken Danish sounds in context. Individual assignments take 30 seconds to 4 minutes.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$1.3 fixed
  • danish
  • danish grammar
  • danish spelling
Language & LinguisticsView job

Ethos

Special Education Specialist / IEP Consultant

Help train a language model for a foundational AI lab by creating, reviewing, and refining professional documents, spreadsheets, and slide decks for special education workflows. Work remotely on your own schedule, with a start date as soon as possible. Task volume varies, and most experts work under 10 hours per week; hours are not guaranteed, with no weekly cap.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$90/hr
  • special education
  • individualized education programs
  • iep writing
Education & TrainingView job

SME Careers

Norwegian Language Expert

Review AI-generated Norwegian and create reference answers that show accurate, natural writing across general knowledge, customer support, education, and conversation. You will assess grammar, meaning, tone, cultural fit, and factual claims, then explain issues in English. This remote, hourly contract has no immediate project; qualified experts may be contacted when relevant work becomes available.

Location and eligibility
Remote · Norway only
Engagement type
Contract
Compensation
Up to $55/hr
  • norwegian
  • bokmål
  • nynorsk
Language & LinguisticsView job

General Translation · via Mercor

Thai Linguistic Expert

Assess Thai-language output from a large language model and help build linguistic data that supports its training and improvement. You’ll review work such as translations, summaries, dialogue, and grammar corrections, then edit or annotate Thai text and share findings with AI researchers. The role calls for strong Thai language judgment, cultural awareness, and professional English fluency.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$40/hr
  • thai
  • english
  • translation
Language & LinguisticsView job

Confidential employer · via Braintrust

Principal Coding Annotator / LLM Evaluation Engineer

Bring hands-on software engineering judgment to the evaluation of large language models. You’ll create coding benchmarks, assess model responses, and investigate security-related tasks and reasoning failures. The contract starts at six months, with potential for a longer engagement. Paris or London candidates are preferred; strong candidates elsewhere in Europe may work remotely.

Location and eligibility
On-site · Eligible in 10 countries
Engagement type
Contract
Compensation
$75–$90/hr
  • software engineering
  • python
  • data annotation
Machine Learning & AIView job

AfterQuery

Content Writer & AI Evaluation Specialist (UK/US)

Create and assess writing for AI training, from prompts and stories to articles, summaries, and dialogue. You’ll draw on at least three years of English content writing or copywriting experience, apply your knowledge of large language models, and judge text for accuracy, quality, and fit with user intent. The independent contractor engagement calls for 40 hours each week over 3–6 months.

Location and eligibility
Remote · United Kingdom, United States
Engagement type
Not specified
Compensation
$43/hr
  • content writing
  • copywriting
  • content editing
Content, Communications & PRView job

Handshake AI

Linguistics Expert (India)

Use your linguistics or computational linguistics expertise on flexible, part-time contract projects supporting AI research. You’ll create realistic linguistic problems, assess AI responses for accuracy and analytical rigor, and write structured feedback. The work is ongoing and project-based, with no fixed schedule or minimum hour requirement, so you can complete assignments alongside your current role.

Location and eligibility
Remote · India only
Engagement type
Part-time
Compensation
$40/hr
  • linguistics
  • computational linguistics
  • morphology
Language & LinguisticsView job

Turing

Biology Expert (PhD/Master’s)

Design and solve complex Biology problems to enhance AI language models.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Freelance
Compensation
Pay not listed
  • biology
  • molecular biology
  • cell biology
Science & ResearchView job

Meridial by Invisible

Audio Evaluations: JSON & Coding Expert (Italian, Italy)

Review AI-generated coding and technical customer service content, including virtual agent replies, JSON, audio-related material, and other technical outputs. You’ll assess accuracy, clarity, natural Italian as used in Italy, and usefulness against project guidelines. Applicants complete an Italian and English speaking introduction plus a JSON Skills Assessment. You do not need to live in Italy.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$11/hr
  • italian
  • english
  • json
Language & LinguisticsView job

Ethos

Japanese Language & Cultural Fluency Consultant

Help a leading foundational AI lab train its language model by creating, reviewing, and refining professional documents, spreadsheets, and slide decks in Japanese and English. You will assess translation quality, localization, tone, cultural context, and business etiquette. Work remotely on your own schedule; task volume varies, and most experts work fewer than 10 hours per week. The role has no weekly cap on hours or earnings, and the start date is ASAP.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$100/hr
  • japanese
  • english
  • translation
Language & LinguisticsView job

SME Careers

Turkish Team Lead

Lead Turkish quality checks across AI training projects by reviewing generated content and contributor work against project rubrics. You will give actionable feedback, guide trainers and QAs through language questions, and keep Turkish style guides and quality records current. The role also covers onboarding and contributor follow-up, with team communication through tools such as Discord and Google Sheets.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $25/hr
  • turkish
  • english
  • llm evaluation
Language & LinguisticsView job

General Translation · via Mercor

Arabic Linguistic Expert

Assess how well a large language model handles Arabic tasks such as translation, summarization, dialogue, and grammar correction. You will also refine Arabic text, annotate linguistic features, and share findings with AI researchers to support model training. The work calls for native or near-native Arabic, professional English, and experience with Arabic language work.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$40/hr
  • arabic
  • english
  • linguistic annotation
Language & LinguisticsView job

AfterQuery

General AfterQuery Expert (Data Annotator)

Bring at least three years of expertise from a professional field to annotation work for text, documents, model outputs, and other domain-specific material. You will assess content with informed judgment, write structured annotations, and support decisions with reasoning and citations. The role offers flexible scheduling, project experience for your portfolio, and exposure to AI development workflows.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$43/hr
  • data annotation
  • content review
  • ai model evaluation
Data Science & AnalyticsView job

Handshake AI

Linguistics Expert (Germany)

Use your linguistics or computational linguistics expertise on flexible, part-time contract projects that support AI research. You will create realistic analytical, corpus, or computational tasks, assess model responses, and write feedback on their accuracy and rigor. Candidates with at least two of the three profile areas described below are encouraged to apply. The work has no fixed schedule or minimum hour requirement.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$65/hr
  • linguistics
  • computational linguistics
  • linguistic analysis
Language & LinguisticsView job

Showing 20 active roles of 195. Dates reflect what the source published, not when AIWorkJobs last synced.

Sources: ten specialist AI work platforms. AIWorkJobs does not verify every claim made by a source. Report a listing that looks wrong through the contact page.