Turn computational research you have run and debugged into evaluation tasks for AI models. Each task centers on a real biological computing problem, with connected subproblems, a Python reference solution, and tests. You will also define evaluation metrics without sharing exact passing thresholds. Work follows a flexible, task-based schedule with no fixed weekly commitment.
Shape how frontier coding agents are tested and improved. You will create evaluation methods, datasets, and tools, then study model performance across software engineering tasks. The work combines Python and software engineering, LLM and agent systems, benchmark design, and model failure analysis. You will also share research findings with technical and client audiences.
Share your experience managing personal finances in Brazil and financial ties to the United States through a structured online survey. The survey takes about 20 minutes and includes multiple-choice questions plus 8–10 short written responses. Participants who submit a response that passes verification receive a one-time payment of $100.
Guide quality across Lithuanian AI training projects by reviewing language output and trainer work against project rubrics. You will coach trainers and QA reviewers, resolve Lithuanian language questions, and keep guidance current in style guides, trackers, and onboarding materials. The role calls for native or near-native Lithuanian, strong English, and at least three years of relevant language or quality-review experience.
Create research-based evaluation tasks that put autonomous AI systems to the test across several subject areas. You will write and refine prompts and responses, check claims against authoritative sources, and record detailed evaluation findings. The role calls for 10 to 20 hours of work each week, with flexible scheduling, as an independent contractor.
Develop and assess LLM-powered agents and AI systems used in enterprise workflows. The role combines applied research with hands-on engineering: you will design evaluations, investigate failures, run experiments, and help improve system performance. You will also build research tools and infrastructure while working with researchers, engineers, domain experts, and enterprise customers.
Help evaluate how advanced AI models respond to sensitive topics in Spanish. You’ll write prompts, assess conversations against structured guidelines, and explain judgments about adversarial wording and escalation patterns. The project starts immediately. No AI or machine learning background is needed; the team provides workflow training.
Lead Urdu quality work across AI training projects by reviewing contributor output against project rubrics, correcting language and formatting issues, and guiding trainers and QA reviewers. You will manage updates and documentation through tools such as Discord, Google Docs, and Google Sheets. The remote contract selection process includes an AI interview, a domain-specific task, and a recruiter interview.
Project Fenrir asks security researchers to uncover real vulnerabilities in private repositories and shape their findings into cybersecurity tasks that test frontier LLMs at different difficulty levels. Contributors earn per approved in-band task and qualifying repository, with referral and milestone bonuses also available.
Realm Lab needs a Member of Technical Staff to evaluate and improve frontier AI systems. You’ll work across machine learning, data research, and evaluation engineering, building benchmarks, reinforcement learning environments, and datasets. The role calls for hands-on experience with LLM evaluation, Python, and experimental methods, plus the ability to turn real-world workflows into structured tests of model capabilities.
Use your existing Linux Ubuntu 24.04 desktop environment for work involving the Ubuntu desktop, including GNOME, Files, Settings, the terminal, and package management. You must already use Ubuntu 24.04 as your primary desktop OS and have a qualifying display setup for screenshots. The role does not provide software licenses, virtual machines, or hardware.
Review AI-generated Malayalam for grammar, meaning, tone, and cultural fit, and create reference answers that demonstrate strong Malayalam writing. You will compare model responses, check claims when needed, and explain language or reasoning issues in written English. The role requires a mathematics degree, native or near-native Malayalam, C1-level English, and at least three years of relevant professional experience.
Create architecture scenarios that help AI labs assess model judgment, then compare model responses with sound design practice and code requirements. You’ll draw on experience from schematic design through construction documents, explain the reasoning behind design choices, and catch proposals that might fail plan review. The work is remote, asynchronous, and ongoing, with a minimum commitment of 10 hours per week.
As an International Humanitarian Law Advisor, you’ll create and assess material that helps train AI systems to apply IHL and human-rights principles. The work centers on lawful conduct, civilian protection, proportionality, and uncertain legal questions. The contractor role is about 15 hours per week. You do not need prior AI experience; the project relies on your legal expertise.
Mercor’s talent network is accepting applications for future opportunities building an AI-native platform that replaces spreadsheet-based operations with live dashboards and agentic workflows. You could deliver backend and frontend features, integrations, analytics, and MCP-compatible tools, then help launch and refine the product with pilot users. Complete your profile and pass an AI interview to become eligible for projects matched to your background as they open.
Build realistic electronics design exercises in KiCad, then record your workflow and document each task in SuperAnnotate. Work may cover schematic capture, footprint assignment, PCB routing, design checks, project organization, troubleshooting, and fabrication file export. You need a configured KiCad environment, clear English communication, and the ability to prepare complete, understandable recordings.
Create mortgage origination scenarios that help AI labs assess model judgment, from prequalification and loan selection through underwriting and closing. You’ll write answers that explain lending decisions, assess model responses against agency and regulatory guidelines, and catch advice that conflicts with lending rules. The work is remote and async, with a commitment of at least 10 hours per week and no stated maximum.
Apply antitrust and competition law expertise to a legal data annotation project supporting AI training. You’ll work with materials on merger reviews, compliance, monopolization, price fixing, and litigation, then help create accurate reference data and guidance. The role seeks US-based legal professionals; prior AI experience is not required.
Help assess how AI models respond to sensitive topics in Hindi. You’ll write prompts, apply review guidelines, and explain your judgments about risky or adversarial language. The role calls for native or near-native Hindi and business-level written English. No AI or machine learning experience is needed; training covers the workflow. The project starts immediately.
Review AI-generated Nepali writing and produce reference content that demonstrates accurate, natural language use. You will assess grammar, meaning, tone, cultural fit, and reasoning, then explain issues in written English and compare candidate responses. The role requires a mathematics degree, advanced Nepali and English proficiency, and at least three years of relevant language-focused experience.
Location and eligibility
Remote · Eligible in 4 countries
Engagement type
Contract
Compensation
Up to $15/hr
nepali
english
devanagari
Language & LinguisticsView job
Showing 20 active roles of 1,907. Dates reflect what the source published, not when AIWorkJobs last synced.
Sources: ten specialist AI work platforms. AIWorkJobs does not verify every claim made by a source. Report a listing that looks wrong through the contact page.