Review real engineering work using the same judgment you bring to pull requests in established open source projects. Each review takes about 30 minutes, and the workload ranges from 5 to 20 hours a week. The remote schedule can fit around your own projects. Applicants should share a GitHub profile.
Use your existing Linux Ubuntu 24.04 desktop environment for work involving the Ubuntu desktop, including GNOME, Files, Settings, the terminal, and package management. You must already use Ubuntu 24.04 as your primary desktop OS and have a qualifying display setup for screenshots. The role does not provide software licenses, virtual machines, or hardware.
Review and create Vietnamese-language material for AI training, assessing response accuracy, reasoning, and compliance with prompts. You will apply senior-level localization judgment, MQM/LQA methods, and CAT tools to check terminology, tone, cultural nuance, and factual details. The role is remote and hourly under a contract arrangement; no immediate project is available, but qualified experts may be contacted for future work.
Bring principal-level engineering or faculty expertise to the design and review of technical problems used to assess AI systems. You’ll work in aerospace, semiconductors, manufacturing and industrial engineering, or materials science, creating realistic challenges with clear standards for correct answers. The contractor role is fully remote and offers flexible commitment alongside your current work. Prior AI experience is not required.
AfterQuery creates evaluation data that AI labs use to assess and improve their models. You’ll draw on your own research to develop and solve machine learning analysis problems covering experiment design, ablations, model evaluation, and behavior. The role suits hands-on researchers rather than engineers focused on deployment, data engineers, or annotators. Work remotely with an asynchronous schedule and no fixed hours.
Help assess how AI models respond to sensitive topics in Hindi. You’ll write prompts, apply review guidelines, and explain your judgments about risky or adversarial language. The role calls for native or near-native Hindi and business-level written English. No AI or machine learning experience is needed; training covers the workflow. The project starts immediately.
Use your finance or accounting expertise to create realistic professional problems and assess AI-generated analyses. You will build scoring rubrics that capture sound calculations, assumptions, methods, and judgment, then review outputs and peer criteria for technical accuracy and completeness across areas such as valuation, reporting, audit, and investment analysis.
Create demanding materials science problems grounded in real constraints and data, then provide the analysis and standards needed to judge each answer. The work draws on your own technical specialty, from processing and characterization to computational materials or failure analysis. Applicants need an advanced degree and principal-, staff-, or fellow-level industry or national lab experience, or a faculty appointment.
AfterQuery hires electrical engineering contractors to create and assess scenarios that AI labs use to evaluate model judgment. You will develop engineering cases, write reasoned reference answers, and review model responses for sound technical decisions and standards compliance. Work runs remotely on an asynchronous schedule, with a 10-hour weekly minimum, no maximum, and ongoing projects matched to your specialty. The application asks for a resume and structured questions; a written assessment may follow.
Mercor keeps this listing open to build a pool of mathematicians for future AI training and evaluation projects; it does not represent one active opening. You may write research- or competition-level problems with solutions, assess model-generated proofs, and identify gaps in reasoning. After you apply, a short AI interview takes about 20 minutes. Matching projects may start within a week or several months.
Review AI-generated Thai responses and create reference answers that demonstrate accurate, natural writing. You will judge grammar, meaning, tone, cultural fit, and prompt compliance, then explain issues in English and compare response quality. The hourly remote contract has no immediate project; qualified experts may hear about relevant work through the expert network.
As a contractor, you’ll create demanding manufacturing and industrial engineering problems, solve them, and set clear standards for judging answers. The work centers on cases where constraints determine the right method and a plausible solution may still be wrong. The schedule can fit alongside your current role, and the position is remote. Selection includes screening questions, an AI interview, profile review, and onboarding.
AfterQuery hires practicing PAs to create and assess clinical reasoning scenarios for AI model evaluations. You’ll write cases from presentation through workup and treatment plan, explain the reasoning behind clinical decisions, and review model answers for diagnostic accuracy, guideline concordance, and patient safety. The work is remote and async, with a minimum commitment of 10 hours per week.
Listen to AI-generated audiobooks in US and North American English and assess whether the narration sounds natural, accurate, and enjoyable. You’ll mark speech and audio issues in review tools, identify where the listening experience falters, and rate listener satisfaction. The role calls for native or near-native English fluency, regular audiobook listening, and availability for about 20 hours each week.
Review AI-generated Catalan and create reference answers that model accurate, natural writing. You will judge grammar, meaning, tone, regional fit, and cultural context, then explain errors and compare response quality. Strong Catalan writing and C1 English are essential for following evaluation guidance and delivering detailed feedback in a remote contract workflow.
As a remote contractor, you’ll create semiconductor engineering problems that test judgment beyond textbook answers, then explain the solution and set standards for evaluating it. The work can fit alongside your current role. Applicants complete screening questions and an AI interview, followed by profile review and onboarding with Talent Ops.
Create and assess advanced chemistry problems that help train AI models. You will write prompts grounded in real chemical principles, develop graduate- and PhD-level questions with solutions, review model responses, and give detailed feedback on question quality and the model’s grasp of chemistry. Assignments are project-based, remote, and offer flexible hours.
Use your research expertise to assess mathematical reasoning in AI model outputs and shape the standards used to judge correctness. You will write rigorous problem specifications, reference solutions, and proofs, then build evaluation sets in your area of specialization. The role is fully remote and requires a United States base.
Based in Germany, you will assess AI-generated German content and create language examples that help improve model responses. Apply MQM/LQA methods to judge accuracy, reasoning, tone, and cultural fit, then document findings and compare candidate answers. The hourly contract has no immediate project; qualified experts may receive invitations when suitable work becomes available.
Work remotely on aerospace technical evaluation by creating challenging problems grounded in real engineering constraints, then documenting the reasoning and standards that distinguish a correct answer from a plausible but flawed one. The commitment is flexible and can fit alongside a current role. Applicants complete screening questions and an AI interview before profile review and onboarding.
Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
$85–$185/hr
Technical Depth in a Subdomain
Engineering Judgment Under Constraints
Knowing When an Answer Is Wrong
EngineeringView job
Showing 20 active roles of 801. Dates reflect what the source published, not when AIWorkJobs last synced.
Sources: ten specialist AI work platforms. AIWorkJobs does not verify every claim made by a source. Report a listing that looks wrong through the contact page.