As a Data Scientist Quality Assurance Lead, you will review AI-generated data science work and contributor submissions for statistical soundness, code accuracy, reproducibility, and rubric fit. You will guide trainers and QAs, resolve technical questions, and maintain calibration and onboarding resources. This remote contract role has no immediate project; qualified experts may be contacted when relevant work becomes available.
Work on AI training data and evaluations for security and responsible AI, creating adversarial tasks, reviewing solutions, and assessing model responses. Cincinnatus LLC employs specialists as W-2 staff and places them with a leading AI lab. The arrangement includes payroll, benefits, and compliance support, with a weekday commitment of at least 20 hours per week.
Use your teaching expertise to assess educational materials and help develop training data for AI systems. You’ll review AI-generated lessons, quizzes, rubrics, student feedback, and parent communications, then flag accuracy, age-appropriateness, and teaching-practice concerns. No AI background is needed, but you must work full time in education and have at least two years of hands-on teaching experience.
Help an AI research team build dependable evaluation workflows by pairing software engineering with preference-data review and rubric development. The remote contract runs for about three months, pays $100 per hour, and is open to candidates in the United States or Canada. The selection process starts with an information form and AI interview; selected candidates may then complete a coding challenge, with scores of 75% or higher advancing to team review and a possible live interview.
Bring your experience with real operational, survey, or insurance data to the creation of research-level problems that test and train AI models. You will use Python or a similar language and explain technical work in your field. The work follows a flexible, task-based contract with no set weekly schedule. AfterQuery requires standard background and identity checks for accepted experts.
Use your quant experience on an ongoing, project-based effort to improve AI research. You’ll create realistic quantitative challenges, assess model answers for accuracy and practical value, and write feedback that reflects how working quants approach research, modeling, and strategy. Set your own pace with no fixed schedule or minimum hours, alongside your current role.
Terac is conducting a paid study on industrial equipment diagnostics and sensor data analysis. You’ll review equipment data and diagnostic outputs, explain how you assess equipment health, and judge whether automated workflows produce accurate, reliable evaluations. The study is remote and open to candidates in the listed eligible countries.
Work with a foundational AI lab to train a language model on software development. You will create, assess, and refine AI-generated work across core engineering workflows. The role is remote with a flexible schedule. Work depends on client task volume, and most experts contribute fewer than 10 hours per week; hours and earnings have no weekly cap but are not guaranteed.
As an independent contractor, you’ll install a specified application version and assess its voice and audio behavior against detailed project instructions. The work covers instruction adherence, model logic, dynamic test scenarios, and the naturalness of recorded audio outputs. A compatible device and operating system are required; the project team will confirm setup during application. Referrals who finish all 10 tasks earn you a $25 USD bonus when you invite them with your unique tracked link.
Lead Turkish quality checks across AI training projects by reviewing generated content and contributor work against project rubrics. You will give actionable feedback, guide trainers and QAs through language questions, and keep Turkish style guides and quality records current. The role also covers onboarding and contributor follow-up, with team communication through tools such as Discord and Google Sheets.
Reviewers help assess AI evaluation material for enterprise product design and manufacturing. You will check technical scenarios, reference answers, and grading rubrics against either the US ASME track, the international ISO track, or both. The work reflects the design complexity, safety demands, and production scale found at Fortune 500 companies and large industrial manufacturers.
Evaluate how personal AI agents handle sensitive information through privacy-focused tests and response reviews. The work centers on spotting improper disclosures and data-protection risks, then recording evidence-based feedback. This is a United States-based, part-time contractor project lasting six weeks. Applicants complete an AI interview; no other assessments are required.
Use your systems or hardware engineering experience to create difficult, clearly specified challenges for AI agents to solve in a Linux terminal. The project is task-based and expected to last a few weeks. Tasks average six to seven hours, with early submissions potentially taking longer. Earn up to $1,350 per task, with higher rates possible based on task quality and submission volume. Onboarding requires two courses, a graded assessment of about 90 minutes, and a live webinar.
Use hands-on power engineering experience to create research-level problems that test and train AI models. The work draws on real relay records and test data from substations, generators, and drives. AfterQuery offers remote, flexible contract tasks with no fixed weekly schedule. Accepted experts complete standard background and identity verification.
Help assess and improve coding agents by giving them real engineering tickets, running their work, and tracing mistakes through a codebase you know. The contract offers 40 hours each week at $6,000, paid each week, with hours you set. It renews each week when your work meets quality standards. Selection involves a proctored coding assessment and a paid trial task.
Lead language quality across Indonesian AI training projects by reviewing model content and trainer or QA work against project rubrics. You will assess grammar, meaning, tone, cultural fit, and instruction-following, then give contributors clear feedback and maintain Indonesian-specific guidance. The role also covers onboarding, quality documentation, and contributor follow-up using tools such as Discord and Google Sheets.
Review repository-level software engineering benchmarks used to train and evaluate a frontier AI lab’s models. You’ll examine task quality, reference patches, test harnesses, Docker isolation, and grading integrity, then write clear feedback against a rubric. The role calls for professional software engineering experience and open-source contribution or maintainer work. The listing is limited to the United States.
Shape how frontier coding agents are tested and improved. You will create evaluation methods, datasets, and tools, then study model performance across software engineering tasks. The work combines Python and software engineering, LLM and agent systems, benchmark design, and model failure analysis. You will also share research findings with technical and client audiences.
Location and eligibility
Remote · Eligibility not specified
Engagement type
Full-time
Compensation
$8–$9/hr
Coding Agent Evaluation & Benchmark Design
Software Engineering & Python
LLM & Agentic Systems
Machine Learning & AIView job
Showing 20 active roles of 113. Dates reflect what the source published, not when AIWorkJobs last synced.
Sources: ten specialist AI work platforms. AIWorkJobs does not verify every claim made by a source. Report a listing that looks wrong through the contact page.