AIWorkJobs
Turing logoTuringPlatform listing

AI Benchmark & Red-Teaming Specialist

Research and build benchmark tasks to expose AI model weaknesses.

Remote · Eligibility not specifiedContract
Work arrangement
Remote
Eligibility
Eligibility not specified
Engagement
Contract
Compensation
Pay not listed
Specialty
AI Training

About the role

Build web-research benchmark tasks that reveal weaknesses in leading AI models. You will uncover hard-to-find facts, verify them against strong sources, test tasks against designated models, and trace failures through search logs and reasoning outputs. The contractor engagement runs for up to 4 weeks, with a commitment of up to 40 hours per week and 4 hours of overlap with PST.

What you would do

  • Research niche topics to uncover facts that basic search queries do not reveal.
  • Confirm findings with primary or credible sources, check for conflicting evidence, and record exact URLs.
  • Write clear benchmark prompts with answer keys supported by primary sources.
  • Run tasks against designated leading models and confirm that they trigger the intended failures.
  • Review model logs, search paths, and intermediate outputs to identify failure causes and distinguish model limitations from unclear prompts.
  • Adapt validated model-breaking research patterns to new subject areas.
  • Review tasks from peer trainers when assigned.

Requirements

  • At least 1 year of experience in investigative research, fact-checking, AI benchmark task authoring, red-teaming, or technical QA review.
  • Experience authoring, testing, or reviewing tasks for an AI benchmark such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench, or Humanity’s Last Exam.
  • Intermediate-to-advanced web search and OSINT skills, including the ability to use academic archives, local records, or obscure databases.
  • Ability to inspect model reasoning chains, search logs, and source choices to find the root cause of a failure.
  • Ability to write unambiguous benchmark prompts and answer keys grounded in primary sources.
  • A bachelor’s degree or higher in a relevant engineering domain.
  • Submit a Job Interest Form and complete the assessment as part of the evaluation process.

Preferred qualifications

The source marks these as preferred, not required.

  • Experience designing prompt-injection or reasoning-trap tasks that challenge leading models.
  • Advanced OSINT skills or published investigative or academic research that demonstrates deep source verification.
  • Expertise in a niche field with hard-to-find records, such as local government filings, regional sports history, legal archives, or historical patents.
  • Experience as a lead reviewer, benchmark auditor, or quality owner on AI evaluation projects.

Who should apply

Candidates with experience in investigative research, benchmark authoring, red-teaming, or technical QA will fit the core work. Strong web-search and OSINT skills, careful source verification, and the ability to diagnose model failures matter most.

How applying works on Turing

Create a profile, finish any required proctored assessments, then complete screening or a role interview.

  1. Apply through work.turing.com and complete your profile.
  2. Finish the required proctored assessments for the role.
  3. Complete shortlisting, screening, or a technical interview.
  4. Confirm rate, availability, and eligibility before an offer.

Typical timeline: Review takes 48 hours to six weeks after all required steps; consideration replies are usually within 12 hours.

Paid steps: Not stated by the source

Not every role requires tests. Turing does not review an application until required assessments are complete.

Process checked Nov 19, 2026. AIWorkJobs does not submit applications or charge a fee. Read the full Turing process and its official source on the Turing page.

Skills mentioned by the source

  • large language models
  • ai benchmarking
  • red teaming
  • open-source intelligence
  • web research
  • fact-checking
  • prompt engineering
  • model evaluation
  • quality assurance
  • source verification
  • search log analysis
  • llm
  • quality management
  • qa

Specialty areas

Specialties come from the source platform’s category taxonomy and are used to filter the job index.

Listing reference: ai-benchmark-red-teaming-specialist-113843. Source: Turing. If something here contradicts the source, the source controls. Report a correction.

Related roles

Other work on the index.

See more in this category

Outlier · via Braintrust

Senior Finance & Operations Experts - AI Benchmarking

Create realistic finance and operations challenges that help benchmark AI systems. You’ll build scenarios from professional workflows, set clear criteria for correct results, and assess proposed answers for accuracy and sound reasoning. The project expects 20 hours of work per week for 1–6 months and pays $1,350 per task.

Location and eligibility
Remote · Eligible in 25 countries
Engagement type
Contract
Compensation
$1,350 fixed
  • finance
  • operations
  • compliance
Machine Learning & AIView job

Mercor

SWE-Bench Task Auditor

Review repository-level software engineering benchmarks used to train and evaluate a frontier AI lab’s models. You’ll examine task quality, reference patches, test harnesses, Docker isolation, and grading integrity, then write clear feedback against a rubric. The role calls for professional software engineering experience and open-source contribution or maintainer work. The listing is limited to the United States.

Location and eligibility
Remote · United States only
Engagement type
Hourly
Compensation
$70–$90/hr
  • python
  • java
  • go
Quality Assurance & TestingView job

micro1

AI Trainer (Earth & Environmental Sciences)

micro1 seeks computational Earth and environmental scientists with paid AI-training or model-evaluation experience for a project that improves advanced AI systems. You will build technical challenges and assess model outputs using scientific and programming expertise. The contractor role has a flexible schedule and requires at least 15 hours per week. Compensation is based on accepted work that meets project specifications and quality standards; task availability and completion time can vary.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
$60–$120/hr
  • Earth & Environmental Sciences
  • AI Training
  • Programming Skills
Machine Learning & AIView job