AIWorkJobs
Micro1 logomicro1Platform listing

Member of Technical Staff, Coding Agent Research & Evaluations

Shape how frontier coding agents are tested and improved. You will create evaluation methods, datasets, and tools, then study model performance across software engineering tasks. The work combines Python and software…

Remote · Eligibility not specifiedFull-time
Work arrangement
Remote
Eligibility
Eligibility not specified
Engagement
Full-time
Compensation
$8–$9/hr

About the role

Shape how frontier coding agents are tested and improved. You will create evaluation methods, datasets, and tools, then study model performance across software engineering tasks. The work combines Python and software engineering, LLM and agent systems, benchmark design, and model failure analysis. You will also share research findings with technical and client audiences.

What you would do

  • Create coding-agent evaluation frameworks, including benchmark specifications, scoring methods, rubrics, and quality standards.
  • Lead research projects that measure and improve coding model performance across varied software engineering tasks.
  • Build datasets, gold-standard examples, and evaluation protocols for assessing frontier coding systems.
  • Investigate model behavior and failure patterns, then turn findings into recommendations for training and evaluation.
  • Develop tools and infrastructure for large-scale experiments, data generation, review workflows, and evaluation pipelines.
  • Set assessment practices that support methodological rigor, reproducibility, and sound measurement.
  • Work with researchers, engineers, and applied AI teams to plan experiments and assess emerging model capabilities.
  • Write technical reports and benchmark studies, and contribute to client-facing research that explains model performance and findings.

Requirements

  • At least 3 years of experience in software engineering, machine learning, AI research, evaluation, or a related technical field.
  • Strong software engineering skills and expertise in Python, C++, or a comparable programming language.
  • Experience designing, reviewing, or validating technical assessments, benchmarks, coding tasks, or evaluation methods.
  • Familiarity with large language models, coding agents, reinforcement learning, model evaluation, or related AI systems.
  • Ability to build tools, automate workflows, and improve technical processes through structured experimentation.
  • Strong analytical skills for examining model behavior and drawing insights from complex technical systems.
  • Strong written and verbal communication skills, including explaining technical findings to varied audiences.
  • Comfort working amid research uncertainty and changing priorities.

Preferred qualifications

The source marks these as preferred, not required.

  • Experience with frontier AI systems, coding agents, or model evaluation research.
  • Interest in how data, evaluations, and feedback mechanisms affect model capabilities.
  • A record of taking ambiguous technical or research projects from initial concept through execution.
  • Experience designing benchmarks or datasets for machine learning systems at scale.
  • Familiarity with agentic workflows, tool use, reinforcement learning, or post-training methods.
  • Publications, open-source work, or demonstrated technical leadership in AI, machine learning, or software engineering.

Who should apply

Candidates with at least three years in software engineering, machine learning, AI research, or evaluation who can build with Python or a comparable language and assess coding systems. Experience with benchmarks, model behavior, and technical communication fits the work.

How applying works on Micro1

Apply through the opportunities portal, complete an AI interview, verify your identity, and join the certified talent pool.

  1. Apply through the opportunities portal or accept a platform invitation.
  2. Complete the AI interview covering communication, reasoning, and role skills.
  3. Verify your government ID and sign the platform contract.
  4. Join the certified talent pool and wait for project matching.

Typical timeline: Micro1 does not publish a fixed interview length. Certification is typically reviewed within 30 days.

Paid steps: Not stated by the source

Certification does not guarantee a project placement.

Process checked Nov 19, 2026. AIWorkJobs does not submit applications or charge a fee. Read the full Micro1 process and its official source on the Micro1 page.

Skills mentioned by the source

  • Coding Agent Evaluation & Benchmark Design
  • Software Engineering & Python
  • LLM & Agentic Systems
  • Model Failure Analysis & Experimentation

Listing reference: member-of-technical-staff-coding-agent-research-evaluations-20d7f8. Source: Micro1. If something here contradicts the source, the source controls. Report a correction.

Related roles

Other work on the index.

See more in this category

micro1

Member of Technical Staff, Cortex Lab (applied AI research)

Develop and assess LLM-powered agents and AI systems used in enterprise workflows. The role combines applied research with hands-on engineering: you will design evaluations, investigate failures, run experiments, and help improve system performance. You will also build research tools and infrastructure while working with researchers, engineers, domain experts, and enterprise customers.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Full-time
Compensation
$100–$130/hr
  • AI Agent Evaluation & Benchmark Design
  • Python & Research Engineering
  • Agent Failure Analysis & Experimentation
AI evaluationView job

Turing

Senior Software Engineer - AI Evaluation / Coding Agents

Evaluate and improve AI coding models by reviewing generated code.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
$100–$150/hr
  • python
  • typescript
  • javascript
Machine Learning & AIView job

Mercor

Applied Engineer, Evaluations Role

Build and refine software engineering evaluations from completed pull requests, using Mercor’s in-house framework to assess frontier coding agents such as Claude Code and Codex. You’ll work with an internal research team whose benchmarks compare frontier large language models, then take ownership of evaluation selection, development, and refinement after onboarding.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$100–$120/hr
  • typescript
  • javascript
  • python
AI evaluationView job

Turing

Senior Python Engineer – LLM Evaluation (US-based)

Create and evaluate datasets to enhance large language model performance.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Pay not listed
  • python
  • c++
  • c
Machine Learning & AIView job