AIWorkJobs
Mercor logoMercorPlatform listing

ML Challenge Task Auditor

Review applied machine-learning tasks that train and evaluate models at a frontier AI lab. You will judge experiment design, model-selection decisions, and evaluation methods, then write rubric-based feedback grounded…

Remote · United States onlyHourly
Work arrangement
Remote
Eligibility
United States only
Engagement
Hourly
Compensation
$70–$90/hr
Specialty
Machine Learning & AI
Weekly commitment
40 hours per week

About the role

Review applied machine-learning tasks that train and evaluate models at a frontier AI lab. You will judge experiment design, model-selection decisions, and evaluation methods, then write rubric-based feedback grounded in evidence. The work focuses on applied and experimental ML rigor, not building LLM applications or managing MLOps.

What you would do

  • Assess the quality and correctness of applied machine-learning tasks.
  • Review experiment design, including choices around model selection and hyperparameter tuning.
  • Evaluate methodology for measuring model performance.
  • Check data practices for leakage, metric gaming, and sound train, test, and cross-validation setup.
  • Compare machine-learning claims with the evidence provided.
  • Reproduce results to assess whether reported findings hold up.
  • Write feedback against the applicable rubric.

Requirements

  • At least 3 years of hands-on experience in applied or experimental machine learning.
  • Experience with experiment design, model selection, hyperparameter tuning, and evaluation methodology.
  • Strong understanding of data-quality practices, including leakage detection, metric gaming, and train, test, and cross-validation hygiene.
  • Proficiency with standard machine-learning frameworks such as PyTorch, TensorFlow, scikit-learn, or XGBoost.
  • Ability to assess machine-learning claims against evidence and reproduce results.

Preferred qualifications

The source marks these as preferred, not required.

  • Competition or benchmark experience, such as Kaggle.
  • Graduate research or publications in applied machine learning.
  • Experience grading tasks or conducting peer review.

Who should apply

Candidates with at least three years of hands-on applied or experimental ML experience should apply if they can scrutinize experiment design, data quality, and evaluation evidence. The role suits US-based reviewers who want to assess ML task rigor rather than build LLM applications or MLOps systems.

How applying works on Mercor

Create a profile, complete the role checklist and AI interview, then verify your identity before a project offer.

  1. Create an account and confirm your résumé.
  2. Complete the role checklist, written assessment, and AI interview where the listing requires them.
  3. Submit the application and wait for review or matching.
  4. Complete a work trial and identity verification before onboarding.

Typical timeline: The AI interview is about 20 minutes; work trials are often 4–6 hours; identity checks are usually minutes.

Paid steps: The work trial is paid.

Assessment and interview requirements vary by role. Check the checklist inside the specific listing.

Process checked Nov 19, 2026. AIWorkJobs does not submit applications or charge a fee. Read the full Mercor process and its official source on the Mercor page.

Skills mentioned by the source

  • machine learning
  • experiment design
  • model selection
  • hyperparameter tuning
  • evaluation methodology
  • data leakage detection
  • cross-validation
  • pytorch
  • tensorflow
  • scikit-learn
  • xgboost
  • result reproducibility

Specialty areas

Specialties come from the source platform’s category taxonomy and are used to filter the job index.

Listing reference: ml-challenge-task-auditor-NFkJZb. Source: Mercor. If something here contradicts the source, the source controls. Report a correction.

Related roles

Other work on the index.

See more in this category

AfterQuery

ML Research Expert — Published First Author (Benchmark Authoring)

AfterQuery seeks Master's- or PhD-level ML researchers to create and validate research-grade benchmark tasks that test frontier AI models. You will draw on at least one first-author paper and hands-on experience writing ML code for training, evaluation, or systems work. The role covers a range of ML research areas and excludes annotation and data-labelling work.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$150/hr
  • machine learning
  • machine learning research
  • benchmark design
Machine Learning & AIView job

Mercor

Machine Learning Expert

Mercor keeps this listing open to build a pool of machine learning practitioners for AI training and evaluation projects. You may write modeling problems from systems you have shipped, assess model-generated code, or identify approaches that could fail in production. Apply with your resume, confirm your work location, and complete an AI interview that takes about 20 minutes. Project invitations may arrive within a week or after several months.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Part-time
Compensation
$70–$120/hr
  • machine learning
  • model training
  • model evaluation
Machine Learning & AIView job

micro1

ML Engineer

Contribute to an AI training project by building, testing, and assessing machine-learning engineering tasks. Work covers model development, training and inference workflows, numerical computing, and performance tuning. This contractor role averages about 15 hours per week, with flexible days and hours, including weekends if you choose. Pay depends on completed tasks that meet project specifications, and minimum submission requirements apply.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Contract
Compensation
$60–$120/hr
  • Machine Learning
Machine Learning & AIView job

SME Careers

Python (ML-Focused) Team Lead

Review Python code and machine learning work for correctness, sound methodology, reproducibility, and compliance with project rubrics. You will guide trainer and QA workflows, give contributors actionable feedback, and maintain resources such as calibration tasks and style guides. This remote contract has no immediate project; qualified candidates may be contacted when relevant work opens. Selection includes an AI interview, a domain task, and a recruiter interview.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $120/hr
  • python
  • machine learning
  • data science
Machine Learning & AIView job