AIWorkJobs
SME Careers logoSME CareersPlatform listing

STEM & Technical Specialist – Rubric Design

Create advanced questions in your technical discipline, assess AI-generated solutions, and build scoring criteria that capture sound reasoning as well as correct results. The work centers on rubric design…

Remote · United States onlyContract
Work arrangement
Remote
Eligibility
United States only
Engagement
Contract
Compensation
Up to $100/hr
Specialty
AI Training
Experience
Not specified
Interview language
English

About the role

Create advanced questions in your technical discipline, assess AI-generated solutions, and build scoring criteria that capture sound reasoning as well as correct results. The work centers on rubric design, partial-credit standards, and close review of technical details such as assumptions, formulas, units, and edge cases. Applicants need a relevant degree, at least three years of domain experience, and C1-level English.

What you would do

  • Write demanding STEM questions that test domain knowledge, multi-step reasoning, and professional judgment.
  • Build evaluation rubrics that specify required concepts, valid methods, acceptable alternatives, and signs of a complete answer.
  • Set partial-credit rules for essential steps, minor mistakes, and errors that invalidate a solution.
  • Judge AI-generated answers for technical accuracy, completeness, reasoning quality, and methodology.
  • Flag solutions that reach a plausible conclusion through flawed assumptions or invalid logic.
  • Review other experts’ rubrics for ambiguity, missing criteria, redundant rules, technical errors, and inconsistent scoring.
  • Prepare expert answers, explanations, and critiques as reference material for model training.
  • Create question, rubric, and evaluation content for RLHF and supervised fine-tuning workflows.
  • Check technical content against professional or academic standards in your subject area.

Requirements

  • Hold a bachelor’s degree or higher in a technical field such as mathematics, physics, chemistry, engineering, computer science, statistics, or applied science.
  • Bring at least three years of professional, academic, research, or industry experience in your technical discipline.
  • Show advanced expertise in a defined technical field or specialization.
  • Use English at C1 level or higher to write precise technical explanations and evaluation criteria.
  • Demonstrate skill in creating challenging questions and detailed scoring rubrics for multi-step technical work.
  • Distinguish a correct conclusion from reasoning supported by valid methods and assumptions.
  • Assess formulas, units, methodology, edge cases, terminology, and logical consistency with close attention.
  • Review technical criteria for clarity, accuracy, completeness, and scoring quality.
  • Work independently and deliver consistent results in an hourly, remote contract arrangement.

Preferred qualifications

The source marks these as preferred, not required.

  • Background in exam writing, academic grading, peer or research review, technical QA, standards development, or assessment design.
  • Prior work in AI evaluation, RLHF, supervised fine-tuning, benchmarking, data annotation, prompt design, or LLM evaluation.

Who should apply

Technical specialists with three or more years in a defined STEM field can apply if they can turn expert judgment into clear questions and measurable scoring criteria. Experience assessing technical work or supporting AI evaluation is an advantage, while C1-level English is required.

How applying works on SME Careers

Create an account and submit the listing application with your CV. The official site does not publish the later screening steps.

  1. Sign up with LinkedIn or email.
  2. Open the listing and submit the application with your CV.
  3. Wait for the platform to contact you about any assessment or next step.

Paid steps: Not stated by the source

SME Careers does not publicly document its full screening process. Some roles may add an assessment; confirm with the platform.

Process checked Nov 19, 2026. AIWorkJobs does not submit applications or charge a fee. Read the full SME Careers process and its official source on the SME Careers page.

Skills mentioned by the source

  • ai evaluation
  • benchmarking
  • chemistry
  • computer science
  • engineering
  • english
  • expert review
  • mathematics
  • model evaluation
  • physics
  • question writing
  • rubric design
  • rubric writing
  • stem
  • technical reasoning

Specialty areas

Specialties come from the source platform’s category taxonomy and are used to filter the job index.

Listing reference: stem-technical-specialist-rubric-design-c26510. Source: SME Careers. If something here contradicts the source, the source controls. Report a correction.

Related roles

Other work on the index.

See more in this category

SME Careers

Finance & Accounting Specialist – Rubric Design

Use your finance or accounting expertise to create realistic professional problems and assess AI-generated analyses. You will build scoring rubrics that capture sound calculations, assumptions, methods, and judgment, then review outputs and peer criteria for technical accuracy and completeness across areas such as valuation, reporting, audit, and investment analysis.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $100/hr
  • accounting
  • finance
  • financial analysis
Finance & AccountingView job

AfterQuery

High School Teacher Expert

Create assessment and learning materials grounded in high school coursework, including rubrics, sample responses, research briefs, and instructional activities. You’ll also assess AI-generated work for sound reasoning, clear organization, and effective teaching choices. The role supports the development of AI educational tools, follows a remote, asynchronous schedule, and expects about 10 hours of work per week when projects are active.

Location and eligibility
Remote · Eligibility not specified
Engagement type
Not specified
Compensation
$55/hr
  • rubric design
  • assessment standards
  • high school pedagogy
AI evaluationView job

Outlier · via Braintrust

STEM PhD Expert for AI Reasoning & Evaluation

Use your doctoral expertise to assess how AI models handle technical questions and domain-specific content. You’ll check factual accuracy and relevance, compare model responses, and contribute questions and answers in machine learning and AI. The remote contract runs through the end of June, with a possible extension based on project needs. Applicants must live in the United States, Canada, Puerto Rico, Mexico, the United Kingdom, Australia, New Zealand, or Argentina.

Location and eligibility
Remote · Eligible in 8 countries
Engagement type
Contract
Compensation
$50–$150/hr
  • machine learning
  • artificial intelligence
  • computer science
Machine Learning & AIView job

SME Careers

Healthcare Specialist – Rubric Design

Apply your healthcare specialty to build questions that test advanced AI systems and scoring rubrics that separate sound clinical reasoning from unsafe or incomplete answers. You will assess model responses, review colleagues’ rubrics, and create reference material for AI evaluation and training workflows. Applicants need a recognized healthcare qualification, at least three years of experience, and English proficiency at C1 or higher.

Location and eligibility
Remote · United States only
Engagement type
Contract
Compensation
Up to $100/hr
  • healthcare
  • clinical pharmacology
  • clinical reasoning
Health Informatics & Medical RecordsView job