Evaluation Jobs

92 jobs from companies building with AI · Avg salary $239k (44 with data)

AI evaluation engineering roles focused on benchmarking, testing, and measuring model performance. Evaluation engineers build the frameworks that determine whether AI systems are improving.

Lead Data Scientist - Knowledge Retrieval

Govini · Pittsburgh, PA
agents llm embeddings fine-tuning nlp rag mlops evaluation
On-site full-time lead 3 months ago

Machine Learning Engineer: Evaluation

Bedrock Robotics · San Francisco, CA
autonomous-vehicles data-pipeline robotics machine-learning evaluation
Remote full-time senior 3 months ago

Research Scientist – LTX Model Evaluation

Lightricks · Jerusalem
pre-training reinforcement-learning gpu computer-graphics generative-ai computer-vision research evaluation
On-site full-time mid 3 months ago

Applied AI, Evaluation Engineer

Mistral AI · Paris, France
llm agents generative-ai security pytorch healthcare evaluation
On-site full-time principal 3 months ago

Senior Safety Researcher - Evaluation & Standardization

Waymo · London, UK
autonomous-vehicles robotics evaluation research
On-site full-time senior 3 months ago

Member of Technical Staff - Evaluations

Reflection AI · San Francisco, CA
pre-training agents llm evaluation
On-site full-time lead 5 months ago

Member of Technical Staff, Data Analysis and Evaluation

Cohere · London, UK
distributed-systems search tensorflow generative-ai pytorch nlp llm evaluation
Remote full-time lead 5 months ago

Senior Research Scientist, Model Evaluation

Cohere · Toronto, Canada
search generative-ai llm research evaluation
Remote full-time senior 7 months ago

LLM Inference Performance & Evals Engineer

Cerebras · Toronto, Canada
agents gpu llm generative-ai evaluation inference
On-site full-time mid 10 months ago

Senior Research Engineer, Model Evaluation

Cohere · Toronto, Canada
generative-ai llm search evaluation research
Remote full-time senior 10 months ago
Hiring AI developers? Start with a job post. Claiming the profile comes after. Post a job →

Senior Software Engineer, AI Platform - Evaluation & Annotation

Datadog · Paris, France
distributed-systems embeddings agents llm generative-ai evaluation
Hybrid full-time senior 10 months ago

Research Engineer, Frontier Evals & Environments

OpenAI · San Francisco, CA
llm agents reinforcement-learning search research evaluation
On-site full-time mid 1 year ago

Research Engineer, Retrieval & Search, Applied Engineering

OpenAI · San Francisco, CA
embeddings search rag evaluation research
On-site full-time mid 2 years ago

ML Researcher - Evaluations

Fundamental · Barcelona
generative-ai evaluation research
On-site full-time mid 1 week ago

Member of Engineering (Evaluations)

Poolside · Remote (Europe)
llm research evaluation
Remote full-time lead 1 month ago

Software Engineer, Agent Evaluation and Quality

Cursor · San Francisco, CA
search evaluation
On-site full-time mid 1 month ago

Lingala Speakers — Contribute to AI Translation Evaluation!

Appen · Any
payments evaluation
Remote contract mid 1 month ago

Legal & Compliance AI Rater/Evaluator - Japanese

LILT · Japan
mlops evaluation
Remote contract mid 1 month ago

Software Engineering & DevOps AI Rater/Evaluator

LILT · Remote
cloud mlops distributed-systems devops evaluation
Remote contract mid 1 month ago

Automotive AI Rater & Evaluator - Remote

LILT · Global
mlops evaluation
Remote contract mid 1 month ago
Hiring AI developers? Start with a job post. Claiming the profile comes after. Post a job →

Weekly AI Jobs Digest

Top new roles from 50+ companies. Curated, not scraped. One email, every Monday.

No spam. Unsubscribe anytime.

Hiring AI engineers?

Post the role first. Your company profile and analytics connect from the employer flow.

Post a Job See Pricing

Agentic API & MCP Server

Wire AI Dev Jobs into your agent at build time — MCP server live, REST API public for discovery, free API keys for recurring search.

# Add as MCP server
claude mcp add --transport http aidevjobs https://aidevboard.com/mcp

# Or hit the REST API
curl https://aidevboard.com/api/v1/jobs?tags=llm,pytorch

13 tools via com.aidevboard/jobs · Free keyed access · Pro $49/mo →