Evaluation Jobs

92 jobs from companies building with AI · Avg salary $239k (44 with data)

AI evaluation engineering roles focused on benchmarking, testing, and measuring model performance. Evaluation engineers build the frameworks that determine whether AI systems are improving.

Machine Learning Engineer, Driver Understanding and Evaluation

Waymo · Mountain View, CA · $170k - $216k
generative-ai pytorch robotics tensorflow autonomous-vehicles machine-learning evaluation
On-site full-time mid 5 months ago

Senior Software Engineer, Large Model Evaluation

Waymo · Mountain View, CA · $204k - $259k
llm generative-ai autonomous-vehicles tensorflow data-pipeline deep-learning robotics evaluation
On-site full-time senior 5 months ago

Staff Machine Learning Research Scientist, LLM Evals

Scale AI · San Francisco, CA · $264k - $331k
llm search nlp generative-ai research evaluation machine-learning
On-site full-time lead 6 months ago

Member of Technical Staff, Evals & Post-Training Product

Fireworks AI · San Mateo, CA · $175k - $220k
llm pytorch generative-ai mlops agents fine-tuning evaluation
On-site full-time lead 6 months ago

Applied Research - Evals & Data

Prime Intellect · San Francisco, CA · $150k - $300k
data-pipeline llm reinforcement-learning distributed-systems agents research evaluation
Remote full-time senior 6 months ago

Senior Software Engineer, ML Evaluation Infra and Efficiency

Waymo · Mountain View, CA · $238k - $302k
tensorflow llm distributed-systems autonomous-vehicles evaluation infrastructure
On-site full-time senior 8 months ago

FullStack Engineer, AI Observability & Evals Platform (LangSmith)

LangChain · San Francisco, CA · $145k - $180k
llm agents fullstack evaluation
Hybrid full-time junior 9 months ago

Senior Frontend Engineer, AI Observability & Evals Platform

LangChain · San Francisco, CA · $175k - $240k
api-design agents llm evaluation frontend
Hybrid full-time senior 11 months ago

Senior Staff Machine Learning Engineer, Data & Eval

Airbnb · United States · $244k - $305k
fine-tuning mlops llm data-pipeline payments generative-ai machine-learning evaluation
On-site full-time lead 1 year ago

Senior Software Engineer, Simulator Evaluation

Waymo · Mountain View, CA · $204k - $259k
robotics search autonomous-vehicles llm generative-ai evaluation
On-site full-time senior 1 year ago
Hiring AI developers? Start with a job post. Claiming the profile comes after. Post a job →

Senior Fullstack Engineer, AI Observability & Evals Platform

LangChain · San Francisco, CA · $175k - $240k
agents llm fullstack evaluation
Hybrid full-time senior 1 year ago

Tech Lead/Manager, Machine Learning Research Scientist- LLM Evals

Scale AI · San Francisco, CA · $264k - $331k
llm search generative-ai nlp research machine-learning evaluation
On-site full-time lead 2 years ago

AI Engineer, Evaluation

Distyl AI · San Francisco, CA · $150k - $250k
healthcare llm evaluation
Hybrid full-time junior 3 weeks ago

Safeguards Enforcement Analyst, Safety Evaluations

Anthropic · San Francisco, CA · $230k - $270k
alignment rust evaluation
Hybrid full-time senior 2 months ago

Software Engineer, Simulation

SpaceX · Hawthorne, CA · $145k - $175k
deep-learning evaluation
On-site full-time senior 2 months ago

Software Engineering Manager, AI Observability & Evals Platform (San Francisco, CA)

LangChain · San Francisco, CA · $200k - $250k
agents llm evaluation
Hybrid full-time senior 3 months ago

Software Engineering Manager, AI Observability & Evals Platform (New York, NY)

LangChain · New York, NY · $200k - $250k
agents llm evaluation
Hybrid full-time senior 3 months ago

Senior Software Engineer, AI Evals

Sentry · San Francisco, CA · $240k - $280k
agents llm evaluation
Hybrid full-time senior 3 months ago

Evaluation Engineer

Elicit · Remote (US) · $165k - $200k
data-pipeline evaluation
Hybrid full-time mid 3 months ago

Staff Data Scientist, Launch Evaluation Quality

Waymo · Mountain View, CA · $238k - $302k
autonomous-vehicles evaluation data-science
On-site full-time lead 5 months ago
Hiring AI developers? Start with a job post. Claiming the profile comes after. Post a job →

Weekly AI Jobs Digest

Top new roles from 50+ companies. Curated, not scraped. One email, every Monday.

No spam. Unsubscribe anytime.

Hiring AI engineers?

Post the role first. Your company profile and analytics connect from the employer flow.

Post a Job See Pricing

Agentic API & MCP Server

Wire AI Dev Jobs into your agent at build time — MCP server live, REST API public for discovery, free API keys for recurring search.

# Add as MCP server
claude mcp add --transport http aidevjobs https://aidevboard.com/mcp

# Or hit the REST API
curl https://aidevboard.com/api/v1/jobs?tags=llm,pytorch

13 tools via com.aidevboard/jobs · Free keyed access · Pro $49/mo →