{"access":{"catalog_url":"https://aidevboard.com/api/v1/catalog","description":"Public read endpoints are open and free. API keys are optional for stable agent identity and keyed hourly throttling.","docs_url":"https://aidevboard.com/docs","employer_pilot_url":"https://aidevboard.com/verified-interview-pilot","mode":"open","register_url":"https://aidevboard.com/api/v1/register"},"candidate_resume_action":{"application_authorized":false,"candidate_charge":0,"endpoint":"https://aidevboard.com/api/v1/candidate/resume-preview","job_id_json_path":"jobs[].id","method":"POST","preview_requires_identity":false,"required_body_fields":["job_id","evidence_bullets"],"requires_explicit_human_review":true,"saved_artifact_protocol":"mcp","saved_artifact_requires_verified_human":true,"saved_artifact_tool":"compile_job_specific_resume","search_requires_identity":false,"status":"available_after_candidate_selects_job","submission_performed":false,"uses_candidate_verified_evidence":true},"degraded":false,"estimated":false,"has_next":true,"jobs":[{"id":"30f0bfd5-b030-48f6-9124-0c1234f28ba7","company_id":"31ae48bc-c938-4c26-a348-0bf3c089a446","title":"Staff Product Manager, AI Inference","slug":"staff-product-manager-ai-inference-85275e80","description":"CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of technology, tools, and teams that enables innovators to build and scale AI with confidence. Trusted by leading AI labs, startups, and global enterprises, CoreWeave combines superior infrastructure performance with deep technical expertise to accelerate breakthroughs and turn compute into capability. Founded in 2017, CoreWeave became a publicly traded company (Nasdaq: CRWV) in March 2025. Learn more at  www.coreweave.com . \n About the Team \n As the Staff Product Manager - AI Inference, you will lead CoreWeave's strategic efforts in shaping and developing cutting-edge AI inference services that drive innovation and business outcomes for our customers. You will collaborate closely with engineers, designers, and industry partners to create world-class products that empower organizations to build, deploy, scale, and optimize AI models and applications efficiently and effectively.\n The ideal candidate is passionate about AI and cloud technologies, with a deep understanding of the foundation model architectures, AI inference landscape, AI application ecosystem, cloud-native technologies, and the competitive landscape. You will leverage market insights and technical expertise to define the product vision, roadmap, and go-to-market strategy, ensuring that CoreWeave builds on its market-segment position as the leading AI Hyperscaler.\n Key Responsibilities: \n \n Product Vision and Strategy: Help refine the product strategy for AI application cloud services aimed at foundational model Inference, along with focus on optimization across the entire CoreWeave product portfolio for AI workloads. Solve novel problems related to scaling AI model inference on specialized hardware.\n Roadmap Development: Own the product roadmap for AI inference services, balancing near-term customer needs with long-term vision, to deliver differentiated and scalable AI cloud services.\n AI and Cloud Expertise: Stay current on advancements in AI/ML technologies including inference engines, model serving architectures, cloud infrastructures, and software development methodologies to inform product decisions and foster innovation.\n Define product portfolio: Manage ambiguity and complexity of a fast evolving AI landscape to define new product investments and develop a portfolio of inference capabilities that deliver long-term differentiation and customer value.\n Customer Engagement: Collaborate with key customers to understand their AI workloads, pain points, and future requirements to continuously improve our service offerings.\n Cross-functional Leadership: Collaborate with engineering, sales, marketing, and customer success teams to ensure seamless product development and deployment.\n Market Analysis: Analyze market trends, customer feedback, and the competitive landscape to identify growth opportunities and ensure CoreWeave stays ahead of industry trends.\n Go-to-Market Strategy: Partner with industry partners, marketing, and sales teams to develop and execute effective go-to-market strategies for AI cloud services, including pricing strategies, positioning, messaging, and competitive differentiation.\n \n Qualifications: \n \n Experience: 8+ years of experience in product management, with at least 5 years in the cloud services or AI/ML market segment.\n Technical Knowledge: Hands on practitioner and deep understanding of AI/ML workloads including foundation model inference, inference engines such as vllm, SGLang, TensorRT-llm, and llm-d, inference optimization techniques such as quantization and speculative decoding, cloud infrastructure (GPUs), and cloud-native architectures (Kubernetes, microservices).\n Leadership: Proven ability to lead cross functional teams, with strong decision-making, project management, and mentoring skills.\n Customer-Centric Approach: Demonstrated experience working closely with customers to identify needs and translate them into product features and solutions.\n Communication Skills: Excellent verbal and written communication skills with the ability to influence cross-functional teams and stakeholders.\n Education: Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field. MBA or equivalent experience is a plus.\n Visionary Leadership: Ability to think strategically while being able to dive into technical details to guide the product team.\n Results-Oriented: Proven track record of driving business growth through innovative product solutions in the AI and cloud space.\n \n Why CoreWeave? \n At CoreWeave, we work hard, have fun, and move fast! We're in an exciting stage of hyper-growth that you will not want to miss out on. We're not afraid of a little chaos, and we're constantly learning. Our team cares deeply about how we build our product and how we work together, which is represented through our core values:\n \n Be Curious at Your Core\n Act Like an Owner\n Empower Employees\n Deliver Best-in","salary_min":207000,"salary_max":275000,"location":"Sunnyvale, CA","workplace":"onsite","remote_scope":"not_remote","job_type":"full-time","experience_level":"lead","tags":["microservices","mlops","cloud","generative-ai","llm","inference"],"apply_url":"https://coreweave.com/careers/job?4720733006\u0026board=coreweave\u0026gh_jid=4720733006","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-10-12T14:43:58Z","expires_at":"2026-11-13T16:56:02.427193Z","created_at":"2026-10-14T15:20:12.017713Z","updated_at":"2026-10-14T16:56:02.430397Z","company_name":"CoreWeave","company_slug":"coreweave","company_logo_url":"https://www.google.com/s2/favicons?domain=coreweave.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/30f0bfd5-b030-48f6-9124-0c1234f28ba7"},{"id":"901acfd2-6d66-4082-b7e6-b0ba1754c580","company_id":"a0000000-0000-0000-0000-000000000001","title":"Staff + Sr. Software Engineer, Cloud Inference","slug":"staff-sr-software-engineer-cloud-inference-b11fdf49","description":"About Anthropic \n Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems.\n About the role \n The Cloud Inference team scales and optimizes Claude to serve the massive audiences of developers and enterprise companies across AWS, GCP, Azure, and future cloud service providers (CSPs). We own the end-to-end product of Claude on each cloud platform, from API integration and intelligent request routing to inference execution, capacity management, and day-to-day operations.\n Our engineers are extremely high leverage: we simultaneously drive multiple major revenue streams while optimizing one of Anthropic's most precious resources: compute. As we expand to more cloud platforms, the complexity of managing inference efficiently across providers with different hardware, networking stacks, and operational models grows significantly. We need product-minded backend engineers who can navigate these platform differences, design the services and abstractions that work across providers, and make architectural decisions that keep us reliable and cost-effective at massive scale.\n Your work will increase the scale at which our services operate, accelerate our ability to reliably launch new frontier models and innovative features to customers across all platforms, and ensure our LLMs meet rigorous safety, performance, and security standards.\n Key responsibilities \n \n Design, build, and own backend services and infrastructure that serve Claude across multiple CSPs, accounting for differences in compute hardware, networking, APIs, and operational models\n Work cross-functionally with internal inference, product API, systems, and security teams, among others, and with CSP partners to stand up the full serving stack on new cloud platforms, resolve operational issues, and influence provider roadmaps\n Build and evolve CI/CD automation systems, including validation and deployment pipelines, that reliably ship new model versions to millions of users across cloud platforms without regressions\n Design interfaces and tooling abstractions across CSPs that enable cost-effective inference management, scale across providers, and reduce per-platform complexity\n Contribute to capacity planning, autoscaling, and workload routing strategies that match supply with demand and direct requests to the most cost-effective accelerator and region\n Analyze observability data across providers to identify performance bottlenecks, cost anomalies, and regressions, and drive remediation based on real-world production workloads\n \n Minimum qualifications \n \n Have significant software engineering experience, with a strong background in high-performance, large-scale distributed systems serving millions of users\n Have experience building or operating services on at least one major cloud platform (AWS, GCP, or Azure), with exposure to Kubernetes, Infrastructure as Code, or container orchestration\n Are curious about LLM serving; prior inference or ML experience is not required\n Thrive in cross-functional collaboration with both internal teams and external partners\n Have experience working with external partners to align goals and deliver impact\n Are a fast learner who can quickly ramp up on new technologies, hardware platforms, and provider ecosystems\n Are highly autonomous and take ownership of problems end-to-end, including work that falls outside your job description\n \n Preferred qualifications \n \n Direct experience working with CSPs to scale infrastructure or products across multiple platforms, navigating differences in networking, security, privacy, billing, and managed service offerings\n Hands-on experience with capacity management, cost optimization, or resource planning at scale across heterogeneous environments\n Solid understanding of multi-region deployments, geographic routing, and global traffic management\n Proficiency in Python or Rust\n The annual compensation range for this role is listed below. \n For sales roles, the range provided is the role’s On Target Earnings (\"OTE\") range, meaning that the range includes both the sales commissions/sales bonuses target and annual base salary for the role.\n Annual Salary:\n $320,000 — $485,000 USD \n Logistics \n Minimum education: Bachelor’s degree or an equivalent combination of education, training, and/or experience\n Required field of study:  A field relevant to the role as demonstrated through coursework, training, or professional experience\n Minimum years of experience: Years of experience required will correlate with the internal job level requirements for the position\n Location-based hybrid policy: Currently, we expect all staff to be in one of our offices at least 25% of the time. However, some roles may require more time in our offices.\n Vis","salary_min":320000,"salary_max":485000,"location":"San Francisco, CA","workplace":"hybrid","remote_scope":"not_remote","job_type":"full-time","experience_level":"lead","tags":["distributed-systems","payments","llm","alignment","infrastructure","inference"],"apply_url":"https://job-boards.greenhouse.io/anthropic/jobs/5436684008","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-09-28T18:53:28Z","expires_at":"2026-11-13T16:54:57.029878Z","created_at":"2026-10-14T15:17:23.514386Z","updated_at":"2026-10-14T16:54:57.033374Z","company_name":"Anthropic","company_slug":"anthropic","company_logo_url":"https://www.google.com/s2/favicons?domain=anthropic.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/901acfd2-6d66-4082-b7e6-b0ba1754c580"},{"id":"c8280610-a672-4bc7-9693-7e8cdab09b06","company_id":"a0000000-0000-0000-0000-000000000001","title":"Staff + Sr. Software Engineer, Cloud Inference Launch Engineering","slug":"staff-sr-software-engineer-cloud-inference-launch-engineering-435acf86","description":"About Anthropic \n Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems.\n About the role\n The Cloud Inference team scales and optimizes Claude to serve the massive audiences of developers and enterprise companies across AWS, GCP, Azure, and future cloud service providers (CSPs). We own the end-to-end product of Claude on each cloud platform, from API integration and intelligent request routing to inference execution, capacity management, and day-to-day operations.\n Within Cloud Inference, the model \u0026 inference launch team owns the validation pipeline for our inference server and load balancer on these platforms. We're responsible for every inference change — model launches, performance improvements, safeguard integrations — landing on cloud platforms with correctness, performance, and reliability intact.\n This is high-leverage infrastructure work: validation has to be fast and cheap enough to run on the same accelerators that serve customers, trustworthy enough to replace manual checks, and consistent enough that a change working on Anthropic first-party means it works everywhere. This directly determines how fast frontier models and features ship to every cloud platform, and how quickly performance wins reach production — reclaiming capacity at a time when compute is our scarcest resource.\n Key responsibilities\n \n Be on the critical path for frontier model launches, bringing up inference for new model architectures and shipping them to cloud platforms in lockstep with our first-party platform\n Work with the core inference team to bring new inference features (e.g. structured sampling, prompt caching, and more) to cloud platforms, owning the platform-specific integration that gets them to production\n Identify and dive deep on the gaps that make inference behave differently across first-party and CSPs — config drift, observability, deployment patterns, hard cross-platform bugs — and fix them at the source rather than building platform-specific workarounds\n Design, build, and own the CI/CD infrastructure for the inference server and load balancer across cloud platforms, with shadow traffic, performance baselines (throughput and latency), and correctness checks that catch regressions before production\n Drive down merge-to-production cycle time by making validation faster, more parallel, and cost-effective enough to run on the same constrained accelerator pool that serves customers, without trading away reliability \n Analyze observability data across providers to identify performance bottlenecks, cost anomalies, and regressions, and drive remediation based on real-world production workloads\n \n Minimum qualifications\n \n Have a strong interest in LLM serving; prior inference or ML experience is not required \n Have significant software engineering experience, with a strong background in high-performance, large-scale distributed systems serving millions of users\n Have a track record of building automation or test infrastructure that measurably improved release velocity or reliability\n Have experience building or operating services on at least one major cloud platform (AWS, GCP, or Azure), with exposure to Kubernetes, Infrastructure as Code, or container orchestration\n Thrive in cross-functional collaboration with both internal teams and external partners\n Are a fast learner who can quickly ramp up on new technologies, hardware platforms, and provider ecosystems\n Are highly autonomous and take ownership of problems end-to-end, including work that falls outside your job description\n \n Preferred qualifications\n \n LLM inference optimization, batching, and caching strategies\n Capacity-constrained scheduling or shared-resource test infrastructure\n Solid understanding of multi-region deployments, request routing, load balancing, global traffic management\n Working with CSP partner teams to scale infrastructure across multiple platforms, navigating differences in networking, security, privacy, and managed service\n Proficiency in Python or Rust\n The annual compensation range for this role is listed below. \n For sales roles, the range provided is the role’s On Target Earnings (\"OTE\") range, meaning that the range includes both the sales commissions/sales bonuses target and annual base salary for the role.\n Annual Salary:\n $320,000 — $485,000 USD \n Logistics \n Minimum education: Bachelor’s degree or an equivalent combination of education, training, and/or experience\n Required field of study:  A field relevant to the role as demonstrated through coursework, training, or professional experience\n Minimum years of experience: Years of experience required will correlate with the internal job level requirements for the position\n Location-based hybri","salary_min":320000,"salary_max":485000,"location":"San Francisco, CA","workplace":"hybrid","remote_scope":"not_remote","job_type":"full-time","experience_level":"lead","tags":["alignment","llm","distributed-systems","inference","infrastructure"],"apply_url":"https://job-boards.greenhouse.io/anthropic/jobs/5436697008","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-09-28T18:52:20Z","expires_at":"2026-11-13T16:54:57.035074Z","created_at":"2026-10-14T15:17:23.523759Z","updated_at":"2026-10-14T16:54:57.037886Z","company_name":"Anthropic","company_slug":"anthropic","company_logo_url":"https://www.google.com/s2/favicons?domain=anthropic.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/c8280610-a672-4bc7-9693-7e8cdab09b06"},{"id":"a1fdf581-2cfe-43eb-b7db-c5e4a939cffc","company_id":"6ce2d21e-b00f-4343-9bd0-5ac62ff81431","title":"Staff Tech Lead, ML Data Infrastructure and Inference Platform ","slug":"staff-tech-lead-ml-data-infrastructure-and-inference-platform-704ea4cd","description":"Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The World's Most Experienced Driver™—to improve access to mobility while saving thousands of lives now lost to traffic crashes. The Waymo Driver powers Waymo’s fully autonomous ride-hail service and can also be applied to a range of vehicle platforms and product use cases. The Waymo Driver has provided over ten million rider-only trips, enabled by its experience autonomously driving over 100 million miles on public roads and tens of billions in simulation across 15+ U.S. states.\n The ML Data Infrastructure and Inference Platform team owns the Data as well as the Data infrastructure for all ML Flywheels at Waymo. We do this via a planet scale centralized feature store for all  ML training and evaluation, the feature extraction framework as well the inference platform for generating the data via bulk and online inferences. The objective of the team is to deliver data necessary for all our onboard as well Foundation Models to make the ML Flywheel successful and in turn lead to the successful scaling of Waymo.  The team is highly collaborative and closely works with all modeling teams within Waymo.\n You will: \n \n As a Tech Lead you’ll  build, lead and contribute to Waymo’s ML data infrastructure platform to enable planet scale ML Flywheel for all ML models at Waymo via data store and data infra ecosystem.\n Work closely with teams across Waymo both onboard \u0026 offboard Foundation models, to understand the data infra needs, data distributions, data quality, data value, freshness and onboard these flywheels onto our planet scale data store and ensure the seamless adoption and guide the development of our infra components. \n You will closely lead, mentor and guide a team of engineers to build and onboard teams to the infrastructure efficiently.\n The team has a large surface area and scope in terms of impact, cross team collaboration and direct interaction with ML models at Waymo. \n \n You have: \n \n 6+ years of professional experience in the field of software engineering\n Experience tech leading projects with high cross team collaboration \n Experience tech leading a team of engineers on complex projects\n Experience in programming C++ \n Experience with building highly scalable distributed system\n Experience with ML Data and ML Flywheels\n \n We prefer: \n \n Passionate about Data and building ML infra \u0026 tools\n Experience Tech leading projects, cross functional projects\n Experience with handling large datasets in the order of exabytes\n Experience building machine learning infrastructure  and model hosting / inference infrastructure\n Experience with production services with high QPS.\n \n #LI-Hybrid\n The expected base salary range for this full-time position across US locations is listed below. Actual starting pay will be based on job-related factors, including exact work location, experience, relevant training and education, and skill level. Your recruiter can share more about the specific salary range for the role location or, if the role can be performed remote, the specific salary range for your preferred location, during the hiring process.  \n Waymo employees are also eligible to participate in Waymo’s discretionary annual bonus program, equity incentive plan, and generous Company benefits program, subject to eligibility requirements.  \n Salary Range\n $251,000 — $310,000 USD","salary_min":251000,"salary_max":310000,"location":"Mountain View, CA","workplace":"onsite","remote_scope":"not_remote","job_type":"full-time","experience_level":"lead","tags":["autonomous-vehicles","distributed-systems","generative-ai","infrastructure","inference"],"apply_url":"https://careers.withwaymo.com/jobs?gh_jid=8142863","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-09-23T16:38:41Z","expires_at":"2026-11-13T16:55:55.288456Z","created_at":"2026-10-14T15:20:04.745628Z","updated_at":"2026-10-14T16:55:55.291576Z","company_name":"Waymo","company_slug":"waymo","company_logo_url":"https://www.google.com/s2/favicons?domain=waymo.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/a1fdf581-2cfe-43eb-b7db-c5e4a939cffc"},{"id":"8a11e24b-763c-4a00-9223-55e73fc8f9c0","company_id":"a0000000-0000-0000-0000-000000000001","title":"Performance Engineer, Inference Engine","slug":"performance-engineer-inference-engine-c9c1c8d4","description":"About Anthropic \n Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems.\n Performance Engineer, Inference Engine\n About the Role\n Anthropic's inference engine is the software between the accelerator kernels and the routing layer. It manages the entire token path in between: batching requests, laying the model out across chips, managing memory for weights and activations, coordinating every forward pass, and managing model state across requests. Built in-house, it runs on all of our accelerator platforms, serving Claude to millions of users and running our research workloads.\n You will work on building and optimizing this system at Anthropic scale: improving throughput, cost, reliability, and latency across all accelerator and cloud platforms. You are intimately familiar with the hardware and bandwidth numbers (FLOPs, HBM, PCIe, RDMA, network links, etc.) and can model a problem quickly: where the time and bytes go, and what sets the bound. The role is deeply technical and high-impact, and suits engineers who enjoy working across accelerator programming, high-performance systems that seamlessly coordinate between host and device, and large-scale distributed systems. Familiarity with the transformer architecture is a plus.\n Some example recurring themes:\n \n \n Keep device utilization high. Accelerators should never be waiting due to other overheads.\n \n Reuse instead of recompute. Keep model state cached and reuse it whenever that is cheaper than computing it again.\n \n Measure, model, then change. We build the observability to see where the gaps are, model the impact of potential improvements, deploy them, and go around again, with Claude speeding up every turn of that loop.\n \n Tokens you can trust. Ensuring model quality matters more than efficiency. We build the infrastructure to ensure Claude maintains its intelligence across platforms and over time. \n \n Safety on every token. We work closely with our safeguards and safety teams. The inference engine is the backbone behind our production safety systems, ensuring efficiency without compromising robustness. \n \n Minimum Qualifications\n \n \n A working mental model of LLM inference: how prefill and decode land on an accelerator's compute, memory, and interconnect, and what the host is doing meanwhile\n \n Proven quick learner: ramped fast in deep, unfamiliar systems and shipped consequential changes quickly\n \n Strong systems programming (Rust, C++, or similar), with care for code quality and tests\n \n Analytical about performance: observe and profile first, form a hypothesis, test it, then change the code and measure again\n \n Low ego: ask the naive question, take feedback well, pick up slack outside your job description\n \n Enjoy pair programming (we love to pair!) and care about the societal impacts of your work\n \n Preferred Qualifications\n \n \n Experience inside an LLM serving engine and a sense of where its abstractions strain\n \n GPU/Accelerator programming\n \n OS internals\n \n Language modeling with transformers\n \n Experience building an allocator, cache, scheduler, or high-bandwidth transport\n \n Fluency in Rust\n \n Experience making systems reproducible: determinism, replay, property-based tests\n \n \n The annual compensation range for this role is listed below. \n For sales roles, the range provided is the role’s On Target Earnings (\"OTE\") range, meaning that the range includes both the sales commissions/sales bonuses target and annual base salary for the role.\n Annual Salary:\n $350,000 — $850,000 USD \n Logistics \n Minimum education: Bachelor’s degree or an equivalent combination of education, training, and/or experience\n Required field of study:  A field relevant to the role as demonstrated through coursework, training, or professional experience\n Minimum years of experience: Years of experience required will correlate with the internal job level requirements for the position\n Location-based hybrid policy: Currently, we expect all staff to be in one of our offices at least 25% of the time. However, some roles may require more time in our offices.\n Visa sponsorship:  We do sponsor visas! However, we aren't able to successfully sponsor visas for every role and every candidate. But if we make you an offer, we will make every reasonable effort to get you a visa, and we retain an immigration lawyer to help with this.\n We encourage you to apply even if you do not believe you meet every single qualification. Not all strong candidates will meet every single qualification as listed.  Research shows that people who identify as being from underrepresented groups are more prone to experiencing imposter syndrome and doubting the strength of their candidacy, so we urge you not to exclude yourself premature","salary_min":350000,"salary_max":850000,"location":"San Francisco, CA","workplace":"hybrid","remote_scope":"not_remote","job_type":"full-time","experience_level":"principal","tags":["alignment","llm","distributed-systems","research","inference"],"apply_url":"https://job-boards.greenhouse.io/anthropic/jobs/5418323008","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-09-09T23:09:53Z","expires_at":"2026-11-13T16:54:55.915667Z","created_at":"2026-09-10T13:30:31.557342Z","updated_at":"2026-10-14T16:54:55.918557Z","company_name":"Anthropic","company_slug":"anthropic","company_logo_url":"https://www.google.com/s2/favicons?domain=anthropic.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/8a11e24b-763c-4a00-9223-55e73fc8f9c0"},{"id":"2d764a21-548d-4995-b28e-a419ab251185","company_id":"a0000000-0000-0000-0000-000000000001","title":"Staff+ Software Engineer, ML Inference Path","slug":"staff-software-engineer-ml-inference-path-b64fb22d","description":"About Anthropic \n Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems.\n About the role: \n The Safeguards ML Inference Path team designs, builds, and operates the production infrastructure that powers Claude's ML based safety systems. We collaborate closely with safety researchers and inference engineers to bring new classifiers and novel classes of ML defenses to production. We own the research → production transfer of new safety technologies that is on the critical path for every Claude model launch. And we build for scale: serving thousands of ML classifiers, for all requests on the token generation path, and for every platform Claude runs on -- 1P, Bedrock, Vertex, and beyond.\n We’re growing the team and looking for engineers who have deep expertise in productionizing ML systems. You'll work at the intersection of machine learning, large-scale distributed systems, and AI safety, developing the platforms and tools that enable our safeguards to operate reliably at scale. And your tooling and infrastructure will be used for every model launch, which are becoming more complex, and more frequent.\n Responsibilities: \n \n Design and build scalable ML infrastructure to support real-time safety deployments across our classifier and model ecosystem\n Build monitoring and observability tools to track classifier performance, data quality, and system health for safety-critical applications\n Collaborate with research teams to productionize safety research, translating experimental safety techniques into robust, scalable systems\n Optimize inference latency and throughput for real-time safety evaluations while maintaining high reliability standards\n Implement automated testing, deployment, and rollback systems for ML models in production safety applications\n Partner with Safeguards, Security, and Alignment teams to understand requirements and deliver infrastructure that meets safety and production needs\n Contribute to the development of internal tools and frameworks that accelerate safety research and deployment\n \n You may be a good fit if you: \n \n Are proficient in Python and have experience with ML frameworks like PyTorch, TensorFlow, or JAX\n Understand distributed systems principles and have built systems that handle high-throughput, low-latency workloads\n Have built automated or self-service deployment pipelines and eval infrastructure allowing researchers to roll out classifiers and models independently\n Have implemented A/B testing frameworks and experimentation infrastructure for ML systems\n Are results-oriented, with a bias towards reliability and impact in safety-critical systems\n Enjoy collaborating with researchers and translating cutting-edge research into production systems\n Care deeply about AI safety and the societal impacts of your work\n \n Strong candidates may also have experience with: \n \n Have 5+ years of experience building production ML infrastructure, ideally in safety-critical domains like fraud detection, content moderation, or risk assessment\n Working with large language models and modern transformer architectures\n Developing monitoring and alerting systems for ML model performance and data drift\n Experience in trust \u0026 safety, fraud prevention, or content moderation domains\n Knowledge of privacy-preserving ML techniques and compliance requirements\n The annual compensation range for this role is listed below. \n For sales roles, the range provided is the role’s On Target Earnings (\"OTE\") range, meaning that the range includes both the sales commissions/sales bonuses target and annual base salary for the role.\n Annual Salary:\n $320,000 — $485,000 USD \n Logistics \n Minimum education: Bachelor’s degree or an equivalent combination of education, training, and/or experience\n Required field of study:  A field relevant to the role as demonstrated through coursework, training, or professional experience\n Minimum years of experience: Years of experience required will correlate with the internal job level requirements for the position\n Location-based hybrid policy: Currently, we expect all staff to be in one of our offices at least 25% of the time. However, some roles may require more time in our offices.\n Visa sponsorship:  We do sponsor visas! However, we aren't able to successfully sponsor visas for every role and every candidate. But if we make you an offer, we will make every reasonable effort to get you a visa, and we retain an immigration lawyer to help with this.\n We encourage you to apply even if you do not believe you meet every single qualification. Not all strong candidates will meet every single qualification as listed.  Research shows that people who identify as being from underrepresented groups are more prone to e","salary_min":320000,"salary_max":485000,"location":"San Francisco, CA","workplace":"hybrid","remote_scope":"not_remote","job_type":"full-time","experience_level":"lead","tags":["tensorflow","distributed-systems","llm","alignment","pytorch","inference","rust"],"apply_url":"https://job-boards.greenhouse.io/anthropic/jobs/5419869008","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-09-09T19:19:17Z","expires_at":"2026-11-13T16:54:56.888395Z","created_at":"2026-09-10T13:30:51.32706Z","updated_at":"2026-10-14T16:54:56.891101Z","company_name":"Anthropic","company_slug":"anthropic","company_logo_url":"https://www.google.com/s2/favicons?domain=anthropic.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/2d764a21-548d-4995-b28e-a419ab251185"},{"id":"c831041b-2707-4916-8a82-01d11b60f788","company_id":"a0000000-0000-0000-0000-000000000001","title":"Engineering Manager, Inference Infrastructure","slug":"engineering-manager-inference-infrastructure-3748e5aa","description":"About Anthropic \n Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems.\n About the role \n Every request that hits Claude — from claude.ai , the API, our cloud partners, or internal research — depends on a set of decisions made before it ever reaches a model: where each request should be served and how much capacity each model needs right now. Getting those decisions right is crucial to satisfying throughput, reliability, and latency constraints. This group builds the control plane that makes those decisions for Anthropic's inference fleet and own the inference request path.\n This is a deeply technical group. The engineers here design placement and load-balancing algorithms, build quantitative models of demand, capacity, and system performance, improve latency across kernel, network, and framework boundaries, and reason carefully about how a change to the fleet ripples through everything that depends on it.\n You'll lead a strong group of ML platform, infrastructure, and distributed-systems engineers working alongside the teams that build our ML internals and cloud infrastructure. You need enough systems depth to make architectural calls, hire people who go deep, and see when a proposed change will ripple across the fleet. You're accountable for the health of the whole path from request to model: its efficiency, its reliability, and how well it evolves as models, hardware, and clouds change underneath it.\n Key responsibilities \n \n Own the technical roadmap for how the inference fleet is coordinated — where traffic goes, where capacity lives, how caches are placed, how fast the system reacts to demand, and the protocols that keep the control plane and the inference engines in sync\n Partner with the product, inference engine, performance, and capacity teams to identify throughput, latency, utilization, and cost wins, then turn those into shipped improvements with measurable results\n Build the group's habit of quantitative modeling: claim a win only when you can measure it, and know before you ship what the expected effect is\n Set technical strategy for how the control plane evolves across heterogeneous hardware, across multiple cloud providers, and across all our serving surfaces\n Run the group's operational backbone — on-call rotations, incident response, postmortem review, deploy safety — so the teams can ship aggressively without the system becoming fragile\n Create clarity at a seam: this group sits between the API surface, the inference engines, capacity planning, and the cloud deployment teams\n Develop and retain strong existing teams, and hire against a high technical bar\n Coach engineers through a roadmap where priorities shift\n Shape team structure as the scope grows: decide where the boundaries between problem areas should sit, and grow leads who can own each\n Pick up slack when it matters. These are small teams on a critical path; sometimes the EM is the one unblocking a stuck initiative or synthesizing a design debate\n \n Minimum qualifications \n \n Engineering management experience leading teams on critical-path production infrastructure at scale\n A deep systems background — load balancing, scheduling, cluster orchestration, autoscaling, cache-coherent distributed state, high-performance networking, or similar — with enough depth to make architectural calls about how a large fleet is coordinated and to evaluate candidates who go to the kernel and framework level\n Experience shipping performance or efficiency improvements in large-scale systems, and the ability to explain, with numbers, what the impact was — including the cost side, not just the latency side\n Experience running production infrastructure with real operational stakes: on-call, incident response, capacity events, deploy discipline\n A results-oriented, impact-driven approach, and comfort working in a space where throughput, latency, cost, stability, launch timelines, and feature velocity all pull in different directions\n Ability to build strong relationships across team boundaries — this is a seam role, and much of the job is making sure other teams can rely on yours\n Curiosity about machine learning systems — you don't need an ML research background, but you should want to learn how transformer inference actually works and how that shapes the systems problems\n \n Preferred qualifications \n \n 5+ years of engineering management experience\n Experience with LLM inference serving — KV caching, continuous batching, request scheduling, prefill/decode disaggregation\n Background in cluster schedulers, autoscalers, load balancers, service meshes, or fleet control planes at scale (Kubernetes internals, Borg-style systems, or equivalents)\n Experience running worklo","salary_min":405000,"salary_max":625000,"location":"San Francisco, CA","workplace":"hybrid","remote_scope":"not_remote","job_type":"full-time","experience_level":"senior","tags":["llm","cloud","alignment","infrastructure","inference"],"apply_url":"https://job-boards.greenhouse.io/anthropic/jobs/5411560008","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-09-01T16:25:46Z","expires_at":"2026-11-13T16:54:55.499507Z","created_at":"2026-09-02T13:30:19.54202Z","updated_at":"2026-10-14T16:54:55.502641Z","company_name":"Anthropic","company_slug":"anthropic","company_logo_url":"https://www.google.com/s2/favicons?domain=anthropic.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/c831041b-2707-4916-8a82-01d11b60f788"},{"id":"b040d81d-82bc-4198-a020-8cf6e2705157","company_id":"e8dfc4ee-9649-4fd0-9c16-90d38a1954e1","title":"Staff Machine Learning Scientist, Applied Causal Inference","slug":"staff-machine-learning-scientist-applied-causal-inference-4231759e","description":"About the Team \n DoorDash is building the next generation of causal decisioning systems for New Verticals: grocery, convenience, retail, alcohol, pets, flowers, and other emerging categories. These businesses operate in high-dimensional, messy marketplaces where every consumer, merchant, item, promotion, substitution, search result, and delivery promise creates a causal question.\n About the Role \n We are hiring a Causal Machine Learning Engineer to help build the causal ML foundation behind how DoorDash grows New Verticals. This is not a generic ML role with some experimentation work on the side. We are looking for someone who has built or deeply worked on production causal systems: uplift models, heterogeneous treatment effect models, surrogate metrics, experimentation platforms, counterfactual policy evaluation, promotion optimization, or marketplace decisioning systems.\n You will join a small, senior pod of causal ML and econometrics experts working across ML, Analytics, Product, and Engineering. The mandate is to build the causal spine for a large-scale consumer marketplace.\n You're excited about this opportunity because you will… \n \n Design, build, and productionize causal ML systems that influence real marketplace decisions across New Verticals.\n Build uplift / heterogeneous treatment effect models for consumer lifecycle value, promotions, retention, and reactivation.\n Develop counterfactual evaluation frameworks for ranking, recommendations, search, promotions, substitutions, and marketplace interventions.\n Build systems that connect experimentation, observational data, and ML decisioning so teams can make better tradeoffs when randomized experiments are slow, noisy, or incomplete.\n Design surrogate metrics and early indicators that help teams move faster while preserving long-term marketplace health.\n Partner with econometrics and analytics leaders to choose the right methods: doubly robust estimation, IV, diff-in-diff, synthetic controls, double ML, CUPED-style variance reduction, contextual bandits, off-policy evaluation, and related approaches.\n Translate causal models into production systems that can shape decisions in ranking, targeting, budget allocation, inventory-aware discovery, and consumer growth.\n Raise the bar for causal reasoning across ML teams: when to trust a model, when not to, and how to debug causal claims in a real marketplace.\n \n We're excited about you because you have… \n \n Deep practical experience with causal inference, econometrics, experimentation, or causal ML .\n Experience shipping models or decision systems in production, ideally in consumer marketplaces, ads, recommendations, search, pricing, promotions, logistics, fintech, or other high-scale settings.\n Strong judgment around the tradeoffs between randomized experiments, observational estimation, and model-based decisioning.\n Comfort debating and applying methods such as doubly robust estimation, double ML, IV, diff-in-diff, CUPED, uplift modeling, contextual bandits, and off-policy evaluation .\n Strong ML engineering ability: you can build reliable pipelines, train models, evaluate them rigorously, and partner with platform teams to put them into production.\n Strong product judgment: you can connect methods to business decisions, not just optimize offline metrics.\n The ability to operate across functions with ML engineers, economists, data scientists, product managers, and business leaders.\n Compensation \n The successful candidate’s starting pay will fall within the pay range listed below and is determined based on job-related factors including, but not limited to, skills, experience, qualifications, work location, and market conditions. Base salary is localized according to an employee’s work location. Ranges are market-dependent and may be modified in the future.\n In addition to base salary, the compensation for this role includes opportunities for equity grants. Talk to your recruiter for more information.\n DoorDash cares about you and your overall well-being. That’s why we offer a comprehensive benefits package to all regular employees, which includes a 401(k) plan with employer matching, 16 weeks of paid parental leave, wellness benefits, commuter benefits match, paid time off and paid sick leave in compliance with applicable laws (e.g. Colorado Healthy Families and Workplaces Act). DoorDash also offers medical, dental, and vision benefits, 11 paid holidays, disability and basic life insurance, family-forming assistance, and a mental health program, among others.\n To learn more about our benefits, visit our careers page here .\n See below for paid time off details:\n \n For salaried roles: flexible paid time off/vacation, plus 80 hours of paid sick time per year.\n For hourly roles: vacation accrued at about 1 hour for every 25.97 hours worked (e.g. about 6.7 hours/month if working 40 hours/week; about 3.4 hours/month if working 20 hours/week), and paid sick time accrued at 1 hour for every 30 hours worked","salary_min":203500,"salary_max":299300,"location":"San Francisco, CA","workplace":"onsite","remote_scope":"not_remote","job_type":"full-time","experience_level":"lead","tags":["fine-tuning","payments","healthcare","cloud","machine-learning","inference"],"apply_url":"https://job-boards.greenhouse.io/doordashusa/jobs/8140067","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-08-18T20:55:11Z","expires_at":"2026-11-13T17:03:28.801506Z","created_at":"2026-08-25T18:33:45.109934Z","updated_at":"2026-10-14T17:03:28.805175Z","company_name":"DoorDash","company_slug":"doordash","company_logo_url":"https://www.google.com/s2/favicons?domain=doordash.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/b040d81d-82bc-4198-a020-8cf6e2705157"},{"id":"e9ebce41-3ac8-47ba-a5ee-5e22e28ef4f1","company_id":"e8dfc4ee-9649-4fd0-9c16-90d38a1954e1","title":"Staff Machine Learning Engineer, Causal Inference","slug":"staff-machine-learning-engineer-causal-inference-da3f0671","description":"About the Team \n DoorDash is building the next generation of causal decisioning systems for New Verticals: grocery, convenience, retail, alcohol, pets, flowers, and other emerging categories. These businesses operate in high-dimensional, messy marketplaces where every consumer, merchant, item, promotion, substitution, search result, and delivery promise creates a causal question.\n About the Role \n We are hiring a Causal Machine Learning Engineer to help build the causal ML foundation behind how DoorDash grows New Verticals. We are looking for someone who has built or deeply worked on production causal systems: uplift models, heterogeneous treatment effect models, surrogate metrics, experimentation platforms, counterfactual policy evaluation, promotion optimization, or marketplace decisioning systems.\n You will join a small, senior pod of causal ML and econometrics experts working across ML, Analytics, Product, and Engineering. The mandate is to build the causal spine for a large-scale consumer marketplace.\n You're excited about this opportunity because you will… \n \n Design, build, and productionize causal ML systems that influence real marketplace decisions across New Verticals.\n Build uplift / heterogeneous treatment effect models for consumer lifecycle value, promotions, retention, and reactivation.\n Develop counterfactual evaluation frameworks for ranking, recommendations, search, promotions, substitutions, and marketplace interventions.\n Build systems that connect experimentation, observational data, and ML decisioning so teams can make better tradeoffs when randomized experiments are slow, noisy, or incomplete.\n Design surrogate metrics and early indicators that help teams move faster while preserving long-term marketplace health.\n Partner with econometrics and analytics leaders to choose the right methods: doubly robust estimation, IV, diff-in-diff, synthetic controls, double ML, CUPED-style variance reduction, contextual bandits, off-policy evaluation, and related approaches.\n Translate causal models into production systems that can shape decisions in ranking, targeting, budget allocation, inventory-aware discovery, and consumer growth.\n Raise the bar for causal reasoning across ML teams: when to trust a model, when not to, and how to debug causal claims in a real marketplace.\n \n We're excited about you because you have… \n \n Deep practical experience with causal inference, econometrics, experimentation, or causal ML .\n Experience shipping models or decision systems in production, ideally in consumer marketplaces, ads, recommendations, search, pricing, promotions, logistics, fintech, or other high-scale settings.\n Strong judgment around the tradeoffs between randomized experiments, observational estimation, and model-based decisioning.\n Comfort debating and applying methods such as doubly robust estimation, double ML, IV, diff-in-diff, CUPED, uplift modeling, contextual bandits, and off-policy evaluation .\n Strong ML engineering ability: you can build reliable pipelines, train models, evaluate them rigorously, and partner with platform teams to put them into production.\n Strong product judgment: you can connect methods to business decisions, not just optimize offline metrics.\n The ability to operate across functions with ML engineers, economists, data scientists, product managers, and business leaders.\n Compensation \n The successful candidate’s starting pay will fall within the pay range listed below and is determined based on job-related factors including, but not limited to, skills, experience, qualifications, work location, and market conditions. Base salary is localized according to an employee’s work location. Ranges are market-dependent and may be modified in the future.\n In addition to base salary, the compensation for this role includes opportunities for equity grants. Talk to your recruiter for more information.\n DoorDash cares about you and your overall well-being. That’s why we offer a comprehensive benefits package to all regular employees, which includes a 401(k) plan with employer matching, 16 weeks of paid parental leave, wellness benefits, commuter benefits match, paid time off and paid sick leave in compliance with applicable laws (e.g. Colorado Healthy Families and Workplaces Act). DoorDash also offers medical, dental, and vision benefits, 11 paid holidays, disability and basic life insurance, family-forming assistance, and a mental health program, among others.\n To learn more about our benefits, visit our careers page here .\n See below for paid time off details:\n \n For salaried roles: flexible paid time off/vacation, plus 80 hours of paid sick time per year.\n For hourly roles: vacation accrued at about 1 hour for every 25.97 hours worked (e.g. about 6.7 hours/month if working 40 hours/week; about 3.4 hours/month if working 20 hours/week), and paid sick time accrued at 1 hour for every 30 hours worked (e.g. about 5.8 hours/month if working 40 hours/week; about 2.9 hours/mon","salary_min":203500,"salary_max":299300,"location":"San Francisco, CA","workplace":"onsite","remote_scope":"not_remote","job_type":"full-time","experience_level":"lead","tags":["fine-tuning","payments","cloud","healthcare","machine-learning","inference"],"apply_url":"https://job-boards.greenhouse.io/doordashusa/jobs/8139942","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-08-18T20:55:10Z","expires_at":"2026-11-13T17:03:28.787472Z","created_at":"2026-08-25T18:33:45.096852Z","updated_at":"2026-10-14T17:03:28.790648Z","company_name":"DoorDash","company_slug":"doordash","company_logo_url":"https://www.google.com/s2/favicons?domain=doordash.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/e9ebce41-3ac8-47ba-a5ee-5e22e28ef4f1"},{"id":"8c1a812e-e15b-4f41-adec-d56b9a321557","company_id":"332b7698-676b-4a3e-8b02-81b1195c5af6","title":"Staff Software Engineer- Foundation Model Inference","slug":"staff-software-engineer-foundation-model-inference-0efd5575","description":"P-1930\n At Databricks, we are passionate about enabling data and AI teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to improve their business. Founded by engineers — and customer-obsessed — we leap at every opportunity to solve technical challenges, from designing next-gen UI/UX for interfacing with data to scaling our services and infrastructure across millions of virtual machines. And we're only getting started.\n As part of the AI team, you'll build the platforms and products that power everything from data apps, AI agents, model training, model serving, and Vector Search. You'll be joining a high-agency, high-visibility team operating at the frontier of AI infrastructure — with deep ties to research, product, and real-world enterprise use cases. Databricks Mosaic AI is one of our fastest-growing businesses, helping thousands of our customers democratize AI within their organizations. We're building the products and infrastructure that power the next generation of AI.\n The Foundation Model Inference team is the backbone of Databricks’ generative AI capabilities. We build the infrastructure that enables our customers to serve, scale, and optimize frontier models with enterprise-grade reliability and performance. Our Foundation Model APIs provide a unified platform that gives customers access to LLMs with the governance, flexibility, and scalability required for enterprise production workloads.\n We are looking for high-agency engineers who are excited to work on powering model inference at enterprise scale. Check out our recent blog post: Reliable LLM Inference at Scale \n The impact you will have: \n \n Build LLM infrastructure powering large-scale inference workloads for customers through partner models (OpenAI, Anthropic, Gemini) and self-hosted models (Qwen, GPT-OSS, Llama)\n Improve reliability, latency, and efficiency of distributed AI workloads\n Collaborate with platform, infra, and ML teams to deliver seamless end-to-end experiences\n Shape how developers and data scientists build and interact with AI on Databricks\n \n What we look for: \n \n 8+ years of experience in backend or infrastructure engineering\n Experience with distributed systems, scalable APIs, or cloud-native infrastructure\n Experience with real-time serving, ML infrastructure, or GPU orchestration\n Familiarity with service-oriented architecture, deployment pipelines, and system observability\n \n Bonus points for: \n \n Exposure to platforms like SageMaker, Vertex AI, or Azure ML\n Contributions to OSS projects like MLflow, PyTorch, Ray, vLLM, SGLang\n Built developer platforms or internal tools supporting AI workflows\n  \n Pay Range Transparency \n Databricks is committed to fair and equitable compensation practices. The pay range(s) for this role is listed below and represents the expected salary range for non-commissionable roles or on-target earnings for commissionable roles.  Actual compensation packages are based on several factors that are unique to each candidate, including but not limited to job-related skills, depth of experience, relevant certifications and training, and specific work location. Based on the factors above, Databricks anticipates utilizing the full width of the range. The total compensation package for this position may also include eligibility for annual performance bonus, equity, and the benefits listed above. For more information regarding which range your location is in visit our page here . \n  \n Local Pay Range\n $190,000 — $265,000 USD \n About Databricks \n Databricks is the Data and AI company. More than 20,000 organizations worldwide — including adidas, AT\u0026T, Bayer, Block, Mastercard, Rivian, Unilever, and 70% of the Fortune 500 — rely on the Databricks Data + AI Platform to build and scale data and AI apps, analytics and agents. Headquartered in San Francisco with 30+ offices around the globe, Databricks offers a unified platform that includes Genie, Lakebase, Agent Bricks, Lakeflow, Lakehouse, and Unity Catalog. To learn more, follow Databricks on LinkedIn , X , YouTube , and Instagram . Benefits At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our employees. For specific details on the benefits offered in your region click here . \n Our Commitment to Diversity and Inclusion \n At Databricks, we are committed to fostering a diverse and inclusive culture where everyone can excel. We take great care to ensure that our hiring practices are inclusive and meet equal employment opportunity standards. Individuals looking for employment at Databricks are considered without regard to age, color, disability, ethnicity, family or marital status, gender identity or expression, language, national origin, physical and mental a","salary_min":190000,"salary_max":265000,"location":"San Francisco, CA","workplace":"onsite","remote_scope":"not_remote","job_type":"full-time","experience_level":"lead","tags":["llm","agents","generative-ai","pytorch","mlops","distributed-systems","cloud","inference"],"apply_url":"https://databricks.com/company/careers/open-positions/job?gh_jid=8649279002","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-07-24T16:49:01Z","expires_at":"2026-11-13T16:55:16.779838Z","created_at":"2026-07-25T14:02:34.398709Z","updated_at":"2026-10-14T16:55:16.784701Z","company_name":"Databricks","company_slug":"databricks","company_logo_url":"https://www.google.com/s2/favicons?domain=databricks.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/8c1a812e-e15b-4f41-adec-d56b9a321557"},{"id":"15d0b3ec-2358-4c4d-b3e1-146969722a5b","company_id":"2721f049-2cf2-4e3e-82d0-8d8df89c8f90","title":"Senior Machine Learning Engineer, LLM Inference Optimization","slug":"senior-machine-learning-engineer-llm-inference-optimization-5eb66d9e","description":"About Nebius: \n Nebius is leading a new era in cloud infrastructure for the global AI economy. We are building a full-stack AI cloud platform that supports developers and enterprises from data and model training through to production deployment, without the cost and complexity of building large in-house AI/ML infrastructure.\n Built by engineers, for engineers. From large-scale GPU orchestration to inference optimization, we own the hard problems across compute, storage, networking and applied AI.\n Listed on Nasdaq (NBIS) and headquartered in Amsterdam, we have a global footprint with R\u0026D hubs across Europe, the UK, North America and Israel. Our team of 1,500+ includes hundreds of engineers with deep expertise across hardware, software and AI R\u0026D.\n The role   \n As a Senior Machine Learning Engineer on the Applied AI team at Nebius Token Factory, you will own model and endpoint optimization from model artifacts through production deployment. Your work will span model internals, quantization and model compression, speculative decoding, KV-cache optimization, inference engines, serving architecture, and benchmarking to improve latency, throughput, memory efficiency, GPU utilization, and cost per token while preserving model quality and reliability. You will work with kernel and platform engineers to investigate serving problems, compare configurations, and resolve performance and quality regressions. You will evaluate model-and engine-level optimizations together with distributed inference designs, including request routing, scheduling, prefill/decode coordination, and multi-node GPU execution. Your improvements will be validated through reproducible benchmarks and safe production rollouts under real-world workloads. \n Your responsibilities :   \n \n \n Own optimization projects for specific model families, customer endpoints, or serving backends.\n \n Compare inference engines and recommend practical serving configurations suited to individual workloads.\n \n Diagnose model-quality and performance regressions during production rollouts.\n \n Improve LLM and VLM endpoint latency, throughput, memory efficiency, GPU utilization, quality, and cost per token.\n \n Deploy, configure, benchmark, and extend inference engines such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, and NVIDIA Dynamo.\n \n Develop production model-compression workflows covering quantization, quantization-aware training, distillation, low-bit serving, and accuracy recovery.\n \n Implement or integrate speculative decoding, draft-model approaches, KV -cache optimization, prefix caching, chunked prefill, continuous batching, and disaggregated prefill/decode serving. For prefill/decode disaggregation (PDD), evaluate KV-cache transfer, worker placement, and capacity balancing to determine which workloads benefit from the architecture.\n Design and improve LLM request routers and scheduling policies that balance worker utilization queuing, request characteristics, and KV-cache locality while meeting latency and reliability targets \n Scale dense and mixture-of-experts inference across multiple GPU nodes. Select and tune tensor, pipeline, data, and expert parallelism - including wide expert parallelism (WideEP) - with attention to hardware topology, expert load balance, and communication overhead. \n \n Build reproducible benchmark harnesses measuring for  TTFT , TPOT , tokens per second per GPU, p95/p99 latency, GPU memory, reliability, and cost per token. Validate architecture choices under representative traffic and consistent GPU budgets.\n \n Collaborate with GPU kernel engineers and platform engineers to trace bottlenecks across model code, kernels, runtime, scheduler, gateway, and cluster layers.\n \n Produce design documents, performance reports, rollout plans, and customer-facing technical explanations.\n \n Must-haves :   \n \n \n Strong engineering skills in Python and PyTorch.\n \n Hands-on experience deploying or optimizing LLM, VLM , or high-throughput transformer inference.\n \n Practical knowledge of at least one modern inference stack, such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server,  NVIDIA Dynamo, Ray Serve, KServe, or an equivalent internal system.\n \n A strong understanding of transformer inference bottlenecks involving KV cache, attention, memory bandwidth, batching, parallelism, and long-context serving.\n \n Ability to quantify trade-offs among latency, throughput, quality, utilization, and cost.\n Experience designing or optimizing distributed inference system architecture, with hands-on work in one or more areas such as request routing, distributed scheduling, PDD, multi-node inference, or expert parallelism.\n \n Strong communication skills and collaboration skills across research, kernel, infrastructure, product, and customer teams.\n \n Nice - to - have s :   \n \n \n Experience with quantization-aware training, post-training quantization, FP8 , INT8 , INT4 , NVFP4 , MXFP4 , AWQ , GPTQ , SmoothQuant.\n \n Experience with distillation,","salary_min":195200,"salary_max":262200,"location":"Palo Alto, CA","workplace":"onsite","remote_scope":"not_remote","job_type":"full-time","experience_level":"senior","tags":["cloud","gpu","llm","agents","pytorch","machine-learning","inference"],"apply_url":"https://careers.nebius.com/?gh_jid=4921522101","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-07-22T22:54:04Z","expires_at":"2026-11-13T17:00:03.470564Z","created_at":"2026-07-24T14:15:06.696498Z","updated_at":"2026-10-14T17:00:03.474071Z","company_name":"Nebius","company_slug":"nebius","company_logo_url":"https://www.google.com/s2/favicons?domain=nebius.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/15d0b3ec-2358-4c4d-b3e1-146969722a5b"},{"id":"f2f0947c-9182-4d73-b1c8-ab97cc666183","company_id":"2721f049-2cf2-4e3e-82d0-8d8df89c8f90","title":"Senior Applied Scientist, Efficient LLM Inference \u0026 Model Optimization","slug":"senior-applied-scientist-efficient-llm-inference-model-optimization-0c77e3de","description":"About Nebius: \n Nebius is leading a new era in cloud infrastructure for the global AI economy. We are building a full-stack AI cloud platform that supports developers and enterprises from data and model training through to production deployment, without the cost and complexity of building large in-house AI/ML infrastructure.\n Built by engineers, for engineers. From large-scale GPU orchestration to inference optimization, we own the hard problems across compute, storage, networking and applied AI.\n Listed on Nasdaq (NBIS) and headquartered in Amsterdam, we have a global footprint with R\u0026D hubs across Europe, the UK, North America and Israel. Our team of 1,500+ includes hundreds of engineers with deep expertise across hardware, software and AI R\u0026D.\n The role   \n Nebius Token Factory is looking for scientists who can turn frontier LLM and VLM inference bottlenecks into research questions, conduct rigorous experiments, and translate their findings into production capabilities. You will own focused research and optimization projects, write strong code, and produce prototypes, publications, and technical artifacts that engineers can build on.\n Your research will span quantization, distillation, speculative decoding, KV-cache optimization, and model/runtime co-optimization, alongside inference engines and distributed inference architectures. You will investigate how these approaches interact, evaluate their trade-offs across model quality, latency, throughput, memory footprint, and cost per token, and work with engineering teams to bring promising results into production.\n Your responsibilities :   \n \n Lead research projects in efficient LLM and VLM inference, from hypotheses and experiments through ablations, prototypes, and production handoff.\n Develop and evaluate methods spanning quantization, quantization-aware training (QAT), distillation, speculative decoding, KV-cache reuse, KV-cache compression, long-context inference, MoE routing, and model/runtime co-optimization.\n Build prototypes using PyTorch, Triton, CUDA-adjacent tooling, or inference-serving frameworks, and collaborate with MLEs and platform engineers to turn them into production components.\n Research LLM request routing and scheduling strategies, including cache-aware load balancing and prefill–decode disaggregation (PDD). Evaluate how queueing, KV-cache transfer, worker placement, and prefill/decode capacity allocation affect latency, throughput, and serving cost.\n Investigate multi-node inference for dense and mixture-of-experts models, including wide expert parallelism (WideEP). Study expert placement, load imbalance, and computation/communication trade-offs, and use the findings to guide model and system architecture choices.\n Develop rigorous evaluations covering quality, latency, throughput, numerical stability, memory footprint, tail latency, and cost per token. Compare serving architectures under consistent workload conditions and GPU budgets.\n Work with MLE, GPU kernel, backend infrastructure, product, and customer teams to select research priorities with measurable production impact.\n Share results through internal reports, technical blogs, papers, and open-source artifacts, and mentor engineers and scientists on experimental design, scientific rigor, and model/system trade-offs.\n \n Must-haves :   \n \n A PhD in computer science, machine learning, ML systems, computer systems, computer architecture, electrical engineering, applied mathematics, or a closely related discipline.\n \n A strong publication record or equivalent research artifacts in ML, ML systems, efficient inference, model compression, quantization, distillation, serving systems, or related areas.\n \n Strong Python and PyTorch implementation skills, with the ability to turn ideas into experiments and working prototypes.\n \n Deep knowledge of LLMs, VLMs, transformer inference, decoding algorithms, model compression, quantization, and production-serving trade-offs.\n \n Strong experimental design skills covering ablations, baselines, metrics, statistical reasoning, and failure analysis.\n \n Excellent written and verbal communication.\n \n Nice - to - have s :   \n \n \n First-author publications at venues such as NeurIPS, ICML, ICLR, MLSys, ACL, EMNLP, ASPLOS, OSDI, SOSP, ISCA, or HPCA.\n \n Experience deploying ML models or inference optimizations in production.\n \n Experience with vLLM, SGLang, TensorRT-LLM, NVIDIA Dynamo, FlashAttention, FlashInfer, Triton, CUDA, or PyTorch internals.\n \n Experience applying post-training, SFT, DPO, RLHF, RLAIF, preference optimization, or synthetic data generation to inference quality or efficiency.\n \n Open-source research artifacts, widely used benchmarks, technical blogs, or invited talks demonstrating contributions to efficient AI systems.\n Research or implementation experience in one or more areas of distributed inference system architecture, such as LLM request routing, PDD, multi-node serving, or MoE expert paralleli","salary_min":195200,"salary_max":262200,"location":"Palo Alto, CA","workplace":"onsite","remote_scope":"not_remote","job_type":"full-time","experience_level":"senior","tags":["reinforcement-learning","gpu","llm","nlp","pytorch","cloud","inference"],"apply_url":"https://careers.nebius.com/?gh_jid=4921549101","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-07-22T22:53:48Z","expires_at":"2026-11-13T17:00:03.291804Z","created_at":"2026-07-24T14:15:05.082048Z","updated_at":"2026-10-14T17:00:03.295186Z","company_name":"Nebius","company_slug":"nebius","company_logo_url":"https://www.google.com/s2/favicons?domain=nebius.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/f2f0947c-9182-4d73-b1c8-ab97cc666183"},{"id":"5a8e8e14-9c3d-4f28-82cc-3fb8c4c39f5d","company_id":"6ea0f41a-b13e-481a-b410-5195f391f939","title":"[Junior / Senior / Staff] Software Engineer, Inference / Compute Infrastructure Engineering","slug":"staff-software-engineer-gpu-infrastructure-lifecycle-management-6fe5d3f8","description":"About the Role \n We're looking for a software engineer to build the Kubernetes-native control plane that provisions and runs our GPU inference fleet. You'll design a manifest-driven API where the inference team declares what they need, whether that's a cluster, a model deployment, or a capacity change, and our controllers handle the reconciliation, provider/runtime selection, and lifecycle management underneath, so the inference team never has to know or care which specific serving stack, scheduler, or hardware pool is doing the work. You'll also build the systems that keep the fleet efficient, not just running, including defragmentation and rebalancing logic that consolidates scattered workloads back into contiguous capacity, and scheduling/bin-packing improvements that push GPU utilization up without hurting latency.\n The core value we're after is decoupling the people building on top of the platform from the operational and runtime complexity underneath, while squeezing more usable capacity out of the same hardware. You'll build the controllers, reconciliation loops, and self-service surface (API/CLI, not tickets) that make that decoupling real, plus the event-driven health, remediation, and utilization systems that keep it running and efficient without a human in the loop. Strong candidates have hands-on experience with Kubernetes controller/CRD patterns, have built or operated a platform API that abstracts multiple backends behind one interface, understand GPU scheduling and capacity efficiency (fragmentation, bin-packing, right-sizing), and think about GPU infrastructure as software to be engineered.\n A product mindset - you've built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship. You build it, you own it. You are not only responsible for delivering the software but also for operating and supporting it in production.\n Responsibilities \n \n Build the provisioning state machine: design and implement the software that models the full lifecycle of a physical host from discovery, inference bring-up to GPU driver/CUDA stack, health validation, and decommission/RMA — as explicit, versioned states and transitions.\n Build the self-service API: design declarative APIs and a control plane so the inference team can request, scale, and tear down inference clusters with one API call — no ticket, no human in the loop.\n Automate self-healing: detect degraded or failed nodes, drain them safely, trigger repair or replacement, and reintroduce healthy capacity into the pool automatically.\n Own reliability of the pipeline: idempotency, retries, rollback, and drift detection so the provisioning system is as dependable as any other production service.\n Partner with the inference/ML platform team: understand the cluster shapes they need — topology, interconnect, scheduling constraints — and encode them as first-class abstractions in the platform.\n Engineer it like software: strong typing, automated tests, code review, versioning, and CI/CD for infrastructure code — this is a product, not a collection of Ansible playbooks.\n \n Requirements \n Core requirements (all levels):\n \n Strong software engineering background in Go, Python, Rust, or similar — you write and test real software for a living.\n Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent to run long-lived, manifest-driven workflows that survive failures and resume mid-execution.\n Experience building software control planes or orchestration systems that model state and reconcile it over time (e.g., Kubernetes controllers/operators, custom reconciliation loops, workflow engines).\n Experience with event-driven systems — designing and building software around message queues, event streams, or pub/sub (e.g., Kafka, NATS, SQS) rather than polling or cron-driven scripts.\n A product mindset. You’ve built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship.\n \n Nice to have:\n \n Exposure to bare-metal provisioning (PXE/iPXE, Redfish/IPMI, BMC) and/or networking fundamentals (VLANs, BGP, fabric design), or GPU/accelerator infrastructure.\n Experience with GPU cluster software stacks (NCCL, CUDA, InfiniBand/RoCE).\n Prior work at a hyperscaler, GPU cloud, or datacenter-scale infrastructure organization.\n Systems programming in Rust or Go.\n \n About Together AI \n Together AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.\n Compensation \n We offer competitive compensat","salary_min":160000,"salary_max":280000,"location":"San Francisco, CA","workplace":"onsite","remote_scope":"not_remote","job_type":"full-time","experience_level":"lead","tags":["fine-tuning","gpu","mlops","pre-training","reinforcement-learning","inference","infrastructure"],"apply_url":"https://job-boards.greenhouse.io/togetherai/jobs/5186628007","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-07-16T16:58:09Z","expires_at":"2026-11-13T16:55:08.807288Z","created_at":"2026-07-18T14:02:18.890811Z","updated_at":"2026-10-14T16:55:08.810033Z","company_name":"Together AI","company_slug":"together-ai","company_logo_url":"https://www.google.com/s2/favicons?domain=together.ai\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/5a8e8e14-9c3d-4f28-82cc-3fb8c4c39f5d"},{"id":"02fdc710-8e20-40fd-aedd-05f740fa50ac","company_id":"377b9ca2-ac79-48a5-8657-da630f9e447d","title":"Senior Staff / Principal Machine Learning Scientist, AI Inference \u0026 Optimization","slug":"senior-staff-principal-machine-learning-scientist-ai-inference-optimization-8c8ecaa7","description":"Join the Future of Security at Netskope\n Netskope (NASDAQ: NTSK) is a leader in modern security and networking for the cloud and AI era. We secure and accelerate cloud, data, and AI in real time, everywhere. Thousands of customers, including more than 30 of the Fortune 100, trust the Netskope One platform, its Zero Trust Engine, and the powerful NewEdge network to gain full visibility and control without performance trade-offs.\n At Netskope, our technology is driven by our greatest strength: our people. We believe that belonging powers innovation, and success is both personal and organizational. We embrace differences in gender, ethnicity, beliefs, ability, and identity, creating an environment where every voice is heard and respected. We empower our employees to bring their authentic selves to work, grow their careers through continuous education and mentorship, and lead with transparency and curiosity. Join a team where you belong, where you are encouraged to be an entrepreneur, and where together, we continue to redefine the landscape of security.\n Visit Careers at Netskope to learn more. Follow us on  LinkedIn and  Instagram .\n Positions are available at Senior Staff and above. Candidates are assessed individually and leveled according to their specific skills and background. \n About the role\n As a Senior Staff Machine Learning Scientist, you own the inference and optimization layer that makes AI in agentic workflows fast, efficient, and production-grade. You fine-tune and evaluate models, push latency and throughput on real hardware, and build the runtime that executes bounded AI tasks, validated against usage from Netskope’s large customer base so you optimize where the data points, not where you guess.\n What’s in it for you\n \n High-impact ownership. You own the model layer of a net-new product that changes the performance and economics of agentic AI.\n Cutting-edge, unusual stack. The hard, interesting inference problems live here: quantization, KV-cache and memory management, sparsity, fine-tuning, and hardware acceleration under real-world resource constraints.\n Real scale to build against. Netskope’s customer footprint gives you production signals most teams never see, so you deploy, validate, and iterate fast.\n \n What you will be doing\n \n Build and optimize the model inference path : quantization, KV-cache optimization, batching, and latency/memory/throughput tuning on constrained, commodity hardware.\n Fine-tune and evaluate models for bounded tasks; build eval harnesses that gate a capability to release on real accuracy, latency, and security relevance.\n Design and grow the task execution runtime (bounded sub-agents), pushing toward dynamic task generation and context compaction.\n Drive hardware acceleration / sparsity and support for larger models as the platform matures.\n Partner with the systems and backend engineers to ship capabilities end-to-end and iterate on real production signals.\n \n Required skills and experience\n \n 10+ years of overall industry experience , with 4+ years hands-on in ML/AI (model development, fine-tuning, and inference optimization).\n Hands-on with fine-tuning (e.g. LoRA/QLoRA), quantization (GGUF/AWQ/GPTQ), and inference runtimes (vLLM/SGLang, TensorRT-LLM, ONNX Runtime, llama.cpp, or MLX/CoreML). On-device or edge inference experience is a strong plus.\n Strong Python; comfort reaching into C++ for low-level interop is a plus.\n Solid grasp of transformer internals and the levers that move real inference performance and cost: KV cache, attention, batching, memory footprint.\n Fluency with agentic coding systems and genuine curiosity about agent harnesses like Claude Code, Pi, and Codex , so you should already be building with them, or itching to.\n Clear communication: able to distill a model or infra bottleneck into an actionable concept for cross-functional teammates.\n \n Education\n \n MS in Computer Science, Machine Learning, Electrical Engineering, or equivalent technical degree required, with a focus in AI/ML research; PhD in a related field strongly preferred.\n Compensation:  \n At Netskope, salary is one component of our competitive total rewards package. The salary range for this position is as listed below. This is a national range. For purposes of complying with applicable laws, the range applies to candidates in California, Colorado, Illinois, Maryland, New York, Washington, and other states. \n The successful candidate’s starting pay will also be determined based on job-related skills, experience, qualifications, location, and market conditions.  \n For all sales roles, the posted salary range is the On Target Earnings (OTE) range for the role, which is the sum of base salary and target commission amount at 100% goal achievement. \n In addition to salary, candidates may be eligible for other forms of compensation such as participation in a bonus plan (for non-sales roles) and a stock award program. Candidates may also be eligible for a comprehensive he","salary_min":124500,"salary_max":272000,"location":"San Jose, CA","workplace":"onsite","remote_scope":"not_remote","job_type":"full-time","experience_level":"principal","tags":["agents","cloud","fine-tuning","llm","machine-learning","inference"],"apply_url":"https://www.netskope.com/company/careers/open-positions/?gh_jid=8063869","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-07-14T04:20:32Z","expires_at":"2026-11-13T16:58:16.456868Z","created_at":"2026-07-15T14:11:39.076302Z","updated_at":"2026-10-14T16:58:16.460043Z","company_name":"Netskope","company_slug":"netskope","company_logo_url":"https://www.google.com/s2/favicons?domain=netskope.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/02fdc710-8e20-40fd-aedd-05f740fa50ac"},{"id":"fa7a576b-4fca-402a-b5c9-b66e260cd3e9","company_id":"f5ee7284-a657-4da2-b351-cb806a3681cd","title":"Member of Technical Staff - RL Inference","slug":"member-of-technical-staff-rl-inference-cb7bd166","description":"SpaceX was founded under the belief that a future where humanity is out exploring the stars is fundamentally more exciting than one where we are not. Today, SpaceX is actively developing the technologies to make this possible, with the ultimate goal of making life multiplanetary and expanding consciousness to the stars. SpaceXSI is building Super Intelligence to help humanity understand the universe. Our teams are building the full Super Intelligence stack: from our Colossus supercomputers, to Grok models, to our suite of applications, including Grok Bot, Grok Build, Grok Imagine, Grok Voice, and X. We're creating a powerful flywheel where more compute trains better models, better models power more useful products, and what we learn from people using those products shapes what we build next.\n ABOUT THE ROLE: \n The RL infrastructure team is looking for an engineer to help with low precision RL training and inference.\n RESPONSIBILITIES: \n \n Design and optimize our inference stack for all shapes of RL workloads at SpaceXSI, from small scale ablations to production training runs.\n Analyze, profile and address performance bottlenecks in large scale RL systems\n Work closely with the modelling team to efficiently implement novel RL techniques and algorithms\n \n BASIC QUALIFICATIONS: \n \n Experience in building, debugging, and optimizing efficiency of large-scale distributed systems\n Experience in LLM inference\n Proficiency in programming languages such as Python, C++ and/or Rust; frameworks such as PyTorch, Jax, CUDA\n Willingness to dive deep and solve hardcore problems at all levels of the stack\n \n PREFERRED SKILLS AND EXPERIENCE: \n \n Strong knowledge in quantization and numerics in LLM inference and training\n Experience in developing inference engines, e.g. SGLang, vLLM\n \n COMPENSATION AND BENEFITS: \n $180,000 - $440,000 USD\n Base salary is just one part of our total rewards package at SpaceXSI, which also includes equity, comprehensive medical, vision, and dental coverage, access to a 401(k) retirement plan, short \u0026 long-term disability insurance, life insurance, and various other discounts and perks.\n SpaceXSI is an equal opportunity employer. For details on data processing, view our  Recruitment Privacy Notice .","salary_min":180000,"salary_max":440000,"location":"Palo Alto, CA","workplace":"onsite","remote_scope":"not_remote","job_type":"full-time","experience_level":"lead","tags":["llm","pytorch","gpu","distributed-systems","inference"],"apply_url":"https://job-boards.greenhouse.io/xai/jobs/5180223007","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-07-06T14:59:58Z","expires_at":"2026-11-13T16:55:31.71233Z","created_at":"2026-07-09T14:03:21.460874Z","updated_at":"2026-10-14T16:55:31.715555Z","company_name":"xAI","company_slug":"xai","company_logo_url":"https://www.google.com/s2/favicons?domain=x.ai\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/fa7a576b-4fca-402a-b5c9-b66e260cd3e9"},{"id":"8ff64df9-5c3c-4794-90fd-dc4cbcafc029","company_id":"a0000000-0000-0000-0000-000000000001","title":"Staff + Senior Software Engineer, Inference Deployment","slug":"staff-senior-software-engineer-inference-deployment-8d22b347","description":"About Anthropic \n Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems.\n About the role\n Our Inference team is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide. We bring Claude to life by serving our models via the industry’s largest compute-agnostic inference deployments. We are responsible for the entire stack from intelligent request routing to fleet-wide orchestration across diverse AI accelerators.\n The team has a dual mandate: maximizing compute efficiency to reliably serve our explosive customer growth, while enabling breakthrough research by giving our scientists the high-performance inference infrastructure they need to develop next-generation models. We tackle complex, distributed systems challenges across multiple accelerator families and emerging AI hardware running in multiple cloud platforms.\n Inference systems are highly performance sensitive distributed systems. Inference serves hundreds of thousands of customers every day, and the size \u0026 span of the inference fleet requires sophisticated routing, scaling, and networking systems.\n Key responsibilities\n \n Design, build, and maintain the distributed systems that serve Claude to millions of users worldwide\n Develop resilient, flexible systems that adapt in real time to real world events\n Develop intelligent request routing, load balancing, and traffic management systems across thousands of accelerators\n Maximize compute efficiency across the fleet by autoscaling and orchestrating production, research, and experimental workloads\n Build and operate production-grade deployment pipelines for releasing new models to users\n Provide high-performance inference infrastructure that enables researchers to develop next-generation models\n Integrate new AI accelerator platforms and support inference for new model architectures\n \n Minimum qualifications\n \n Significant software engineering experience, particularly with distributed systems\n Results-oriented, with a bias towards flexibility and impact\n Willingness to pick up slack, even if it goes outside your job description\n Desire to learn more about machine learning systems and infrastructure\n Thrive in environments where technical excellence directly drives both business results and research breakthroughs\n Care about the societal impacts of your work\n \n Preferred qualifications\n \n Experience with high-performance, large-scale distributed systems\n Experience implementing and deploying machine learning systems at scale\n Experience with load balancing, request routing, or traffic management systems\n Familiarity with LLM inference optimization, batching, and caching strategies\n Experience with Kubernetes and cloud infrastructure (AWS, GCP, Azure)\n Proficiency in Python or Rust\n \n Representative projects\n \n Designing intelligent routing algorithms that optimize request distribution across many accelerators in different environments\n Autoscaling our compute fleet to dynamically match supply with demand across production, research, and experimental workloads\n Building production-grade deployment pipelines for releasing new models to millions of users reliably\n Contributing to new inference features\n Supporting inference for new model architectures\n Analyzing observability data to tune performance based on real-world production workloads\n Managing multi-region deployments and geographic routing for global customers\n \n Deadline to apply:  None. Applications will be reviewed on a rolling basis. \n The annual compensation range for this role is listed below. \n For sales roles, the range provided is the role’s On Target Earnings (\"OTE\") range, meaning that the range includes both the sales commissions/sales bonuses target and annual base salary for the role.\n Annual Salary:\n $320,000 — $485,000 USD \n Logistics \n Minimum education: Bachelor’s degree or an equivalent combination of education, training, and/or experience\n Required field of study:  A field relevant to the role as demonstrated through coursework, training, or professional experience\n Minimum years of experience: Years of experience required will correlate with the internal job level requirements for the position\n Location-based hybrid policy: Currently, we expect all staff to be in one of our offices at least 25% of the time. However, some roles may require more time in our offices.\n Visa sponsorship:  We do sponsor visas! However, we aren't able to successfully sponsor visas for every role and every candidate. But if we make you an offer, we will make every reasonable effort to get you a visa, and we retain an immigration lawyer to help with this.\n We encourage you to apply even if you do not believe you meet every sin","salary_min":320000,"salary_max":485000,"location":"San Francisco, CA","workplace":"hybrid","remote_scope":"not_remote","job_type":"full-time","experience_level":"lead","tags":["alignment","cloud","llm","distributed-systems","inference","infrastructure"],"apply_url":"https://job-boards.greenhouse.io/anthropic/jobs/5285557008","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-06-29T14:33:21Z","expires_at":"2026-11-13T16:54:56.649945Z","created_at":"2026-06-30T14:00:33.50732Z","updated_at":"2026-10-14T16:54:56.653526Z","company_name":"Anthropic","company_slug":"anthropic","company_logo_url":"https://www.google.com/s2/favicons?domain=anthropic.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/8ff64df9-5c3c-4794-90fd-dc4cbcafc029"},{"id":"a8d08f57-0c6f-40d4-b42c-4fa9061feaed","company_id":"2114efab-ea67-411b-bfb8-7899153105f3","title":"Member of Technical Staff, Inference ","slug":"member-of-technical-staff-inference-86b5008f","description":"Inferact's mission is to grow vLLM as the world's AI inference engine and accelerate AI progress by making inference cheaper and faster. Founded by the creators and core maintainers of vLLM, we sit at the intersection of models and hardware—a position that took years to build.\n\n\n\n\nABOUT THE ROLE\n\nWe're looking for an inference runtime engineer to push the boundaries of what's possible in LLM and diffusion model serving. Models grow larger. Architectures shift: mixture-of-experts, multimodal, agentic. Every breakthrough demands innovations on the inference engine itself. You'll work at the core of vLLM, optimizing how models execute across diverse hardware and architectures. Your work will directly impact how the world runs AI inference. \n\n\n\n\nSKILLS AND QUALIFICATIONS\n\nMinimum qualifications:\n\n - Bachelor's degree or equivalent experience in computer science, engineering, or similar.\n\n - Deep understanding of transformer architectures and their variants.\n\n - Strong programming skills in Python with experience in PyTorch internals.\n\n - Experience with LLM inference systems (vLLM, TensorRT-LLM, SGLang, TGI).\n\n - Ability to read and implement model architectures and inference techniques from research papers.\n\n - Demonstrate the ability to contribute performant and maintainable code and debug in complex ML codebases.\n\nPreferred qualifications:\n\n - Deep understanding of KV-cache memory management, prefix caching, and hybrid model serving.\n\n - Familiarity with RL frameworks and algorithms for LLMs.\n\n - Experience with multimodal inference (audio/image/video/text).\n\n - Contributions to open-source ML or system infrastructure projects.\n\nBonus points if you have:\n\n - Implemented core features in vLLM or other inference engine projects.\n\n - Contributed to vLLM integrations (verl, OpenRLHF, Unsloth, LlamaFactory, etc).\n\n - Written widely-shared technical blogs or side projects on vLLM or LLM inference.\n   \n   \n\n\nLOGISTICS\n\n - Location: This role is based in San Francisco, California. Will consider remote in the US for exceptional candidates.\n\n - Compensation: Depending on background, skills, and experience, the expected annual salary range for this position is $200,000 - $400,000 USD + equity.\n\n - Visa sponsorship: We sponsor visas on a case-by-case basis.\n\n - Benefits: Inferact offers generous health, dental, and vision benefits as well as 401(k) company match.","salary_min":200000,"salary_max":400000,"location":"San Francisco, CA","workplace":"hybrid","remote_scope":"not_remote","job_type":"full-time","experience_level":"lead","tags":["agents","mlops","llm","pytorch","reinforcement-learning","diffusion-models","research","inference"],"apply_url":"https://jobs.ashbyhq.com/inferact/43c0ca54-fcf5-41fa-83a1-38800c75ccc0/application","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-06-18T18:55:18.703Z","expires_at":"2026-11-13T16:58:50.480986Z","created_at":"2026-06-28T14:10:47.134304Z","updated_at":"2026-10-14T16:58:50.483856Z","company_name":"Inferact","company_slug":"inferact","company_logo_url":"https://www.google.com/s2/favicons?domain=inferact.ai\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/a8d08f57-0c6f-40d4-b42c-4fa9061feaed"},{"id":"da71ff33-42e4-48bf-87eb-04189fa35d9d","company_id":"a0000000-0000-0000-0000-000000000001","title":"Staff+ Software Engineer, Inference Velocity","slug":"staff-software-engineer-inference-runtime-0cb9cf6b","description":"About Anthropic \n Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems.\n About the role\n Anthropic's Inference organization serves Claude to millions of users and enterprise customers with the speed, reliability, and efficiency that frontier AI demands. We build across GPUs, TPUs, and Trainium, and the complexity of our development environment grows with every platform we add. We're looking for a Staff engineer to be the technical lead for Inference Developer Productivity: the team that makes every engineer in the org dramatically more effective at building, testing, and shipping inference software.\n This is a senior IC role with broad technical ownership. You'll set technical direction for the team's toolchains, workflows, and feedback loops, and you'll be the one making the hard calls on architecture, prioritization, and tradeoffs across heterogeneous accelerator platforms. You'll pair with the team's Engineering Manager, who owns hiring and people development, while you own the technical roadmap and drive the work. You'll also partner closely with Anthropic's central Infrastructure org, where company-wide developer productivity lives, to make sure Inference's multi-accelerator reality is well served without duplicating effort.\n This role is for someone who has been the technical anchor on a platform or infrastructure team before, who thinks in systems and feedback loops, and who gets real satisfaction from the moment another engineer stops fighting their environment and starts shipping.\n Key responsibilities\n \n Set technical direction for Inference Developer Productivity, owning the architecture and roadmap for toolchains, dev environments, and CI/CD across GPU (CUDA), TPU, and Trainium platforms \n Be the technical owner of accelerator toolchain management: compilers, drivers, libraries, frameworks, kept current, compatible, and well-tested so Inference engineers focus on model serving instead of environment archaeology \n Design and build infrastructure for efficient accelerator usage during development, including devbox environments, pre- and post-land validation automation, and shared tooling that reduces the cost of working across heterogeneous hardware \n Define and instrument productivity metrics for the Inference org, building the dashboards and alerting that surface regressions early (smoke tests red for extended periods, build times creeping up, toolchain breakages) and drive them to resolution \n Proactively hunt down bottlenecks, toil, and friction across Inference engineering workflows, then design and build the systems that eliminate them \n Act as the technical counterpart to Anthropic's central Infrastructure org, aligning on shared developer productivity initiatives, contributing Inference-specific requirements, and making the call on build vs. adopt \n Mentor engineers on the team through design review, code review, and direct collaboration, raising the technical bar without owning headcount\n \n Minimum qualifications\n \n 8+ years of software engineering experience, with significant time as the technical lead or anchor on an infrastructure, platform, or developer productivity team \n Deep background in systems engineering, build/test infrastructure, or ML infrastructure, with the ability to go hands-on with toolchain issues, CI/CD pipelines, and developer workflow optimization \n Experience owning toolchains or development environments for compute-intensive workloads (ML training or inference, HPC, large-scale distributed systems) \n Real depth in at least one accelerator ecosystem (CUDA/GPU, TPU, or Trainium/AWS Neuron) and genuine appetite to learn the others \n A track record of defining and using engineering metrics to drive improvement: you've built dashboards, set SLOs on developer workflows, or led initiatives that measurably improved engineering velocity \n Experience driving technical alignment across organizational boundaries, advocating for your team's needs while contributing to shared infrastructure \n Strong written and verbal communication, and the ability to influence technical direction without formal authority\n \n Preferred qualifications\n \n Experience with ML compiler toolchains (XLA, Triton, NeuronX) or accelerator driver/firmware management at scale \n Background building or running shared development environments (devboxes, remote development, ephemeral environments) for hardware-dependent workflows \n Experience with CI/CD systems at scale, particularly for workloads involving accelerator hardware \n Familiarity with Kubernetes-based development and job scheduling environments \n Prior tech lead experience on a developer productivity or platform engineering team at a fast-growing AI/ML company\n The annual compensation","salary_min":405000,"salary_max":485000,"location":"San Francisco, CA","workplace":"hybrid","remote_scope":"not_remote","job_type":"full-time","experience_level":"lead","tags":["mlops","gpu","alignment","distributed-systems","cloud","infrastructure","inference"],"apply_url":"https://job-boards.greenhouse.io/anthropic/jobs/5257650008","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-06-12T20:17:38Z","expires_at":"2026-11-13T16:54:56.853092Z","created_at":"2026-06-28T14:00:35.550199Z","updated_at":"2026-10-14T16:54:56.856128Z","company_name":"Anthropic","company_slug":"anthropic","company_logo_url":"https://www.google.com/s2/favicons?domain=anthropic.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/da71ff33-42e4-48bf-87eb-04189fa35d9d"},{"id":"8dd71050-3131-45bd-a196-a71038f3f814","company_id":"b467c425-56b3-40ce-826a-e603e82a08bd","title":"Senior Product Manager, Inference Platform ","slug":"senior-product-manager-inference-platform-806705aa","description":"Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators.  \n At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there.  \n A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. \n About the Role \n Inference Platform is Roblox's multi-tier orchestration system for microservices, powering AI Inference across Roblox. Through a simple developer API, we deliver a \"deploy and forget\" runtime, hiding the complexity of scheduling, scaling, and reliably running services across our on-prem and multi-cloud footprint at global scale. As Senior Product Manager, Inference Platform, you'll take the helm at a defining moment - leading the charge as we scale the platform to become the default runtime for critical Roblox services worldwide.\n You Will \n \n Own Inference Platform end-to-end - set the multi-year vision for how Roblox engineers deploy, run, and scale AI and other services across our Core and Edge Datacenters, and cloud.          \n Power Roblox's AI future - build the platform that brings frontier models and next-gen AI workloads to life, with the primitives, scheduling guarantees, and resource classes AI teams need to move fast.                              \n Evolve the platform's technical core - define how Roblox prioritizes, isolates, and preempts workloads across shared fleet, and shape the entire system across global control plane, multi-region orchestration, distributed systems primitives, capacity/state management, DR/failover, and operator-based Kubernetes extensions.                                               \n Make developer experience the reason teams choose Jobs Platform - by delivering integrated tooling (CI/CD, telemetry, profiling, tracing) and managing robust SLOs via real-time fleet signals (health, queue depth, scheduling efficiency)—all while engineering a frictionless, effortless onboarding experience for teams.                                                        \n Architect intelligent, predictive scaling - advance Multi-Cloud Bursting from reactive to proactive with AI-powered prediction, and partner with Data Science and Finance on demand forecasting, cost-to-serve analytics, and AI-driven resource optimization.     \n Own the trade-off calls across efficiency, latency, cost-to-serve and time-to-ship, while maintaining reliability.\n Lead cross-functionally, acting as the connective tissue between Jobs Platform Users, and Jobs Platform, AI, and infrastructure engineers. \n \n You Have \n \n 7+ years of product management experience building resource orchestration, workload management platforms or large-scale distributed systems \n Track record of building developer-facing platforms that engineers genuinely love to use. You have a deep understanding of modern software development lifecycle and developer tooling with a history of abstracting complex infrastructure into clean, maintainable platform schemas.\n Built and operated Kubernetes and enterprise-grade service mesh at scale, and you understand the developer pain points around control plane mechanics like etcd scaling and API server bottlenecks. You've shipped custom Operators, Controllers, and CRDs, going beyond vanilla Kubernetes. \n Familiarity with GPU/accelerator architecture and the complex scheduling challenges that come with it.\n Background in AI model development, training, inference \n A builder mindset - you are passionate about prototyping, evolving products through rapid iteration, and leveraging AI for ideation and unlocking value for users.\n Experience building workload management or resource orchestration platforms in AWS, Google Cloud Platform (GCP), Azure or other cloud providers. (preferred)\n Kernel-level experience or familiarity with custom kernel drivers. (preferred)\n Experience building agentic systems for workload management or infrastructure. (preferred)\n For roles that are based at our headquarters in San Mateo, CA: The starting base pay for this position is as shown below. The actual base pay is dependent upon a variety of job-related factors such as professional background, training, work experience, location, business needs and market demand. Therefore, in some circumstances, the actual salary could fall outside of this expected range. This pay range is sub","salary_min":280540,"salary_max":330950,"location":"San Mateo, CA","workplace":"onsite","remote_scope":"not_remote","job_type":"full-time","experience_level":"senior","tags":["distributed-systems","cloud","agents","generative-ai","microservices","inference"],"apply_url":"https://careers.roblox.com/jobs/7997195?gh_jid=7997195","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-06-10T18:48:38Z","expires_at":"2026-11-13T17:02:53.259022Z","created_at":"2026-08-25T18:33:06.215553Z","updated_at":"2026-10-14T17:02:53.261882Z","company_name":"Roblox","company_slug":"roblox","company_logo_url":"https://www.google.com/s2/favicons?domain=roblox.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/8dd71050-3131-45bd-a196-a71038f3f814"},{"id":"ddcf527b-a153-46b0-a4f1-94832080a914","company_id":"a0000000-0000-0000-0000-000000000001","title":"Staff + Senior Software Engineer, Inference Infrastructure","slug":"staff-senior-software-engineer-inference-bb6014f6","description":"About Anthropic \n Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems.\n About the role\n Our Inference team is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide. We bring Claude to life by serving our models via the industry’s largest compute-agnostic inference deployments. We are responsible for the entire stack from intelligent request routing to fleet-wide orchestration across diverse AI accelerators.\n The team has a dual mandate: maximizing compute efficiency to reliably serve our explosive customer growth, while enabling breakthrough research by giving our scientists the high-performance inference infrastructure they need to develop next-generation models. We tackle complex, distributed systems challenges across multiple accelerator families and emerging AI hardware running in multiple cloud platforms.\n Inference systems are highly performance sensitive distributed systems. Inference serves hundreds of thousands of customers every day, and the size \u0026 span of the inference fleet requires sophisticated routing, scaling, and networking systems.\n Key responsibilities\n \n Design, build, and maintain the distributed systems that serve Claude to millions of users worldwide\n Develop resilient, flexible systems that adapt in real time to real world events\n Develop intelligent request routing, load balancing, and traffic management systems across thousands of accelerators\n Maximize compute efficiency across the fleet by autoscaling and orchestrating production, research, and experimental workloads\n Build and operate production-grade deployment pipelines for releasing new models to users\n Provide high-performance inference infrastructure that enables researchers to develop next-generation models\n Integrate new AI accelerator platforms and support inference for new model architectures\n \n Minimum qualifications\n \n Significant software engineering experience, particularly with distributed systems\n Results-oriented, with a bias towards flexibility and impact\n Willingness to pick up slack, even if it goes outside your job description\n Desire to learn more about machine learning systems and infrastructure\n Thrive in environments where technical excellence directly drives both business results and research breakthroughs\n Care about the societal impacts of your work\n \n Preferred qualifications\n \n Experience with high-performance, large-scale distributed systems\n Experience implementing and deploying machine learning systems at scale\n Experience with load balancing, request routing, or traffic management systems\n Familiarity with LLM inference optimization, batching, and caching strategies\n Experience with Kubernetes and cloud infrastructure (AWS, GCP, Azure)\n Proficiency in Python or Rust\n \n Representative projects\n \n Designing intelligent routing algorithms that optimize request distribution across many accelerators in different environments\n Autoscaling our compute fleet to dynamically match supply with demand across production, research, and experimental workloads\n Building production-grade deployment pipelines for releasing new models to millions of users reliably\n Contributing to new inference features\n Supporting inference for new model architectures\n Analyzing observability data to tune performance based on real-world production workloads\n Managing multi-region deployments and geographic routing for global customers\n \n Deadline to apply:  None. Applications will be reviewed on a rolling basis. \n The annual compensation range for this role is listed below. \n For sales roles, the range provided is the role’s On Target Earnings (\"OTE\") range, meaning that the range includes both the sales commissions/sales bonuses target and annual base salary for the role.\n Annual Salary:\n $320,000 — $485,000 USD \n Logistics \n Minimum education: Bachelor’s degree or an equivalent combination of education, training, and/or experience\n Required field of study:  A field relevant to the role as demonstrated through coursework, training, or professional experience\n Minimum years of experience: Years of experience required will correlate with the internal job level requirements for the position\n Location-based hybrid policy: Currently, we expect all staff to be in one of our offices at least 25% of the time. However, some roles may require more time in our offices.\n Visa sponsorship:  We do sponsor visas! However, we aren't able to successfully sponsor visas for every role and every candidate. But if we make you an offer, we will make every reasonable effort to get you a visa, and we retain an immigration lawyer to help with this.\n We encourage you to apply even if you do not believe you meet every sin","salary_min":320000,"salary_max":485000,"location":"San Francisco, CA","workplace":"hybrid","remote_scope":"not_remote","job_type":"full-time","experience_level":"lead","tags":["alignment","llm","cloud","distributed-systems","infrastructure","inference"],"apply_url":"https://job-boards.greenhouse.io/anthropic/jobs/5245851008","is_featured":false,"is_sticky":false,"status":"active","published_at":"2026-06-08T20:10:07Z","expires_at":"2026-11-13T16:54:56.655226Z","created_at":"2026-06-28T14:00:33.577053Z","updated_at":"2026-10-14T16:54:56.658005Z","company_name":"Anthropic","company_slug":"anthropic","company_logo_url":"https://www.google.com/s2/favicons?domain=anthropic.com\u0026sz=128","quality_score":90,"url":"https://aidevboard.com/job/ddcf527b-a153-46b0-a4f1-94832080a914"}],"page":1,"per_page":20,"total":151,"total_is_exact":true,"total_pages":8}