Site Reliability Engineer
full-time
senior
Posted 1 year ago
About this role
Runpod is the foundational platform for developers to build and run custom AI systems that scale. With over 500,000 developers worldwide and an annual recurring revenue run rate exceeding $120M, Runpod operates at the intersection of developer velocity and production-scale AI. Founded in 2022, we’ve grown rapidly by building infrastructure purpose-built for modern AI workloads. Our platform enables teams to move from experimentation to deployment with flexibility across cloud, on-prem, and hybrid environments. As a remote-first, globally distributed company, we are building the infrastructure layer that powers the next generation of AI systems.
The Reliability team owns the availability, performance, and operational excellence of Runpod’s global platform. While infrastructure teams build the systems, the Reliability team ensures those systems remain resilient, observable, and scalable under real-world production conditions.
This team is responsible for:
Defining and enforcing reliability standards across engineering
Designing incident response processes and improving recovery times
Building observability systems and reliability tooling
Driving SLO adoption and production readiness reviews
Reducing operational toil through automation
The Reliability team works cross-functionally with Infrastructure, Product Engineering, and Support to ensure our systems remain stable and performant as we scale rapidly. We value proactive problem solving, automation-first thinking, and strong ownership of production systems.
As a Site Reliability Engineer on the Reliability team, you will focus on ensuring the stability and resilience of Runpod’s distributed platform. You will partner with engineering teams to improve system design, strengthen observability, and prevent incidents before they happen.
This role blends software engineering with production operations. You’ll work on reliability frameworks, SLO design, automation, and production hardening, reducing errors and improving performance across different services and infrastructure.
This is a high-impact role central to maintaining trust with developers running critical AI workloads on Runpod.
Your Impact
Increase platform uptime and reduce incident frequency and duration
Establish and operationalize SLIs/SLOs across services
Improve MTTR through better tooling, automation, and runbooks
Strengthen production readiness standards
Drive long-term systemic reliability improvements
You will influence how reliability is defined and measured across Runpod and help build the operational backbone of the company.
Responsibilities:
Reliability Engineering
Define and implement SLIs/SLOs for critical services
Lead incident response and coordinate cross-team mitigation efforts
Conduct blameless postmortems and ensure corrective actions are completed
Perform production readiness reviews for new services and features
Identify systemic risks and drive preventative improvements
Observability & Monitoring
Design and improve monitoring, alerting, and dashboards (Prometheus, Grafana, etc.)
Improve signal-to-noise ratio in alerts and reduce alert fatigue
Build internal tooling for reliability tracking and reporting
Improve visibility into GPU performance and distributed systems health
Automation & Toil Reduction
Automate recurring operational workflows
Build tools and scripts (Python, Go, Bash) to eliminate manual processes
Improve deployment safety through automation and guardrails
Strengthen CI/CD reliability and release processes
Cross-Functional Reliability Advocacy
Partner with engineering teams to improve system resilience
Provide guidance on fault tolerance, scalability, and failure handling
Contribute to architectural discussions with a reliability-first mindset
Requirements:
5+ years of experience in SRE, Reliability Engineering, or Production Engineering
Strong Linux systems and Networking expertise
Experience managing containerized production systems
Strong understanding of distributed systems and failure modes
Experience defining and managing SLIs/SLOs
Proven incident response and postmortem leadership experience
Strong scripting or programming skills
Experience with monitoring and alerting systems
Excellent written communication skills
Successful completion of a background check
Preferred:
Experience with GPU infrastructure or AI/ML platforms
Experience improving reliability in high-growth or large scale environments
Familiarity with GPU observability tooling
Experience with Infrastructure as Code
Experience working in startup environments
Experience building internal reliability platforms or frameworks
What You’ll Receive:
The competitive base pay for this position ranges from $150,000- $200,000 usd. This salary range may be inclusive of several career levels at Runpod and will be narrowed during the interview process based on a number of factors, including the candida
Similar Jobs
Related searches:
Get jobs like this delivered weekly
Free AI jobs newsletter. No spam.