Staff Engineer - Site Reliability

Reliability is the most important feature of Hudl.com. If our site isn’t available and performant, then nothing else matters. We have a strong DevOps culture at Hudl, our development teams directly support, fix, and scale our production systems and we’re now looking for a Lead Engineer, Site Reliability Specialist to improve this culture and enhance the tooling we use to maintain sitewide reliability. You’re an experienced engineer, with a passion for observability and the autonomy to drive measurable improvements in Hudl’s availability, production operations and system architecture. In this role you will be part of the Platform Engineering team but will work really closely with the wider product team, to successfully build adoption for new observability technologies, process and system architecture. #LI-Remote

• Knowledgeable in monitoring and observability technology. We’re investing in OpenTelemetry and tools that support it. 
• Experienced with public clouds, preferably AWS. You understand the state of operating in the cloud and know what every company should be doing in them. You are excited by cloud services and the reliability and scaling benefits of adopting managed services. 
• Believe in the “DevOps” philosophy. You want to empower feature teams to own their services, from provisioning to operation and incident resolution.
• A mentor at heart. You find mentoring others rewarding and seek out opportunities to level up others in your area of expertise.
• Dedicated to delivering quality work. You want your team to get things done and done well and are willing to wear many hats to make this happen.
• Data driven. You understand that you can’t solve everything at once and like to target your investments using a data driven approach. 
• Autonomous and self led. You know how to seek out valuable initiatives and can independently plan a roadmap to deliver on these. 

• Define, document, and drive adoption for the processes and tools used to improve production alerting and incident response sitewide at Hudl. 
• Understand, evangelize, and help implement the best strategies and tools for faster discovery and resolution of production incidents.
• Help Hudl define and measure reliability metrics, such as MTTD, MTTR and availability. You’ll help teams become more accountable for individual microservice metrics.
• Collaborate and embed with teams to eliminate architectural weaknesses or anti-patterns across our systems.
• Take on-call shifts a few times a year. All engineers at Hudl participate in ops duty, not just infrastructure engineers. Expect about one 24-hour on-call shift out of business hours per quarter.