Senior Cloud Infrastructure Engineer - Observability (FedRAMP)

Join us as we pursue our ground-breaking vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people who are passionate about our product and seek to deliver the best experience for our customers. At Splunk, we are committed to our work, customers, having fun, and most significantly to each other's success.

The Splunk Observability Cloud provides full-fidelity monitoring and fixing across infrastructure, applications, and user interfaces, in real-time and at any scale, to help our customers keep their services reliable, innovate faster, and deliver great customer experiences. Infrastructure Software Engineers at Splunk are cloud-native systems engineers who use infrastructure-as-code, microservices, automation, and efficient design to build, operate, and scale our products.

Role

You will help us run one of the largest and most sophisticated cloud-scale, bigdata, and microservices platforms in the world. You will be responsible to maintain and provision the base infrastructure that runs Splunk Observability Cloud, including cloud compute platforms, managed services, Kubernetes, and required tooling. You will be a key part of the team that strives to offer a fully automated and self-serve, secure, performant, compliant, and cost-efficient Platform-as-a-Service that follows cloud-native best practices and empowers product teams to easily and quickly deploy and operate workloads. You are passionate about automation, infrastructure-as-code, microservices, and getting rid of tedious, manual tasks.
You will:

• Design new services, tools, and monitoring to be implemented by the entire team.
• Analyze the tradeoffs of the proposed design and make recommendations based on these tradeoffs.
• Mentor new engineers to achieve more than they thought possible. You enjoy making other teams successful and are fulfilled through the success of others.

You will work on infrastructure projects, including:

• Adopting new cloud-native frameworks and services
• Automating cloud provider infrastructure via Terraform, Kubernetes, and Helm
• Developing code for tools and automation to reduce manual tasks and reduce human error
• Establishing and documenting run books and guidelines for using the multi-cloud infrastructure and microservices platform.
• Improve the resiliency of multi-cloud microservices platform
• Networking, routing, and load balancing
• Security vulnerability remediation and patching automation
• Automating deployment of our services in new provider zones/regions
• Designing and productionizing access tiers to provide appropriate permissions across roles

Qualifications
Must-Have:

• Bachelors/Masters in Computer Science, Engineering, or related technical field, or equivalent practical experience.
• 7+ years of solid hands-on cloud infrastructure experience on public cloud platforms specifically AWS or GCP.
• 3+ years of strong hands-on experience deploying, managing, and monitoring large-scale Kubernetes clusters in the public cloud
• Experience with Infrastructure-as-Code using Terraform and/or Helm.
• Experience with infrastructure automation and scripting using Python and/or bash scripting.
• Knowledge of microservices fundamentals including Service Mesh using Istio, service discovery, deployment strategies, monitoring, scheduling, and load balancing
• Excellent problem-solving, triaging, and debugging skills in large-scale distributed systems
• Candidate must be a US citizen or a permanent resident and must reside on US soil

Preferred:

• AWS Solutions Architect certification preferred.
• CKA and HashiCorp-certified Terraform Associate certifications preferred
• Experience with Infrastructure-as-Code using Terraform, CloudFormation, Google Deployment Manager, Packer, Pulumi, ARM, etc.
• Experience with developing infrastructure or platform services using GoLang or Python.
• Experience with CI/CD frameworks and Pipeline-as-Code such as Jenkins, Spinnaker, Gitlab, Argo, Artifactory, etc.
• Exposure to monitoring tools such as Splunk, Prometheus, Grafana, ELK stack, etc. in order to build observability for large-scale microservice deployments.
• Proven skills to effectively work across teams and functions to influence the design, operations, and deployment of highly available software.
• Experience handling cloud infrastructure and operations in strict security, compliance, and regulatory environments such as FedRAMP.

We value diversity, equity, and inclusion at Splunk and are an equal employment opportunity employer. Qualified applicants receive consideration for employment without regard to race, religion, color, national origin, ancestry, sex, gender, gender identity, gender expression, sexual orientation, marital status, age, physical or mental disability or medical condition, genetic information, veteran status, or any other consideration made unlawful by federal, state, or local laws. We consider qualified applicants with criminal histories, consistent with legal requirements.