Infrastructure Engineer

The infrastructure team is the center of our business. We build and maintain the hardware and software that all our customers run on. The team has a broad mandate, allowing people with different skill sets to contribute while providing plenty of room to grow and learn.

• Kubernetes underpins our stack. Both containerized workloads and Virtual Machines are orchestrated by Kubernetes. Our clusters have several thousand bare-metal nodes in them. The team spends a lot of time on scaling, optimizing and troubleshooting the Kubernetes control plane.
• Automation. We strictly follow GitOps, and heavily leverage Helm and Argo CD to manage all components of our stack. Images for bare metal machines and VMs are built with Packer.
• Open Source contributions. As we face unique challenges due to our scale, we often make tweaks or fix bugs in the Open Source software we utilize. As a principle, we PR all changes upstream that will benefit the broader community. Our engineers contribute to projects like Calico, Kubevirt and Argo Workflows.
• Tooling. We built a lot of custom controllers to wrangle our large set of hardware.
• Observability. We leverage Prometheus and Loki for early problem detection. The infrastructure team maintains the observability stack, as well as the alerts and metrics used to detect problems and track regressions.
• Solution design. The CoreWeave infrastructure is unique in the breadth of hardware options we offer to clients, as well as it’s scale. The infrastructure team assists in solution-design for large scale clients to make sure their applications play seamlessly on our platform.

We're looking for engineers with experience in building infrastructure at scale. We expect a candidate to have production experience in some but not all of the following areas.

• Kubernetes "the hard way" - We run fully managed control planes for our customers. Kubernetes clusters contain up to 1,000 nodes at a time. Knowledge of Kubernetes internals such as the scheduler, modifying scheduler policies, pod security policies, admission controllers are areas a candidate should have experience in.
• Continuous Deployment via tools such as Ansible and Argo CD
• Linux - Bare metal, Ubuntu based images booted over iSCSI and PXE. Understanding of CPU sets and NUMA topologies.
• Go Development - We are contributing to open source projects such as Kubeflow, Argo and write our own admission controllers, CLIs and scheduler plugins.