Principal ML Ops Engineer

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Principal ML Ops Engineer based in South Africa.

This is a highly technical leadership opportunity to build and scale the infrastructure powering next-generation AI applications.
The role focuses on designing reliable, efficient, and production-grade machine learning platforms capable of supporting large-scale AI workloads.
You will take ownership of model serving systems, GPU-powered infrastructure, deployment pipelines, and operational excellence.
Working closely with infrastructure, platform, and AI teams, you will help shape the foundations of a modern AI-native cloud ecosystem.
This position offers the chance to solve complex distributed systems challenges while improving performance, scalability, and cost efficiency.
You will play a key role in defining engineering standards and building critical ML infrastructure from the ground up.

• Design, build, and operate production-grade ML inference infrastructure using modern model serving frameworks such as vLLM, TGI, Triton, or equivalent solutions.
• Develop scalable deployment pipelines supporting reliable model releases through strategies such as blue/green deployments and canary rollouts.
• Build and maintain auto-scaling systems, multi-model serving architectures, and intelligent request routing mechanisms.
• Optimize GPU utilization, memory efficiency, network performance, and model artifact storage to improve system reliability and cost effectiveness.
• Implement observability solutions to monitor inference latency, throughput, GPU usage, operational health, and infrastructure costs.
• Manage model registries, CI/CD workflows, and automation processes to enable reproducible and efficient model deployments.
• Own the complete lifecycle of ML systems, from development and deployment through production operations and ongoing support.
• Establish engineering best practices and contribute to platform architecture decisions in a fast-moving, remote-first environment.
• Collaborate with infrastructure, platform, and applied AI teams to deliver scalable and reliable AI systems.

Requirements:

• 4+ years of experience in ML Ops, Platform Engineering, SRE, or similar infrastructure-focused roles supporting machine learning systems.
• Strong hands-on experience with production model serving frameworks such as vLLM, TGI, Triton, or comparable technologies.
• Proven experience operating GPU-based workloads and managing containerized environments in production.
• Strong understanding of MLOps practices, including model registries, experiment tracking, automated deployment pipelines, and lifecycle management.
• Proficiency in Python and infrastructure-as-code tools such as Terraform, Helm, or similar technologies.
• Solid knowledge of distributed systems, performance optimization, scalability, and reliability engineering principles.
• Experience using AI coding assistants to accelerate software development, troubleshooting, and debugging workflows.
• Ability to work independently with strong ownership and accountability in a remote-first environment.
• Experience with ML platforms such as Kubeflow, MLflow, or KubeAI is a plus.
• Knowledge of GPU scheduling, CUDA/ROCm optimization, multi-tenant inference systems, and infrastructure cost optimization is advantageous.
• Previous experience building greenfield infrastructure projects or working in early-stage technology environments is highly valued.

Benefits:

• Opportunity to own and shape critical ML infrastructure for a rapidly scaling AI-focused technology platform.
• Fully remote working environment with flexibility to work from Romania.
• Chance to build foundational systems from the ground up rather than maintaining legacy infrastructure.
• Exposure to cutting-edge technologies across distributed systems, GPU computing, and large-scale AI model serving.
• High level of ownership and influence over technical decisions and engineering practices.
• Opportunity to work with experienced professionals solving complex AI infrastructure challenges.
• Dynamic startup environment with strong growth opportunities and meaningful technical impact.