Software Engineer, Conduit Flume Pipelines

Minimum qualifications:

• Bachelor's degree or equivalent practical experience.
• 2 years of experience in software development in one or more general purpose programming languages.
• Experience programming in one or more of the following languages: Ruby, Python, Java, C++.

Preferred qualifications:

• Experience with Engineering Tools and Infrastructure.
• Experience in running and managing Flume pipelines.

About the job

The Core team builds the technical foundation behind Google's flagship products. We are owners and advocates for the underlying design elements, developer platforms, product components, and infrastructure at Google. These are the essential building blocks for excellent, safe, and coherent experiences for our users and drive the pace of innovation for every developer. We look across Google's products to build central solutions, break down technical barriers and strengthen existing systems. As the Core team, we have a mandate and a unique opportunity to impact important technical decisions across the company.

Responsibilities

• Build components of a managed system that works on deployment, scheduling, launching, monitoring, and debugging of Flume jobs.
• Deliver new features to expand the set of use cases that Conduit Flume Pipelinescan serve.
• Build core orchestration libraries in C++, define and extend public APIs in C++, Python, Java and Go.
• Work with the underlying infra teams (Conduit, Flume, Data Engineering Console, DPPE, Table Service) to integrate with their APIs, define requirements for new features, and make direct code contributions to their codebases.
• Make smaller contributions to Dreampipe to continue supporting the most important new use cases.