Data Engineer

Locations - Austin Highly referred, must be willing to relocate.
About the team
The Business Intelligence team at Cloudflare is responsible for building a centralized cloud data lake and an analytics platform that enables our internal Business Partners and Machine Learning teams with actionable insights and also provides a 360 view of our business. Our goal is to democratize data, support Cloudflare's critical business needs, provide reporting and analytics via self-service tools to fuel existing and new business critical initiatives.
About the role
We are looking for an experienced Data Engineer to join our Business Intelligence Data Science team. The role involves designing, building, and maintaining data pipelines and infrastructure to support data science and machine learning initiatives. You will work closely with data scientists and other stakeholders to ensure data accessibility and quality, enabling the creation and operation of machine learning models and analytics solutions. You will help support our data science team to build solutions using Large Language Models, AI services, and machine learning solutions following industry AI standards and best practices.
What you will do

• Collaborate with data scientists and business stakeholders to design datasets and engineer features for advanced analytical models.
• Develop, test, and manage robust and scalable data pipelines and systems.
• Collect, explore, validate, and prepare data for comprehensive analysis.
• Monitor, optimize, and support development, testing, and production environments to maintain efficiency.
• Ensure adherence to data governance and compliance standards and processes.
• Design application components and evolve architecture, including APIs, services, data access, and integration.
• Implement automation tools and frameworks, including CI/CD pipelines, to streamline processes.
• Develop tools to automate workload monitoring and take proactive measures to scale the platform or resolve issues.
• Mentor and guide junior data engineers, fostering a culture of continuous learning and development.

Desired skills, knowledge, and experience

• Bachelor's or Master's degree in Computer Science, Engineering, Information Technology or related field
• 3+ years of experience designing data solutions, modeling data, and developing ETL/ELT pipelines at large scale.
• 3+ years of experience in a programming language (e.g., Python, Java, Scala, Golang).
• Experience in developing large-scale data solutions using cloud platforms (e.g., AWS, Azure, Google Cloud) and big data technologies (e.g., Apache Spark).
• Experience in data cataloging, classification, data quality, metadata management, and data lifecycle.
• Experience with SQL and data visualization/analytics tools (e.g., Tableau, Looker Studio).
• Experience with CI/CD pipelines and source control.
• Experience with Infrastructure as Code tools like Terraform
• Proficiency in API design and development of RESTful web services or GraphQL.
• Working knowledge of containerization technologies like Kubernetes and Docker.
• Experience with machine learning techniques and tools is a plus