Staff Data Engineer

Job Summary:

We are seeking a highly experienced Staff Data Engineer to lead the design development and optimisation of our data architecture pipelines and workflows. This role will serve as a technical lead within the organisation setting best practices mentoring team members and solving complex data challenges to enable data-driven decision-making at scale.

As a Staff Data Engineer you will collaborate with cross-functional teams including data scientists analysts and software engineers to design systems that transform raw data into actionable insights while ensuring scalability security and reliability.

Key Responsibilities:

Technical Leadership

• Design and implement scalable and reliable data pipelines ensuring the processing of large volumes of structured and unstructured data.

• Define and enforce data engineering best practices coding standards and architectural principles across teams.

• Conduct code reviews and provide mentorship to junior and senior data engineers.

Data Pipeline Development

• Build and maintain batch and real-time data pipelines using tools such as Apache Spark Kinesis and AWS services.

• Works with multiple teams to coordinate the event-driven architecture managing inter-dependencies and promoting consistency.

• Ensure data quality governance and security by implementing monitoring validation and compliance tools.

Collaboration & Cross-Functional Engagement

• Partner with product analytics and data science teams to understand business requirements and translate them into technical solutions.

• Work closely with DevOps and software engineering teams to deploy and maintain production-ready data infrastructure.

Innovation & Scalability

• Evaluate and recommend emerging technologies and frameworks to ensure the data platform remains future-proof.

• Drive initiatives to improve the performance scalability and efficiency of existing systems.

Required Skills & Experience

• 12+ years of experience in data engineering field with at least 2 years in a senior or staff-level role.

• Expertise in designing and implementing scalable data architectures for big data platforms.

• Strong programming skills in Python Scala.

• Deep experience with distributed data processing systems such as Apache Spark Databricks Delta Lake.

• Proficiency with relational databases (e.g. PostgreSQL MySQL) and NoSQL databases (Dynamo).

• Strong understanding of ETL/ELT workflows data warehousing concepts and modern data lake architectures.

• Employ the established Data Governance model to sustain Data Quality for the data objects and implement the necessary operating mechanisms to ensure compliance

• Knowledge of CI/CD practices.

• Excellent problem-solving skills and the ability to design creative efficient solutions for complex data challenges.

• Background in AI machine learning pipelines is a plus

• Proactive self-driven and detail-oriented with a strong sense of ownership.