Staff Engineer, Big Data

Company Description

We are a Digital Product Engineering company that is scaling in a big way! We build products, services, and experiences that inspire, excite, and delight. We work at scale — across all devices and digital mediums, and our people exist everywhere in the world (15000+ experts across 26 countries, to be exact). Our work culture is dynamic and non-hierarchical. We are looking for great new colleagues. That is where you come in!

Job Description

Must have Skills: Apache Spark (Strong), Azure Data Factory, Azure Table storage,

Good To Have Skills: Databricks,

Job Summary:

We are seeking a skilled Data Engineer with a minimum of 8+ years experience. The ideal candidate will have a strong background in batch data processing, developing and maintaining scalable ETL/ELT pipelines, and working with cloud platforms like Azure.

This role focuses on building efficient data solutions that ensure high performance and data quality.

Key Responsibilities:

• Design, develop, and maintain scalable batch ETL/ELT pipelines using Azure Data Factory and other Azure services.
• Collaborate with data architects and cross-functional teams to translate business requirements into technical solutions.
• Ensure data processing pipelines are optimized for performance, reliability, and cost-effectiveness.
• Perform data integration, cleansing, and validation to ensure high data quality.
• Monitor and troubleshoot batch processing jobs to ensure timely and accurate data delivery.
• Ensure compliance with data governance and security standards.

Qualifications:

• Minimum of 8+ years of experience in Data Engineering, focusing on batch data processing.
• Hands-on experience with Azure data services (Azure Data Factory, Data Lake, etc.).
• Strong understanding of ETL processes, data integration, and data modeling.
• Experience with data storage solutions such as Azure Data Lake and SQL databases.
• Strong problem-solving and communication skills.