Senior Data Engineer (Scala)

Company Description

Are you passionate about data and eager to make a significant impact in the ad tech industry? We're seeking a Senior Data Engineer to be a key player in our dynamic team. Our focus is on transforming and optimizing high-velocity ad log streams, creating a unified data lake for seamless downstream data consumption. 

Our client processes billions of ad events daily, providing both real-time and batch data to deliver comprehensive reporting and troubleshooting solutions. As a Senior Data Engineer, you'll be at the forefront of developing, deploying, and maintaining robust data pipelines and infrastructure. You'll work closely with cross-functional teams to design, implement, and refine data solutions that enhance the efficiency of data processing, storage, and analysis. 

If you're excited about working with cutting-edge technology in a fast-paced environment, we’d love to have you on board. Don’t hesitate—join our stellar team today! 
CUSTOMER
Our client has superior end-to-end technology, a premium marketplace, and best-in-market advisory services that power the world’s advertising businesses of the largest media and entertainment companies. For example, Fox, NBC Universal, Viacom in the USA, Sky, Channel 4, RTE, and Mediaset in Europe. 

Job Description

• Design, implement, and maintain scalable data pipelines to ingest and transform large volumes of data from various sources 
• Optimize streaming and batch data processing and storage solutions for performance and scalability 
• Monitor system performance, troubleshoot issues, and implement solutions to ensure high availability and reliability 
• Collaborate with product managers and other stakeholders to understand data requirements and deliver solutions that meet business needs 
• Communicate technical concepts and solutions effectively to non-technical stakeholders 
• Stay abreast of emerging technologies and best practices in data engineering. Identify opportunities to improve data processes, tools, and infrastructure to enhance efficiency and effectiveness 
• Document data pipelines, processes, and systems to ensure clarity and maintainability 
• Share knowledge and best practices with team members to foster a culture of learning and collaboration 
• Work with and mentor junior Data Engineers 
• Other duties and responsibilities as assigned 

Qualifications

• 5+ years of hands-on experience in the Software Development field and/or Big Data 
• 2+ years of hands-on experience in building and operating large-scale data processing system 
• Solid programming skills, fluent with Scala 
• Familiar with big data processing Framework/tools Spark, Spark Streaming, Databricks, Flink 
• Experiences with Data Lakehouse technics Delta Lake and Iceberg 
• Solid SQL skills with experience in writing complex queries, stored procedures, and optimizing querying performance 
• Experiences with data warehouse, data modeling techniques, ETL processes, and relational databases (MySQL, PostgreSQL) 
• Familiarity with AWS (EC2, S3, etc.), and proficiency in managing cloud-based data solutions (Snowflake) 
• Upper-Intermediate level of English

Additional Information

PERSONAL PROFILE

• Excellent problem-solving skills and attention to detail 
• Strong communication skills, with the ability to collaborate effectively in a team environment 
• Proven ability to deliver results in a fast-paced environment