Lead Data Engineer

Lead Data Engineer

Important Information

Location: Kuala Lumpur Malaysia

Experience: 8-10 years

Job Mode: Full-time Contract

Work Mode: On-Site

Responsibilities and Duties

Data pipeline development and operationalization

• Design data pipelines to ingest data from multiple sources and implement quality processes
• Design processes to create data insight, data features and operationalize analytics models
• Develop and implement those data pipelines
• Operationalize data pipelines for automated and semi-automated execution
• Implement reporting of data universe, data quality across the organization

Data platform innovation

• Own the Data Platform, suggest, specify, and implement enhancements to the platform
• Suggest and implement technology components relevant to the data platform
• Support key initiatives including migration of CTOS data services to the cloud

Qualifications and Skills

• Hands on working experience on Data Strong knowledge of data platforms, data pipelines, data related development including platforms such as Hadoop, Spark, Jupyter, cloud native services
• Hands on experience on cloud services and knowledge of data related cloud native services (AWS)
• tools and programming languages required to implement data pipelines: SQL, Python / Pandas
• tools required to implement automation and monitoring: Shell scripts, scheduling tools such as AirFlow
• ETL tools such as Talend and/or cloud native services
• Experience implementing reporting using a data platform or reporting software

About Encora

Encora is the preferred digital engineering and modernization partner of some of the world’s leading enterprises and digital native companies. With over 9,000 experts in 47+ offices and innovation labs worldwide, Encora’s technology practices include Product Engineering & Development, Cloud Services, Quality Engineering, DevSecOps, Data & Analytics, Digital Experience, Cybersecurity, and AI & LLM Engineering.

At Encora, we hire professionals based solely on their skills and qualifications, and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.