Lead Data Software Engineer

We are seeking a Lead Data Software Engineer to take ownership of designing and implementing advanced data engineering solutions.
Our client specializes in delivering cutting-edge property insights and data analytics, enabling clients in real estate, financial services and insurance to make informed decisions backed by comprehensive data. This role requires both hands-on expertise and the ability to guide a team of engineers. You will work in a dynamic and challenging environment, collaborating with a transitioning client team to ensure seamless knowledge transfer and successful project delivery.
The ideal candidate will have extensive experience with Python, Spark and cloud platforms and strong leadership and problem-solving skills.

Want more jobs like this?

Get jobs in Kuala Lipis, Malaysia delivered to your inbox every week.

Get Jobs

Send me The Muse newsletters for the best in career advice and job search tips.

By signing up, you agree to our Terms of Service & Privacy Policy.

• Lead the design and development of scalable data pipelines and architectures using Python, Spark, Airflow and Kafka
• Architect and implement robust data storage and processing solutions leveraging AWS (DynamoDB, Glue, S3, Lambda, Fargate) or GCP and Snowflake
• Provide technical leadership and mentorship to a team of engineers
• Drive knowledge transfer from the client's onshore team and ensure continuity in project delivery
• Collaborate with stakeholders to define technical requirements and deliver effective data solutions
• Oversee the implementation of containerized solutions using Docker, with a focus on scalability and reliability
• Ensure adherence to best practices in code quality, performance, and security
• Work within an Agile framework (e.g., Scrum, Kanban) and lead the team in adopting DevOps practices such as CI/CD pipelines and automated testing
• Use GitFlow or similar version control workflows for development and deployment

• Extensive experience with Python, Spark and SQL in large-scale data environments
• Proven ability to lead technical teams and deliver complex projects
• Deep understanding of cloud-based ecosystems and distributed data systems
• Strong interpersonal and stakeholder management skills to address challenges in a transitioning client team
• Exposure to advanced technologies, such as serverless architecture and data lake design
• Highlight time zone flexibility to collaborate effectively with an Australian client
• Understanding and proficiency in general big data concepts

• By choosing EPAM, you're getting a job at one of the most loved workplaces according to Newsweek 2021 & 2022&2023
• Employee ideas are the main driver of our business. We have a very supportive environment where your voice matters
• You will be challenged while working side-by-side with the best talent globally. We work with top-notch technologies, constantly seeking new industry trends and best practices
• We offer a transparent career path and an individual roadmap to engineer your future & accelerate your journey
• At EPAM, you can find vast opportunities for self-development: online courses and libraries, mentoring programs, partial grants of certification, and experience exchange with colleagues around the world. You will learn, contribute, and grow with us

• EPAM is a leader in the fastest-growing segment (product development/digital platform engineering) of the IT industry. We acquired Just-BI in 2021 to reinforce our leading position as a global Business Intelligence services provider and have been growing rapidly. With a talented multinational team, we provide data and analytics expertise
• We are currently involved in end-to-end BI design and implementation projects in major national and international companies. We are proud of our entrepreneurial start-up culture and are focused on investing in people by creating continuous learning and development opportunities for our employees who deliver engineering excellence for our clients