Lead Software Engineer (DSE)

As a Lead Data Engineer, you will play a leading role in designing, building, and optimizing our data infrastructure, ensuring that it supports the advanced analytics need of the bank. You will oversee a team of data engineers, working closely with data analysts, DevOps team, infrastructure engineers, and other stakeholders to deliver high-quality data solution.

#LI-DNI

Responsibilities

• Lead the design, development and implementation of high-performance Spark Scala applications and data processing pipelines for large-scale data
• Integrate Elasticsearch with Spark for efficient data indexing, querying and retrieval
• Optimize Spark jobs for performance and scalability on OpenShift Container Platform (OCP)
• Collaborate with cross-functional teams to translate data requirements into robust and scalable solutions
• Implement and maintain CI/CD pipelines for data engineering workflows, ensuring platform stability and efficient resource utilization
• Monitor and optimize data pipeline performance, troubleshoot issues and implement necessary enhancements
• Establish and maintain data quality and integrity throughout the data processing lifecycle
• Document data engineering processes, workflows and infrastructure configurations
• Stay abreast of emerging technologies and best practices in data engineering and DevOps
• Mentor and guide junior team members, fostering a culture of continuous learning and innovation

Requirements

• Bachelor's degree in Computer Science, Data Engineering, Information Technology or a related field
• 10+ years of experience as a Data Engineer with expertise in Hadoop, Spark and related technologies (HDFS, Hive, Ranger)
• Strong proficiency in Spark, Scala, and Python for data engineering tasks
• Experience with containerization (Docker, Kubernetes) and orchestration platforms like OpenShift
• Knowledge of DevOps practices, CI/CD pipelines and automation tools (Jenkins, Ansible, BitBucket)
• Experience with monitoring and logging tools (Grafana, Prometheus, Splunk) is a plus
• Strong problem-solving, troubleshooting, and communication skills
• Ability to work effectively in a fast-paced, collaborative environment
• Experience with cloud platforms (AWS, Azure, GCP) is a plus

We offer

• By choosing EPAM, you're getting a job at one of the most loved workplaces according to Newsweek 2021 & 2022&2023
• Employee ideas are the main driver of our business. We have a very supportive environment where your voice matters
• You will be challenged while working side-by-side with the best talent globally. We work with top-notch technologies, constantly seeking new industry trends and best practices
• We offer a transparent career path and an individual roadmap to engineer your future & accelerate your journey
• At EPAM, you can find vast opportunities for self-development: online courses and libraries, mentoring programs, partial grants of certification, and experience exchange with colleagues around the world. You will learn, contribute, and grow with us

Life at EPAM
- EPAM is a leading global provider of digital platform engineering and development services. EPAM has been expanding in Singapore since 2013 and delivering the best solutions to our customers. As a recognized leader, EPAM Singapore achieved Great Place to Work® Certification in 2023 and is committed to providing our team with inspiring careers.
- You will have the opportunity to work with fellow talented technologists and accelerate your career by participating in our numerous upskilling, training, and certification programs. That is why EPAM Singapore was awarded Gold for Best In-House Certification Programmes in the Employee Experience Awards 2023 and Silver in the SkillsFuture Employers Awards 2022 for our efforts in championing employees' skills development and building a lifelong learning culture at the workplace. You can also look forward to developing holistically with the multiracial festivals and various wellness and cultural activities organized by our passionate colleagues here.