We are currently hiring a Data Engineer with Spark to join our IT Digital Marketing Solutions team to develop dynamic software solutions.
In this role you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers) where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology.
In this role you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers) where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology.
Your responsibilities will include:
* Writing effective and scalable data processing code using Spark with Scala / Java
* Designing and implementing robust solutions
* Debugging applications to ensure low-latency and high-availability (Docker / Kubernetes)
* Interfacing with developed API and building Rest API
* Implementing and refactoring ETL processes (Airflow)
* Implementing security and data protection (HVault)
* Accommodating various data storage solutions (COS API Cassandra)
* Proficiency in data processing with Apache Spark (Scala / Java)
* Work experience with Docker / Kubernetes
* Experience in implementing and refactoring ETL processes (Arflow / Python)
* Experience in implementing security and data protection (HVault)
* Previous work experience with: SQL COS API Cassandra
Nice to have: Kafka GitLab Jenkins