Data Engineer

Introduction
At IBM work is more than a job – it’s a calling: To build. To design. To code. To consult. To think along with clients and sell. To make markets. To invent. To collaborate. Not just to do something better but to attempt things you’ve never thought possible. Are you ready to lead in this new era of technology and solve some of the world’s most challenging problems? If so lets talk.

Your Role and Responsibilities
As a Data Engineer in IBM’s CIO organization you will be supporting a data warehouse that consolidates data for entitlements usage and customer health across many different IBM Offerings into one trusted source. This data is used to help drive our Customer Success applications and strategy.

Responsibilities:
As a Data Engineer you will:

• Manage integrations for data ingestion from multiple source systems via APIs queries Apache Spark and Airflow workflows and Kafka streaming.
• Support data transformations and aggregations in a Cloud Object Storage (COS) integration zone and subsequent data feeds to a DB2 warehouse.
• Assist in working incoming support tickets for the application and provide necessary feedback/resolutions.
• Contribute to innovation and development of new ETL and Data Transfer processes through Spark/Airflow APIs SQL etc while protecting data quality and integrity during movement.
• Perform comprehensive testing of individual components as well as end-to-end solution.

Required Technical and Professional Expertise

• Bachelor’s degree or higher in Computer Science Engineering or related field or Master’s degree preferred.
• 5+ years of experience with managing data warehouses and integration solutions
• API integration experience
• Apache Spark and Airflow experience
• SQL development relational database table structure and database design
• Expertise in working with structured and unstructured data
• Excellent communication skills (written and verbal). Able to clearly communicate with leadership and colleagues about technical capabilities limitations issues and recommendations.
• Highly organized detail oriented independent and resourceful.
• Able to manage complex technical projects with diverse global stakeholders and detailed interdependent requirements
• Experience with Agile practices and associated tools including Jira

Preferred Technical and Professional Expertise

• None