Sr. Data Engineer

Job Description

DUTIES:

Build, enhance and support data ingestion pipelines, delta lakes and data warehouse across a variety of infrastructure types, both on premises and cloud.

Collaborate with infrastructure and IT team to design, create and maintain optimal data pipeline architecture and data structures for Enterprise Data Platform.

Assemble and consolidate complex data sets from business and reporting systems such as SAP, Salesforce, E1, Genesys, Cloud for Service, Business Objects, and Cognos/EDW.

Automate manual data acquisition processes and optimize data delivery.

Perform data integration in Apache Spark-based platform to ensure the technology solutions leverage cutting edge integration capabilities.

Develop customized SQL queries for database solutions & business ad hoc requests.

Ensure data integration and data integrity to improve data fidelity for real time analytics.

Navigate and work independently in a large enterprise and distributed set up.

REQUIREMENTS: Bachelor's degree in Computer Science, Electronic Engineering, or a related field of study plus 5 years of experience in data engineering, or a related field. Employer will also accept a Master's degree in Computer Science, Electronic Engineering, or a related field of study plus 3 years of experience in data engineering, or a related field.

Must have experience or knowledge of:

Data Integration;

Databricks;

Delta Lake;

AWS Cloud;

Kafka;

Shell Scripting;

Informatica power center or IICS;

Power BI or Cognos;

Power Automate or Airflow;

SQL;

Python and PySpark; and

Apache Spark.

Travel: Up to 25% domestic travel required. Can work remotely or telecommute.