Senior Integration Engineer - Python - SSIS - Data Bricks

SoHo Dragon represents an investment bank with offices in New York, NY that needs to hire a Senior Integration Engineer - Python - SSIS - Data Bricks.

Work schedule - Hybrid - 3 days onsite - Mon, Wed, Thurs or Mon, Tues, Thurs.

Job Description:

We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. As a Senior Data Engineer, you will play a crucial role in designing, implementing, and maintaining data pipelines and infrastructure for our big data projects. Your expertise in Java, Python, Spark cluster management, data science, big data, REST API development, and knowledge of Databricks and Delta Lake will be essential in driving the success of our data initiatives.

Responsibilities:

- Design, develop, and implement scalable data pipelines and ETL processes using Java, Python, and Spark.

- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and design efficient solutions.

- Manage and optimize Spark clusters to ensure high performance and reliability.