Sr. Data Engineer

Iron Systems is an innovative, customer-focused provider of custom-built computing infrastructure platforms such as network servers, storage, OEM/ODM appliances & embedded systems. For more than 15 years, customer have trusted us for our innovative problem solving combined with holistic design, engineering, manufacturing, logistic and global support services.

Job Title: Sr. Data Engineer

Location: Remote

Summary:

• We are seeking a highly experienced Senior Data Engineer with a deep understanding of PySpark using Databricks or AWS Glue or AWS EMR and cloud-based databases such as Snowflake. Proficiency in workflow management tools like Airflow a is essential.
• Healthcare industry experience is a significant advantage.
• The ideal candidate will be responsible for designing, implementing, and maintaining data pipelines, while ensuring the highest levels of performance, security, and data quality.

Responsibilities:

• Design, develop, and maintain scalable, reliable, and secure data pipelines to process large volumes of structured and unstructured healthcare data using PySpark and cloud-based databases.
• Collaborate with data architects, data scientists, and analysts to understand data requirements and implement solutions that meet business and technical objectives.
• Leverage AWS or Azure cloud services for data storage, processing, and analytics, optimizing cost and performance.
• Utilize tools like Airflow for workflow management and Kubernetes for container orchestration to ensure seamless deployment, scaling, and management of data processing applications.
• Develop and implement data ingestion, transformation, and validation processes to ensure data quality, consistency, and reliability across various healthcare datasets.
• Monitor and troubleshoot data pipelines, proactively identifying and resolving issues to minimize downtime and ensure optimal performance.
• Establish and enforce data engineering best practices, ensuring compliance with data privacy and security regulations specific to the healthcare industry.
• Continuously evaluate and adopt new tools, technologies, and frameworks to improve the data infrastructure and drive innovation.
• Mentor and guide junior data engineers, fostering a culture of collaboration, learning, and growth within the team.
• Collaborate with cross-functional teams to align data engineering efforts with broader organizational goals and strategies.
• Is familiar with SOC 2 compliance and its impact on company policies and processes.
• Understands importance of adhering to SOC 2 requirements and maintains an effort to do so.
• Reviews and understands the Employee Handbook, and internal policies that define individual security responsibilities, and maintains segregation of duties in accordance to their role requirements

Requirements:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• 12+ years of experience in data engineering, with a strong background in Apache Spark and cloud-based databases such as Snowflake.
• Strong Knowledge in Big Data Technologies, PySpark, thorough in one or more programming language like Python
• Proven experience with AWS or Azure cloud services for data storage, processing, and analytics.
• Expertise in workflow management tools like Airflow and container orchestration systems such as Kubernetes.
• Strong knowledge of SQL and NoSQL databases, as well as data modeling and schema design principles.
• Familiarity with healthcare data standards, terminologies, and regulations, such as HIPAA and GDPR, is highly desirable.
• Excellent problem-solving, communication, and collaboration skills, with the ability to work effectively in cross-functional teams.
• Demonstrated ability to manage multiple projects, prioritize tasks, and meet deadlines in a fast-paced environment.
• A strong desire to learn, adapt, and contribute to a rapidly evolving data landscape.