Data Integration Engineer

Job Summary 

As a Data Integration Manager, you’ll be responsible for automating data ingestion for production information systems. The ability to automate the ingestion of member, claims and HIE data is essential. In this role, you own the data ingestion processes from end to end.

 Responsibilities 

Work closely with product and engineering to develop strategy for data ingestion architecture, tools and processes

Design, implement, test and monitor automated system for ingesting healthcare member, claims and quality data

Automate read, extract, transform, stage and load data processes using state of the art cloud-based tools

Develop and build custom software and integrations for EMR interoperability with Hospitals and PCP offices

Develop, construct, test and maintain data management systems using appropriate tools to perform data ingestion, transformation, cleansing and loading

Integrate up-and-coming data management and software engineering technologies into existing data structures.

Implement processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it.

Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues.

Gather Data - Collected data from HIE, EMR, Flat files or other structured and unstructured sources.

Clean Data - Once the data has been collected, in order to allow for proper comparison, the data engineer must filter and "clean" the data to remove errors and ensure consistency and completeness of data. Cleansing routines should be used to identify like matches where data values may have been keyed incorrectly, may have partially matching strings or values may be missing. The Data Integration Engineer will need ETL tools as well as programming skills to write cleansing scripts to correct errors through automation.

Structure Data – Once the data is extracted and cleaned it must be organized and stored in database structures to optimize performance and maintain history.

Project Management - Works with the project team to define tasks and create work plans with moderate supervision. Identify issues affecting work progress and recommends solutions. Communicate variances and potential scope changes in status reports. Control project costs, communicating any project-related expenses and recommend ways to control costs. 

Educational and Knowledge Requirements

• Bachelor's degree in Mathematics, Computer Science, Information Management or similar required 
• 5+ years of experience as a data integration engineer or in a similar role
• 5+ years of MySQL experience (No-SQL experience is a plus)
• 5+ years of programming/scripting experience (Python preferred)
• 5+ years of experience designing, building, and maintaining data processing systems in a cloud architecture (AWS Glue / PySpark preferred)
• Experience with data integration tools (Talend and MuleSoft )
• Experience with cloud based data warehouses (Redshift and Snowflake)
• Experience with processing healthcare data