Data Engineer

As a Data Engineer at Gore Mutual Insurance, you have a strong technical background in software engineering / computer science and will be responsible for building, and maintaining our data platform. Your work will facilitate data accessibility, accuracy, and accountability, enabling data-driven decision-making across our organization. You will collaborate closely with cross-functional teams to ensure our data processes are robust and scalable.

What will you be doing in this role?

Developing Systems for Collecting and Storing Data

• Build data pipelines and technical components

Automating Data Management Processes and Handling Data Security

• Automated data management processes.
• Follow security protocols and best practices to protect against potential security threats.

Testing and Optimizing Data Pipelines

• Optimize data pipelines to ensure efficient data flow.
• Ensure that the data extracted from sources is accurate, complete, and usable. This might involve checking for missing values, inconsistent formats, or anomalies that could indicate errors.
• Test the efficiency and speed of data pipelines and databases. This can help identify bottlenecks and optimize performance.
• Verify that different components of the data infrastructure work together as expected. This includes checking that data flows correctly from sources to databases, and from databases to applications.

Maintenance of Data Pipelines

• Regularly check the health and performance of the data processes.
• Identify and resolve issues including debugging code, optimizing queries, or adjusting configurations.

What will you need to succeed in this role?

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Software Engineering or a related field.
• A minimum of 1-3 years of relevant experience as a data engineer is required. This includes experience in data engineering, data system development, or related roles.
• Good understanding of data structures, data modeling, and SQL.
• Exposure to Cloud based Services, preferably Microsoft Azure.
• Understanding with software design patterns and test-driven development (TDD)
• Proficiency in one programming language, preferably Python, including a strong grasp of Object Oriented and Functional programming paradigms.
• Exposure to Apache Spark concepts and distributed systems, including data transformations, RDDs, DataFrames, and Spark SQL.
• Strong problem-solving and critical-thinking abilities.
• Strong communication and collaboration skills.

#LI-Hybrid

#IndHP