(Cloud) Data Platform Engineer

Job Type:

Permanent

Build a brilliant future with Hiscox
 

About us:

HISCOX is an international insurer specialized in the market of luxury goods & property, and high end professional insurance. As experts in risk we give people and businesses the confidence to realise their ambitions. With offices in 9 European countries we have a very international staff and encourage a culture that fosters inclusion and collaboration, offering exciting opportunities for personal and professional growth.

In our Lisbon office we have the privilege of employing approximately 500 exceptional professionals representing 29 diverse nationalities. Despite our central location within the city, we recognize the importance of maintaining a healthy work-life balance. As part of our commitment to our employees’ well-being, we provide a 35-hour workweek along with the option for a hybrid working schedule.

Key Responsibilities:

• Provide expertise in Databricks architecture, configuration, and administration, following best practices and industry standards.

• Provide consultancy to the business for Databricks infrastructure design, configuration, administration, deployment and support.

• Where necessary or required, fulfil the responsibility for Databricks infrastructure design, configuration, administration, deployment and support for Business Unit change projects/programs

• Collaborate with data engineering, data science, and analytics teams to understand their requirements and provide technical guidance and solutions.

• Optimize Databricks workloads and processes for performance, cost-efficiency, and scalability.

• Support and own centralised features for Databricks, such as Unity Catalog, on behalf of the business, ensuring resilience, governance and availability, and minimising disruption to Business Change requirements

• Design, define and enforce standards for Data Gateways across the business, providing automation where appropriate, support, consultancy and assistance for new implementations and migrations.

• Ensure compliance with industry standards related to data security, privacy, networking on Data replated Cloud infrastructure.

• Collaborate with the Platform Services Group team on the development of reusable Terraform modules for Data related Cloud infrastructure and adhere to central Cloud standards set for the wider business.

• Ensure cohesive integration between existing systems on the Hiscox estate and the IT Services instance of Databricks

• Manage costs, capacity, efficiency, and forecasting for the Databricks estate

• Research, evaluate, and adopt new and emerging tools and technologies with a ‘right tool for the job’ mentality.

• Partner with IT Services Data Engineers to assess and implement suitable data solutions for central infrastructure MI and capacity management.

• Maintain a comprehensive documentation suite to standardise operational practices, and provide training where necessary to support team members and stakeholders to self-serve their own usage of Databricks and PowerBI within governance.

Qualifications:

• Strong knowledge of Azure cloud services and infrastructure, particularly Databricks.

• Familiarity with using Terraform for infrastructure as code.

• Understanding with continuous integration/continuous development principles.

• Experience with data engineering principles (data ingestion, data transformation, etc.)

• Strong team collaboration practices, both in person and in code.

• Effective communication skills.

• A Computer Science degree or data related qualification would be advantageous.

Technical Skills:

• Practical proficiency in Databricks, Git, Terraform, Azure DevOps, SQL

• Experience in Python, PowerShell

• Knowledge of networking / security practices would be advantageous

Work with amazing people and be part of a unique culture