Data Engineer (Azure Expert)

Job Brief:
We are looking for a skilled Azure Expert Data Engineer to design and implement robust Azure-based data architectures for our customers, primarily within the government sector. The ideal candidate will have extensive experience with the Azure data ecosystem, including Azure Data Factory, Azure Databricks, and other related tools. This role demands a strong technical background, a passion for building scalable data solutions, and the ability to navigate complex requirements within highly regulated environments.
VentureDive Overview:
Founded in 2012 by veteran technology entrepreneurs from MIT and Stanford, VentureDive is the fastest growing technology company in the region that develops and invests in products and solutions that simplify and improve lives of people world-wide. We aspire to create a technology organization and an entrepreneurial ecosystem in the region that are recognized as second to none in the world.
Key Responsibilities:

• Architect and implement Azure-based data solutions tailored to meet customer requirements, ensuring scalability, reliability, and security.
• Leverage the Azure data ecosystem, including Azure Data Factory (ADF) and Azure Databricks, to build data pipelines, perform transformations, and enable advanced analytics.
• Collaborate with customers to understand their data needs, define architecture strategies, and provide technical recommendations.
• Design and implement data ingestion, transformation, and storage pipelines for structured and unstructured data sources.
• Ensure adherence to data governance policies and compliance requirements specific to government projects.
• Optimize data workflows and processes to improve performance, reduce costs, and enhance scalability.
• Develop and maintain technical documentation, including architecture diagrams, data flow models, and operational procedures.
• Provide guidance and mentoring to junior team members on Azure best practices and data engineering methodologies.
• Stay updated with the latest advancements in Azure services and assess their relevance to customer projects.

Qualifications and Experience:

• 5+ years of hands-on experience in data engineering and designing cloud-based data architectures.
• 3+ years of experience working extensively with the Azure data ecosystem, including:
• Azure Data Factory (ADF) for data integration and orchestration.
• Azure Databricks for data processing, transformation, and machine learning.
• Other services such as Azure Synapse Analytics, Azure Storage, and Azure SQL Database.
• Proven experience in building and optimizing large-scale ETL/ELT pipelines.
• Strong understanding of data warehousing and data lake architecture principles.
• Familiarity with programming languages such as Python, Scala, or SQL for data transformation and analytics.
• Experience with data governance, including security, compliance, and access management, especially for government customers.
• Strong communication skills to effectively engage with stakeholders, document solutions, and deliver presentations.
• Ability to design solutions that balance performance, cost-efficiency, and maintainability.

Preferred Skills:

• Knowledge of government compliance frameworks or other relevant standards.
• Familiarity with DevOps practices and CI/CD pipelines in Azure.
• Experience integrating Azure data solutions with Power BI or other visualization tools.
• Hands-on experience with real-time data processing using Azure Stream Analytics or similar tools.
• Exposure to large-scale migration projects from on-premises to Azure.

In order to thrive at VentureDive, you
…are intellectually smart and curious
…have the passion for and take pride in your work
…deeply believe in VentureDive’s mission, vision, and values
…have a no-frills attitude
…are a collaborative team player
…are ethical and honest
Are you ready to put your ideas into products and solutions that will be used by millions?
You will find VentureDive to be a quick pace, high standards, fun and a rewarding place to work at. Not only will your work reach millions of users world-wide, you will also be rewarded with competitive salaries and benefits. If you think you have what it takes to be a VenDian, come join us ... we're having a ball!
#LI-Hybrid