Senior Databricks Engineer

Information Security Responsibilities

• Promote and enforce awareness of key information security practices including acceptable use of information assets malware protection and password security protocols
• Identify assess and report security risks focusing on how these risks impact the confidentiality integrity and availability of information assets
• Understand and evaluate how data is stored processed or transmitted ensuring compliance with data privacy and protection standards (GDPR CCPA etc.)
• Ensure data protection measures are integrated throughout the information lifecycle to safeguard sensitive information

Role and Responsibilities:

Architect build and maintain scalable ETL/ELT pipelines on the Databricks Lakehouse Platform using PySpark Spark SQL and Delta Lake.
Design and implement medallion (bronze/silver/gold) data architectures and enforce data quality governance and lineage standards.
Optimize Spark jobs and cluster configurations for performance and cost including partitioning caching and autoscaling strategies.
Implement and manage Unity Catalog for access control data governance and cross-workspace asset sharing.
Build and orchestrate workflows using Databricks Workflows Delta Live Tables and CI/CD pipelines.
Collaborate with data scientists analysts and business stakeholders to translate requirements into reliable data products.
Establish engineering best practices conduct code reviews and mentor junior data engineers.
Monitor production pipelines troubleshoot failures and drive root-cause analysis and continuous improvement. 

Required Qualifications:

5+ years of data engineering experience with 3+ years building production solutions on Databricks and Apache Spark.
Expert proficiency in Python (PySpark) and advanced SQL.
Deep hands-on experience with Delta Lake Unity Catalog and the medallion architecture pattern.
Strong experience with at least one major cloud platform (AWS Azure or GCP) and its core data services.
Proven track record optimizing Spark performance and managing cluster cost.
Experience with data modeling warehousing concepts and building dimensional/analytics-ready datasets.
Proficiency with Git-based version control CI/CD and infrastructure-as-code.
Bachelor's degree in Computer Science Engineering or equivalent practical experience. 

Preferred Qualifications

Databricks certification (Data Engineer Associate/Professional).

Experience with Delta Live Tables structured streaming and real-time data processing.

Familiarity with MLflow and supporting machine learning workflows in production.

Experience with orchestration tools (Airflow dbt) and data observability platforms.

Exposure to data governance security and compliance frameworks (e.g. GDPR HIPAA SOC 2).

Hands-on experience using AI coding assistants (e.g. Claude Code GitHub Copilot Cursor) to accelerate development refactoring and code review.

Familiarity with large language model APIs and SDKs (e.g. Anthropic Claude OpenAI) and prompt engineering for data and analytics use cases.

Experience integrating GenAI capabilities into data pipelines or applications including retrieval-augmented generation (RAG) and vector search.

Awareness of responsible AI practices including evaluation guardrails and cost/latency trade-offs when deploying LLM-based solutions. 

 

What We Do

Bounteous is a global AI Services firm where agentic engineering and human experience converge to deliver transformative business outcomes for the enterprise. We help organizations design build and scale AI-driven products platforms and processes. With more than 5000 team members worldwide Bounteous delivers AI that sticks powering adoption and outcomes that move organizations from experimentation to true transformation.

Why Work With Us

Bounteous combines strategy experience design engineering data and AI to help leading brands build intelligent systems that are practical scalable and measurable. People join us to work on meaningful transformation initiatives collaborate with smart and supportive teams and help shape what’s next in AI and digital experience.

Gallery