Databricks Solution Architect

Information Security Responsibilities

• Promote and enforce awareness of key information security practices including acceptable use of information assets malware protection and password security protocols
• Identify assess and report security risks focusing on how these risks impact the confidentiality integrity and availability of information assets
• Understand and evaluate how data is stored processed or transmitted ensuring compliance with data privacy and protection standards (GDPR CCPA etc.)
• Ensure data protection measures are integrated throughout the information lifecycle to safeguard sensitive information

Role and Responsibilities

Architect and lead the implementation of an enterprise lakehouse on Databricks (Delta Lake Unity Catalog Photon Workflows) across one or more major clouds (AWS Azure or GCP).
Design scalable batch and streaming data pipelines using PySpark Spark SQL Structured Streaming and Delta Live Tables; establish patterns for ingestion from operational systems event streams and third-party APIs.
Define and enforce platform standards for data modeling (medallion architecture) CI/CD code quality testing observability and cost optimization.
Lead the governance strategy using Unity Catalog — fine-grained access control data lineage audit and PII handling — in partnership with security and compliance.
Optimize Spark workloads for performance and cost: cluster sizing Photon autoscaling file layout Z-ordering caching and query tuning.
Partner with ML engineers and data scientists to operationalize models using MLflow feature stores and model serving on Databricks.
Own the cloud infrastructure footprint for the platform: networking IAM secrets encryption and Terraform/IaC for Databricks workspaces and supporting services.
Mentor a team of data engineers; lead architecture reviews code reviews and technical design sessions; raise the bar on engineering practices.
Engage with stakeholders across analytics product and finance to translate business needs into a roadmap for the data platform. 

Required Qualifications

10+ years of data engineering experience with 4+ years building production workloads on Databricks.
Deep expertise in Apache Spark (PySpark and Spark SQL) — including performance tuning partitioning strategy and the Catalyst/Photon execution model.
Strong hands-on experience with Delta Lake Unity Catalog Databricks Workflows and Delta Live Tables.
Production experience on at least one major cloud (AWS Azure or GCP) including networking IAM storage (S3/ADLS/GCS) and compute primitives.
Proficiency in Python and SQL; comfort with Scala is a plus.
Experience designing medallion (bronze/silver/gold) architectures and dimensional models for analytics.
Strong CI/CD and DevOps practice: Git Terraform Databricks Asset Bundles or dbx automated testing of data pipelines.
Track record of leading technical projects end-to-end and mentoring engineers.
Excellent written and verbal communication; able to drive alignment with both engineering and business stakeholders. 

Preferred Qualifiactions

Databricks certifications (Data Engineer Professional Machine Learning Professional or Solutions Architect).
Experience with streaming architectures (Kafka Kinesis Event Hubs Pub/Sub) and Structured Streaming at scale.
Experience with MLflow feature stores and ML model deployment patterns.
Familiarity with dbt Airflow or other orchestration/transformation tools alongside Databricks.
Experience migrating legacy data warehouses (Teradata Netezza on-prem Hadoop Snowflake) to Databricks.
Multi-cloud or hybrid-cloud experience.
Experience operating under regulated frameworks (SOC 2 HIPAA GDPR PCI). 

What You'll Bring

An architect's instinct for trade-offs — balancing speed cost complexity and long-term maintainability.
A bias toward writing things down: design docs ADRs runbooks and onboarding guides.
Comfort operating with ambiguity and shaping problems before solving them.
A collaborative mentorship-oriented leadership style. 

What We Do

Bounteous is a global AI Services firm where agentic engineering and human experience converge to deliver transformative business outcomes for the enterprise. We help organizations design build and scale AI-driven products platforms and processes. With more than 5000 team members worldwide Bounteous delivers AI that sticks powering adoption and outcomes that move organizations from experimentation to true transformation.

Why Work With Us

Bounteous combines strategy experience design engineering data and AI to help leading brands build intelligent systems that are practical scalable and measurable. People join us to work on meaningful transformation initiatives collaborate with smart and supportive teams and help shape what’s next in AI and digital experience.

Gallery