Sr Big Data Engineer / Architect

You will be part of the Professional Services team and work with clients on short to medium term customer engagements solving their big data problems using Apache Spark and the Databricks platform. You will be in a customer-facing role that requires deep hands-on expertise in Apache Spark™ and data engineering, along with a variety of knowledge of the big data ecosystem. You will guide our largest customers, implementing pipelines from data engineering through model building and deployment, plus other technical tasks to help customers to get value out of their data with Databricks. You will report to the Regional Leader of the Resident Solutions Architect team.

The impact you will have:

• You will guide customers as they implement transformational big data projects, including end-to-end development and deployment of industry-leading big data and AI applications
• You will assure that Databricks best practices are being used within all projects and that our quality of service and implementation is strictly followed
• You will facilitate technical workshops, discovery and design sessions, customer requirements gathering and scoping for new and existing strategic customers
• Assist the Professional Services leader and project managers with level of effort estimation and mitigation of risk within customer proposals and statements of work
• Architect, design, develop, deploy, operationalize and document complex customer engagements individually or as part of an extended team as the technical lead and overall authority
• Knowledge transfer, enablement and mentoring of other team members, customers and partners, including developing reusable project artifacts
• Provide experience to the consulting team, provide best practices in client engagement to other teams
• Travel would be 30-40% regionally

What we look for:

• Deep hands-on expertise in Apache Spark (Scala or Python)
• 6+ years experience in Design and implementation of Big Data technologies (Apache Spark, Hadoop ecosystem, Apache Kafka, NoSQL databases) and familiarity with data architecture patterns (data warehouse, data lake, streaming, Lambda/Kappa architecture)
• 5+ years experience working as :
• Software Engineer / Data Engineer / Big Data Engineer: query tuning, performance tuning, troubleshooting, and debugging Spark and other big data solutions.
• Familiarity with a full range of data engineering and data science approaches, covering theoretical best practices and the technical applications of these methods
• Experience building and deploying a range of data engineering pipelines into production, including using automation best practices for CI/CD
• Familiarity with databases and analytics technologies in the industry including Data Warehousing / ETL, Relational Databases, or MPP
• Experience with cloud providers such as AWS, Azure or GCP
• Familiarity with AWS/EC2 cloud deployment models (Public vs. VPC)

Benefits

• Private medical insurance
• Accident coverage
• Employee's Provident Fund
• Equity awards
• Paid parental leave
• Gym reimbursement
• Annual personal development fund
• Work headphones reimbursement
• Business travel insurance