Senior Data Engineer

Data is the foundation of Compass. Our Org is responsible for evaluating, accelerating, building, and maintaining a unified, scalable and cost effective data ecosystem including infrastructure, data lake, data warehouse and tooling. We are a horizontal org within Compass handling data related to subject areas such as Agent, Listings, Segment, Finance etc.

We are looking for a self-driven Senior Data Engineer who can lead key initiatives and help us achieve more with the data. You will collaborate with multiple stakeholders from partner teams such as Analytics, Growth, Finance, Product etc to understand their business needs and recommend an optimal solution. The ideal candidate is an experienced data wrangler who can understand and optimize data systems from the ground up. We are looking for someone who stays on top of the things and effectively communicates on the progress/blockers to the leadership team.

Responsibilities

• Develop & Operationalize scalable and reliable data pipelines.
• Assemble large, complex data sets that meet functional/non-functional business requirements.
• Experience in database management techniques including logical data modeling, physical database optimization and security.
• Keep improving the integrity of data pipelines by setting up Data Quality checks to provide a comprehensive data service.
• Work with Data and Analytics experts to strive for greater functionality in our data systems.
• Experience with On-Call support and unblock users based on severity of the issue.

Qualification

• 7+ Years of experience in Engineering and Operationalizing Data Pipelines with large and complex dataset.
• 3+ Years of experience working with Unstructured data (JSON).
• Experience working with batch and stream processing.
• Extensive experience in Advanced SQL.
• 3+ Years of experience in Python/PySpark.
• 3+ Years of experience with transforming data using Databricks.
• 3+ Years of experience with AWS.
• 2+ Years of experience in orchestrating data pipelines using Airflow.
• Experience with GIT (CI/CD).
• Good to have DBT experience.
• Good to have Tableau experience.

Compensation: The base pay range for this position is $140k-$170k annually; however, base pay offered may vary depending on job-related knowledge, skills, and experience. Bonuses and restricted stock units may be provided as part of the compensation package, in addition to a full range of benefits. Base pay is based on market location. Minimum wage for the position will always be met