Senior Data Engineer

THE ROLE

At Sotheby’s, we’re transforming the way we drive business decisions through data with the adoption of a fully Cloud Native environment. This hands-on role will support the efforts of maintaining and growing an enterprise data lake on which this team and others perform their work. You will interface with technical system owners, security, and IT teams to understand and support their efforts. You will be responsible for implementing new datasets and data services to support our various applications and business functions. You will partner with development teams to create ETL processes that automate the transformation and centralization of data from all corners of the enterprise.

RESPONSIBILITIES

• Implement new ETLs for data collection from internal systems into GCP
• Help evangelize high-quality software engineering practices toward building data infrastructure and pipelines at scale
• Work within and across agile teams to design, develop, test, implement, and support technical solutions across a fullstack of cloud development tools and technologies
• Create monitoring and alerting solutions for data pipeline statuses
• Implement specific Google Cloud data security and governance controls
• Maintain access controls for the data lake and associated Google Cloud products
• Ensure the quality of these solutions is robust, scalable, and efficient to meet the needs of the business

IDEAL EXPERIENCE & COMPETENCIES

• Strong programming skills preferably in Python
• 3+ years experience supporting production cloud environments
• Strong understanding of IAM and cloud-based access and security controls
• Familiarity with ETL pipeline orchestration frameworks, such as Luigi or Airflow
• Experience with data processing and storage frameworks like Apache Beam, BigQuery, BigTable, Redshift, Kinesis, etc.
• Experience with log management and monitoring tools, including tools within Amazon Web Services and Google Cloud Platform as well as open source and third-party monitoring tools.
• Experience in managing projects and infrastructure for cloud-based platforms (security, authentication, monitoring, data governance)
• Experience working with containers and container services is also a significant plus

The proposed base salary for this position ranges from $135,000-$185,000. In addition to base salary successful candidates are eligible to receive a discretionary bonus, as well as a competitive benefits package. Salary offers are based on a wide range of factors including relevant skills, training, experience, education, and, where applicable, licensure or certifications obtained. Market and organizational factors are also considered.