Senior Data Engineer, Platform Engineering

The Data Platform team at Stack Overflow builds and maintains the data processing and analytic pipelines that democratize access to data across our organization.  Your cross-functional team will optimize flow and collection of all of our product and business data, enabling data at scale for Stack Overflow.  You’ll ensure that our data delivery architecture is optimized and consistent, enabling software engineers, database architects, data analysts, and data scientists to do their best work.  Data engineers have several key responsibilities:

• Create and maintain optimal data pipeline architecture to extract, transform, and load data from a wide variety of data sources using SQL and Azure “big data” technologies
• Assemble large, complex data sets that meet functional / non-functional business requirements
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Keep our data separated and secure across national boundaries through multiple data centers and Azure regions
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader

Skills and Requirements

We expect to see:

• 5+ years experience building and optimizing “big data” pipelines, architectures, and data sets
• 5+ years working SQL knowledge and experience working with relational databases, query authoring (SQL), as well as working familiarity with a variety of database platforms
• 5+ years working with Python
• Strong analytic skills related to working with free-form text
• 5+ years experience building processes supporting data transformation, data structures, metadata, dependencies, and workload management 
• A track record of leading and mentoring less experienced developers. You are eager to teach others and invested in the growth of your team.
• Self-motivating, self-directing, and a great communicator (written and oral). You thrive in an environment that grants you a lot of autonomy to explore creative solutions.
• Excellent problem solving skills. You excel at analyzing and solving problems using technology.
• Living and working within GMT-7:00 (US) to GMT+2:00 (Europe) time zones.

We like to see (but not required):

• Experience with Microsoft technologies and Azure cloud services for building and operating data pipelines
• Experience working remotely and/or working with teams that are distributed geographically.
• Experience with Agile methodologies such as Scrum, XP, or Kanban. Certification is a plus, but not a requirement.
• An active Stack Overflow profile, open source code, example projects that you're proud of (whether open source or worked on at a previous job), or any other evidence of your passion for building great software.
• Knowledge of how Stack Overflow works from our blog, podcasts, and other public artifacts. Ideas about how to evolve the platform and increase our impact on the developer community are even better.
• Experience with leveraging cloud-native technologies and techniques to build product ecosystems