Staff Data Engineer, Infrastructure

Role.  An overview of the opportunity:

At Skupos, data is everything. Our data integrates the forgotten, fragmented world of mom-and-pop corner with the glitzy, gigantic world of CPG conglomerates.

With the volume of incoming transactions growing everyday, Skupos is looking for a s engineer to wrangle and tame our data flows, expand and optimize our data ingestion and process platform architecture. If you are passionate about building scalable solutions that process terabytes of data in a matter of minutes, we look forward to hearing from you ! 

Team.  The team and our people:

This role will be part of the core functional team building our data ingestion/processing  architecture. This team works closely with the Data Infrastructure and the Data Platform.

Responsibilities.  Your responsibilities will include:

• Develop core capabilities of our data ingestion and data processing platform, with an architecture goal of scaling data flows to 10x current volumes
• Build a team of data engineers as the lead for multiple data initiatives
• Work with the core function team members to build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using AWS big data technologies (EMR/MSK/Spark/Kafka)
• Collaborate closely with Data Infrastructure to build CI/CD pipelines. Maintain and deliver continuous improvement of core projects through automation and process enhancement
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Partner with engineers, product managers, and data scientists to breakdown data requirements, analyze source data sets, address data quality issues and effectively build and automate ETL pipelines at scale.
• Partner with ML engineers, data infrastructure team to define and own Data engineering tools, products and processes in place and define/set SLAs for each.
• Set policies and systems for data governance especially data quality monitoring
• Architect and implement best practices of big data tools from scratch & also using tools such as Spark, Airflow, Kafka, Snowflake

Experience and Skills. Candidates should have:

• Bachelor’s degree or equivalent, ideally in a technical or quantitative field (advanced degree is a plus)
• Past experience with developing enterprise quality solutions in an iterative or Agile environment
• 4+ years of experience building and optimizing large-scale data pipelines, architectures and data sets on the public cloud, preferably AWS
• Experience with Apache Spark in large scale production environment
• Experience with AWS MWAA / Apache Airflow
• Experience with Terraform or Kubernete or other infrastructure as code
• Advanced proficiency with one of Python / Java / Scala in a production environment
• Experience with data warehousing architecture, data modeling and SQL
• Experience developing and supporting data intensive applications within AWS is a plus

Salary is based on experience and location.

Salary range based on Denver Market: $150,000 - $170,000.