Data Engineer

About The Role:

As a Data Engineer you will work on developing a modern data pipeline integrating data from a variety of data sources in order to bring the data into the analytics platform using highly scalable and extensible pipeline technologies. 

What You'll Do:

• Research various data tools and gain consensus on a tool which will be used not only for a particular use case, but generically for other upcoming use cases.
• You’ll not only build the data pipeline, but think hard about how data pipelines can be standardized and tools created with minimal amount of customization per pipeline.
• Understand the overall need, design a solution, and develop a data pipeline to integrate data from a source to a target.
• Develop automated test cases to validate data integrity and consistency.
• Think about how to handle security issues surrounding access, PII, and business-sensitive data.
• Design for scalability and robustness of availability.
• Document and maintain data systems; research and analyze application behaviors to improve performance and stability.
• Be a team player in contributing your thoughts and ideas to the overall goals of the team.

What We Look For:

• Bachelor's degree in Computer Science or other technical field or equivalent work experience
• 2+ years of developing in the cloud 
• 3+ years of relevant experience working with data tools which might include Airflow, Lambda’s, Kafka, DBT, Presto, MapReduce, AWS Glue, or similar platforms
• Experience using cloud analytics platforms which could include Google Big Query, AWS Redshift or Snowflake and standard relational databases such as Postgres, MySQL or other transactional databases
• Exposure in developing batch systems and real time streaming platforms.
• Excellent SQL experience with working through complex queries.  
• Solid Python or Java skills; with UNIX shell scripting being helpful
• Exposure to configuration-as-code tools such as Ansible, Terraform, or Chef.
• Experience with continuous integration, testing, and deployment using tools such as Git and Jenkins; nice to have some Docker experience as well.
• Expertise in the design, creation, management, and business use of large datasets.
• A willingness to being in an on-call rotation
• Strong multitasking skills with the ability to balance competing priorities
• Ability to work in a fast-paced environment where continuous innovation is desired, and ambiguity is the norm
• Experience with agile or other rapid application development methods
• Ability to coach / mentor other team members as appropriate