Senior Data Engineer

Team Introduction

The data team today comprises of 4 data scientists, 2 data engineers and 1 business analyst. Our focus is to procure the data and make it accessible, visualize and draw insights from the data and eventually use it for ML and data driven decisions that create high impact. We are a very cross functional team that impacts many different parts of the business including Credit Risk, Product, Operations, Marketing and Legal.

The Role

The data infrastructure at Possible has evolved tremendously over the last couple years. Our data lake now captures data from various different functions and sources and brings it all in one place for access. However, in its raw form the data has limited value and benefit and requires significant effort for new users to explore. This role will help build the required data models and subsequent layers that will help consolidate the data and make it more useful and easy to understand.

Responsibilities

The new hire will focus on formalizing and standardizing our data models. The first 3 months will focus on understanding the current uses of our most commonly used data in its raw form across various teams and then designing and building the data models using modern tools like dbt. Over the next year this role will be responsible to standardize the data modeling process for all existing and new data sources such that the models are reliable, maintainable and extendable. Long term the role will grow to define our data strategy and identify areas of the business which the data team can impact.

• Design our data models for optimal storage and retrieval and to meet critical product and business requirements
• Build, analyze and present actionable data to Marketing/Credit Risk/Finance/Product teams
• Engineering solutions to aggregate and automate large scale data flows from varying sources
• Collaborate with Engineers and Scientists in the organization to construct complex data models for analytics and machine learning
• Explore and learn the latest AWS technologies to provide new capabilities and increase efficiency
• Help continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers.
• Contribute to shared Data Engineering tooling & standards to improve the productivity and quality of output for Data Engineers across the company

Qualifications & Skills

• 5+ years of experience as a Data Engineer or a similar role
• Bachelor’s and/or Master’s degree, preferably in CS, or equivalent experience
• Demonstrated strength in data modeling, ETL development, and data warehousing
• Knowledge of data management fundamentals and data storage principles
• Knowledge of distributed systems as it pertains to data storage and computing
• Knowledge of batch and streaming data architectures
• Experience with AWS technologies including Redshift, RDS, S3, EML, Glue, Athena or similar solutions built around Hive/Spark etc. and data modeling tools like dbt.
• Knowledge of software engineering best practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, and operations
• Experience communicating with senior management as well as with colleagues from engineering, analytics, and business backgrounds
• Experience providing technical leadership and mentoring other engineers for best practices on data engineering
• Proficiency in Python, SQL or other similar languages.