Software Engineer I - Data Engineering

About Us

YipitData is the leading market research firm for the disruptive economy and recently raised $475M from The Carlyle Group at a valuation of over $1B.

We analyze billions of data points every day to provide accurate, detailed insights on ridesharing, e-commerce marketplaces, payments and more. Our on-demand insights team uses proprietary technology to identify, license, clean and analyze the data many of the world’s largest investment funds and corporations depend on.

We are one of Inc’s Best Workplaces - a fast-growing technology company with offices located in NYC (where we are based in), Hong Kong, and Shanghai, backed by Norwest Venture Partners and The Carlyle Group with a strong culture focused on mastery, ownership, and transparency.

About the Data Engineering Department:

Data Engineering’s mission is to create the best-in-class data analytics platform to support YipitData’s current and future data needs. Our self-service data platform empowers our Investor and Corporate product teams to analyze billions of data points every day to provide accurate, granular insights to their clients.

The Data Engineering Department is composed of 4 teams, including Data Infrastructure, Data Platform Engineering, ETL Engineering, and Analytics Engineering (~15 engineers). We offer a highly collaborative work environment where Data Engineering teams meet regularly to review architectures and strategies to empower a technical audience of data users at the company. Each team has a high degree of ownership and opportunity to work with state of the art tools in the data industry to reach their objectives. We offer the flexibility to switch teams based on your skills and career aspirations, a career ladder with growth opportunities, good work/life balance, and we have a very high employee retention rate.

About The Role:

We are looking for a Software Engineer I to join our Data Infrastructure team.

Data Infrastructure’s mission is to securely ingest business-critical data into our petabyte-scale data lake, and to enable the Data Engineering department to deploy solutions more effectively through cloud infrastructure. We onboard a wide range of datasets including third-party vendors, SaaS platforms, and internal applications to power thousands of analytics workflows at the company. We collaborate heavily with Product teams to fuel Investor and Corporate data products. We also interface with external vendors to make third-party and licensed datasets accessible in our data platform (AWS S3 and Databricks). Our work is highly specialized and essential to our product given the wide range of data structures, access requirements, and urgency of product development.

This is a remote-friendly opportunity that can be based in NYC, where our headquarters is located, or anywhere in the US (we expect Eastern Time working hours).

As a Software Engineer I you will:

• Learn and deploy cutting edge cloud infrastructure using best practices such as infrastructure-as-code.
• Be responsible for the ingress of all data sources that power our investor and corporate products.
• Design new infrastructure patterns to be used by other engineers.
• Build tooling that enables the department to deploy new cloud resources.
• Work directly with engineering stakeholders to understand their workflows.
• Write documentation, create architecture diagrams, and help drive the adoption of new technologies and best practices.
• Collaborate with engineers and business stakeholders at external companies to come up with the best solution for ingesting particular datasets.

On a given day, you might:

• Work directly with our CISO to push forward on security initiatives to help our organization reach a higher level of cloud maturity.
• Work with the Partnerships team to understand a vendor’s new 3rd-party dataset, and what options we have to ingest it into our data lake.
• Work with the Data Integration team to improve an existing pipeline by refactoring the ingestion method.
• Meet with AWS Product Managers to learn about new features for a service we use, and write a recommendation for our department on how to incorporate it into our workflows.
• Meet with the ETL team to discuss what infrastructure would enable them to maintain data pipelines more reliably.
• Meet with the Analytics Engineering team to sift through our billing and usage metrics for S3, to investigate a recent spike in storage cost.

As long as you’ve worked with modern data tools, we’re positive that you will learn and understand our technology stack:

• AWS: S3, CloudFormation (CDK), SQS, Lambda, and many more
• Databricks, Fivetran, Snowflake, CircleCI, Terraform
• Python, PySpark, Spark, SQL, Git, Unix command line
• For business tools we use: GSuite, Slack, Asana, Zoom

You Are Likely To Succeed If:

• Bachelor’s, Master’s or PhD degree in Computer Science or a related technical discipline, or equivalent experience
• 1-2+ years of experience as a Software Engineer or Data Engineer
• You are comfortable working with large-scale datasets using PySpark or Pandas
• You are a self-starter who enjoys working collaboratively with stakeholders
• You are excited about solving data challenges and learning new skills
• You have strong verbal and written communication skills
• Nice to have: experience with AWS, Databricks, IaC tools

What We Offer:

• The annual base salary for this position is anticipated to be $100-130K. The final offer may be determined by a number of factors, including, but not limited to, the applicant's experience, knowledge, skills, and abilities, as well as internal team benchmarks.
• We care about your personal life. We offer flexible work hours, open vacation policy, a generous 401K match, parental leave, team events, wellness budget, learning reimbursement, and more.
• Your growth at YipitData is determined by the impact that you are making, not by tenure, unnecessary facetime, or office politics. Everyone at YipitData is empowered to learn, self-improve, and master their skills in an environment focused on ownership, respect, and trust.
• To learn more about our culture and values, check out our Glassdoor page.

We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, marital status, disability, gender, gender identity or expression, or veteran status. We are proud to be an equal opportunity employer.

Job Applicant Privacy Notice