Software Engineer (Data Engineering)

What we are looking for

Equilibrium was founded with a vision for building a company where data-centric decision-making drives our operations. Our reliance on closed-loop ML-infused autonomous workflow decision-making puts data and data management at the foundation of everything we build and operate. We are looking for talented and passionate software engineering professionals to design lay and support the technical foundation of our company – data – and thus drive our innovative collaborative and agile delivery culture.

What you will do

• Support the design development implementation and maintenance of our data engineering tools workflows processes and platforms that underpin our data engineering efforts

  • Design and build infrastructure for optimal extraction transformation and loading of data from various data sources (e.g. AWS tooling Databricks DBT Snowflake Temporal and SQL technologies)

  • Identify design and implement internal process improvements including re-designing infrastructure for greater scalability optimizing data delivery and automating manual processes

  • Design and engineer performant high-reliability systems to handle petabytes of data

• Architecting mentoring implementing and supporting a data ingestion platform for cross company use and enablement

  • Building and designing the long term vision for data ingestion at the company to enables cross team collaboration and end to end data ownership

  • Assemble large complex sets of data that meet non-functional and functional business requirements and build large scaled data pipelines

  • Design and build analytical tools to accelerate and automate the implementation of new and changing data pipelines

• Nurture improve and maintain our data quality including data accuracy availability performance and ease of access / understanding of the data in our data engineering ecosystem

  • Ensure data pipelines provide the intended actionable insights into key business performance metrics including operational efficiency

  • Design and implement scalable data validations per business and domain requirements.

  • Support a culture of continuous data quality improvement

• Serve as a collaborative subject matter expert on our data and accelerate our understanding and use of our data

  • Work with stakeholders including executive product data and design teams to support their data engineering needs while assisting with data quality and comprehension

The minimum qualifications you’ll need

• Passion for clean energy and fighting climate change

• An advanced degree in computer science data science machine learning artificial intelligence operations research engineering or related quantitative discipline

• 8+ years of experience developing a globally distributed data services platform

• 8+ years coding experience (Python Go Java or other applicable languages)

• Expertise in data analysis and ETL/ELT

• Experience developing and operating large-scale data pipelines demonstrating knowledge with technologies such as Airflow Luigi Spark Kafka ElasticSearch Cassandra RDBMS and SQL

• Advanced working SQL knowledge and experience working with relational databases query authoring (SQL) as well as working familiarity with a variety of databases

• Experience building and optimizing big data data pipelines architectures and data sets

• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement

• Strong analytic skills related to working with unstructured datasets

• Build processes supporting data transformation data structures metadata dependency and workload management

• A successful history of manipulating processing and extracting value from large disconnected datasets

• A collaboration-first mentality with a willingness to teach as well as learn from others

Nice to have additional skills

• Familiarity with time series data transformation and ingestion

• Strong understanding of data science concepts

• Expertise in distributed database systems data lakes and data meshes