Senior Data Engineer - Data Insights

You’ll Succeed With:

• BS or MS in computer science or a related technical field.

• Proven experience as a big data engineer or a similar role with a deep understanding of big data technologies frameworks and best practices.

• Designing and implementing large-scale distributed data processing systems using technologies such as Apache Hadoop Apache Spark or Apache Flink.

• Developing and optimizing data pipelines and workflows for ingesting storing processing and analyzing large volumes of structured and unstructured data.

• Building and maintaining data infrastructure including data lakes data warehouses and real-time streaming platforms.

• Designing and implementing data models and schemas for efficient data storage and retrieval.

• Ensuring the scalability availability and fault-tolerance of big data systems through proper configuration monitoring and performance tuning.

• Identifying and evaluating new technologies tools and frameworks to improve the efficiency and effectiveness of big data processing.

• Implementing data security and privacy measures to protect sensitive information throughout the data lifecycle.

• Knowledge of distributed computing principles and technologies such as HDFS YARN and containerization (e.g. Docker Kubernetes).

• Understanding of real-time streaming technologies and frameworks such as Kafka (Confluent) or Apache Flink.

• Strong problem-solving skills and ability to optimize and tune big data processing systems for performance and scalability.

• Excellent communication and teamwork skills to collaborate with cross-functional teams and stakeholders.

Nice to Haves:

• Leadership skills: guiding and mentoring junior members coordinating projects and collaborating with other teams.

• Problem solving: Dealing with complex challenges optimizing data pipelines and ensuring data quality and troubleshooting issues.

• Domain Knowledge: Having understanding of the hospitality industry can greatly enhance your ability to design and build effective data pipelines.

• Data Architecture Expertise: Contributing to designing data architecture and system and having deep understanding of data storage and ability to design a robust scalable and maintainable architecture is a valuable skill.

• Performance Optimization: Being able to optimize the performance of data pipelines is a crucial skill such as query optimization indexing caching and data partitioning and sharding.

• Data Governance and compliance: having an understanding of different data governance principles and compliance regulations such as GDPR and HIPAA is very important.

• Programming knowledge: Knowledge of Python and SQL is essential familiarity with Java and other languages is a plus.

• Experience working with a remote-first and globally distributed team

• Experience with CI/CD tooling including GitHub Actions and Build Workflows

• Knowledge of Confluent and AWS

#LI-IK1