Senior Data Engineer

Deimos is a Cloud-native Developer and Security Operations technology services company.  We help companies of all sizes adopt the Cloud for improved service delivery to their clients. We’re a fully remote African-based team of engineers who are passionate about implementing engineering best practices. We leverage the latest technologies while building globally competitive solutions for our clients. With Deimos being one of the two moons of Mars we refer to ourselves as “Martians” who are on a mission to Mars together.

Our teams value the ability to learn and adapt to technology changes while appreciating solid foundational design and the craft of software engineering. As such our engineers enjoy working with various clients who have different problems to solve. If this sounds like you then you would be an ideal fit for our environment. However it is ideal for you to be based in one of the countries we currently hire in which are as follows: Kenya Ghana Nigeria South Africa and Senegal.

Role Overview

We are seeking a seasoned Senior Data Engineer to join our Engineering division and work with an established team dedicated to data-centric solutions. In this role you will establish and enforce best practices to facilitate the effective delivery of high-quality data products. You will play a crucial role in a high-performing interdisciplinary team and will report directly to an Engineering Manager.

What you will be doing:

• Helping to build & maintain the data lake of an African e-commerce giant.

• Collaborating closely with the team on data requirements their implementation process enhancements and domain-specific analysis.

• Conducting critical evaluations of the data systems and architectures aiming to identify opportunities for improved data usability efficiency and customer satisfaction.

• Designing and developing robust maintainable and efficient code that aligns with best practices in data engineering.

• Identifying and addressing performance bottlenecks resolving data-related issues and optimizing the performance of data processing and storage systems.

What you must have:

• At least five (5) years of experience in data engineering with specific expertise in big data technologies and cloud-based data warehousing.

• Experience managing data storage and optimisation in AWS S3 for high-performance data warehousing of large volumes of raw and processed data.

• Extensive knowledge of implementing scalable and efficient data processing workflows using Apache Spark on AWS EMR.

• A strong understanding of data integration and orchestration using AWS services such as Managed Workflows for Apache Airflow (MWAA) and AWS Lambda.

• Well-versed in data modelling and data architecture principles suitable for large-scale data environments.

• A strong understanding of programming principles with proficiency in Python or similar languages used in data processing.

• Good working knowledge of version control using Git in a collaborative project setting.

• Experience working within an agile development environment particularly with a focus on iterative and incremental data projects.

• A solid understanding of automated release management for data applications utilising CI/CD pipelines for efficient deployment and operations.

• Experience creating and maintaining data catalogs using AWS Glue Catalog to organise metadata

• Nice to have - Experience with resource provisioning and deployments on AWS with Terraform;

About you

For us to achieve our ambitious vision together as a team it is important for our Martians to lead at all levels be self starters who take initiative and put their hands up for challenging tasks. A Growth Mindset is important to us and we encourage all our Martians to openly share knowledge support and help each other ask questions get creative with new technologies and learn from setbacks.

Becoming a Martian means:

• Comfortably working and learning from a fully remote culturally diverse team based predominantly in South Africa Kenya Nigeria and Ghana.

• Being an open honest and respectful communicator.

• You enjoy asking questions identifying areas of improvement and proposing solutions no matter your job title or whether you have been with us for a day a month or years!

• You are comfortable taking initiative and operating independently.

• You thrive in a fast paced environment where change is constant.

• You find it exciting to work with various clients from different industries each with a different problem for you and your team to solve.

• Intentionally sharing tech and industry trends that excite you with your peers.

• Seeking continuous feedback and actively taking steps to continuously grow personally and professionally.

Want to know what you get by joining us?

• Become a member of a team where we value each individual's contribution from day 1 and empower you to make suggestions get involved and do what you love most!

• Flexibility and the freedom to work remotely.

• Work-life balance where you are not expected to work over weekends or after hours.

• A forward thinking remote company that knows how important it is to stay connected as one team by providing virtual social platforms for employee engagement.

• A monthly work from home allowance which you can use to set yourself up to work comfortably from home. Whether that is pens notebooks new headphones or work snacks!

• A MacBook or Windows laptop for you to do your best work on.

• Become part of a team of exceptionally clever and talented people who like to share their knowledge and learnings.

• We support your career growth and love to celebrate your successes and advancement!