Data Engineer - Azure
Company
Rackspace
Location
Viet Nam
Type
Full Time
Job Description
-
Design develop and maintain real-time data streaming pipelines using Spark Streaming or Azure Functions.
-
Load merge and process machine logs from Kafka ensuring efficient data flow and transformation.
-
Integrate processed data into Redis cache and send it to the data lake for long-term storage and analysis.
-
Implement and optimize data processing solutions using Python.
-
Apply software engineering best practices including code reviews version control and continuous integration/deployment (CI/CD).
-
Collaborate with cross-functional teams to understand requirements and deliver high-quality data solutions.
Proven experience in real-time data streaming using Spark Streaming or Azure Functions. Strong proficiency in Python for data processing and automation tasks. Hands-on experience with Kafka for data ingestion and message queuing. Familiarity with Redis for caching and fast data retrieval. Knowledge of data lake architectures and best practices for data storage and retrieval. Solid understanding of software engineering principles including design patterns testing and documentation. Strong problem-solving skills and ability to work in a fast-paced collaborative environment.
Preferred Qualifications:
Experience with cloud platforms such as Azur Knowledge of other programming languages and frameworks related to data engineering. Familiarity with data modeling ETL processes and data lake concepts.
Date Posted
07/29/2024
Views
1
Similar Jobs
Data Engineer III - Rackspace
Views in the last 30 days - 0
The job description outlines a Data Engineer role with a focus on big data technologies SQL and data pipeline management The role involves project man...
View DetailsProject & Data Coordinator - Enveritas
Views in the last 30 days - 0
Enveritas is a nonprofit organization that builds solutions to improve sustainability in agricultural supply chains They are looking for exceptional t...
View DetailsSenior Machine Learning Engineer - Rackspace
Views in the last 30 days - 1
The job posting is for a Machine Learning Engineer position at Rackspace Technology a multicloud solutions expert The role involves building and deplo...
View DetailsSEO Lead - Canva
Views in the last 30 days - 0
Canva is seeking a Vietnamese SEO lead to expand their SEO presence in Vietnam The role involves leading SEO strategy building and managing a local SE...
View DetailsExecutive Assistant to the Vice President - ABC Education
Views in the last 30 days - 0
The job posting is for a fulltime executive assistant to support the Vice President of a thriving international education firm The role is fully remot...
View DetailsSenior Art Director - Canva
Views in the last 30 days - 0
The job description is for a Creative LeadSenior Art Director role at Canva based in Ho Chi Minh Vietnam The role involves shaping creative direction ...
View Details