Sr Data Engineer - Python, Spark, AWS, Big Data
Job Description
Duration: 6 Months
Location: Beaverton, OR
Key Skills: Python, AWS, Spark, Big Data, Hadoop, Hive, S3, SQL
Must Have Skills:
- 5+ years' experience in Big Data stack environments like AWS EMR, Clourdera, Hortonworks
- 3+ years of SPARK in batch mode
- 3+ years of experience in scripting using Python
- 3+ years of experience working on AWS Cloud environment
- In-depth knowledge of Hive and S3
- Strong understanding of Hadoop, MPP systems and data structures
- Strong understanding of solution and technical design
- Experience building cloud scalable high-performance data lake solutions
- Experience with relational SQL & tools like Snowflake
- Aware of Datawarehouse concepts
- Performance tuning with large datasets
- Experience with source control tools such as GitHub and related dev processes
- Experience with workflow scheduling tools like Airflow
- Strong problem solving and analytical mindset
- Able to influence and communicate effectively, both verbally and written, with team members and business stakeholders
Responsibilities:
- The Data Engineer will collaborate with product owners, developers, database architects, data analysts, visual developers and data scientists on data initiatives and will ensure optimal data delivery and architecture is consistent throughout ongoing projects.
- Must be self-directed and comfortable supporting the data needs of the product roadmap. The right candidate will be excited by the prospect of optimizing and building integrated and aggregated data objects to architect and support our next generation of products and data initiatives.
- Create and maintain optimal data pipeline architecture,
- Assemble large, complex data sets that meet functional / non-functional business requirements.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing for greater scalability
- Comprehensive documentation and knowledge transfer to Production Support
- Work with Production Support to analyze and fix Production issues
- Participate in an Agile / Scrum methodology to deliver high -quality software releases every 2 weeks through Sprint
- Refine, plan stories and deliver timely
- Analyze requirement documents and Source to target mapping
Please visit https://mavensoft.catsone.com/careers/?portal= for additional job openings.
Date Posted
10/27/2022
Views
0
Similar Jobs
Software Engineer - DAT
Views in the last 30 days - 1
DAT is looking for a Software Engineer to join their team in Beaverton OR or Denver CO The role involves working in a full stack TypeScript ecosystem ...
View DetailsIT Engineer - DAT
Views in the last 30 days - 0
DAT is seeking a strong IT Engineer to join their growing technical team and drive the evolution of their technology infrastructure and end user exper...
View DetailsSenior Structural Engineer - HDR
Views in the last 30 days - 5
HDR is a company that specializes in engineering architecture and construction services They believe in diversity and collaboration and offer employee...
View DetailsSenior Frontend Engineer - ICIS
Views in the last 30 days - 0
Cirium is a company that provides data and aviation analytics solutions to various industries They are looking for a senior frontend software engineer...
View DetailsSoftware Engineer (Mid-level) - Act-On Software
Views in the last 30 days - 7
ActOn is a leading marketing automation company looking for a Software Engineer to join their team The company offers a supportive and fun culture com...
View DetailsAssociate Actuary - Unum
Views in the last 30 days - 6
Unum offers a comprehensive employee benefits package including competitive benefits generous PTO and opportunities for career advancement The company...
View Details