Map R_python_spark Developer

Title: PySpark Developer

Location: Charlotte, NC

Requisite Skills:

Mapr, Python, Spark, PySpark and SQLServer

Good to have Snowflake

At least 2 years of experience as Application Developer

Participate in detailed technical design development implementation and support of Big Data applications using existing and emerging technology platforms

Develop understanding of business processes objectives and solution requirements

Participate in project work groups with subject matter experts and stakeholders to understand data specific needs

Demonstrable experience engineering scalable data processing pipelines

Develop fast data infrastructure leveraging data streaming and batch processing

Experience in Data Lake Apache Drill PySpark MapR MapR DB

Demonstrable expertise with Python Spark Framework and wrangling of various data formats Parquet CSV XML XSD JSON

Must have experience with batch process design implementation scheduling monitoring enhancement perform core development scripting and performance tuning

Experience with RDBMS platforms including Microsoft SQL Server and Oracle Lead work and deliver elegant and scalable solutions

Work and collaborate with a nimble autonomous cross functional team of makers breakers doers and disruptors who love to solve real problems and meet real needs

Execute performance test and performance tuning activities

Experience with Error handling to records all error in parquet files Know how to Store files in S3 bucket instead of network path

Develop Email notification if file not generated using PySpark script

Experience with Performance improvement of Mapr jobs if it takes more time to load data Creative and innovative approach to problem solving

Good Understanding of CI/CD pipeline and Agile methodologies

About Capgemini:

Capgemini is a global leader in partnering with companies to transform and manage their business by harnessing the power of technology. The Group is guided everyday by its purpose of unleashing human energy through technology for an inclusive and sustainable future. It is a responsible and diverse organization of over 360,000 team members in more than 50 countries. With its strong 55-year heritage and deep industry expertise, Capgemini is trusted by its clients to address the entire breadth of their business needs, from strategy and design to operations, fueled by the fast evolving and innovative world of cloud, data, AI, connectivity, software, digital engineering and platforms. The Group reported in 2022 global revenues of €22 billion.

Get The Future You Want | www.capgemini.com

Disclaimer:

Capgemini is an Equal Opportunity Employer encouraging diversity in the workplace. All qualified applicants will receive consideration for employment without regard to race, national origin, gender identity/expression, age, religion, disability, sexual orientation, genetics, veteran status, marital status or any other characteristic protected by law.

This is a general description of the Duties, Responsibilities and Qualifications required for this position. Physical, mental, sensory or environmental demands may be referenced in an attempt to communicate the manner in which this position traditionally is performed. Whenever necessary to provide individuals with disabilities an equal employment opportunity, Capgemini will consider reasonable accommodations that might involve varying job requirements and/or changing the way this job is performed, provided that such accommodations do not pose an undue hardship.

Click the following link for more information on your rights as an Applicant - http://www.capgemini.com/resources/equal-employment-opportunity-is-the-law

Applicants for employment in the US must have valid work authorization that does not now and/or will not in the future require sponsorship of a visa for employment authorization in the US by Capgemini.

Ref: 1677199

Posted on: May 23, 2023

Experience level: Experienced (non-manager)

Contract Type: Permanent

Location: