Hadoop/Big Data Engineer

Description
Not Applicable

Qualification:

Post-grad in one of the following fields with strong academic credentials:

• Computer Science/IT.

• Operations Research/Applied Math.

• Engineering.

• Statistics.

Responsibility:

Business:

• 'Works with the business team to identify the right business objective and data required to answer the same.

• Executes a data collection plan from both structured and unstructured sources that helps in data exploration , hypothesis testing and statistical modeling.

• Analyzes data and generates insights that can articulated to business stakeholders.

• Develops POCs that can be used to generate business decisions.

• Simulates several scenarios through an interactive visualization that deepens the business understanding with underlying trends and root causes.

Stakeholder Management:

• '• Communicate analytical results in a way that is meaningful for business stakeholders and provides actionable insights.

• Coordinates in communicating the data needs with both technology and business teams to ensure that right data is captured for analysis and modeling.

Project Management:

• '• Ensure that the work goes ahead in a way that conforms to DS delivery framewrok and fulfills all outputs specifically outlined by stakeholders.

• Work with Business Operations and Delivering teams on designing, building and deploying data analysis systems for large data sets.

• Execute flawlessly on the DS project plan by adhering to timelines.

• Execute the design, analysis, or evaluation of assigned projects.

Data Analytics and Reporting:

• '• Explore and examine data from multiple disparate sources.

• Prepare a data collection plan from both structured and unstructured sources.

• Collaborate and coordinate with Technology and Business teams for all data needs.

• Expert level proficiency in data handling (SQL).

• Data Discovery & Profiling.

Data Discovery & Profiling:

• '• Perform exploratory data analysis and generate insights.

• Validate hypothesis developed during exploration phase.

• Present initial results to business stakeholders and identify the next steps.

Data Modelling:

Create models using one or more of the platforms like R, SAS, Python, Matlab Model creation would involve one or more of the following technqiues:

• 1 Classification.

• 2 Clusterning.

• 3 Time Series.

• 4 Forecasting.

• '• Testing and validating the model.

• Deriving insights and recommendations from the models.

• Performing data visualization and presentation to clients.

Innovation & Thought Leadership:

• '• Provide thoughtleadership and dependable execution on diverse projects.

• implement best practices and technology.

• Discover new opportunities where advanced analytical techniques can be leveraged for solving business problems.

Qualifications

Bachelor's Degree required

5-11 years solid experience

Experience working in Data warehousing projects - Hadoop Developer

Desired Skills

• Hadoop Developer with strong experience in Python Pyspark, and SQL
• Solid experience in Hadoop tech stack - Hive, Spark, Shell Scripting
• Development experience in Java technologies and Hadoop ecosystem

• Able to work with Business teams to document requirements, work with IT Operations/shared service teams for developing and implementing solutions, conduct (UAT) user acceptance test, and provide the weekly status.
• Able to prioritize and work on multiple tasks involving with other multiple Client application teams simultaneously.
• Able to learn and implement the solutions with new technologies as per business requirement.

• Page Break

Roles & Responsibilities:

Within the AIA team, a phenomenal opportunity awaits with in a group that is responsible for driving connected vehicle strategy by managing global data streams that integrate with internal Client systems shaping the future of vehicles. The Hadoop Developer role is critical in ensuring data integrity and availability. This position requires a Hadoop developer with strong Java, Pyspark, and SQL skills who is familiar with Spring boot API development. Collaborate closely with other seasoned developers and data scientists to create and maintain a single, complete, and actionable view of the vehicle data by supporting the landing, transforming, discovery, design, marketing, and development of our current and future products. Maintain leadership around enabling connectivity in the next generation of vehicles allowing Client to remain a technical leader in the automotive space.

Our strength is built on our ability to work together. Our diverse backgrounds offer different perspectives and new ways of thinking. It encourages lively discussions, inspires thought leadership, and helps us build better solutions for our clients. We want someone who thrives in this setting and is inspired to craft meaningful solutions through true collaboration.

Excellent communication and presentation skills.

Qualifications
Technical Skills
SNo Primary Skill Proficiency Level * Rqrd./Dsrd. 1 Big Data Management PL1 Required 2 Hadoop Distributed File System PL1 Required 3 SQL Developer PL1 Required

Domain Skills
SNo Primary Skill Proficiency Level * Rqrd./Dsrd. 1 Mutual Fund/Securty Accounting NA Required 2 Hedge Fund Accounting NA Required

* Proficiency Legends
Proficiency Level Generic Reference PL1 The associate has basic awareness and comprehension of the skill and is in the process of acquiring this skill through various channels. PL2 The associate possesses working knowledge of the skill, and can actively and independently apply this skill in engagements and projects. PL3 The associate has comprehensive, in-depth and specialized knowledge of the skill. She / he has extensively demonstrated successful application of the skill in engagements or projects. PL4 The associate can function as a subject matter expert for this skill. The associate is capable of analyzing, evaluating and synthesizing solutions using the skill.