Lead Data Engineer

We are looking for experienced Lead Data Engineer to join our international team of professionals!

Work at Exadel - Who We Are: 
Since 1998, Exadel has been engineering its own software products and custom software for clients of all sizes. Headquartered in Walnut Creek, California, Exadel currently has 2700+ employees in development centers across the Americas, Europe, and Asia. Our people drive Exadel’s success, and they are at the core of our values.

About Our Customer:
Top 10 North American bank

About Our Project:
The client needs to build a system for tracking deal workflows during collaborations between investors and issuers. One of the challenges is to create a Data Lake with elements of data virtualization and data governance.

Data will be used for further analytics in reporting systems and for feeding internal web applications. Web applications will be able to perform data enrichment. Initial data source processing will be done on the client side. Our team will need to set up file processing and ingestion, create data quality checks, normalize data and setup lineage, and create a data catalog.

Project Team:
When you join our team, you'll be immersed in a culture where teammates always help each other achieve better results. We believe that together we are greater and that we can find brilliant solutions by sharing ideas. 

Project Tech Stack:

• AWS: S3, Glue, Lambda, EMR, RDS, Neptune
• Programming languages: Java/Scala, SQL

Requirements:

• 6+ years of experience working with data (experience working on Lead position)
• 3+ years of experience in Java/Scala
• Hands-on experience implementing Data Lake and ingestion processes using AWS Services: S3, RDS, Glue
• Hands-on experience in setting up AWS services
• Hands-on experience in relational databases architecture and SQL
• Experience in normalizing data and setting row-based security

English level:
Upper-Intermediate

Responsibilities: 

• Communication with the client and formalizing requirements in cooperation with the business analyst
• Discuss architectural approach with the client’s architecture team and perform suggestions
• Create a Glue ingestion layer
• Setup Clue Crawler
• Create Glue Data Catalog
• Create custom data quality checks rules
• Create file processing: normalization, enrichment, and lineage using Spark
• Setting row-based security rules
• Participate in data QA automation setup

Advantages of Working with Exadel:

• You can build your expertise with our Client Engagement team, who provide assistance with existing and potential projects
• You can join any Exadel Community or create your own to communicate with like-minded colleagues
• You can participate in continuing education as a mentor or speaker. You will not only be emotionally but also financially rewarded for mentoring
• You can take part in internal and external meetups as a speaker or listener. We support you in broadening your horizons and encourage knowledge sharing for all of our employees
• You can learn English with the support of native speakers
• You can take part in cultural, sporting, charity, and entertainment events
• Working at Exadel means always upgrading your skills and proficiency, so we provide plenty of opportunities for professional development. If you’re looking for a challenge that will lead you to the next level of your career, you’ve found the right place 
• We work hard to ensure honest and open relations between employees and leadership, so our offices are friendly environments