Senior Big Data Engineer

Life at ICUBE CSIAre you interested in working with a startup in cutting edge Technology space ? Are you excited about Artificial Intelligence, Machine Learning, Data Analytics ? Come join our ICUBE CSI team!Our team comes from all over the world with backgrounds in different types of industries. We are building a close knit team and meaningful culture. Our benefits:We’re happy when you’re happy. To make this happen, we offer competitive compensation, big-company benefits and a startup culture with vibrant energy and cool perks of a startup. • Health care (medical and dental) • 401k/Retirement Benefits • Life/LTD Insurance • Flexible schedules • Paid time off • Training & development • Fun, diverse and intellectually eager coworkers • Team happy hours and retreats • Workplace perks such as food/coffee We are building innovation around Data Analytics, Data Management using open source technologies. Our company thrives on selling big data analytics solutions. Our in-house advanced big data analytics team comes with backing of its leadership team with 100+ combined years of experience in data engineering, data processing, infrastructure designs, machine learning and visualization. We are an equal opportunity employer. Everyone can push forward in good times and when all is going perfectly as planned. But when it doesn’t go perfectly, you’re ready to attack problems head on. Business isn’t always easy, but you’re known for being there through thick and thin. Responsibilities: • Design, Develop and Implement Big data engineering projects in Hadoop ecosystem. • Engineer solutions with Cloudera, MapR or HDP for both batch & streaming data with high quality and with a sense of urgency. • Develop application and custom integration solutions using spark streaming and Hive. • Understand specifications, plan, design and develop software solutions, adhering to process – either individually or collectively within a project team • Work in state-of-the art programming languages and utilize object-oriented approaches in designing, coding, testing and debugging programs. • Work with support teams in resolving operational & performance issues • Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities • Integrate data from multiple data sources, Implementing ETL process using APACHE NIFI • Monitoring performance and advising any necessary infrastructure changes • Management of Hadoop cluster, with all included services such as Hive, HBase, mapReduce and Sqoop • Cleaning data as per business requirements using streaming API’s or user defined functions. • Build distributed, reliable and scalable data pipelines to ingest and process data in real-time, defining Hadoop Job Flows. • Managing Hadoop jobs using scheduler. • Apply different HDFS formats and structure like Parquet, Avro, etc. to speed up analytics. • Work with various hadoop ecosystem tools like Hive, pig, Hbase , spark etc. • Reviewing and managing Hadoop log files. • Assess the quality of datasets for a hadoop data lake. • Fine tune Hadoop applications for high performance and throughput. • Troubleshoot and debug any Hadoop ecosystem run time issues. Being a part of a POC effort to help build new Hadoop clusters Education: Bachelor’s Degree or higher in Computer Science, Information Systems or related engineering disciplines General Knowledge, Skills & Abilities • Be a good detail-oriented data engineer • Systematic and organizational skills important. • Willing to commit for completing deliverable on time. Preferred Qualifications: • Must have experience with Spark, Hive, Scala or py spark. • Preferred experience in one of the following technologies: Nifi, Kafka, or any other streaming technologies. • 3+ years experience in data engineering building ETL pipelines using JAVA or Python or Scala • Should be good at Pig, HIVE scripting. • Solid understanding of HDFS is important. • Work experience within a Data Warehousing/Business Intelligence/Data analytics group, and have hand’s-on experience with Hadoop • Create tables/views in Hive or other relevant scripting language • Have experience with Agile development methodologies • Experience with NoSQL databases, such as HBase, Cassandra, MongoDB. Experience Architecting Solutions Utilizing any of the following: • JAVA or Python or Scala programming languages • Nifi, Kafka-topics, or any other streaming technologies • Parquet/Avro/ORC/XML/JSON/ORC/CSV/TXT formats Location:Jacksonville, FL Send your resumes tocareers@ICUBE CSI.com