Senior Software Engineer

This is a Sr.  Software Engineer role to fulfill Syndigo’s growing crawling needs. This role drives the development of cutting-edge data-collecting technology, from thousands of websites, with data crawling software using thousands of bots.

 HOW YOU MAKE AN IMPACT

The role will require knowledge of overall scraping architecture, understanding sophisticated crawlers to scrape large volumes of unstructured data from public resources. The role will help ensure the stability and high performance of the platform.

HOW WE’LL BE WINNING TOGETHER DAY BY DAY

• The ideal candidate should have in-depth experience in Python and Scrapy, JavaScript, Puppeteer/Playwright, Linux and comfortable working in an agile environment and is experienced in continuous-delivery processes.
• Applicants must also be excellent problem solvers who know how to detect and evade anti scraping/bot technologies
• Use third-party APIs and web scraping tools to source data at scale.
• Develop and implement standards for clean code that maintain modularity, clarity, and portability.
• Timely escalation with Product and Engineering leaders in grooming of backlog items
• Demonstrate passion for continued learning by staying abreast of new technology and trends.

WE SHOULD TALK IF THIS SOUNDS LIKE YOU

• 5-7 years’ experience in Python.
• 2+ years of Scrapy framework development
• Crawlers and web scraping experience
• SQLnoSQL databases experience
• Experience with at least two queue managers or message brokers, necessary AWS-SQS
• Unix: command line, IPC, shell scripting
• Advanced in Docker and docker-compose
• Experience with at least one container orchestrator
• Knowledge of Browser-Web Server interaction including DNS, HTTP, SSL, GET vs POST.
• Command of software engineering principles, frameworks, and technologies.

• Plus:
  
  • Knowledge of e-commerce platforms.
  • Knowledge of Proxies..

 #LI-REMOTE