Web Crawling Engineer

Centric Pricing™ (formerly StyleSage), is an AI driven competitive assortment benchmarking and market trend insights solution for fashion, beauty and home goods brands and retailers.

We are a key innovation partner for iconic and emerging brands across the world. 

Our Platform is able to analyze the info of more than 1.000 retailers, processing data from more than 600.000 brands, tracking millions of products! 

You will be part of the Data Collection Team, formed by a group of motivated individuals that focus on crawling services. This team is the key to unblock other teams and machine learning and artificial intelligence processes, translating raw data into a valuable piece of information of the analytic pipeline.

As Crawling Engineer, your main mission is:

• To create, own and maintain Spiders
• To contribute to general improvements such as shared libraries & frameworks
• You will be closely working and communicating across different teams

What You'll Do:

• Build sustainable solutions used to crawl websites and extract data from their pages
• Delivering software systems focused on fast high-level web crawling using web scraping frameworks, analyzing patterns/state/next steps on how to get around bot protection. 
• Applying anti-bots mechanisms, workarounds and different proxy strategies.
• Collaborate with the rest of the technical team to ensure the Data-Collection solutions align with the organization’s goals, as well as customer needs.
• Remain up-to-date with technology trends to keep our software as innovative as possible.

• Review software code written by other team members to identify bugs and improve the code quality.

Desired Technical Skills:

• Experience building automated data collection scripts (spiders) to collect structured data.
• Experience using scraping frameworks. Scrapy, based on Python, is our main one, but you may also be proficient in other modern frameworks and that’s fine, we will help you get up to speed with Scrapy, and you’ll love it!
• Knowledge of the Web environment (model, standards, DOM, Request-Response, Cookies, Javascript, Browsers, Headers, XHR, etc.).
• Building well documented and organized systems, following common coding conventions.
• Troubleshooting and debugging skills.
• Bonus extra point: Knowledge and agility working on low level TCP/IP protocols (TLS, HTTP(S), SSL, etc) 

Soft Skills:

• Your job will require written and spoken communications in English
• Collaborative skills and teamwork mindset. We have people in many different countries and timezones.
• Analytic orientation, able to decompose complex problems and projects into manageable pieces; comfortable suggesting and presenting solution