Senior Python Data Scraping Engineer

This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Senior Python Data Scraping Engineer in Israel.

This role is designed for a highly technical professional who specializes in building robust, scalable, and accurate web data extraction systems within a hybrid AI-human workflow environment. You will work on complex scraping challenges involving dynamic websites, large datasets, and structured data delivery at scale. The position combines hands-on engineering with problem-solving, requiring strong Python expertise and the ability to adapt scraping strategies to evolving web environments. You will collaborate with AI-driven agents while ensuring human-level accuracy, validation, and quality control. This is a remote freelance opportunity where independence, precision, and technical creativity are essential. Your work will directly contribute to building reliable datasets that support advanced AI and analytics applications.

• Own end-to-end web scraping and data extraction workflows across complex and dynamic websites.
• Build, maintain, and optimize Python-based scraping systems using tools such as BeautifulSoup, Selenium, and API integrations.
• Extract structured datasets from dynamic content sources including JavaScript-rendered pages, AJAX calls, and infinite scroll interfaces.
• Utilize internal tools (such as Apify and OpenRouter) alongside custom-built automation workflows to improve efficiency and scalability.
• Ensure high data quality through validation checks, normalization processes, and cross-source consistency verification.
• Handle large-scale scraping operations using batching, parallelization, and performance optimization techniques.
• Monitor scraping stability and adapt quickly to changes in website structure or anti-bot mechanisms.
• Deliver clean, structured datasets in formats such as CSV, JSON, or spreadsheet-based outputs.
• Apply systematic quality control processes before final data delivery.

Requirements:

• Minimum 5+ years of experience in data engineering, web scraping, automation, or software development.
• Strong hands-on experience with Python scraping frameworks such as BeautifulSoup, Selenium, or equivalent tools.
• Proven ability to extract data from complex and inconsistent web structures, including dynamic and JavaScript-heavy sites.
• Experience working with APIs, proxies, and anti-bot mitigation techniques.
• Strong background in data cleaning, normalization, transformation, and validation.
• Experience delivering structured datasets in formats such as CSV, JSON, or Google Sheets.
• Familiarity with cloud environments such as AWS or equivalent platforms.
• Experience using containerization tools like Docker in production workflows.
• Exposure to LLM-based frameworks such as LangChain, OpenRouter, or similar automation tools.
• Strong analytical mindset with exceptional attention to detail and data accuracy.
• Ability to work independently, troubleshoot issues, and manage tasks without supervision.
• Upper-intermediate or higher English proficiency (B2+ required).
• Nice to have: GitHub portfolio showcasing relevant scraping or automation projects.

Benefits:

• Fully remote freelance opportunity with flexible working schedule.
• Work on cutting-edge hybrid AI + human systems in a fast-growing technical field.
• Opportunity to collaborate with advanced AI tools and automation frameworks.
• Competitive compensation of up to $30/hour depending on experience and performance.
• Performance-based bonuses rewarding quality and consistency.
• Exposure to complex real-world data extraction and AI-driven workflows.
• High autonomy with the ability to choose your working pace and environment.
• Opportunity to build highly valuable expertise in AI-assisted data engineering.