Web Scraper

$99,999 yearly

Job Description

Web Scraper

$20 - $50/hourpay

Required Skills

python
scrapy
selenium
html
css
javascript
rest-api
json
xml
data-cleaning
data-manipulation
nosql
relational-databases
web-security

About micro1

micro1 is the leading AI data lab for training frontier models and evaluating AI agents. Experts contribute their diverse subject matter knowledge across domains such as finance, healthcare, STEM engineering, and more. micro1 transforms that real-world expertise into high-quality training data, evaluations, and feedback loops that improve how AI systems learn, reason, and perform.

Our platform identifies and vets top talent through an AI recruiter, enabling high-quality expert contributions at scale. We aim to enable 1 billion people to do meaningful work by applying their expertise to AI. As our global expert network grows, micro1 is building the human intelligence layer for frontier AI.

 

Job Title: Web Scraper

Job Type: Contractor (Part-time)

Location: Remote

 

Job Summary: In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters.

 

Key Responsibilities:

- Design, develop, and maintain robust web scraping scripts and applications

- Perform data extraction from multiple websites, handling both static and dynamic content

- Clean, validate, and structure large volumes of scraped data for further analysis

- Monitor scraping pipelines for data quality and troubleshoot scraping failures

- Implement solutions to bypass anti-scraping mechanisms and maintain scraping effectiveness

- Collaborate with customer’s team members to define data requirements and deliverables

- Document scraping processes and maintain code for scalability and reusability

 

Required Skills and Qualifications:

- Proven experience in building and maintaining web scrapers using tools such as Python, Scrapy, Selenium, or similar technologies

- Strong understanding of HTML, CSS, JavaScript, and web protocols

- Expertise in handling APIs, RESTful interfaces, and parsing JSON/XML data

- Excellent written and verbal communication skills, with the ability to convey technical information clearly

- Proficiency with data cleaning, manipulation, and storage using relational or NoSQL databases

- Knowledge of web security concepts and best practices for ethical scraping

- Demonstrated ability to troubleshoot and resolve issues autonomously

 

Preferred Qualifications:

- Experience with cloud-based scraping and deployment (AWS, Azure, or GCP)

- Familiarity with version control systems like Git

- Background in large-scale data extraction projects