Job Summary
GrabOn is looking for a highly skilled Python Developer with deep expertise in large-scale web scraping, browser automation, and distributed systems . The ideal candidate will design and maintain high-resilience, fault-tolerant scraping infrastructure capable of operating reliably against modern anti-bot defenses.
You will work on distributed, cloud-native scraping pipelines , collaborate on agentic and
autonomous systems , and continuously optimize for scale, cost, and reliability .
Key Responsibilities
Scraping & Automation
Design, build, and maintain high-resilience web scraping systems at scale
Implement advanced Selenium / Playwright automation (headless, stealth, browser fingerprint control)
Handle anti-bot mechanisms , including:
IP rotation & proxy orchestration
CAPTCHA detection & mitigation
Browser fingerprinting and evasion strategies
Distributed Systems & Cloud
Architect and maintain distributed scraping pipelines using:
AWS Lambda
SQS
EC2 (auto-scaling worker fleets)
Build retry-safe, idempotent, and fault-tolerant pipelines
Ensure graceful handling of failures, throttling, and partial outages
Performance, Monitoring & Optimization
Monitor scraping throughput, failure rates, and infrastructure health
Debug production issues across distributed workers
Optimize AWS cost, latency, and system throughput
Agentic & Autonomous Systems
Collaborate on agent-based architectures for:
Autonomous task execution
Decision-making workflows
Self-healing data pipelines
Engineering Practices
Write production-grade Python code (clean, testable, maintainable)
Work comfortably in Linux server environments
Follow best practices for concurrency, rate-limiting, and queue-based systems
Mandatory Requirements (Non-Negotiable)
Strong Python expertise (production systems, not notebooks or scripts)
Deep hands-on experience in web scraping at scale
Expert-level Selenium and/or Playwright knowledge
Proven experience with AWS Lambda, SQS, and EC2
Strong understanding of:
Concurrency & parallelism
Queues and distributed workflows
Retries, backoff strategies, and rate limits
Comfortable managing and debugging Linux servers & cloud infrastructure
or continue with email
GrabOn is looking for a highly skilled Python Developer with deep expertise in large-scale web scraping, browser automation, and distributed systems . The ideal candidate will design and maintain high-resilience, fault-tolerant scraping infrastructure capable of operating reliably against modern anti-bot defenses.
You will work on distributed, cloud-native scraping pipelines , collaborate on agentic and
autonomous systems , and continuously optimize for scale, cost, and reliability .
Key Responsibilities
Scraping & Automation
Design, build, and maintain high-resilience web scraping systems at scale
Implement advanced Selenium / Playwright automation (headless, stealth, browser fingerprint control)
Handle anti-bot mechanisms , including:
IP rotation & proxy orchestration
CAPTCHA detection & mitigation
Browser fingerprinting and evasion strategies
Distributed Systems & Cloud
Architect and maintain distributed scraping pipelines using:
AWS Lambda
SQS
EC2 (auto-scaling worker fleets)
Build retry-safe, idempotent, and fault-tolerant pipelines
Ensure graceful handling of failures, throttling, and partial outages
Performance, Monitoring & Optimization
Monitor scraping throughput, failure rates, and infrastructure health
Debug production issues across distributed workers
Optimize AWS cost, latency, and system throughput
Agentic & Autonomous Systems
Collaborate on agent-based architectures for:
Autonomous task execution
Decision-making workflows
Self-healing data pipelines
Engineering Practices
Write production-grade Python code (clean, testable, maintainable)
Work comfortably in Linux server environments
Follow best practices for concurrency, rate-limiting, and queue-based systems
Mandatory Requirements (Non-Negotiable)
Strong Python expertise (production systems, not notebooks or scripts)
Deep hands-on experience in web scraping at scale
Expert-level Selenium and/or Playwright knowledge
Proven experience with AWS Lambda, SQS, and EC2
Strong understanding of:
Concurrency & parallelism
Queues and distributed workflows
Retries, backoff strategies, and rate limits
Comfortable managing and debugging Linux servers & cloud infrastructure
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
Similar jobs
Show more