Agent Directory

Scrapers

• 008
  008 is a scraper operated by 80legs.
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• AdsTxtCrawler
  AdsTxtCrawler is a bot operated by OneTag that crawls and retrieves ads.txt files from websites. These files are part of an Interactive Advertising Bureau standard that p…
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• dcrawl
  dcrawl is a scraper.
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• HelloworkJobPostingBot
  HelloworkJobPostingBot is a web crawler that aggregates job listings by scanning company career pages and job posting websites to collect employment opportunities for the…
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• HTTrack
  HTTrack is a scraper.
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• HTTrack 3.0
  HTTrack 3.0 is a scraper.
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• IndeedJobBot
  IndeedJobBot is Indeed's web scraper that crawls and indexes job postings across the internet to populate the Indeed job search platform with comprehensive employment opp…
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• KrawlerBot
  KrawlerBot is a web scraping bot operated by Krawler that crawls and extracts data from websites for its customers.
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• MetaInspector
  MetaInspector is a scraper.
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• MetaJobBot
  MetaJobBot is a web scraper operated by METAJob, a job search platform based in Germany. The bot collects job listings and employment information to populate their job bo…
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• newspaper
  newspaper is a scraper.
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• Nutch
  Nutch is an open-source web crawler developed by the Apache Software Foundation. It is commonly used for large-scale web scraping.
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• nyt_scraping
  nyt_scraping is a web scraping bot operated by The New York Times. It is used to collect content from websites for The New York Times' purposes.
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• Offline Explorer
  Offline Explorer is a scraper operated by MetaProducts.
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• OpenindexSpider
  OpenindexSpider is a scraper operated by Openindex.
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• Potions
  Potions is a web crawler that fetches product feeds and crawls data from e-commerce websites to collect product information for commercial purposes.
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• Scrapy
  Scrapy is an open source web scraping framework written in Python that allows developers to extract data from websites. It is maintained by Zyte and used by millions of d…
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• ServerHunterSpider
  ServerHunterSpider indexes hosting-plan prices, specifications, and availability across providers for ServerHunter's server comparison database.
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• StatsDroneBot
  StatsDroneBot is StatsDrone's web scraper that collects and aggregates affiliate marketing statistics and performance data from various websites and platforms.
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• Thinkbot
  Thinkbot is a scraper.
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →
• web-image-collection-research
  web-image-collection-research is a web scraper that collects public images for a research dataset.
  Scraper

  Scraper

  Extracts large amounts of web data, often without explicit website permission

  See More →