Agents & Bots
Add an Agent
All Categories
A directory of every known AI agent, crawler, scraper, and other bot on the internet.
AI Search Crawlers
AddSearchBot
AddSearchBot is a web crawler that indexes website content for AddSearch's AI-powered site search solution, collecting data to provide fast and accurate search results.
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
amazon-kendra
amazon-kendra is an Amazon web crawler that indexes authorized web pages for Amazon Kendra search applications.
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
Amzn-SearchBot
Amzn-SearchBot is an AI search crawler operated by Amazon that indexes web content for use in improving Alexa and other Amazon services.
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
Anomura
Anomura is Direqt's web crawler that discovers and indexes links and metadata from websites for inclusion in Direqt's AI search results.
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
Applebot
Applebot is Apple's web crawler that gathers data to power search features across Apple's ecosystem including Spotlight, Siri, and Safari search functionality.
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
atlassian-bot
atlassian-bot is Atlassian's AI web crawler that indexes custom websites for Rovo, allowing the indexed content to appear in Rovo Search results and be used by Rovo Chat …
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
AzureAI-SearchBot
AzureAI-SearchBot is an AI search crawler operated by Microsoft that indexes web content to support Azure AI services and improve search capabilities across Microsoft's A…
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
Channel3Bot
Channel3Bot is Channel3's web crawler that visits publicly accessible product detail pages to index product information into a universal product catalog. The bot collects…
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
Claude-SearchBot
Claude-SearchBot is used to create an index of websites that can be surfaced as results in Anthropic's Claude AI assistant search feature.
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
Cloudflare-AutoRAG
Cloudflare-AutoRAG is Cloudflare's web crawler that indexes websites for AI Search (formerly AutoRAG), a managed retrieval-augmented generation service. The crawler autom…
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
Google-CloudVertexBot
Google-CloudVertexBot is a web crawler operated by Google that collects content for Google Cloud's Vertex AI Search service. This crawler indexes web pages to power enter…
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
Kimi-SearchBot
Kimi-SearchBot is a web crawler operated by Moonshot AI that powers Kimi's search functionality. It analyzes and indexes web pages so they can appear in Kimi search resul…
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
LinkupBot
LinkupBot is a web crawler operated by Linkup that indexes enterprise web content for its AI search platform. The bot visits websites to collect relevant business informa…
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
meta-webindexer
meta-webindexer is Meta's web crawler that browses the internet to improve search results for Meta AI users. It analyzes online content to make Meta AI's responses more r…
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
MistralAI-Index
MistralAI-Index is one of Mistral AI's automated web crawlers. It indexes web content to power search capabilities in Vibe, Mistral's AI assistant product.
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
OAI-SearchBot
OAI-SearchBot is OpenAI's web crawler that indexes websites for SearchGPT, collecting and analyzing web content to power AI-driven search results and real-time informatio…
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
PerplexityBot
PerplexityBot is the web crawler for Perplexity AI that indexes web content to power the AI search engine's real-time information retrieval and answer generation capabili…
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
PetalBot
PetalBot is Huawei's web crawler that indexes PC and mobile websites to build search databases for Petal Search engine and provide AI-powered content recommendations for …
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
xAI-SearchBot
xAI-SearchBot is a web crawler operated by xAI, the company behind Grok, to collect information from across the web for use in their search and chatbot products. This bot…
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
YandexAdditional
YandexAdditional is a web crawler operated by Yandex that retrieves website content for YandexGPT quick answers in Yandex Search.
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
YandexAdditionalBot
YandexAdditionalBot is a web crawler operated by Yandex that retrieves website content for YandexGPT quick answers in Yandex Search.
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
ZanistaBot
ZanistaBot is an AI search crawler operated by Zanista.
AI Search Crawler
AI Search Crawler
Indexes website content to possibly include as citations in AI-powered search results
See More →
Archivers
Archive-It
Archive-It is a web archiving crawler operated by Internet Archive that preserves copies of web pages for long-term digital preservation and historical record-keeping. It…
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
archive.org_bot
archive.org_bot is the Internet Archive's web crawler for the Wayback Machine, systematically crawling and preserving publicly accessible web pages for historical record …
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
Arquivo-web-crawler
Arquivo-web-crawler is the Portuguese web archive's bot that systematically crawls and preserves Portuguese websites for historical research, creating a comprehensive dig…
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
Authory
Authory is an automated content archiving crawler that systematically searches for and backs up published articles, podcasts, and videos by journalists and content creato…
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
bl.uk_lddc_bot
bl.uk_lddc_bot is operated by the British Library as part of their legal deposit web archiving program, which collects and preserves UK web content for the national archi…
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
bne.es_bot
bne.es_bot is an archiver.
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
bnf.fr_bot
bnf.fr_bot is the official web crawler of the Bibliothèque nationale de France (BNF), systematically collecting and archiving digital content from French websites to pres…
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
CUCH.org Archive Bot
CUCH.org Archive Bot is a web crawler that archives web content on behalf of C U Cyber History.
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
heritrix
heritrix is an archiver operated by Internet Archive.
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
ia_archiver
ia_archiver is an archiver operated by Internet Archive.
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
ia_archiver-web.archive.org
ia_archiver-web.archive.org is an archiver operated by Internet Archive.
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
IABot
IABot (InternetArchiveBot) is a Wikipedia bot operated by the Internet Archive that combats link rot by finding dead links on Wikipedia articles and adding archived versi…
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
Internet Archive
Internet Archive is an archiver operated by Internet Archive.
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
mirrorweb
mirrorweb is a web archiving crawler that captures websites in their entirety, including dynamic content and interactive elements, creating tamper-proof WARC archives for…
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
netarkivindsamling
netarkivindsamling is a web crawler operated by the Royal Danish Library that collects Danish internet content according to the Danish Legal Deposit Act for preservation …
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
Nicecrawler
Nicecrawler is an archiver operated by NiceCrawler.
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
SmarshBot
SmarshBot is an archiving bot operated by Smarsh that crawls and captures web content for compliance management and regulatory recordkeeping purposes. Organizations use t…
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
special_archiver
special_archiver is a web crawler operated by Internet Archive as part of their Archive-It service, which creates and preserves collections of web content. This bot visit…
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
Turnitin
Turnitin crawler gathers web content to build a comprehensive database for plagiarism detection services, comparing student papers against internet content for academic i…
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
XY-Archive-Compliance
XY-Archive-Compliance creates website archives to help customers meet regulatory and legal compliance requirements for data retention and historical website preservation.
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
XY-Archive-Compliance-Archiver
XY-Archive-Compliance-Archiver is an archiving bot operated by XY Planning Network, a membership organization for fee-only financial advisors. This bot archives web conte…
Archiver
Archiver
Captures and stores historical website snapshots for long-term digital preservation
See More →
Automated Agents
Amazon Bedrock AgentCore Browser
Amazon Bedrock AgentCore Browser is a cloud-based browser service that runs on AWS infrastructure. It enables AI agents and applications to interact with websites, naviga…
Automated Agent
Automated Agent
Automates browser interactions programmatically without direct human supervision
See More →
Anchor Browser
Anchor Browser is Anchor's cloud-hosted browser designed to enable AI agents to interact with websites in a human-like manner while maintaining verified, legitimate autom…
Automated Agent
Automated Agent
Automates browser interactions programmatically without direct human supervision
See More →
Browserbase
Browserbase is a cloud browser platform that enables AI agents and applications to autonomously read, write, and perform tasks on the web. It provides scalable, fast, and…
Automated Agent
Automated Agent
Automates browser interactions programmatically without direct human supervision
See More →
Cloudflare Browser Rendering
Cloudflare Browser Rendering is a serverless browser automation service that runs on Cloudflare Workers. It enables AI agents and applications to control a headless brows…
Automated Agent
Automated Agent
Automates browser interactions programmatically without direct human supervision
See More →
HeadlessChrome
Headless Chrome is a tool provided by Google's open source Chromium project that gives software the ability to render web pages in full detail. One of the most common use…
Automated Agent
Automated Agent
Automates browser interactions programmatically without direct human supervision
See More →
Known Agents Browser
Known Agents Browser is an automated agent operated by Known Agents.
Automated Agent
Automated Agent
Automates browser interactions programmatically without direct human supervision
See More →