Agent Directory
The definitive guide to every known AI agent, crawler, scraper, and other bot on the internet, updated daily.
SEO Crawlers
-
SemrushBot-BASemrushBot-BA is a specialized web crawler used by Semrush's Backlink Audit tool to analyze website backlinks and identify potentially harmful or toxic links for SEO purp…SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SemrushBot-CTSemrushBot-CT is an SEO crawler operated by Semrush.SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SemrushBot-OCOBSemrushBot-OCOB is the web crawler component used by Semrush's Content Toolkit for gathering and analyzing website content, supporting content marketing research, competi…SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SemrushBot-SASemrushBot-SA crawls websites to perform comprehensive SEO analysis and identify technical issues for site audit and optimization recommendations.SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SemrushBot-SISemrushBot-SI is Semrush's specialized Site Intelligence crawler for On Page SEO Checker tool, analyzing website content and technical SEO factors to provide comprehensiv…SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SemrushBotBacklinksSemrushBotBacklinks is an SEO crawler operated by Semrush.SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SenutobotSenutobot is an SEO crawler operated by Senuto.SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
seo-audit-check-botseo-audit-check-bot is a web crawler used by SEO audit platforms like SEO Audit Check and WebCEO to perform comprehensive SEO analysis and audits on websites for optimiza…SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
seo4ajaxseo4ajax is an SEO crawler operated by Capsule Code.SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
seo4ajax.comseo4ajax.com is a dynamic rendering crawler that helps make JavaScript-heavy single-page applications (SPAs) crawlable by search engines by pre-rendering pages into stati…SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SeobilityBotSeobilityBot crawls websites to gather SEO information and provide comprehensive SEO analysis including website audits, ranking monitoring, and optimization insights to i…SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SEOJuice-SearchBotSEOJuice-SearchBot crawls public web pages to map links between websites. SEOJuice uses this link graph for marketing intelligence and SEO analysis.SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SEOkicksSEOkicks is an SEO crawler operated by SEOkicks.SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SEOlizerSEOlizer is an SEO crawler operated by SEOLizer.SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SEOToolsOutreachSEOToolsOutreach discovers websites and extracts public contact details for SEOTools.info's link-building outreach workflow.SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SERankingBacklinksBotSERankingBacklinksBot is a web crawler operated by SE Ranking, an SEO software platform. It collects backlink data to build and maintain their database, which customers u…SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SerpMinerBotSerpMinerBot is the web crawler for Serp Miner, a platform that provides SEO keyword research tools including bulk allintitle checking, Keyword Golden Ratio calculation, …SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
serpstatbotserpstatbot is an SEO crawler that continuously crawls the web to discover new links and track changes for Serpstat's backlink database. The bot builds one of the largest…SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SiteAuditBotSiteAuditBot crawls websites to analyze over 130 SEO and technical issues, examining crawlability, markups, internal linking, performance, HTTPS, and international SEO fa…SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SitebulbSitebulb is a website crawler used for conducting technical SEO audits, available in both desktop and cloud versions. It crawls websites to analyze their structure and pr…SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SiteCheck-sitecrawlSiteCheck-sitecrawl is Siteimprove's crawler that performs comprehensive website audits for their content intelligence platform, scanning sites for accessibility, SEO, qu…SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SiteCheckerBotCrawlerSiteCheckerBotCrawler is the web crawler operated by Sitechecker, an SEO monitoring and analytics platform designed for agencies and teams. The bot analyzes SEO performan…SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SiteGuruCrawlerSiteGuruCrawler is an SEO crawler operated by SiteGuru, an SEO tool that provides website optimization, technical audits, content analysis, and performance tracking servi…SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
Siteimprove CrawlSiteimprove Crawl is an SEO crawler operated by Siteimprove.SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SitelinerSiteliner is an SEO crawler that analyzes websites to identify duplicate content, broken links, internal page rank, redirections, and generates XML sitemaps.SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
SpyglassesBotSpyglassesBot visits websites for Spyglasses to analyze how site content appears to AI assistants and search services, supporting visibility reporting.SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
StatusNestBacklinkSpiderStatusNestBacklinkSpider is an SEO crawler operated by StatusNest, a domain monitoring service. This bot likely crawls websites to discover and track backlinks as part of…SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
Storebot-GoogleStorebot-Google is Google's web crawler that analyzes e-commerce websites, product pages, and checkout processes using machine learning to gather information on pricing, …SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
W3 Validator ServicesW3 Validator Services is an SEO crawler operated by World Wide Web Consortium (W3C).SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
woorankreviewwoorankreview is an SEO crawler operated by WooRank that analyzes websites to provide search engine optimization insights and recommendations. The bot collects data on SE…SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
XoviOnpageCrawlerXoviOnpageCrawler is the web crawler component of XOVI, an all-in-one SEO software suite that analyzes and monitors websites for search engine optimization purposes. This…SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
-
ZoombotZoombot is an SEO crawler operated by SEOZoom.SEO CrawlerSEO CrawlerAnalyzes website structure and content to identify SEO improvement opportunitiesSee More →
Uncategorized Agents
-
007ac9 Crawler007ac9 Crawler is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
2ip.ru2ip.ru is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
360Spider-Image360Spider-Image is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
360Spider-Video360Spider-Video is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
5emeRue5emeRue is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
5erue5erue is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
7Siters7Siters is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
A Patent CrawlerA Patent Crawler is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
A6-IndexerA6-Indexer is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
ABEvalBotABEvalBot is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
AboundexAboundex is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
absearch-crawlerabsearch-crawler is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
AcademicBotRTUAcademicBotRTU is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
acapbotacapbot is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
Accessible Web BotAccessible Web Bot is an uncategorized agent operated by Accessible Web.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
acoonbotacoonbot is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
Acunetix Security ScannerAcunetix Security Scanner is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →
-
Acunetix Web Vulnerability ScannerAcunetix Web Vulnerability Scanner is an uncategorized agent.UncategorizedUncategorizedNot yet assigned a typeSee More →