Agent Directory
The definitive guide to every known AI agent, crawler, scraper, and other bot on the internet, updated daily.
Fetchers
-
Slack-ImgProxySlack-ImgProxy is a web crawler used by Slack to fetch, process, and cache images that are shared in Slack channels, ensuring they can be displayed properly within the me…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SlackbotSlackbot is operated by Slack and is used to fetch link previews and unfurl content from websites when users share URLs in Slack conversations. When a link is posted, thi…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Slackbot-LinkExpandingSlackbot-LinkExpanding is Slack's link unfurling bot that fetches web page content when users post links in channels, extracting meta tags like oEmbed, Twitter Cards, and…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SlackbotLinkExpandingSlackbotLinkExpanding is a fetcher operated by Slack.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Snap URL Preview ServiceSnap URL Preview Service is a bot operated by Snap that fetches web page content to generate URL previews and rich link cards for links shared in Snapchat conversations. …FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SnapchatSnapchat is a fetcher operated by Snapchat.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Sottopop-user-rssSottopop-user-rss is a feed fetcher operated by UpContent that retrieves RSS content to support UpContent content discovery and curation features.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SottopopNoneSottopopNone is a web crawler operated by UpContent, a content curation and discovery platform that helps users find and share relevant content for their marketing and so…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
StartmeBotStartmeBot is a fetcher bot operated by Start.me, a personalized startpage and bookmarking service. The bot fetches content from websites to generate previews, thumbnails…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
StatabotStatabot is StataCorp's web crawler that searches for and indexes Stata statistical software table of contents files (stata.toc) for software package management.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SteamChatSteamChat is Valve's web crawler that fetches URL previews and metadata for links shared within Steam's chat functionality, enabling rich link previews in conversations b…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SubstackContentFetchSubstackContentFetch is a bot operated by Substack, a newsletter and subscription publishing platform. This bot fetches content from external websites to generate link pr…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SuperfeedrSuperfeedr is a fetcher operated by Superfeedr.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Superfeedr botSuperfeedr bot is a feed fetching crawler operated by Superfeedr, a real-time RSS and feed API service. The bot retrieves and parses feeds on behalf of subscribers, provi…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SurdotlyBotSurdotlyBot is the web crawler for Sur.ly, a content preview and link shortening service. This bot fetches content to generate previews and thumbnails for links shared th…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SynapseSynapse is a fetcher operated by Matrix.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Telegram BotTelegram Bot is a fetcher operated by Telegram.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
TelegramBotTelegramBot is Telegram's web crawler that fetches webpage content to generate Instant View pages and link previews for users, enabling rich content display directly with…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
TextRazor DownloaderTextRazor Downloader is a web fetcher operated by TextRazor that retrieves URLs submitted to its natural-language-processing API for analysis.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
TikTokSpiderTikTokSpider is a web crawler operated by TikTok that fetches content from websites when users share external URLs on the platform. The bot collects information for link …FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Tiny Tiny RSSTiny Tiny RSS is a fetcher operated by Tiny Tiny RSS.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
TumblrTumblr is Automattic's link unfurling bot that fetches URL content to create rich Link Blocks in Tumblr posts, parsing responses to display titles, descriptions, and prev…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
TwitterbotTwitterbot crawls web pages to generate Twitter Cards, fetching metadata like titles, descriptions, and images for link previews when URLs are shared on X (formerly Twitt…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
updayupday crawls news websites and sources to aggregate articles for its news app platform, collecting and indexing content to provide personalized news recommendations.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
ViberViber is a fetcher operated by Viber.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
videootvBotvideootvBot is a crawler that extracts the newest articles from publisher websites via feeds or HTML parsing to create carousels with images, links and text for native ad…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
vkSharevkShare is a fetcher bot operated by VK, a social networking platform. It visits web pages when users share links on the VK network to collect preview information such as…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
WhatsAppWhatsApp is a fetcher operated by Meta.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
WireReaderBotWireReaderBot is the crawler for Wire RSS Reader, a modern news reader application that aggregates RSS feeds. The bot fetches full article content from websites so that W…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
WordCountBotWordCountBot is a website analysis crawler that scans public web pages to count words in HTML source code, including SEO tags, to help estimate website translation costs …FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
WovnCrawlerWovnCrawler is a web crawler used by WOVN.io website localization service to retrieve source language content from client websites for translation and multilingual websit…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
WP.com mShotsWP.com mShots is a web fetcher operated by Automattic that captures webpage screenshots and thumbnails for the WordPress.com mShots service.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Yahoo Link PreviewYahoo Link Preview bot fetches metadata and preview data from URLs shared on Yahoo platforms and services. Supports Yahoo's social features, news aggregation, and content…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
YahooMailProxyYahooMailProxy retrieves and caches webpage content from URLs embedded in Yahoo Mail emails, improving security and reducing bandwidth usage for email users.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
YelpBotYelpBot is a web crawler operated by Yelp, the popular business review and local search service. This bot fetches content from business websites to gather information suc…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
ZoteroTranslationServerZoteroTranslationServer extracts citation metadata from URLs, DOIs, and ISBNs using Zotero translators to power Wikimedia's Citoid service for automated reference generat…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
Intelligence Gatherers
-
A360-SearchA360-Search is a web crawler used by Area360 for collecting and analyzing property information and real estate data from websites.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
ActiveComplyActiveComply is a compliance monitoring crawler that systematically scans social media platforms and websites to identify regulatory compliance violations for businesses …Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
AdagiobotAdagiobot is a web crawler that analyzes websites for advertising demand optimization, helping publishers maximize revenue through real-time bidding analysis and performa…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
adbeat_botadbeat_bot is a web crawler operated by Adbeat, a competitive intelligence platform that tracks and analyzes digital advertising campaigns. The bot collects data about di…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
adidxbotadidxbot is Microsoft's Bing Ads crawler that monitors ads and follows ad destination websites for quality control to ensure advertising standards and policy compliance.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
AdkernelTopicCrawlerAdkernelTopicCrawler is a web crawler operated by Adkernel, a white-label ad tech solutions provider. This bot gathers information to support the company's ad network, DS…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
admantxadmantx is an intelligence gatherer operated by Integral Ad Science.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
AdmixerBotAdmixerBot is a web crawler operated by Admixer, a digital advertising technology company. It visits web pages to analyze and categorize their content, enabling contextua…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
AdpResearchBotAdpResearchBot is operated by ADP and collects publicly available government legal and payroll documents to support regulatory research and guidance for its customers.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
AdsBot-GoogleAdsBot-Google is a specialized Google crawler that monitors and analyzes websites where Google Ads are served to ensure quality and policy compliance.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
AdsBot-Google-MobileAdsBot-Google-Mobile is a crawler operated by Google that specifically checks mobile web pages for ad quality purposes. The bot ensures ads display correctly on mobile de…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
AffsignalCrawlerAffsignalCrawler is a web crawler that collects information about affiliate marketing relationships by mapping affiliate links across websites. It gathers data to help id…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
aiHitBotaiHitBot is a web crawler operated by aiHit that collects business intelligence data about companies and tracks company change events. The collected data is compiled into…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
Amazing-SearchBotAmazing-SearchBot is operated by Amazing, an e-commerce education and software platform that provides tools, training, and community resources for entrepreneurs building …Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →