Agent Directory
The definitive guide to every known AI agent, crawler, scraper, and other bot on the internet, updated daily.
Fetchers
-
rss2tg botrss2tg bot is an automated service that monitors RSS feeds and sends content updates to Telegram channels or users, enabling real-time notifications for feed content.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
RSSAPIRSSAPI is a service that periodically monitors RSS, ATOM, and JSON feeds for customers, parsing and normalizing content, then delivering updates via webhooks in near real…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
RSSExtractorRSSExtractor is a web fetcher that retrieves and extracts RSS feeds from websites.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
scoopit-crawlerscoopit-crawler is a web crawler operated by Scoop.it, a content curation platform that helps users discover, curate, and share content on various topics. The bot fetches…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
scour.ingscour.ing is Scour's AI-powered web crawler that fetches content from RSS feeds and websites to create personalized content recommendations based on user interests.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
ScourRSSBotScourRSSBot is a fetcher bot operated by Scour that retrieves RSS feed content from websites. The bot is likely used to collect and aggregate syndicated content for the S…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
serendeputybotserendeputybot is a fetcher operated by Serendeputy.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SFDC-CalloutSFDC-Callout is the user agent used by Salesforce when making HTTP callouts and API requests from Apex code, workflow rules, and other Salesforce automated processes to e…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
ShortPixelShortPixel is a bot from an image optimization service that compresses and converts images to formats like WebP and AVIF. It visits websites to fetch and optimize images …FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Shortwave Image FetcherShortwave Image Fetcher is a privacy-focused service that proxies images from HTML emails through Shortwave's servers to protect users' IP addresses and maintain email pr…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SimplePieSimplePie is a fetcher operated by SimplePie.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Skroutz ImageBotSkroutz ImageBot crawls and fetches product images from e-commerce websites to display in Skroutz's price comparison platform, enabling visual product browsing and compar…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SkroutzBotSkroutzBot is a specialized web crawler used by Skroutz to download XML data feeds, product images, and sample product pages from merchant websites for quality control an…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SkypeSkype is a fetcher operated by Skype.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SkypeUriPreviewSkypeUriPreview is Skype's link preview bot that fetches page content when users share URLs in Skype messages, extracting metadata to display rich previews with titles, d…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Slack Image ProxySlack Image Proxy is a fetcher operated by Slack.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Slack-ImgProxySlack-ImgProxy is a web crawler used by Slack to fetch, process, and cache images that are shared in Slack channels, ensuring they can be displayed properly within the me…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SlackbotSlackbot is operated by Slack and is used to fetch link previews and unfurl content from websites when users share URLs in Slack conversations. When a link is posted, thi…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Slackbot-LinkExpandingSlackbot-LinkExpanding is Slack's link unfurling bot that fetches web page content when users post links in channels, extracting meta tags like oEmbed, Twitter Cards, and…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SlackbotLinkExpandingSlackbotLinkExpanding is a fetcher operated by Slack.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Snap URL Preview ServiceSnap URL Preview Service is a bot operated by Snap that fetches web page content to generate URL previews and rich link cards for links shared in Snapchat conversations. …FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SnapchatSnapchat is a fetcher operated by Snapchat.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Sottopop-user-rssSottopop-user-rss is a feed fetcher operated by UpContent that retrieves RSS content to support UpContent content discovery and curation features.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SottopopNoneSottopopNone is a web crawler operated by UpContent, a content curation and discovery platform that helps users find and share relevant content for their marketing and so…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
StartmeBotStartmeBot is a fetcher bot operated by Start.me, a personalized startpage and bookmarking service. The bot fetches content from websites to generate previews, thumbnails…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
StatabotStatabot is StataCorp's web crawler that searches for and indexes Stata statistical software table of contents files (stata.toc) for software package management.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SteamChatSteamChat is Valve's web crawler that fetches URL previews and metadata for links shared within Steam's chat functionality, enabling rich link previews in conversations b…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SubstackContentFetchSubstackContentFetch is a bot operated by Substack, a newsletter and subscription publishing platform. This bot fetches content from external websites to generate link pr…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SuperfeedrSuperfeedr is a fetcher operated by Superfeedr.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Superfeedr botSuperfeedr bot is a feed fetching crawler operated by Superfeedr, a real-time RSS and feed API service. The bot retrieves and parses feeds on behalf of subscribers, provi…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SurdotlyBotSurdotlyBot is the web crawler for Sur.ly, a content preview and link shortening service. This bot fetches content to generate previews and thumbnails for links shared th…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
SynapseSynapse is a fetcher operated by Matrix.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Telegram BotTelegram Bot is a fetcher operated by Telegram.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
TelegramBotTelegramBot is Telegram's web crawler that fetches webpage content to generate Instant View pages and link previews for users, enabling rich content display directly with…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
TextRazor DownloaderTextRazor Downloader is a web fetcher operated by TextRazor that retrieves URLs submitted to its natural-language-processing API for analysis.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
TikTokSpiderTikTokSpider is a web crawler operated by TikTok that fetches content from websites when users share external URLs on the platform. The bot collects information for link …FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Tiny Tiny RSSTiny Tiny RSS is a fetcher operated by Tiny Tiny RSS.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
TumblrTumblr is Automattic's link unfurling bot that fetches URL content to create rich Link Blocks in Tumblr posts, parsing responses to display titles, descriptions, and prev…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
TwitterbotTwitterbot crawls web pages to generate Twitter Cards, fetching metadata like titles, descriptions, and images for link previews when URLs are shared on X (formerly Twitt…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
updayupday crawls news websites and sources to aggregate articles for its news app platform, collecting and indexing content to provide personalized news recommendations.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
ViberViber is a fetcher operated by Viber.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
videootvBotvideootvBot is a crawler that extracts the newest articles from publisher websites via feeds or HTML parsing to create carousels with images, links and text for native ad…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
vkSharevkShare is a fetcher bot operated by VK, a social networking platform. It visits web pages when users share links on the VK network to collect preview information such as…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
WhatsAppWhatsApp is a fetcher operated by Meta.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
WireReaderBotWireReaderBot is the crawler for Wire RSS Reader, a modern news reader application that aggregates RSS feeds. The bot fetches full article content from websites so that W…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
WordCountBotWordCountBot is a website analysis crawler that scans public web pages to count words in HTML source code, including SEO tags, to help estimate website translation costs …FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
WovnCrawlerWovnCrawler is a web crawler used by WOVN.io website localization service to retrieve source language content from client websites for translation and multilingual websit…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
WP.com mShotsWP.com mShots is a web fetcher operated by Automattic that captures webpage screenshots and thumbnails for the WordPress.com mShots service.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
Yahoo Link PreviewYahoo Link Preview bot fetches metadata and preview data from URLs shared on Yahoo platforms and services. Supports Yahoo's social features, news aggregation, and content…FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →
-
YahooMailProxyYahooMailProxy retrieves and caches webpage content from URLs embedded in Yahoo Mail emails, improving security and reducing bandwidth usage for email users.FetcherFetcherRetrieves web page metadata to power app features like link previews or feedsSee More →