Agent Directory
The definitive guide to every known AI agent, crawler, scraper, and other bot on the internet, updated daily.
Intelligence Gatherers
-
KonturbotKonturbot is a web crawler operated by Kontur that collects public business information for its company-data and business-intelligence services.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
KStandBotKStandBot is an intelligence gatherer operated by URL Classification.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
LeadFinderBotLeadFinderBot is a web crawler that collects public company and contact information for B2B research.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
LeadForgeLeadForge scans newly discovered websites for NewSiteLead, recording technical, SEO, accessibility, and quality signals. Agencies and consultants use the resulting data t…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
LeikibotLeikibot is a web crawler operated by DoubleVerify that analyzes webpage content for digital advertising verification, brand safety, and ad fraud detection services.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
LightspeedSystemsCrawlerLightspeedSystemsCrawler is an intelligence gatherer operated by Lightspeed Systems.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
linkfluencelinkfluence is an intelligence gatherer operated by Meltwater.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
LinkWalkerLinkWalker is an intelligence gatherer operated by Fortra.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
ListSignalBotListSignalBot visits a small number of public pages per site to identify its technology stack, language, and whether it represents an active business. The data powers Lis…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
MacrobondbotMacrobondbot implements custom web crawlers to fetch macroeconomic and financial data from 2,400+ official data sources for Macrobond's integrated research platform. Gath…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
magpie-crawlermagpie-crawler indexes web content and social media posts for Brandwatch's social media monitoring and consumer intelligence platform to analyze brand mentions.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
MantisbotMantisbot is operated by Mantis. It's used to crawl web pages for contextual advertising and brand safety signals.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
MapTheNetBotMapTheNetBot records links between domains to build MapTheNet's web map, rankings, and link data.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
McontextualBotMcontextualBot is a web crawler operated by MContextual that collects content from websites for contextual advertising targeting and analysis. The company uses the crawle…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
mediaboardbotmediaboardbot is a web crawler operated by Mediaboard, a PR and media intelligence platform. It monitors online media sources for brand mentions, news coverage, and senti…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
MedialogiaBotMedialogiaBot is a media monitoring crawler that collects content from online news sources and social media platforms to provide comprehensive media analytics and brand m…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
MediaMonitoringBotMediaMonitoringBot crawls news and media publisher websites to index new content and match it against customer-provided keywords, delivering automated media monitoring up…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
Mediapartners-GoogleMediapartners-Google crawls website content to analyze page context and keywords for Google AdSense ad targeting, determining appropriate advertisements to display based …Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
MediatoolkitbotMediatoolkitbot is Determ's media monitoring crawler that scans the open internet searching for brand mentions, keywords, and phrases that users track, helping marketers …Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
Mediavine Medatada ParserMediavine Medatada Parser is an intelligence gatherer operated by Mediavine.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
MediavineMetadataParserMediavineMetadataParser is Mediavine's content analysis tool for their Grow platform that enables publishers to add bookmarking, social sharing, and search functionality …Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
meta-externaladsmeta-externalads is a web crawler operated by Meta that scans websites to gather information about content that may be shared or advertised on Facebook's platform. The bo…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
MgidBotMgidBot is MGID's web crawler that analyzes website content to detect context categories and topics for targeted native advertising recommendations and content discovery.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
Missinglettr BotMissinglettr Bot is an intelligence gatherer operated by Missinglettr.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
MixrankBotMixrankBot is a web crawler operated by MixRank that collects data about websites, companies, mobile apps, and technologies. The bot gathers information such as technolog…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
MRLeadsBotMRLeadsBot collects public information about local businesses, deals, coupons, and events for the SavingsSites directory.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
MuckRackMuckRack is a bot operated by Muck Rack, a platform that connects journalists with public relations professionals. This bot likely crawls websites to gather information a…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
NanoInteractiveNanoInteractive is a web crawler operated by Nano Interactive, a digital advertising and targeting company. The bot likely crawls websites to gather intelligence for audi…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
NetcraftNetcraft is a web crawler that conducts comprehensive internet surveys, analyzing web server technologies and infrastructure across millions of websites to provide market…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
NetcraftSurveyAgentNetcraftSurveyAgent is operated by Netcraft, a cybersecurity company specializing in threat intelligence and digital risk protection. The bot crawls websites to build thr…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
netEstate NE CrawlernetEstate NE Crawler systematically crawls websites to extract public contact information and business data for the netEstate business directory platform.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
Neticle CrawlerNeticle Crawler is a web crawler operated by Neticle, a browser-based social and online media monitoring service. The bot collects web content to quantify online opinions…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
NetSeer crawlerNetSeer crawler is a web crawler operated by NetSeer that collects and analyzes web content for intelligence gathering purposes. This bot systematically visits websites t…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
NetvibesNetvibes is a web crawler operated by Netvibes, a dashboard and intelligence gathering service. The bot collects publicly available web content to aggregate and display i…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
NeuwoBotNeuwoBot is a web crawler operated by Neuwo, a company that provides AI-powered contextual advertising solutions. It analyzes and classifies web content to generate conte…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
NewsRoom.BINewsRoom.BI is a web crawler operated by Marfeel, a company that provides analytics and personalization solutions for publishers. This bot likely collects data to support…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
NoahBotNoahBot is a web crawler operated by Noah Wire that collects public news and web content for its real-time news intelligence and editorial services.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
NostoCrawlerBotNostoCrawlerBot is operated by Nosto, a company that provides ecommerce personalization solutions including product recommendations, search personalization, and merchandi…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
OAI-AdsBotOAI-AdsBot is OpenAI's bot that validates the safety and policy compliance of web pages submitted as advertisements on ChatGPT.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
OmvionLeadLakeOmvionLeadLake collects public company and website information for OMVION's lead-generation database.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
osentix-crawlerosentix-crawler collects public web information for Osentix's AI-powered open-source intelligence platform, supporting social media, network, and blockchain investigation…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
outboundbuddy-enrichmentoutboundbuddy-enrichment is operated by Outbound Buddy, a B2B go-to-market growth agency that specializes in helping SaaS and agency teams connect with ecommerce brands. …Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
outbrainoutbrain is Outbrain's web crawler that analyzes content on publisher websites to understand context and topics for content recommendation and advertising purposes.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
OwlerOwler is a business intelligence bot that gathers competitive intelligence and company information data. It crawls websites to collect publicly available information abou…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
page-preview-toolpage-preview-tool is SalesViewer's web crawler that visits websites to generate page previews and enrich company information for sales and marketing intelligence services…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
PandalyticsPandalytics is a web crawler operated by Domainsbot, a company that provides domain-related data and business intelligence services to clients in the domain, hosting, and…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
panscient.companscient.com is an intelligence gatherer operated by Panscient.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
parse.ly scraperparse.ly scraper is a web crawler operated by Parse.ly, a content analytics platform used by newsrooms and content marketers. The bot collects web content data to provide…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
PartnerOptimizer-botPartnerOptimizer-bot is a web crawler operated by PartnerOptimizer, a partner ecosystem intelligence platform that discovers, profiles, and analyzes B2B technology compan…Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →
-
peer39_crawlerpeer39_crawler analyzes publisher pages for context and suitability to support programmatic advertising decisions.Intelligence GathererIntelligence GathererAnalyzes web content for brand safety, competitive insights, and ad targetingSee More →