Known Agents for Publishers
See all your bot traffic, block scraping, reduce CDN costs, and grow AI chat referrals.
A simple, all-in-one solution to manage bot traffic
Easy setup on any platform
Don't see yours? Let us know, and we'll add it for you.
AI made bot traffic business-critical
- Almost half of your traffic now comes from bots, including AI agents, scrapers, assistants, search crawlers, and data providers.
- Bots are scraping your content and driving up server costs. Full visibility and control over their activity is strategically necessary.
- Humans will increasingly send AI agents to your website on their behalf. If you don't support them, you will lose real customers.
- AI chat is becoming a major acquisition channel like search and social. If you can't measure referrals, you can't grow them.
Why publishers choose Known Agents
- Known Agents is often significantly less expensive than enterprise alternatives.
- Get granular breakdowns by bot, category, and page, with two-year retention, exportable CSV reports, and MCP access.
- Unlike JavaScript-based analytics, Known Agents uses server-side tracking, which is more accurate when ad blockers are enabled.
- Known Agents works with your existing paywall or firewall without risky DNS changes.
Products
Agent Analytics
AI Chat Referral Tracking
# Known Agents (updated 1 hour ago)
User-agent: GPTBot
Disallow: /
User-agent: DeepSeekBot
Disallow: /
User-agent: CCBot
Disallow: /
...Automatic Robots.txt
// Response
{
"agent_token": "Claude-User",
"agent_type_name": "AI Assistant",
"operator_name": "Anthropic",
"result": "verified"
}Agent Identification API
How to start your free trial
We're happy to offer a free trial to all potential Enterprise customers.
This process is self-serve, so feel free to connect your website immediately. You can also request a demo or contact us directly.
Step 1: Create an Account
- Sign up with your work email
- Create a project with your website's name and domain
- Invite team members to help manage the project
A Known Agents account manager will see that you created an account, and remove the Free plan event limit before you hit it.
Step 2: Connect Your Website
Each option below is independent and optional. Set up only the products you want to use, in any order.
-
Start Tracking Agent Visits and AI Chat Referrals
- Follow the Send Visit Events guide to enable Agent Analytics and AI Chat Referral Tracking.
-
Get a Full-Coverage Robots.txt
- Follow the Set Up Automatic Robots.txt guide to enable Automatic Robots.txt and keep your rules current.
-
Start Blocking Bad Bots
- Follow the Block Bad Bots guide to block unknown, spoofed, and disallowed bots.
Frequently asked questions
Can I get started without talking to sales?
Yes. Sign up with your work email to start a free Enterprise trial. You can also request a demo or contact us directly.
Which platforms does Known Agents work with?
Known Agents works with any platform. Use a setup guide for WordPress, WordPress VIP, Cloudflare, Fastly, Akamai, AWS, Google Cloud, Vercel, or Node.js, or connect any backend through the REST API. No DNS changes are required.
How long does setup take?
Most integrations involve copying and pasting a simple configuration, which takes only a few minutes. Once connected, data starts appearing immediately.
Do I have to set up every product?
No. Each connection is independent and optional. Set up only the products you want to use, in any order.
Does Known Agents work with my existing paywall or firewall?
Yes. Known Agents works alongside existing analytics, monetization platforms, paywalls, and firewalls.
How much does Known Agents cost?
The vast majority of Known Agents users are on the Free plan. Standard plans are available for medium publishers, with usage-based Enterprise plans for large publishers.
How does Known Agents handle data processing and subprocessors?
Known Agents acts as a data processor for customer analytics and event data. Our Privacy Policy explains how data is handled, and our standard DPA covers security, subprocessors, international transfers, and deletion.