← Back

Generate a User Agent String List With the REST API

Request

Call this endpoint to get all known user agent strings for agents in the requested categories.

Endpoint

HTTP Method POST
URL https://api.knownagents.com/user-agent-string-lists
Availability Enterprise plans

Headers

Header Value
Authorization A bearer token with your project's access token (e.g. Bearer YOUR_ACCESS_TOKEN). You can get your project's access token by navigating to the Known Agents Projects page, opening your project, and opening its settings page.
Content-Type Set to application/json.

Body

Field Type Required Description
agent_types string[] Yes An array of agent types whose user agent strings should be included. Allowed agent types include:
  • "AI Agent"
  • "AI Assistant"
  • "AI Coding Agent"
  • "AI Data Provider"
  • "AI Data Scraper"
  • "AI Search Crawler"
  • "Archiver"
  • "Developer Helper"
  • "Fetcher"
  • "Automated Agent"
  • "Intelligence Gatherer"
  • "Scraper"
  • "SEO Crawler"
  • "Search Engine Crawler"
  • "Security Scanner"
  • "Undocumented AI Agent"
  • "Uncategorized"
excluded_agent_ids string[] No An array of agent IDs that should be excluded. You can find an agent's ID in the Metadata section on its page in the Agent Directory. You must be logged in to see the Metadata section.
excluded_operator_ids string[] No An array of operator IDs whose agents should be excluded. You can find an operator's ID in the Metadata section on its page in the Agent Directory. You must be logged in to see the Metadata section.

Example

curl -X POST https://api.knownagents.com/user-agent-string-lists \
-H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
-H "Content-Type: application/json" \
-d '{
    "agent_types": [
        "AI Data Scraper",
        "AI Data Provider",
        "Scraper"
    ]
}'

Response

The endpoint returns an array of user agent strings in application/json format.

Example

[
    "Mozilla/5.0 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot",
    "ClaudeBot/1.0",
    "meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)",
    "CCBot/2.0 (https://commoncrawl.org/faq/)",
    "Scrapy/2.17.0 (+https://scrapy.org)",
    "Mozilla/5.0 (compatible; Bytespider; spider-feedback@bytedance.com)",
    "Mozilla/5.0 (compatible; cohere-ai/1.0; +https://cohere.com/crawler)",
    "Mozilla/5.0 (compatible; Diffbot/0.1; +http://www.diffbot.com)",
    "omgili/0.5 +http://www.omgili.com/Crawler.html",
    "Mozilla/5.0 (compatible; ImagesiftBot; +https://imagesift.com)",
    ...
]