What is cohere-training-data-crawler?

Block cohere-training-data-crawler if you want more control over whether your work is used for AI training. Allowing it may increase the chance that your brand appears in AI-generated answers, but either choice leaves traditional search rankings unchanged. For context, 13% of the top websites we track currently have robots.txt rules for cohere-training-data-crawler.

Yes. cohere-training-data-crawler is expected to follow robots.txt rules, so a disallow rule is the appropriate first step. Automatic Robots.txt can add and maintain that rule, while Agent Analytics lets you verify whether cohere-training-data-crawler respects it.

Not through legitimate access. cohere-training-data-crawler primarily targets public content, but some training-data scrapers also attempt to collect gated or paywalled pages. If content loads without authentication, assume it can be collected.

Agent Analytics tracks cohere-training-data-crawler visits in real time. You can also check your server logs for requests whose user-agent string contains "cohere-training-data-crawler". Look for high page counts, short gaps between requests, and deep traversal through linked content. Because cohere-training-data-crawler does not publish a verification method, any client can claim its identity and a log match is only a clue.

Your content matched the criteria Cohere set for a training dataset. cohere-training-data-crawler typically discovers pages through external links, sitemaps, and seed lists rather than because someone selected your site individually.