What is scraping@nytimes.com?

scraping@nytimes.com's traffic pattern depends on the intelligence being collected. It may revisit specific pages on a schedule, crawl broader sections of your site, or appear only when information is requested. Its request scope and frequency reflect the underlying objective.

scraping@nytimes.com is expected to follow robots.txt rules, so a disallow rule is the appropriate first step if you want to block it. Automatic Robots.txt can add and maintain that rule, while Agent Analytics lets you verify whether scraping@nytimes.com respects it.

Possibly, but its category does not establish that purpose. scraping@nytimes.com's operator may use or provide the collected content for AI training or for unrelated automated tasks.

Not directly. scraping@nytimes.com's expected purpose is not to supply an AI search index or retrieve pages for AI-generated answers.

No special access. scraping@nytimes.com can read publicly available pages but cannot bypass authentication. Assume anything you publish openly may be collected.

Agent Analytics tracks scraping@nytimes.com visits in real time. You can also check your server logs for requests whose user-agent string contains "scraping@nytimes.com". Traffic may range from repeated requests to the same pages to broader crawls or one-off fetches. Because scraping@nytimes.com does not publish a verification method, any client can claim its identity and a log match is only a clue.

The New York Times operates separate agents for different purposes, so each agent should be evaluated and controlled independently.

• nyt_scraping: nyt_scraping is a web scraping bot operated by The New York Times. It is used to collect content from websites for The New York Times' purposes.
• New York Times Newsgathering: New York Times Newsgathering is an uncategorized agent operated by The New York Times.