Apple

Applebot-Extended is an Apple robots.txt control token that lets publishers opt out of using Applebot-crawled content to train Apple foundation models. It does not crawl …

Applebot is Apple's web crawler that gathers data to power search features across Apple's ecosystem including Spotlight, Siri, and Safari search functionality.

AASA-Bot is Apple's crawler that fetches Apple App Site Association files to support Universal Links functionality, allowing iOS apps to handle specific URL patterns.

iMessage is Apple's fetcher that retrieves web page metadata to generate link previews for URLs shared in the Messages app.