← Home
Product Overview
Bad Bot Blocking
Stop unwanted bots from accessing your content, even when they ignore robots.txt or spoof trusted identities.
Unidentified Bot
/pricing
Pageview
GET
HTTP 403
Forbidden
The request was rejected by your server.
This visit was blocked
5 seconds ago
Disallowed
Non-spoofed pageviews from agents that ignored your robots.txt rules.
20.15.240.80
China
Unidentified Bot
/docs/api
Pageview
GET
HTTP 403
Forbidden
The request was rejected by your server.
This visit was blocked
9 seconds ago
Disallowed
Non-spoofed pageviews from agents that ignored your robots.txt rules.
20.15.240.80
China
Some bots will ignore your robots.txt rules
Robots.txt sets the rules for well-behaved bots. Bad Bot Blocking enforces them against bots that refuse to comply.
Choose exactly which requests to reject
Robots.txt Enforcement
Reject bots that ignore your disallow rules
Spoof Protection
Block requests that fail identity verification
Unidentified Bots
Stop automated traffic that cannot be identified
Path Protection
Protect sensitive or expensive routes
Keep unwanted bots away from your content
Block disallowed, spoofed, and unidentified traffic before it can scrape content or consume server resources.
Enforce your policy with confidence
HTTP 403
Returned to disallowed bot requests
Robots.txt Enforcement
Return HTTP 403 to bots that violate your disallow rules.
Spoof Protection
Reject requests that fail verification instead of trusting a claimed identity.
Known Agent IP
The IP address is associated with a known automated agent.
Inconsistency
The request contained conflicting browser or protocol signals.
Non-Browser
The user agent identifies software other than a conventional web browser.
Automated Browser
The request showed signs of a browser being controlled by automation software.
Residential Proxy
The IP address was identified as a residential proxy.
VPN
The IP address was identified as a VPN exit node.
Cloud Service Provider
The IP address is associated with a cloud service provider.
Unidentified Bot Fingerprinting
Detect automated traffic even when it does not identify itself.
Protect your content from bad bots in under 5 minutes
Enforce your policy with a platform integration or the REST API. Available with every paid plan. New accounts automatically receive a premium features trial.
