There’s a pretty significant difference in request rate. A tool trying to search and summarize will hit a search engine once, and each website maybe 5 times (if every search engine link points to the site).
A bot trying to scrape content from a website can generate thousands or tens of thousands of requests per second.
It isn’t opt in.
You can block all bot page scraping, and also block user initiated AI tools or you can block no traffic.
There isn’t an option to block bot page scraping but allow user initiated AI tools.
Because, as the article points out, Cloudflare is not able to distinguish between the two
Thats not true, I just viewed my panel in CF, and Perplexity is an optional block, which by default is off.
They must be A/B testing a new feature then, it’s not on mine
Log into your dashboard, click “AI Audit”, and you’ll see the toggles.
There’s no appreciable difference on how they affect systems between the two for site owners.
There’s a pretty significant difference in request rate. A tool trying to search and summarize will hit a search engine once, and each website maybe 5 times (if every search engine link points to the site).
A bot trying to scrape content from a website can generate thousands or tens of thousands of requests per second.