2026-09-02 09:51 UTC
@simonzerafa@infosec.exchange @anthony@indieweb.social it's a bit of both really. Just today I was looking for some stuff for a friend to buy and being barred from a page prevented me from doing it, that's the downside for the business. The upside is that crawlers can literally make your web page functionally offline, effectively closing it down for any customer. Barring access from abroad might limit your market a bit but if it keeps your regular customers around then that's a reasonable compromise.
Replies (1)
-
@anthony@indieweb.social 2026-09-02 09:54
@gabrielesvelto@mas.to @simonzerafa@infosec.exchange I don't see how doing country based filtering helps with AI crawlers. A lot of them are using botnets (also called "residential proxy") to get around any IP based filtering. There are other techniques to block AI crawlers that are less disruptive for your audience.