2026-09-02 09:48 UTC
@anthony@indieweb.social @gabrielesvelto@mas.to
Allowing malicious crawlers to steal content and use local resources, without permission, is a very bad business and security position.
Replies (1)
-
@gabrielesvelto@mas.to 2026-09-02 09:51
@simonzerafa@infosec.exchange @anthony@indieweb.social it's a bit of both really. Just today I was looking for some stuff for a friend to buy and being barred from a page prevented me from doing it, that's the downside for the business. The upside is that crawlers can literally make your web page functionally offline, effectively closing it down for any customer. Barring access from abroad might limit your market a bit but if it keeps your regular customers around then that's a reasonable compromise.