2026-09-15 16:38 UTC
@internetarchive@mastodon.archive.org
Too late. A site I use regularly has already blocked the internet archive, because too many AI scrapers were using the archive as a vehicle to scrape sites (which increased the cost of running the site).
So now I can no longer archive this site for human purposes, even on special occasions.
This has seriously damaged your ability to preserve what is left of the human created web. (As I am sure this is not the only site to respond this way.)
Replies (2)
-
@aburka@hachyderm.io 2026-09-15 16:40
@5ciFiGirl@indieverse.social @internetarchive@mastodon.archive.org that doesn't make any sense... scraping a site by looking at the archive wouldn't incur any load on the original site, so blocking the archive wouldn't help
-
@zer0unplanned@friendica.rogueproject.org 2026-09-15 17:32
@5ciFiGirl@indieverse.social @internetarchive@mastodon.archive.org unless you are a robots.txt it won't block you at all with that function it blocks big corp AI crawlers. Get things right please.