The Bots Are Ruining the World Wide Web
TODAY the site is serving about 10 requests per second, but most of them are requested by annoying bots. Thankfully, however, this time they interact with and receive static pages, not PHP and there's no back-end database. It makes the nuisance less costly, but that does not make it go away.
The amount of bots out there - whether they're search bots (e.g. googlebot) or for some chatbot - is simply not acceptable. Last week we quoted a new article as saying that about half of all traffic is bots (citing a firm that studies this because it tackles traffic shaping).
Boasting about bot traffic is easy; Twitter did for many years to fake its worth (valuation) and when Elon Musk put a bid to take over we agreed with his assessment that he had been suckered into buying somewhat of a botfarm, feeding machines more than it feeds people. He has since then claimed to be tackling the issue, but paywalls and walled gardens shut out plenty of actual humans who just wish to access the site anonymously.
Musk is a fool (full of it!) and a fraud, no less than Twitter is.
After upgrading the site we rarely have to deal with RAM and CPU issues; even lots of bots hammering away at the back end cannot generate much but/except bandwidth excesses, sans the Apache overhead (requests over TCP/IP still take some computation to serve).
The way things are looking on today's Web, as noted moments ago in the sister site, is rather discouraging. Gemini has its share of bot traffic too, but the overhead associated with Gemini requests is low (little footprint).
We encourage people to talk more about this "Web pandemic" (and picking ClownFlare, which is cancer on the Web, only creates additional issues!). The mainstream media does not like highlighting the issue because that lessens its appeal or its ability to fake its reach (audience), hence its supposed importance (which matters to their advertisers). These things will continue to exacerbate, partly because of so-called "solutions" like "attestation". The future looks ugly and people-hostile.
More and more sites will shut down unless this issue is brought under control. Imposing challenges and JavaScript on visitors isn't a solution but an additional (new) problem. █