LLMs Versus Search (Not Replacing Search But Engaging in DDoS Attacks Against Web Sites That Permit Searching)
Someone asked us about search yesterday. The question then became, "what do we use for search?", knowing we already have search working internally with plans of releasing it to the public (for public access) some time this year.
Bugs aren't the main concern, nor is security. Seeing how LLM slop bots hammer/bang on sites, exposing a search interface (clean and without JavaScript) means exposing oneself to bots generating lots of wasteful CPU and RAM workloads/payloads. We definitely don't want to get back to that.
The state of the Web isn't just bad; it's utterly terrible. Yesterday we mentioned how even the main GNU site was becoming inaccessible or barely accessible. We used to have similar problems before the SSG; there was constantly very high CPU load, mostly due to bots that hammered away all day long and all night long at random pages that aren't (or cannot be) cached. █