Chatbots (or LLMs) Are Killing Us, and We Ought to Talk About It
Not an entirely new problem [1, 2]. We need to talk (to each other, not to bots).
There has long been this issue with people who "google" (v.) their symptoms or "ask Dr. Google" for what to do regarding health. When Google was a Search Engine, it would at least prioritise good resources like WebMD, not slop.
From my understanding, some people nowadays go to pharmacies and ask things based on what ChaffGPT "told them" or what Google slop blurted out. They even follow hallucinated brand names, not just bad/invalid advice.
This is worrying. We're already past the beliefs in witches and all sorts of "alternative medicines" are regarded as a crank thing.
LLMs that lie with confidence and are marketed by Scam Altman (who knows how to lie with confidence; he cannot code, cannot explain what he talks about; Garrett-like personality disorder) as "intelligence". They lead people to self-harming actions which moreover become a burden on real medical professionals. NHS staff is compelled to "argue" against people who "know better" because some chatbot said this and that (even if it lacks any comprehension and merely parrots a bunch of inputs based on word correlation weights).
Two days ago (there are other reports like this):

Do you trust this man to explain your symptoms to you and tell you how to tackle life-threatening conditions?
If you experience health problems, go see a doctor. Do not bother with chatbots as they hurt more than yourself; they're a large systemic problem.
That Microsoft is trying to sell chatbots as a medical solution (while buying companies to gear up towards it with voice dictation, recognition, and synthesis) is worrying given that Microsoft now formally admits its chatbots are only "for entertainment".
Worryingly enough, due to Microsoftism, the NHS has added LLM/chatbot garbage to its Web site. So it is feeding this problem in order to keep feeding Microsoft.
Is anyone "entertained"? █

