Another Failed Use Case for Chatbots (LLM): Legal Advice and Analysis
People who try slop get in trouble and professionals who dabble in it get caught
Even before the LLM hype many health professionals cautioned people not to google their medical symptoms (among other stuff) or rely on "Doctor Google" (with or without "summaries", which were 'borrowed' and quoted verbatim from sites). We recently wrote about how chatbots were failing millions of people with phony and very dangerous "medical advice" (slop disguised or falsely marketed as "intelligence"), noting that reckless slop pushers had removed warning messages about their output not constituting medical advice but merely toy output. In our latest daily links we moreover mention a story about how slop pushers not only cause medical problems [1] but are moreover covering that up. "In response to some of our subsequent reporting on ChatGPT-induced psychosis," it says, the Microsoft-connected company pretended to have done something. "In April, it rolled back an update that caused ChatGPT to be egregiously sycophantic, even by its standards."
So basically, to summarise: using chatbots causes medical problems and using chatbots causes people to do the wrong things when they already have medical problems. Worst of both worlds.
What about legal advice? Having written about this a few times lately [1, 2, 3], consider "If You've Asked ChatGPT a Legal Question, You May Have Accidentally Doomed Yourself in Court" [2].
This is also applicable to reporting online. So far today we called out two Web sites that do this, WebProNews and sometimes Linuxiac. "It's rude to show AI output to people," argued a new article [3].
I was rather disappointed when the editor of Linuxiac disagreed with me on this point.
Apparently there are some sites out there that think it's acceptable to spew out LLM output as if it's journalism. The only solution is to shun such sites.
So it remains unclear if there's any real use case for chatbots; they already failed in many businesses, including large ones like McDonalds (both for service and for staff recruitment). They're not going to really replace doctors, lawyers, and reporters. They're just some self-discrediting toy that costs way too much to operate. █
Related/contextual items from the news:
-
OpenAI Is Giving Exactly the Same Copy-Pasted Response Every Time Time ChatGPT Is Linked to a Mental Health Crisis
It's baffling because, on a certain level, OpenAI is acting like it's serious about the issue. Or at least, it wants us to think it's serious. In response to some of our subsequent reporting on ChatGPT-induced psychosis, it said it hired a full-time clinical psychiatrist with a background in forensic psychiatry to help research the effects of its chatbot on users' mental health. In April, it rolled back an update that caused ChatGPT to be egregiously sycophantic, even by its standards.
-
If You've Asked ChatGPT a Legal Question, You May Have Accidentally Doomed Yourself in Court
In response to that massive acknowledgement, Jessee Bundy of the Creative Counsel Law firm pointed out that lawyers like her had been warning "for over a year" that using ChatGPT for legal purposes could backfire spectacularly.
"If you’re pasting in contracts, asking legal questions, or asking [the chatbot] for strategy, you're not getting legal advice," the lawyer tweeted. "You’re generating discoverable evidence. No attorney-client privilege. No confidentiality. No ethical duty. No one to protect you."
-
It's rude to show AI output to people
Whoa, let me stop you right here buddy, what you're doing here is extremely, horribly rude.