Bonum Certa Men Certa

The Great LLM Delusion - Part IV: Academic Papers as Microsoft Marketing for LLMs

posted by Roy Schestowitz on Jan 31, 2024

Photo capturing two very different architectural styles for student buildings at the UCLA Campus, California

"Strange and misleading article about LLMs," as explained by an anonymous contributor

THIS series coincides with Microsoft hype and vapourware (much-needed distractions). This part is a guest post of sorts, an unedited version of something a reader sent us. The supporting material is in there.

To be clear, you can talk to a chatbot. The chatbot won't talk back to you. It'll just spew out words, some of them merely plagiarised based on words similar to what you said (which the chatbot does not grasp, it's more like a Web search, except there's no attribution/link to source). Chatbots might lower the bar for journalism - to the point where Web as a whole will lose legitimacy, trust, value etc. Then what? Going back to physical libraries? Saying you can compose a physical book using a chatbot is like saying you can make a very large meal by assembling trash and cooking parts of it (LLMs are "digital pagpag"). Given reports like "Scammy AI-Generated Book Rewrites Are Flooding Amazon", this is already a real problem. "These 'AI' stock increases based on fake increases in revenue," an associate has remarked, "appear funded by mass firings to appease the LARPers in the financial community, That can only go on so long before they run out of people to take care of the core income-generating activities, a line which I suspect they have already crossed."

Will Microsoft also start spewing out "papers" or "publication" made by its chatbots, in order to generate hype about chatbots? That probably would not work, as the quality would not meet basic criteria.

Without further ado, here is the contributor's message:


I stumbled upon a recent article you may find curious.

While reading comments on a post at Bruce Schneier's blog, I saw a user who posted the following link as a kind of "proof" that conversations with LLM-based chatbots can be "useful" and "interesting."

Of course, it sparked my interest at first, but as I started reading it, red flags started to pop up here and there.

I do not know much about Quanta Magazine's credibility. At first, I thought that it was some semi-crackpot pop science news site, but after a shallow search, I saw a good rank from a fact-checking site.

The article was published on January 22, 2024, and the research it discussed was released on October 26, 2023. May be it does not mean much—just a few months—but it is a bit suspicious that the research paper is apparently not peer reviewed (just published on arXiv and cited in ~2–3 sources), and the article about it came out in parallel with "AI" swindle failure unraveling.

It seems like the article is desperately trying to spark new interest in readers regarding LLMs and chatbots, saying that there is some evidence that there is "much more than just autocomplete."

Following are some dubious parts.

1. The article talks about the "understanding" of something by LLMs but presents no clear definition of it.

The thing that can pass as a semi-definition (from the research paper)—"combinations that were unlikely to exist in the training data"—is, in my opinion, misleading for ordinary people. Much like other misnomers in the field (e.g., "hallucinations").

I guess it may be suitable to talk about "competence" instead, as in the "competence without comprehension" phrase from Dennet's writings.

2. The paper described in the article seems to support (or go in the direction of) the vague idea that if you shovel a lot of data and complexity into "AI" (LLM in this case), then "something" will emerge ("skills" and "ability to generalize" in this case, as stated in the paper and researcher's comments in the article). I find it concerning.

3. "Research scientist at Google DeepMind" among the authors of the paper, so it is probably not clearly independent (from corporate influence) research.

4. “[They] cannot be just mimicking what has been seen in the training data,” said Sébastien Bubeck, a mathematician and computer scientist at Microsoft Research who was not part of the work. “That’s the basic insight.”

Wait, what? Why is this part inserted in the article at all? Some guy from Microsoft is eager to tell us that LLMs are "something more." No bullshit. What a surprise!

5. The paper starts with this passage: "With LLMs shifting their role from statistical modeling of language to serving as general-purpose AI agents..."

I mean, what the fuck?! LLMs are not "shifting" anywhere; they are poorly shoehorned into use cases where a "general-purpose AI agent" is required (whatever it is, it does not exist in our reality anyway) by people who want to reap profits from selling half-assed "products" based almost entirely on lies! LLMs are definitely not suitable for general-purpose tasks other than text manipulation or some kinds of entertainment where facts, preciseness, and responsibilities do not matter at all.

One of the researchers acknowledges that it is not about accuracy.

"Arora adds that the work doesn’t say anything about the accuracy of what LLMs write. “In fact, it’s arguing for originality,” he said. “These things have never existed in the world’s training corpus. Nobody has ever written this. It has to hallucinate.”

I need to make it clear: I have no competence to review the actual paper; this task requires actual experts in the field.

As far as I understand the paper, the researchers devised some abstractions to describe observations they already made and try to construct a method that would be useful to work with their definitions and hypotheses that have a little in common with laymen's definitions (e.g., for terms like "understanding" and "creativity") and perceptions of the matter.

I tried to read the paper with an open mind to avoid at least some obvious biases. I have no problems with the paper; maybe it is actual useful research that will serve to advance the field (and not the companies of con artists)—I cannot say for sure.

What bothers me are the misnomers, misleading, and vague terms and descriptions in the paper (less) and the article (a great deal) based on it. In my opinion, the article commits the crime of severely misinforming the reader. █

Other Recent Techrights' Posts

“Losing Huang Would Introduce a Lot of Uncertainty"
Huang won't last much longer
Intercontinental Brigading - Part XXXXV - The Brand (and Reputation)
"Russell Brand sentenced to 10 years and six months for rape and sexual assault"
Protestants at the EPO Fight On, Rising in Opposition to and Against Corruption, Also Against GAFAM
Don't say we didn't warn you...
When Some Footage Causes Your 'Company' to Lose Almost Half a Trillion Dollars in 'Value' in a Few Minutes Your Company is Fictionally Valued
stock slips 8% because the CEO cheats on the wife
Links 09/11/2026: Nvidia CEO Jensen Huang Cheats on Wife With A Suspected Chinese Spy, "400,000 Russians Killed in Battle Since 2022 Invasion"
Links for the day
 
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Monday, November 09, 2026
IRC logs for Monday, November 09, 2026
Gemini Links 10/11/2026: Open Problem in Computer Science Due to Bloat and "Queueing Nine Hours for an iPhone"
Links for the day
GNU/Linux Rose to 6% as Laptop/Desktop Operating System in Azerbaijan
In the past two weeks the average measure as seen by Clownflare is 5.7%
Microsoft Will Pay $11,742 USD for Workers to Leave
11 lakh Indian Rupees
Since Records Began 18 Months Ago, statCounter Saw Chatbot Usage Falling Sharply
Chatbots are not becoming more popular and the commercial world is having second thoughts
Losing Their 'Faith' in Slop
Are leather jackets becoming a "signal" of "cheating on the wife"?
Boosters of Non-reciprocal Licensing Are Just Trying to Cancel Richard Stallman (RMS) and His Baby, GNU
If only they liked freedom as much as they love Microsoft (or Microsoft's GitHub, which is proprietary and unsafe)
Gemini Links 09/11/2026: Zephyr, Wireless Planet, and Old E-mail
Links for the day
IBM: From Illegal NDAs to Illegal Secret Settlements (Money in Exchange for Promise Not to Sue)
Insiders also explain why RAs may come weeks from now [...] IBM has marked 40,000 workers it wishes to get rid of
Links 09/11/2026: 'Humanitarian' Putin Knocks 600,000 Civilians Off the Grid, Eminem’s Publisher Sues Suno
Links for the day
Like Crooked Politicians, The Register MS is Selling "Dinners" (Access, Sponsors)
Talk about "hey hi".
The Register MS is Not Objective Regarding "AI", It's Paid by Pushers of the Pyramid Scheme (including Nvidia)
Some time soon we'll name some companies that pay millions to The Register MS in order to cement their role in the house of cards (while it lasts)
GAFAM Moribund
A lot of what we say about Microsoft in respect to debt is equally applicable to the rest of GAFAM
NVIDIA as a Scam Company Run by Scammers Who Fake Demand for Their Waste of Energy (Subsidising GPU Sales for Use Cases That Don't Exist)
First they targeted gamers, then fake 'currencies' (which turned out to be a bust), now it's slop (obfuscation of plagiarism using brute force and fusion) [...] We're quite glad that Huang publicly broadcasts how untrustworthy he is
Zero Tolerance for Slop
very few sites that dabbled in slop ended up well off (for it)
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Sunday, November 08, 2026
IRC logs for Sunday, November 08, 2026
Gemini Links 09/11/2026: "The Blue Wave Won't Save Us" and "Discoverability Is Only Solved If You Can Be Discovered"
Links for the day
IBM 'Robbery': No More Car Allowance in the UK, Severance Shredded, CEO Pay Up 51% in One Year While Thousands Laid Off in Secret
Why does a CEO reward himself so much when the company does so badly?
Digitalisation Is Not Inevitable or Unavoidable
Nor inescapable
Gemini Links 08/11/2026: "Back", Zot, and Information Overload
Links for the day
Intercontinental Brigading - Part XXXXIV - Why the Swatting Failed
After that we only ever heard from or spoke to cops when we invited them to our home to collect evidence
“Concerned Microsoft Workers” Say “Working at Microsoft Has Become Joyless and Demoralizing.” (That's What Microsoft Wants, More Will Leave That Way)
Many Microsoft employees are going to leave
The Register MS is Losing Hundreds of Thousands of Pounds a Year While Directors Pay Themselves Hundreds of Thousands of Pounds a Year in Dividends
they take money from slop pushers and have clearly compromised some journalistic standards in the process
GNU/Linux Measured at 7.4% in the UK This Weekend: Clownflare Radar
I hope that one day GNU/Linux will be everywhere here
Never a One-Person Project
Just because I'm the main "public face" of the site (who takes all the insults and SLAPPs) doesn't mean I deserve all the credit
“I Always Expect to Lose. I Fight Anyway, and Sometimes I Win.” — Richard Stallman
To change the world meaningfully people must think big
Stop Looking for News About "Layoffs", Layoffs Are Over (They've Been Rebranded)
To properly assess the status quo we need other yardsticks
Links 08/11/2026: Bosses Who Were Forcing Their Workers to Use Slop Change Mind, "Rapper RBX Files Ninth Circuit Appeal in Fraud-Focused Class Action Lawsuit Against Spotify"
Links for the day
Links 08/11/2026: Why Vikings Stayed in Iceland and "TikTok Accused Of Promoting Sex Industry To Young Girls"
Links for the day
Gemini Links 08/11/2026: "Before Sunrise" and LLM Scrapers Versus Self-Hosting (or Web in General)
Links for the day
Intercontinental Brigading - Part XXXXIII - Weaponising Cops in Another Continent (Europe), Knowing They Cannot Come to Your Home in America, While Threatening Women
This year it culminated in death threats
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Saturday, November 07, 2026
IRC logs for Saturday, November 07, 2026