Bonum Certa Men Certa

The LLM Ouroboros Phenomenon

posted by Roy Schestowitz on May 19, 2025,
updated May 19, 2025

An ouroboros in a 1478 drawing in an alchemical tract

Ancient Greek mythology came up with this concept of an ouroboros, wherein some animal - typically a snake for a feasible "IRL" (in real life) metaphor - eats itself by eating its own tail. We would not be the first to point out the analogy here for LLMs because an ouroboros is a good parable. This morning we catalogued two BSD and Linux sites complaining about desperate LLM scrapers staging a DDoS attack in pursuit of original, as in human-written, code or words. This isn't a new problem for us and in the past few days we served about half a million pages in Gemini Protocol, likely due to to LLM scrapers. It's obnoxious to say the least, but distinguishing benign from malicious (or worthless junk) requests is hard and a "moving target" (it's never enough as parasites learn to adapt).

This morning in IRC we made an assertion about LLMs and fake (slop) images. We also made several observations. Fact #1: over time slop gets worse (training set is like some blurry JPEG). Fact #2: People's "smell" for slop improves over time, as they 'train' on slop and can detect it based on prior encounters. Put 1 and 2 together.

Are LLMs bound to not only get worse but also more easily detectable by an increasingly sceptical general public? TheLayoff.com has just responded to this.

An associate opines that fact #1 (that slop gets worse over time) is exacerbated by the flood of slop on the Net being snarfed up by newer bots and mistaken for training data. "Thus the feedback loop I mentioned a long time back and which Andy wrote about in depth." (He was referring to Dr. Farnell's good writings about this dilemma - as he did several times in The CyberShow's blog)

To a certain extent my Ph.D. thesis (dissertation) covered this about two decades ago. The associate says that it's a "well-known problem from days of old".

There are several unique aspects to this, including validation bias. To me it seemed a bit related to but not the same as over-training because, as an associate explains, "overtraining is something else: too much data and the patterns become locked too tightly to the training set and less useful for new data".

For an LLM to scan online its own output serves to affirm the mistakes, or the errors, often euphemised as mere "hallucinations", which are innocent, not libellous, and by no means "intentional" and "harmful". Dr. Farnell and Dr. Kate Brown responded to this last October in "Radical disbelief and its causes".

In the context of my thesis (dissertation), a concern was raised about what we back then called "synthetic data" finding its way "back" into the training set. So when you check brain MRI scans (which is what we did back then) you must ensure you only ever deal with real data, not mock or manipulated data that can confirm your own biases and "fit into" the model that generated it in the first place (in generative mode). To use the analogy of text-based LLMs, your BS is "truth" if your input is your own BS (output/s) and it would be deemed accurate, based on you (opposite of the notion of peer review in science). The associate correctly points out, based on a scan of my thesis (dissertation), that the strings "overtraining" and "over-training" are not in the dissertation, but we used different terms back then.

A squat toilet (also known as an Eastern, Turkish, Iranian or Natural-Position toilet). This one is in Turkey

"An LLM Ouroboros of shit", as the associate dubs it, would be statistical models (such as PDMs or AAMs*) treating computer-generated images as something from "the real world".

The so-called "generative hey hi" (genAI) "bros" won't allow the media to talk about such issues, at least if they can downplay the issues and deny/misportray them (in the media). But it's a real and growing problem. Its magnitude likely grows quadratically, not linearly. Just like other bubbles (overabundance based around hype), don't expect linear implosions. When it's gone (poof!), it's gone.

____

* PDM and AAM need expansion in the explanatory sense, not just words (in the acronyms). PDMs go back several decades ago they were invented or pioneered by the people who tutored me. They use mathematical, statistical models to perform multidimensional analysis of data variations, based upon principal component analysis (PCA). AAMs are an extension but with textures, not only points. This is really old stuff; even AAMs are over 23 years old; now the mainstream media pretends those are some kind of "revolution".

Other Recent Techrights' Posts

Microsoft Media Operatives and Bill Epsteingate-Funded Sites Said Microsoft Lays Off 9,000, But Other Sites Say More (Including 2,300 in Redmond Alone)
We might never know the real number/s (Microsoft will keep the cards close to its chest) until there are leakers or unless there are whistleblowers with hard proof
GNU/Linux Reaches All-Time High in the United States of America
Windows is trending down
Why the Microsoft People Who Started SLAPPs Against Techrights Could Very Well be Sent Back to Prison
White-collar crime is also a crime
 
Links 02/07/2025: Deep Microsoft Cuts, Macron Speaks to Putin
Links for the day
Confirmed: Microsoft Shutdowns Today, Not Only Mass Layoffs
"The Initiative is the only studio closure planned today, although some other teams have seen cuts of varying degrees."
GNU/Linux is Replacing Microsoft Windows. But We Need to Eradicate Microsoft, It's a Hub of Crime.
I have been writing about Microsoft since the 1990s when I was in school
Microsoft Windows Nosedives in Switzerland While GNU/Linux Leaps Above 6%
sooner or later they might have to make the move anyway
Anxiety at Microsoft: Many Workers (Maybe Over 10,000) Still Don't Know They're Being Laid Off Just Before US Independence Day
"Has anyone gotten the notification yet?"
Microsoft "Declined to Say How Many People Would be Laid Off," According to Associated Press
Some other prominent publications said they reached out for comment from Microsoft and received none
The X War is Over and the "Wayland People" Lost
People will gravitate towards what works for them
20 Years Since My Thesis
It's still online
GNU/Linux is Replacing Windows in Laptops/Desktops
The world will move on while Windows and Microsoft shrink
Now Comes the Expected Webspam, Framing Microsoft Layoffs as "Hey Hi" Success Story (False Marketing That's Piggybacking the Layoffs)
falsely marketed as "intelligence"
Hungary: Microsoft Windows Sinks to 17% "Market Share"
In many nations in Europe it seems like the era of Windows is coming to an end
Microsoft Layoffs in Spain, Portugal Record for GNU/Linux
in Portugal we see GNU/Linux at record levels
Yes, Microsoft is Again Using Its Favourite Liars (Stenographers) to Seed Fake Layoff Numbers, Much Lower Than What's Really Happening
It is Jordan Novet again, just as we predicted
Will Microsoft Once Again Choose Its Favourite Liar to Spread Lies About Today's Layoffs, Quickly to be Replicated and Spread by Slopfarms?
What lies is Microsoft briefing its media moles to tell today?
"OSS Fetishism" Wins After Ferenc Zsolt Szabó Ousted (Microsoft Mole From Capgemini)
Many people said 2025 would be the "year of Linux on the desktop"
There is Nothing That LLMs Can Offer Honest People
LLMs are a passing fad; they're expensive and offer poor "value" for energy; they usually offer no value at all unless you are a cheater, spammer, and liar
What statCounter Shows Today Helps Explain Microsoft's Helplessness, Mass Layoffs
Since many US journalists are already away on holiday almost nobody will dare ask the difficult questions or give a voice to whistleblowers
Microsoft Gets the Chop in South America
The notion of digital sovereignty gained a lot of popularity
Europe Has an 'Exit'
Let's see what happens the rest of this year
El Presidente Talks, Canada Walks (Away From Windows)
GNU/Linux rising
Cities in France and Germany Move to GNU/Linux and statCounter Detects Big Differences
Will governments lead by example?
Microsoft Lost Its Foothold in Africa
How many of these are "old" Windows machines converted to GNU/Linux? Probably a lot.
Led by Europe, GNU/Linux Makes Big Gains This Month
statCounter started showing new/fresh stats
Links 02/07/2025: Massive Microsoft Layoffs About to Commence, "Tesla's Robotaxi Program Is Failing"
Links for the day
The Company Run by Former (and Last Proper) Red Hat CEO, Promoting Microsoft Mono, Faces Shock as Senior Partner Jailed for 33 Sexual Offenses Including Pedophilia
"As reported by The Oxford Mail in April 2025, the offenses include rape, sexual assault, engaging in non-penetrative activity with a child, and more."
Microsoft Lost 29% of Windows Users, Based on Microsoft, Now Come Massive Layoffs
Microsoft collapse is today
Slopwatch: Google Serves to People Linux Slop and Linux FUD (Made by Bots)
"Slopwatch" finds it difficult to ignore Google's role in encouraging LLM slop
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Tuesday, July 01, 2025
IRC logs for Tuesday, July 01, 2025
"Wayland People" Behave Like the Googles and Microsofts of This World
Published yesterday by Igor Ljubuncic
Gemini Links 02/07/2025: Arch Linux and Fulfillment in Gemini
Links for the day
Links 01/07/2025: "Independence Day in Taiwan", Bounties on Software Patents
Links for the day
What Happens When Your Law Firm is Preoccupied With Harassing and Trying to Extort a Humble Couple in Manchester, Even on Behalf of Violent Microsoft Staff From Another Continent
It's good to see that law firms which operate in bad faith are perishing
Lawyer X, Law Firm X and Elon Musk's X: scandals linked by Old Xaverian
Reprinted with permission from Daniel Pocock
Gemini Links 01/07/2025: Distraction-Free Writing and Hytale Mismanagement
Links for the day
Links 01/07/2025: "Beauty of Blogging" and "Etiquette of Collapse"
Links for the day
Microsoft Uses LLM Slop to Defraud (or Rob) Shareholders
Microsoft is basically defrauding its shareholders by LLM slop
The Web is a Dead End
We need to adopt alternatives
When Words Lose Their Intended Meaning
examples of words that, at least in the technical spheres, don't mean what they sound like
People Who Disagree With You on Technical Matters May or May Not Agree With You on Political Things (But Usually They Do)
What bothers me a great deal is seeing left-leaning people accusing other left-leaning people of being "nazis"
"Too Much Choice" and "Too Many Programming Languages"
What IBM and its apologists aim for was attempted in the 1930s and it failed
Microsoft Lost 400,000,000 Windows Users, According to Microsoft
more people adopt smaller computers and many people replace Windows with GNU/Linux, as they don't really need a new computer
The "Davos Effect": Tarnishing the Reputation of Places Not by Overtourism But by Oligarch Infestation
The last Venice needs is an affiliation with Venetian oligarchs
Half a Year Gone, What's to Come Next
In the second half of 2025 we expect to be done with the Microsoft SLAPPs
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Monday, June 30, 2025
IRC logs for Monday, June 30, 2025
People at the Very Top of Microsoft Know How Bad Things Really Are
There's no product that can replace the former profitability of Windows licensing and stuff that went on top of Windows
Gemini Links 01/07/2025: Mid Year and a Tour of Old Languages
Links for the day
EPO Presentation Bemoans Misuse of Slop in Decision-Making on Patents and in Classification (Which is Likely Illegal Too)
We habitually mention failed use cases of LLMs on the Web
Mass Layoffs at Microsoft Confirmed, "XBox Hardware Is Dead"
It's possible that over 20% of the staff will be laid off
Links 30/06/2025: Kyrgyzstan vs Media Freedom, Dalai Lama Succession
Links for the day
Gemini Links 30/06/2025: Backend Programs in Gemini and Dynamic Content Without The Scripting
Links for the day
Links 30/06/2025: Zuckerberg’s Tax-Evading Scheme Harms Kids, US Copyright Office Lacks Leadership
Links for the day
Microsoft Isn't Laying Off Tens of Thousands to 'Invest' in Slop ('Hey Hi'), It's Laying Off Tens of Thousands Because It's Running Out of Money (and Willing Lenders)
the layoffs are a sign of the business failing, not "hey hi" (whatever that is) replacing staff
Intel Lays Off 20% of Its Workforce, Microsoft is Doing the Same This Year
Like a yoyo, whatever goes up will come back down
Microsoft XBox Layoffs: Almost 2,000 Layoffs Became "Over 2,000"? (Over 20% of the Staff)
over 20% of staff will be let go, not counting staff that leaves voluntarily
GNU/Linux Rises to New Highs in Angola, Africa in General is Abandoning Windows
Western media barely covers Microsoft layoffs in Africa, but in recent years Microsoft culled the workforce and even shut down entire operations
Summer Plans in Techrights and Elsewhere
massive layoffs at Microsoft
Destination Geminispace (in the Age of LLM Slop and Slop Images That Infest the Web and Social Control Media)
Geminispace isn't vast, but at least it is - on average - a lot "cleaner"
GNU/Linux Growing in Sierra Leone This Year
Based on what statCounter is seeing, this year there are more and more people there who adopt GNU/Linux
Serial Sloppers Gonna Slop
More sites out there ought to call out the cheaters
Quartz (qz.com) is Spam and a Slopfarm
It used to be OK. Then they fired the staff.
Links 30/06/2025: US Economic Woes, Extreme Heat
Links for the day
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Sunday, June 29, 2025
IRC logs for Sunday, June 29, 2025
Gemini Links 30/06/2025: "The AI Hype" and New AuraGem Ask
Links for the day