In His Departure Post, Poul-Henning Kamp (phk) Explains Why Linus Torvalds is Full of "Hot Air" and Slop Bubbles
tl;dr Uneconomic garbage, fuelled by hype and bribes (to the media, to Linux Foundation etc.) is a passing fad
phk is signing off at ACM. Before he goes, however, he leaves the words we mentioned earlier about complete system lock-down and the following tidbit about LLMs in software development or fuzzing and plagiarism (unattributed, low-quality copycatting) spun as "hey hi":
There are two hot envelopes in the FOSS inbox right now: LLM-assisted code reviews and age verification.
The LLM-assisted code reviews are causing a lot of headlines because this comes out of one of those economic bubbles that seem to automatically inflate as soon as most people have forgotten how utterly stupid and predictable the previous bubble was.
I’m not particularly worried about this.
I have never felt particularly confident in myself as a programmer, so I have always applied all the “code-quality” tools I could lay my hands on, and the pattern has always been the same:
— Day 1-2. Tried
$toolout. Oh boy! We have some work to do.— Day 3-5. OK, there were a couple of solid bugs there, and a fair number of what were technically bugs, but not actually all that bad.
— Week 2. I guess that was it?
This pattern first manifested when I read the Zilog Zeus reference manual from cover to cover in 1984 (while walking barefoot in the snow on the freeway, etc.) and read about
lint(1).I saw the pattern most recently with Clang’s quite impressive static analysis facilities.
Just on that repeated experience, I suspect we have already seen more than half of the “worst software bugs found with LLM-tools” list.
Supporting that suspicion is that it would beggar belief that AI bros so desperate to blow hot air into their bubbles would not try to make as big a splash as possible, as soon as possible.
(May I kindly suggest that they give their next LLM model the entire design and as-built documentation for a nuclear power plant? That is guaranteed to give them a lot of publicity and will maybe teach us something about the limits of human engineering complexity.)
When I look at the nature of the bugs found by LLM tools, I see clear parallels to computers playing chess: The LLMs are able and willing to explore the tree both wider and deeper in detail than human brains evolved to economize energy and time, by probabilistically and heuristically searching for food, sex, and comfort only where the odds look most promising.
That’s good.
Cyberattacks and defense are by nature a chesslike game, and we also badly need better tools to write the programs that we will increasingly trust with our lives in the future.
The only real question for me is: Are the LLM-code-review tools economically viable outside the bubble?
All the work, and all the cost, comes up front, during “training” of the model. But the resulting artifact, the weights of the model—which you have to sell many million times over before you turn a profit—easily fits on contemporary pocket-sized storage devices.
This is not unlike a major film or video production where millions are spent on sets, actors, costumes, special effects, and marketing to create a 4K video stream that fits on pocket-sized storage devices, and profit is made in increments of fractions of a teenager’s allowance.
Where [H/B]ollywood has somewhat successfully managed to wrap itself in a righteous cloak of copyright protections, the LLM industry may have trouble doing that, given how religiously they recite “fair use” in the many lawsuits where they are accused of infringement of copyrights.
So, it is not obvious to me who will be training new iterations of these models once the current bubble explodes, in particular if the returns are diminishing the way I have experienced.
Having lived through previous bubbles as a UNIX geek, phk cannot be ignored. He knows very well why the Web/Internet/dotcom bubble happened. █
Image source: Sony