The Great LLM Delusion - Part I: How Microsoft is Looking to Distract From Profound Erosion of Real Value (Not Pump-and-Dump)
Today we start a new series. Over the coming few days we'll highlight views and some pseudo-'news' about LLMs, which are marketed as "AI" rather than just chatbots which aren't even that good and lack commercial value in most areas they're being marketed for.
Microsoft's quarterly festival of fake 'results' is only a day away. People whom I spoke to over the telephone tell me Microsoft will probably fixate on "HEY HI" (AI) hype to mostly distract from how badly almost every division has been doing, continuing a downward trend (rapid decline). What is this "HEY HI" thing anyway? Just about anything they wish to label or relabel as "HEY HI", just like they did with servers and "clown computing". Faking "growth" by nymshifting is a form of commercial cannibalism.
This week's series will thus coincide with a sea of puff pieces about "HEY HI". Recommended readings on this kind of hype include this recent (last month) article about "HEY HI" being a bubble, based purely on hype.
But heck, what do I know about it anyway? It's not like I'm a coder or something, right? It's not like I spent over 20 years doing machine learning, right? Actually, I did.
As a student and as a postdoc I worked a lot with what the media now calls "AI", going back to 2002 when I made computer games with OpenGL and needed to work on some "logic", e.g. in my undergraduate project. So I think I have a good grasp of this area. My Ph.D. thesis (which I started writing in 2005, i.e. 19 years ago) covers the mathematics behind machine learning as applied to analysis of brain images, acquired typically by MRI.
Later work I did included more hands-on applications with OpenCV, Android, and Octave.
I can agree wholeheartedly with the text contributed to us by another person who understands the subject matter and doesn't just "name-drop" the term, in effect parroting narratives from the mainstream media, sponsored primarily though advertisers, i.e. Microsoft, Google and so on. The contributor's words will form future parts of this week's series.
Regarding the whole "illusion" (we shall use the word delusion in this series), Baldur Bjarnason did a whole book about it and, quoting his outline: "Most of the hype is bullshit. AI is already full of grifters, cons, and snake oil salesmen, and it’s only going to get worse."
This past weekend was the first time I heard of this book, but so many con artists infested the media with "AI" hype. So the summary seems like a fresh breath of air.
In December I overheard two men in the tram talking about "AI", so the media clearly controls people's offline vocabulary and that alone is rather troubling. By changing language they can change perceptions and conversations. Suddenly servers become "the cloud" and security breaches are painted as just "fucking Russians". Breaches are now "attacks". We try not to cover the buzzwords too often as it mostly magnifies the 'hype cycle' and the newspeak.
In the next part, scheduled for tomorrow, I shall explain my own understanding and interpretation of LLMs as a concept. Sentence generators are something I've studied for about 20 years, not counting rudimentary things like tokenisation, which I dealt with as a programmer 23 years ago. The next part, "What Are LLMs Anyway?", won't use very technical language. Alex Oliva, who is a programmer, followed Richard Stallman's take on this matter two months later with a metaphor ("Stochastic Parrots"). We have heard of other metaphors such as "plausible sentence generators" and LLMs being a unconventional "digital pagpag". "The more I read about 'pagpag' the more the metaphor makes sense for LLMs," one person told us after the analogy had been floated last week. █