Akira Urushibata on the Likely False (Unverifiable) Claims Anthropic Makes About Defects for Marketing/Hype
One month ago: Akira Urushibata on Misleading Numbers From Anthropic's Project Glasswing (False Marketing by FUD Tactics) | LLM Slop Attacks Not Only Sites of Free Software Projects But Also Bug Reporting Systems (Time-wasting, in Effect "DDoS")
Is "Glasswing" just a passing fad? Like most chatbots which turn out not to work as advertised?
Some pro-LLM person has managed to derail the discussion on this topic, so Akira Urushibata has posted (yesterday morning) a brief outline of things to watch out for. It's nice and short (with the numbers put in context). We reproduce this below:
I started this thread to share my thoughts on Anthropic's Claude Mythos and Project Glasswing. Unfortunately the discussion is veering off topic.
Here is a summary of the points I have discussed thus far:
1. Project Glasswing's purpose is to enhance OS/critical system component security through "AI" powered screening. Anthropic/Project Glasswing has not contacted FSF/GNU.
2. Project Glasswing started with thirteen member firms and organizations. Early on 40-50 members joined. The member list has not been made public. We don't even know the exact number of members. (As of this writing it seems there are 150-200 members.)
3. The Linux Foundation is a member of Glasswing. How exactly it is coordinating efforts with system critical "open source" projects is unclear. Ordinary people who have been misled into believing that "Linux" is the entire OS cannot be expected to comprehend the limitations of the Linux Foundation.
4. The Linux Foundation provides a list of what they consider critical system components. The list contains projects that are far from essential while omitting many packages which are well known and rank high in Debian Popcon.
5. Glasswing produced an initial report on May 22. About 1,000 "open source" projects were scanned. Claude Mythos reported 23,019 problems including those of low severity.
6. There are reports of vulnerabilities found in proprietary software in the May 22 report, but unlike "open source" software, summarized figures are not provided.
7. It is unclear to what extent "open source" project maintainers agree with the severity assessments of the discoveries they received. The May 22 report offers some numbers but they are presented in a manner which obscures this issue.
8. Daniel Stenberg, lead developer of cURL, wrote a blog article which offers insight into some of the points raised above.
I am providing information and analysis in the hope that they will help people who have heard in mainstream media reports about the "dangerous" capabilities of Claude Mythos which likely apply to modern "AI" systems in general. Those who work in fields such as government, finance, transportation, and logistics are concerned. But the available information is limited and often provided in summarized form by (or via) experts whose impartiality is questionable, and politicians and business leaders who do not understand software systems quite well.
I would like to point out here we have an example of what happens when Freedom 0, the freedom to run programs, is restricted. The owner can run the program and assess the quality. Those others, deprived of the right to execute can't get the necessary information independently. The owner may be honest, or may be compromised due to a desire to increase revenue. In the latter case the need to stay abreast with rivals may be given as an excuse.
If your friends in say finance aren't begging for an objective evaluation of Claude Mythos it is understandable. They know that outsiders are not well informed. When Freedom 0 does not exist the owner acquires monopoly rights on quality statements. This leads to a perverse condition in which reports, possibly tailored to suit someone's interest, are eagerly sought and consumed.
For those who desire to discuss models which are better than Anthropic's, or "AI" issues in general, I would advise doing so in a separate thread with an appropriate title.
Akira Urushibata
Meanwhile, Anthropic is losing so much money that it makes an exit (letting others shoulder the massive debt and massive losses) and is now accusing other companies of "stealing", which is deeply ironic given what Anthropic does. █
Image source: Young woman with magnifying glass