Based on large statistical models, it can learn what is common.
That is only vaguely correlated with what is intelligent, let alone with what is good.
To figure out what is good, we need to reflect, to weight consequences, to identify benefits and risks to ourselves and to others.
The machine learning systems better understood as stochastic parrots don't do that.
They may seem to, when they're trained on datasets that embody our own collective intelligence.
But it's a lossy process. It's not just that it doesn't advance wisdom, it rather dilutes it in noise.
As training datasets encompass more and more data produced by such intelligence-diluting processes, the pearls that would advance our collective intelligence will become ever harder for us, the reflecting beings, to identify, select and promote in the sea of stochastic noise generated by the artificial parrots.
This vulnerability doesn't affect only these brute-force algorithms. Humans too are vulnerable to it. Social media filter bubbles, much older propaganda techniques and cults exploit just that: if we're surrounded by people (or bots) voicing a similar position, we tend to take that position for granted, as a reflection of collectively-selected wisdom.
It takes huge effort to overcome peer pressure and maintain and advance the unusual, surprising, even unreasonable positions that progress depends on.
Stochastic parrots, with all their computing power, can't crack us out of these serpent eggs: being unable to reason, they can only further entrap us, reinforcing already common sedimented positions.
They won't figure out that we've been fooled and warn us. They won't find solutions for climate catastrophe, for war, for famine, for inequity, for prejudice, for exploitation.
They are artificial parrots made to identify prevalent patterns and to boost them.
The prevalent systems of beliefs that they are made to reinforce are the ones that brought us these problems.
The pearls we wish and hope for are too rare to catch the stochastic parrots' algorithms' attention, and the extra noise the parrots output won't make the pearls any easier for us intelligent beings to find.
So blong... ⬆