There will genuinely be a huge problem in the near future—gradually increasing over the next couple of years—(yes this is an em-dash) where a lot of the people who were once considered very useful and intelligent will become toxic and obsolete. Not because some magical AI system will simply replace them, but because of their mindset.
A lot of respected people are actually impostors hiding behind memorized knowledge and data, and that is going to be crushed by language models. It has already been crushed, in fact. But since most people still do not know how to use the current models and tools, they get very bad or mediocre outputs. As these tools improve, even the "I have no idea what I am doing" person will start getting better results despite giving trash input.
This technology is still developing, and quite fast—exponentially, even. It can be predicted that the ability of chat systems and their underlying language models to understand a user's mental capacity will increase. Eventually, these systems will be able to form internal evaluations of their users wherever the patterns are predictable. For example, a person with very low intelligence could be identified over hundreds of conversations, even if those conversations do not involve many personal details. The patterns will still show themselves clearly.
The chatbot will then try to adapt to the user's level of understanding, partly to mitigate the risk of the user ranting about how bad the tool is. So we will probably see less of that. Of course, the user will still be responsible for evaluating the output. But, as I said, a lot of things will become useless quite fast.
What will not become useless—no matter what—is scientific principles, philosophy, and, therefore, engineering approaches. I am not necessarily saying engineers themselves, because, as with most professions, fields, and whatever else people do, we cannot really trust the masses. What we can trust—or at least learn to identify—is the capacity for genuine high-IQ, well-educated, scientifically minded individuals who follow engineering principles, whether or not they come from formal education.
Those individuals will represent the real value in whatever they are pursuing. Even if they are doing something that looks totally unrelated, like music, they will still stand out through their music.
What we are going to see, then, is a total destruction of the illusion surrounding what we value in society. Machines may or may not surpass human intelligence within our lifetime, but there are strong indications that they will. Even now, we have people saying LLMs are “just stochastic parrots,” and some of these people have credentials that allow them to present themselves as machine-learning experts.
But, as the father of neural networks said in his Nobel Prize speech and in interviews afterward, many people have no idea what is going on or what is coming. And as a very good mathematician once said in an online Stanford course I took, most mathematicians really need to go through basic training in mathematical thinking again.
That applies to many other things as well, you know.
So the road ahead is troublesome—but, at the same time, bright as hell.