Why AI is not intelligent at all, and therefore can’t speak, reason, hallucinate, or make errors

(This is a slightly extended version of my June column as “der Wissenschaftsnarr” in the German Laborjournal: “Kritik der schwätzenden Vernunft“)

The ongoing debate whether ChatGPT et al. are a blessing for mankind or the beginning of the reign of the machines is riddled with metaphors and comparisons. The products of Artificial Intelligence (AI) are “humanized“ by means of analogy: They are intelligent, learn, speak, think, reason, judge, infer decide, generalize, feel, hallucinate, are creative, (self-)conscious and make errors, are based on neuron-like structures, etc. At the same time, functions of the human brain are described using terms like computer, memory, storage, code, algorithm, and we are reminded that electric currents flow in the brain, just like in a computer. Befuddled by the astounding achievements of chatting and painting bots, many now argue that generative AI displays features of “real” intelligence, and that it is just a matter of more programming and time until AI surpasses human cognition.

The camp of those who think AI is intelligent proves its point with a long list of what AI can do that all look pretty intelligent. The doubters, however, are not convinced; they complain that AI still lacks certain “functionalities” of intelligence, following Tesler’s theorem: “AI is whatever hasn’t been done yet.”

In the following, I will argue that the current AI debate is missing the point, completely.  Instead of simply marveling at AI’s putative intelligence, we should ask what intelligence, thinking, language, consciousness, etc. actually are – to measure AI against them.    

Fortunately, more than 200 years ago, someone had already  very deep thoughts about these mental activities. It was not Immanuel Kant, as the title of this post might suggest, but his critic, Georg Wilhelm Friedrich Hegel. Unfortunately, his writings come in a language that is quite difficult to digest. I will, nevertheless, try to pick some of his thoughts apart to accomplish something, that – Spoiler Alert! – AI cannot do: To deduce from the concepts of language and thinking, why AI cannot speak and think.

The question at stake is: Can a computer think with zeros and ones, can it develop into a mental subject by means of AI (or has it perhaps already done so)? A “mind” which might evolve an understanding of the world? And consequently, its own free will, with all its potentially catastrophic consequences for mankind?

Let us start on the level of the computer, more precisely on the level of the transistor. For the computer, and thus AI, a “word“ is nothing but a sequence of two physical states, the “on” or “off” of a silicium switch on a semiconductor. The engineers who built and programmed the chips have assigned symbols to these states, namely 0 and 1. Numbers because we can calculate with them. This is also the reason why the device is called computer: Because it – and that includes your cell phone, is nothing else than a programmable calculating machine. In many intermediate steps, which only have meaning for us humans, the program code assigns words to certain sequences of the number symbols. To call the algorithms performing the final steps of these operations “neural networks” is a brilliant marketing ploy, just as effective and false as the term artificial “intelligence” itself. Why? Because in fact, artificial neural networks are nothing but mathematical formulas that compute with symbols which have no intrinsic content, and were inspired by very simple, outdated models of how “real“ neurons might work. The calculations performed by the AI do not involve any understanding or concept of the symbolic representations it uses. Even in its most elegant expressions, for the AI they are just meaningless physical states, coded in zeros and ones. This becomes even more obvious with the representation of pictures in the computer – their pixels are nothing but matrices of binary on-off states of transistors.

So, in the previous paragraph, I have derived, simply from the notion of what a computer does, why AI can‘t be “intelligent“, thus also does not speak, think or judge. We could stop here, but you might not be convinced yet, and argue that “somehow“, in the black box of Large Language Models (LLM), miraculously unpredictable “emergent abilities”, including a “Geist“ (mind) can evolve by adding and subtracting zeros. It is easy but meaningless to pull white rabbits out of a hat, let us instead carry on with the argument and delve into the question of what speaking, thinking, and judging entail.

Unfortunately, none of the popular “functional” definitions help us understand these mental activities. Here is, just as an example, how the consensus group of leading international psychologists defines intelligence: “Intelligence is a very general mental ability that includes, among other things, the ability to think, plan, solve problems, think abstractly, understand complex ideas, learn quickly, and learn from experience.” Alas, this is not a definition, but a rather arbitrary list of capabilities. It does not say what intelligence is, but merely what it can (possibly) be used for. The same goes for the definition of language as “a structured system of communication that consists of grammar and vocabular”, or thinking as “as conscious processes that can happen independently of sensory stimulation”. These are the definitions one finds in psychology textbooks, or through a Google search. Erroneously defining mental activities as “capabilities“ is the original sin of the misguided discussion on the “intelligence“ of generative AI. A discussion which must get stuck in the search for those abilities in the products of AI – an endless folly.

O.K., let us first hear from those who believe that the Rubicon has finally been crossed and that Large Language Models (LLM) possess human-like general intelligence. In its purest form, the misconception of the intelligent computer can be found in the recently published 155-page preprint “Sparks of Artificial General Intelligence: Early experiments with GPT-4“. With almost childlike glee, scientists at Microsoft’s research division report on their “experiments” with a number of LLMs. Among them of, course, the current champion, GPT-4. The bots are asked questions and tasks, and lo and behold, the results look quite like the LLMs are capable of judgment, sympathetic and creative (they paint and make music!), as well as having a “Theory of Mind”. Of course, the researchers attest that the LLMs still have some room for improvement: sometimes they “hallucinate,” or make significant errors, even in the simple math. Of all things, GPT-4 fails math because it outputs 88, when asked to calculate 7 x 4 + 8 x 8 ! But for the authors, even this is also proof of intelligence: How human, all too human! And hey, this is only GPT-4, just wait for GPT-5 and its competitors…

The Microsoft researchers were intoxicated by the perfect grammar of the LLMs, and their exceedingly polite language, which can effortlessly switch between rap, Shakespeare and Python. But because they have no concept of their (research) subject, they must miss the point. This is doubly tragic, because not only do the authors therefore come to a wrong conclusion (computer = intelligent). They have also failed to do precisely what is one of the essential achievements of human intelligence, namely to develop a concept of the subject matter (in this case, of AI): That is to grasp what the subject really is, not just how it appears – what is a necessary and essential feature, and not only an accidental and external one. By distinguishing these features one then understands what the subject matter really represents and therefore has cognitively “internalized the world“.

Arguing only with superficial analogies, the authors are therefore misled in their conclusion that LLMs demonstrate general, or any other form of intelligence. They fail to realize that the only intelligence responsible for the amazing output of the AIs was human– namely the human(s) who programmed the AI, as well as the intelligence that had been generated and accumulated by the humans in the material used for the training of the AI. Thus, it was only human intelligence, that allowed the AI to simulate recognition, understanding, and decision-making, without having any concept or idea of what it is doing, simply by statistically extrapolating from the training material. The LLM algorithms merely establish statistical references and correlations between features of the input, regardless of whether these consist of tweets by Elon Musk, Goethe’s Faust, or Wikipedia. These references between the contents of the training material are purely stochastic, they are not based on physical or logical correlations or related to its content. Contrary to popular claims, the AI and its language model do not generalize, they merely create concept-free labels, classifications and rules. These are not based on general properties of the subject matter, but are merely the result of statistical similarities between individual cases and the training data.

A great example for this semantics-, concept- and content-lessness of AI is that it can translate languages perfectly without knowing or understanding the vocabulary and grammar of even one of these languages, i.e. without being able to speak them. For us humans, however, the latter is the basic prerequisite for learning a foreign language. Contrary to another popular claim, the AI does not learn, unless one, like many psychologists, understands learning to be merely conditioning, imitation, or habituation. Indeed, according to their definition, learning is mindless repetition (“cramming”). Real learning, however, means grasping the object of learning by reflection or comprehension, or even more abstract (and this is what AI can’t do): In learning we grasp the general properties of a subject matter.

Take language acquisition: A child does not learn to speak by listening to millions of texts and subsequently analyzing them statistically. It learns a language (and at the same time complex thinking, see below) by “storing” concepts in its mind through personal experience and observation, and connecting them in a firm association with linguistic signs and words it hears. These can also be the gestures that deaf-mute individuals see and learn as their language. For this, a child needs surprisingly little material, in any case not terabytes of world literature. The child’s brain learns the language by using it according to the example it hears, and acquires its grammatical rules without ever consulting a grammar.

The child’s intelligence recognizes a thing by its name (e.g., a word or concept) and lets both the “thing“ and the name become one in thinking. We no longer need to imagine a tree in order to understand what is meant by the word “tree” – one could casually say that the word “tree” has become a tree in the brain. How the brain accomplishes this with its synaptic electrochemical storms is completely unknown – but we don’t need to know this either, because this neurobiological knowledge does not contribute anything to the question of how computers can behave intelligently. Neurobiology “only” describes the underlying physiological, anatomical, and biochemical foundations of thinking, not its concept or what thinking is. As a corollary, while work on artificial neural networks may in the future provide some hints on how complex network structures in the brain interact to recognize patterns in auditory or visual input, it also means that computers can’t teach us anything about cognition or the mind.

The merging of subject matter and name in the brain during thinking (“embodiment”) allows functional magnetic resonance imaging (fMRI) to “read” brain oxygenation patterns that occur during speaking words, looking at, or imagining images or language. These patterns, which have been assigned their meaning through prior training with these specific images or words, then allow for partial reconstruction of those words or images, but only in the same, trained individual and with a high error rate. These are fantastic feats of engineering and programming; they could also be useful for rudimentary communication (brain-computer interface, BCI) with individuals who are paralyzed and unable to express themselves. However, this is not mind-reading, nor does it make the machine more intelligent in any way: In a BCI, the computer extracts patterns without meaning or content, the content (meaning) is assigned to the pattern by humans.

Through language, we can contemplate things without necessarily engaging in an internal monologue. Of course, conducting such a monologue can be helpful at times, especially when we ponder complex thoughts like: What is language, or thinking? Even without such a monologue, our thinking, which allows us to comprehend the world and expand this knowledge in our daily lives and our research, is based on language. In comparison, AI can (often) flawlessly and more comprehensively “state” what a tree is than some humans. However, AI does not speak or think because, for AI, a tree is just a signifier, a mere entry in a lexical list of content-empty determination definitions, gathered from myriad sources, coded in 0s and 1s. AI therefore can’t, as Yuval Harari argues, “hack the operating system of human civilization”, i.e. language. When AI outputs this information, perhaps even with a resonant voice, it may appear intelligent to some. Yet, would you consider Wikipedia (artificially) intelligent merely because you (as well as the AI) find correct descriptions in it at the entry “tree“?

There are, indeed, many reasons why AI cannot make judgments and form concepts. Fundamentally, this is because the world is represented within it by concept-empty symbols. Hence, AI cannot speak – and what is presented to us as spoken language is merely the translation of symbols into sounds. However, since the symbol itself lacks content for AI, the resulting sound naturally lacks content as well. And because AI cannot speak, it cannot think, as language is the medium of conceptual thinking.

This is also why AI fails when it comes to inference. Through language, intelligence separates the subject from its determination (e.g., “The rose is fragrant,” “The computer is a programmable calculating machine”). By inference (“syllogism“) we then prove the identity of subject and predicate – thereby, if the inference is correct, we have determined the substance of a subject matter. We have explained it, distinguishing it from how it merely appears or seems. Hegel would have said that we have grasped the concept of the subject matter, we comprehend reality in thought. We humans can do this – AI cannot. Animals, by the way, cannot either, because although they can think and communicate, they do not have language. But this could be subject matter for another post!

Without language (and thus complex thinking), AI can’t develop free will – and come after us, like the AI SkyNet in the movie “Terminator.” But of course, this does not mean that AI cannot be very dangerous. Its long-established use in military technology proves this, as well as Tesla vehicles that in autopilot sometimes kill their owners and a few pedestrians. But here, the human being is always the subject, that is, the one endangering others. The same applies to deep fakes, plagiarism, and other criminal activities, for which humans make skillful use of AI.

From the above, it should be clear that all existing LLMs so far only perform ‘next word’ or ‘next pixel prediction’, a purely stochastic approach applied to preexisting knowledge, which therefore cannot create new knowledge. AI synthesizes everything that humans have put into digital form, provided it is accessible through the internet or proprietary databases. There, AI finds what is correct and useful, but also a lot of nonsense, misinformation, and falsehoods, and replicates all prevalent biases. This is why hordes of programmers must censor the AI, often via crowd sourcing a la “Amazon mechanical turk”, trying to iron out the resulting obscenities, incitements to violence, or instructions how to build bombs.  Or they censor the user, by not allowing queries which could trick the AI to reveal how to successfully commit suicide, or list strategies for the perfect murder – information harvested from the darker corners of the internet.

Through AI, we are confronted with the achievements, follies, and excesses of our own intelligence. That’s why AI is not “artificial stupidity,” as some critics believe. Also, because stupidity, which is nothing but the wrong use of intelligence, requires a good dose of intelligence itself, and AI completely lacks that. AI is excellent for writing reflective essays and poems. It is also useful for everything in which human intelligence does nothing but recognize patterns, encode, sort or classify. And there are plenty of such tasks in medicine, government offices, journalism, programming, language translation, or, on the battlefield. In fact, AI shows us how mindless many of our professional activities actually are. These will be replaced by AI in the near future.

But why do promoters and beneficiaries of AI, of all people, foster angst about AI and publicly warn against their own products? And why do they, like Elon Musk, even call for a training pause for their best language models, “because they could take control of our civilization”? Or compare themselves to the “fathers of the atomic bomb,” like Sam Altman, the founder of OpenAI? One reason is probably that they themselves have no concept of what AI really is: Intoxicated by the hype that they have created, they actually believe that their language models possess general intelligence. They make us believe that only they have the knowledge and power to keep the genie in the bottle. In addition, they present themselves – in anticipation of government regulation – as responsible benefactors of humanity, while further accelerating the hype around their products. Regulatory hurdles fortify existing market structures and thereby help the tech giants to safeguard their businesses against new kids on the block. Most importantly, regulation may shield them against open-source AI, which is already threatening their business model.

What really worries me is human intelligence, which generates, controls, and uses the artificial one, but not the prospect of being subjugated by computers.

Further reading (mostly in German):

Hegel, G.W.F. (1812): Wissenschaft der Logik. Bd. 1,1. Nürnberg. Retrieved April 14, 2023, from https://www.deutschestextarchiv.de/book/show/hegel_logik0101_1812

Hegel, G.W.F. (1817). The Doctrine of the Notion. Part One of the Encyclopedia of Philosophical Sciences: The Logic. https://www.marxists.org/reference/archive/hegel/works/sl/slsubjec.htm

Hegel, G.W.F. (1817) Enzyklopädie der philosophischen Wissenschaften im Grundrisse. Heidelberg. Retrieved April 14, 2023, from http://www.zeno.org/Philosophie/M/Hegel,+Georg+Wilhelm+Friedrich/Enzyklopädie+der+philosophischen+Wissenschaften+im+Grundrisse

Schröder, W. (2009). Zum Begriff der Sprache und des Zeichens in Hegels Enzyklopädie der philosophischen Wissenschaften. GRIN Verlag. https://www.grin.com/document/211419

Werckmeister, G. (2010) Die Begriffsmomente in Hegels Urteilslogik. Retrieved April 27, 2023, from https://hegel-system.de/de/v1312werckmeister.htm