People thinking without speaking

The science of human thought deals a fatal blow to the quest for artificial general intelligence through large-language models

Artificial intelligence emulates only one of these networks in the brain

The most important research article I’ve read about AI this year fails to mention “artificial intelligence” even once. Doesn’t even nod in AI’s direction. And yet I submit to you that understanding the implications from the research I’m about to describe is vital to understanding why claims about AI soon becoming as smart or smarter than humans are deeply scientifically misguided.

Buckle up as I take you on a brief journey across the lands of cognitive science, neuroscience, linguistics, philosophy (oh yes), and cultural evolution. Will it be wonky? Yes. Will it be worth it? I hope so.

The central claim of this essay that you will resist believing because it is so counterintuitive

Our cognitive capabilities do not dependent on human language. We use language to transmit our knowledge, but language does not give rise to thought. Language is simply a communicative tool.

My summary of the new article in Nature that cuts off the quest for artificial general intelligence at the kneecaps

The article is titled, with admirable clarity, “Language is primarily a tool for communication rather than thought,” it was published last week in Nature, and it’s authored by Evelina Fedorenko, Steven T. Piantadosi & Edward A. F. Gibson. A highly readable summary is here and (shhh) you can access the paywalled version by clicking the [PDF] link here.

The article is a tour-de-force summary of decades of scientific research regarding the relationship between language and thought, and has two purposes: (1) to tear down the notion that language gives rise to our ability to think and reason; and (2) to build up the idea that language evolved cultural as a tool to share our thoughts.

Let’s take each of these in turn.

When we contemplate our own thinking, it often feels as if we are thinking in a particular language, and therefore because of our language.¹ Until relatively recently, science didn’t have much to say about this, and so arguments about the interrelationship between thought and language were mostly left to philosophy (we’ll come back to this).

But now science has weighed in. If it were true that language is essential to thought, then taking away language should take away our ability to think. This does not happen. This does not happen. Over the past decades, studies of humans who have lost their language ability through brain damage or disorders, combined with imagining of neural activity in the brain, have demonstrated empirically that taking away language does not fundamentally impair the ability to think. Here's the key quote from the article:

The evidence is unequivocal—there are many cases of individuals with severe linguistic impairments, affecting both lexical and syntactic abilities, who nevertheless exhibit intact abilities to engage in many forms of thought—they can solve mathematical problems, perform executive planning and follow non-verbal instructions, engage in diverse forms of reasoning, including formal logical reasoning, causal reasoning about the world and scientific reasoning, to understand what another person believes or thinks and perform pragmatic inference, to navigate in the world, and to make semantic judgements about objects and events. (oodles of citations omitted)

Ok, so. Having torn down the notion that language is essential to thinking, the authors then build up the case for language as a tool that has been “optimized” for so that we can our ideas and thoughts to one another – an “efficient communication code,” as they call it. This is evidenced by the fact, across the broad diversity of human languages, there are common features they share – they are “easy to produce, easy to learn and understand, concise and efficient for use, and robust to noise.” (You’ll have to read the article to dive into the linguistics details of this.)

Lastly, but certainly not leastly, all of this directly ties into the evolution of human cultures. In the view of the researchers, and a view shared by yours truly, the human capacity to share our thoughts across generations via language – not because of language, but via language – is the key driver of what makes humans unique. As they put it: “The cumulative effect of this transmission—knowledge building on knowledge—along with increased sophistication of our social and problem-solving abilities is plausibly what enabled us to create human civilizations.”

A brief philosophical interlude that I know some of you will skip to get to the AI section that follows (but you shouldn’t)

What is extraordinarily satisfying about this scientific evidence is that it generally accords with what philosophers have been contending about language for oh, the past 75 years or so. You may be dimly familiar with someone called “Ludwig Wittgenstein” but have avoided figuring out why he’s important, and I don’t blame you – his prose is dense and confusing, and many of the people who talk about him are (a) annoying and (b) prone to misunderstanding his key insight.

To summarize way too briefly: Wittgenstein, along with other philosophers that include Donald Davidson, Richard Rorty, Hilary Putnam, Gilbert Ryle, and Daniel Dennett – to name only a few, and let’s acknowledge here that the lack of diversity in philosophy is a real and ongoing problem – they all argued that the claims we make about the world, what we consider “true,” arise from conversations we humans have with each other. This moved the entire field of philosophy away from trying to establish certain universal and inexorable propositions, and to instead focus – some might say navel gaze – on the role of language in shaping our understanding of the world through our descriptions of it.

This has been disastrously interpreted by some so-called postmodernists as suggesting anything can be “true,” that truth is essentially an arbitrary creation, but that was not what these guys meant. Like, at all. What they were arguing instead is, well, what science now confirms – that language is how we express our ideas to one another, and that this ability to share (and debate) claims about the world is extraordinarily useful. It’s how we transmit knowledge across generations. Put another way, our ability to think and reason is necessary to civilization, but not sufficient – for that, we need language too.²

With that philosophical detour complete, let’s get to the good stuff – what all this means for AI.

Why the effort to create artificial general intelligence through ever-more-powerful models of human language appear doomed to fail, scientifically

The premise of many AI enthusiasts who believe we are on the verge of creating artificial general intelligence – something as or more intelligent than humans – is that “scale is all you need.” What does that mean? It’s a simple story, really: If we gather tons of data about the world, and combine this with ever more powerful computing power (read: Nvidia chips) that can make connections between words, then presto, we’ll have AGI.

If you’ve read this far, I hope you already see the outlines of why this theory of AGI is seriously scientifically flawed. Because if it’s true that human thinking is largely independent of human language – and remember, the evidence for this is unequivocal – then we will not be able to recreate the powers of human thinking by modeling human language, no matter how powerful such models become. The entire effort will fail because LLMs are simply tools that emulate the communicative function of language, not the cognitive process of thinking and reasoning.

The artist Angie Wang, who not coincidentally studied linguistics, captured this elegantly in her illustrated essay for The New Yorker contrasting the difference between her toddler learning to talk and an LLM “learning” language:

Human lives are enriched by language – deeply enriched – but they are not defined by language. Take away our ability to speak, and we can still think, reason, form beliefs, fall in love, move about the world, our range of what we can experience and think about remains vast.

Take away language from a large-language model…and you are left with literally nothing.

I’m making a strong claim here. I’m suggesting in no uncertain terms that no matter how adept LLMs become at making statistical correlations between words, they will not be capable of thinking the way that humans think. Perhaps you are quietly thinking to yourself, Ben is getting out over his skis, but you’ll have to trust me when I tell you some of the leading thinkers in AI space are coming around to this view. That’s the subject of a future post.

What I am left wondering: The implications of this science of thinking for AI seem so obvious to me, I’m surprised that the authors of Nature piece did not include a sidebar or some form of commentary on the topic. So, I’m getting in touch with them and aiming to interview at least one on whether they draw the same conclusions that I am. That too may be the subject of a future post.³

But in the meantime, the 4^th of July approaches in America, so I’m taking next week off from Substacking – I know, you’re devastated. Why not use your free time to read one of the greatest essays ever written about America, written by a truly great American? His thoughts, powerfully communicated through his masterful use of language, are always worth reflecting on.

1. The last part of the last chapter of James Joyce’s Ulysses, the chapter that is written from the perspective of Molly Bloom and contains a single sentence that runs for 50-odd pages, is the single best literary representation of the stream of consciousness of human thought that I’ve ever come across. Yes I said yes.

2. On the other side of this debate is one Noam Chomsky, who claims that humans possess an innate, genetically bestowed language capacity called Universal Grammar. Last week reports surfaced that Chomksy had died, but he’s still very much alive. The same can’t be said for his theory.

3. After publishing this post, Ev Fedorenko (lead author of the Nature article) interacted with me briefly on Twitter and pointed me to another article she co-authored on the relationship of language and thought that’s specific to LLMs. It’s behind a paywall but I will try to write about this soon. https://www.sciencedirect.com/science/article/abs/pii/S1364661324000275