I’ve had the damn song “I am the very model of a modern major general” running through my head all week, because I’ve been thinking about model organisms and whether and how we might determine if AI systems can be models of the human mind. Apparently the tune is from The Pirates of Penzance, and, uh, I have no idea what’s going on in this clip but it feels very colonial and imperial. Dislike.1
Gilbert & Sullivan opera aside, in many ways the idea that AI might prove a useful way of helping us understand ourselves is what led me into this whole career pivot. Some brief history: Several years ago circa 2018, I became dimly aware that something big was happening in AI research and development, something referred to as “deep learning.” At the time, I was the CEO of an education nonprofit organization dedicated to improving the preparation of classroom teachers, and so the notion that we had developed artificial [something] capable of deep learning intrigued me—if it had been labelled “deep connections,” I doubt I would have cared. But if we’d manage to create something artificial that learned the way that we humans do, well, that’d be incredibly valuable as a tool for preparing teachers, a flight simulator of the mind essentially. My curiosity was piqued.
So, I went on a mini-learning excursion to figure out what deep learning entailed, which in turn led me to connect with Gary Marcus, who graciously agreed to chat with me for reasons unknown. His take then was that the deep learning process using artificial neural networks was significantly different from human cognition, and thus unlikely to be useful as a training model for teaching. And so that basically ended my interest in AI and deep learning until ChatGPT crashed into our lives like the Kool-Aid Man.
Flash forward to the present and the question still remains: To what extent might AI systems serve as “model organisms” of the human mind? My friend Sean Trott has written about this extensively (see here and here), with a particular interest in comparing and contrasting the underlying processes used by large-language models to produce their output with what’s happening when we humans think and act—he calls this the search for “mechanistic congruence,” also a great name for a metal band (nods to Dave Barry). There are reasons to doubt we can do this, which we’ll get to momentarily, but the possibility remains scientifically tantalizing. As Trott observes, “extrapolating from LLMs to humans would represent a major leap forward in our ability to characterize the mechanisms underlying interesting human cognitive behaviors.”
Learning is one of those interesting human behaviors, and so I was very interested to come across a new research paper titled, “When can AI models explain learning? Validity criteria for AI as cognitive models in education,” authored by Dr. Peng Wang and a medley of international scholars.2 As the title suggests, the goal of Wang et al. is to set forth the conditions that AI models would have to satisfy in order for us to treat them as model organisms for human learning. As they put it, “in practical terms, building an AI tool as a cognitive model means shaping the model’s architecture, constraints, and learning rules to reflect theoretically motivated assumptions about cognition, then analyzing whether its learning dynamics and failure modes resemble those of humans on comparable tasks.”
So how might we do that? Here, Wang et al. set a high bar for AI to clear to via a set of validity criteria with eight components:
Theoretical grounding. Do we have a theory of how the AI model process aligns (or not) to human learning?
Construct validity. Does using the AI model explain how learning unfolds, rather than just redescribe outcomes?
Mechanistic transparency. Can we examine the internal processes of the AI model?
Alignment with human learning trajectories. Does the AI learning process mirror the path of human learning?
Error-signature matching. Does the AI model make the same sort of learning mistakes that humans make?
Causal-intervention tests. Can we disable components of the AI model and thereby determine their causal role in the model’s behavior?
Ecological validity. Are AI models trained in ways akin to human learning, or have they “learned” from unrealistic quantities of data?
Instructional usefulness. Can insights from AI models regarding learning be explained to human educators using validated inferences, not just statistical correlations?
This is…a great list! In many ways, this approach exemplifies what I’ve called the “adversarial attitude” toward evaluating AI models, an approach borrowed from animal-cognition research. Start with a default presumption that these tools operate in ways that are very different from how we humans think, and then try to falsify that hypothesis through careful and deliberate means. This cautious approach is reflected throughout Wang et al.’s discussion of these criteria, and I’m a fan.
Of course me being me, I can’t resist poking at this a bit. The first question I have is regarding that fourth component, aligning AI models with human learning trajectories, because I’m not at all convinced we have very robust understanding of those paths across a wide range of subjects. What’s more, to the extent we become capable of developing such learning trajectories for humans, it’s not clear whether AI models would add that much value beyond what we’d already know about our own learning.
As Sean Trott observes, drawing on Daniel Steel’s book Across the Boundaries, there’s a circular trap of sorts here: We employ model organisms in order to draw inferences from one species to another—e.g., we test carcinogens on rats and then apply the resulting insights to the “target species” of humans. The challenge, however, is determining when we can extrapolate from the model to the target given our inability to directly test the latter. Thus, Trott asks, “if we haven’t done the right kinds of empirical confirmation in the target, how can extrapolation be licensed from the model? And if we have done the right kinds of empirical confirmation in the target, what good is the model?”
So that’s one issue. My second major wondering is whether it’s ever fruitful to use a non-organism (AI) as a model organism for actual organisms (humans). When it comes to cognition, I’m of the view that there is something fundamentally important about being a living thing with goals and needs that’s developed against the backdrop of billions of years of evolutionary pushes and pulls. An organism, after all, is something fundamentally separated from the broader world that acts on its own behalf because it wants to live.
AI is not this. AI is software we’ve developed that is ostensibly designed to solve certain problems we feed to it. It does not have self-driven goals and it is not self organized; it is a tool of human creation. Can tools serve as model organisms of actual living things? Can airplanes be model organisms of birds? You see my point.
That said, I continue to see AI as a useful foil to contrast human cognition against—my experience with my workshops is that people enjoy contemplating their own thinking but they need something to play off of, to make abstract questions more concrete and visible. So although I’m skeptical of AI serving as model organism of the mind, there’s educational value in seeing it as an anti-model organism too.
This also sent me down a rather grim wormhole on what happened to George Rose, the actor playing the major general. Details here, if you’re morbidly curious.
Shout out to Jeff Greene at UNC-Chapel Hill for putting this on my radar.
Good little post here.
Last couple paragraphs make me wonder if you have read Hans Jonas.