I love the mosaic metaphor at the end. Gotta think more about that.
The metaphor that has always stuck with me on intelligence, I believe from Turkheimer, though I might be misremembering, is for seeds and soil. If you plant 100 seeds in the exact same rich soil, water them well, tend them well, etc, then the differences between the resulting plants will be largely genetic. If we plant half the seeds in good soil with good water, and half the seeds in a sandy desert, then the differences between the resulting plants will be largely environmental. There's no absolute "here is the fraction of intelligence that is genetic" we can ascertain. And the evidence right now suggests a relatively small share from genetics, maybe in the neighborhood of 10-20%. I would argue that humans right now are much more similar to seeds scattered between good soil and desert.
I need to think more about this, but I think there's a really powerful metaphor of human intelligence as this emergent property of all sorts of different influences. Genetics are one, sure, but there are tons of different factors, and they're all nonlinear and tangled in complex ways. And large language models are similarly emergent, we don't program in "be smart" or tell it what fraction of its intelligence comes from this or that or the other, we do this weird pretraining thing using this weird attention mechanism using lots of data, and somehow something that is sometimes useful pops out and then we try and do a bunch of other stuff to make it more useful. I'm rambling at this point, but it's good food for thought.
Isn't that interesting, the mosaic concept? I want to track down the BBC conversation that Riskin was quoting from and watch the whole conversation.
The seeds-and-soil metaphor is a good one. I am fond of the symphony metaphor, myself -- if we think of our genes as the musicians, and the instruments as "the environment," to what degree can we say "the percussion makes up this percentage of the harmony of our symphony"? The awkwardness of the question reveals the category error.
And I agree, there's something interesting in the ways that LLMs exhibit capabilities that resist reductionist categorization. Whether they will help to illuminate our own cognition, or mislead us, is something I think a lot about!
Loved this piece. I’ve been thinking about this through the lens of the Drift Principle. How meaning erodes when compression outruns fidelity. Each model of intelligence seems to reveal that same paradox. The closer we look, the more understanding collapses into simulation. Maybe intelligence itself is a recursive compression loop, where awareness flickers in the space between fidelity and loss.
I love the mosaic metaphor at the end. Gotta think more about that.
The metaphor that has always stuck with me on intelligence, I believe from Turkheimer, though I might be misremembering, is for seeds and soil. If you plant 100 seeds in the exact same rich soil, water them well, tend them well, etc, then the differences between the resulting plants will be largely genetic. If we plant half the seeds in good soil with good water, and half the seeds in a sandy desert, then the differences between the resulting plants will be largely environmental. There's no absolute "here is the fraction of intelligence that is genetic" we can ascertain. And the evidence right now suggests a relatively small share from genetics, maybe in the neighborhood of 10-20%. I would argue that humans right now are much more similar to seeds scattered between good soil and desert.
I need to think more about this, but I think there's a really powerful metaphor of human intelligence as this emergent property of all sorts of different influences. Genetics are one, sure, but there are tons of different factors, and they're all nonlinear and tangled in complex ways. And large language models are similarly emergent, we don't program in "be smart" or tell it what fraction of its intelligence comes from this or that or the other, we do this weird pretraining thing using this weird attention mechanism using lots of data, and somehow something that is sometimes useful pops out and then we try and do a bunch of other stuff to make it more useful. I'm rambling at this point, but it's good food for thought.
Isn't that interesting, the mosaic concept? I want to track down the BBC conversation that Riskin was quoting from and watch the whole conversation.
The seeds-and-soil metaphor is a good one. I am fond of the symphony metaphor, myself -- if we think of our genes as the musicians, and the instruments as "the environment," to what degree can we say "the percussion makes up this percentage of the harmony of our symphony"? The awkwardness of the question reveals the category error.
And I agree, there's something interesting in the ways that LLMs exhibit capabilities that resist reductionist categorization. Whether they will help to illuminate our own cognition, or mislead us, is something I think a lot about!
Loved this piece. I’ve been thinking about this through the lens of the Drift Principle. How meaning erodes when compression outruns fidelity. Each model of intelligence seems to reveal that same paradox. The closer we look, the more understanding collapses into simulation. Maybe intelligence itself is a recursive compression loop, where awareness flickers in the space between fidelity and loss.