Love what you write, Benjamin. (Have been a fan since I picked up 'deliberate practice' from you guys when I was still a teacher.) But it feels like a dangerous topic&time about which to be using words like 'never'. Absolutes are the death of the credbile, after all 😉
And on 'Her': we still need real world LLMs to speak in our language... who's to say they wouldn't already make new metaphors if they could converse in vectors, instead of forcing (and probably distorting) useful connections through the bottleneck of English..?
Thanks Dominic, nice to have a fan from my older world stay active in the new. And while I take your point, I am unafraid to say that LLMs will not develop the capacity for irony, in the same way I am unafraid to make that claim about my toaster. Don't get me wrong, it's fun to imagine the alternative possibilities, including a post-verbal form of communication -- check out the movie Arrival for wonderful exploration of this with aliens.
But in terms of the AI tools we build, sorry, there is no irony forthcoming!
p.s. Have you read the Ted Chiang short story that Arrival is based on? If not I strongly recommend it. It's a much deeper and more satisfying exploration of that idea, including how physics often has two entirely separate explanations of the same phenomenon.
(Which, come to think of it, is exactly the sort of thing that might make novel 'analogy' accessible to a synthetic mind that—unlike a human—can hold the entirety of scientific exploration in its working memory at once...)
Well in a sense I envy your surety—I'm working deeply with these technologies, and I have no such clarity.
If you'd like to stick firmly to the definition of 'LLM', I'm closer to agreeing with you. But we shouldn't, particularly given most are already 'LMM's (multimodal), and even more so given they will all soon be 'very-LMM's.
I don't have a specific prediction that contradicts "no irony forthcoming". But for what it's worth, I don't think you're thinking exponentially enough, and my bet is you'll be revising that position a lot sooner than you think.
One way or the other, it's going to be fascinating to find out!
Re "Now, finally, with all this as backdrop, let’s bring in AI, and large-language models specifically. What are they? Among other things they are attempts to predict what a generally intelligent human would say in response to a given prompt from another human."
And just how do they determine what is posted on the internet is by a "generally intelligent" person and a racist or a raving maniac?
Re "Seen this way, LLMs are mere common-sense repositories. Yes, they can remix and recycle our knowledge in interesting ways. But that’s all they can do. They are trapped in the vocabularies we’ve encoded in our data and trained them upon. Large-language models are dead-metaphor machines."
They seem to be a remix of all of our foibles, errors, biases, prejudices, etc. In the image generators, Jesus the Christ is always rendered as an Anglo, not as a Middle Easterner.
Very interesting post. I haven't sent any money your way (I am a retired teacher and have to watch my budget) but this is the first piece of yours I have read.
Thanks for reading Steve, and the kind words. I think you make an important point: What Rorty calls "common sense" may nonetheless be racist or biased or otherwise reprehensible -- and this is what drives a "liberal ironist" (Rorty's term) to try to *change* the world and our vocabulary to describe it. These tools make our foibles and prejudices unambigously visible, but they can't work to change that -- only we humans can.
Love what you write, Benjamin. (Have been a fan since I picked up 'deliberate practice' from you guys when I was still a teacher.) But it feels like a dangerous topic&time about which to be using words like 'never'. Absolutes are the death of the credbile, after all 😉
And on 'Her': we still need real world LLMs to speak in our language... who's to say they wouldn't already make new metaphors if they could converse in vectors, instead of forcing (and probably distorting) useful connections through the bottleneck of English..?
Thanks Dominic, nice to have a fan from my older world stay active in the new. And while I take your point, I am unafraid to say that LLMs will not develop the capacity for irony, in the same way I am unafraid to make that claim about my toaster. Don't get me wrong, it's fun to imagine the alternative possibilities, including a post-verbal form of communication -- check out the movie Arrival for wonderful exploration of this with aliens.
But in terms of the AI tools we build, sorry, there is no irony forthcoming!
p.s. Have you read the Ted Chiang short story that Arrival is based on? If not I strongly recommend it. It's a much deeper and more satisfying exploration of that idea, including how physics often has two entirely separate explanations of the same phenomenon.
(Which, come to think of it, is exactly the sort of thing that might make novel 'analogy' accessible to a synthetic mind that—unlike a human—can hold the entirety of scientific exploration in its working memory at once...)
Well in a sense I envy your surety—I'm working deeply with these technologies, and I have no such clarity.
If you'd like to stick firmly to the definition of 'LLM', I'm closer to agreeing with you. But we shouldn't, particularly given most are already 'LMM's (multimodal), and even more so given they will all soon be 'very-LMM's.
I don't have a specific prediction that contradicts "no irony forthcoming". But for what it's worth, I don't think you're thinking exponentially enough, and my bet is you'll be revising that position a lot sooner than you think.
One way or the other, it's going to be fascinating to find out!
Re "Now, finally, with all this as backdrop, let’s bring in AI, and large-language models specifically. What are they? Among other things they are attempts to predict what a generally intelligent human would say in response to a given prompt from another human."
And just how do they determine what is posted on the internet is by a "generally intelligent" person and a racist or a raving maniac?
Re "Seen this way, LLMs are mere common-sense repositories. Yes, they can remix and recycle our knowledge in interesting ways. But that’s all they can do. They are trapped in the vocabularies we’ve encoded in our data and trained them upon. Large-language models are dead-metaphor machines."
They seem to be a remix of all of our foibles, errors, biases, prejudices, etc. In the image generators, Jesus the Christ is always rendered as an Anglo, not as a Middle Easterner.
Very interesting post. I haven't sent any money your way (I am a retired teacher and have to watch my budget) but this is the first piece of yours I have read.
Thanks for reading Steve, and the kind words. I think you make an important point: What Rorty calls "common sense" may nonetheless be racist or biased or otherwise reprehensible -- and this is what drives a "liberal ironist" (Rorty's term) to try to *change* the world and our vocabulary to describe it. These tools make our foibles and prejudices unambigously visible, but they can't work to change that -- only we humans can.
Just my kind of wonkery! Also: *chef's kiss* on the image.
Ah thank you, so gratifying that you get that.