Interesting stuff, Ben. I appreciate the guide to work so far from my disciplinary home.
It seems to me you are describing a group of researchers who are skeptical about the intelligence of the latest LLMs, yet are nonetheless excited about the fact that transformer-based models are producing such impressive cultural outputs. They seem to think that the jump we say in 2022 may provide insights useful to the four avenues you outline, but not a meaningful step in that direction. Gotta say, #4 seems more dangerously silly than ambitious.
My window into this is Cosma Shalizi's musings, "You Can Do That with Just Kernel Smoothing!?!" and "You Can Do That with Just a Markov Model!?!!?!" at http://bactra.org/notebooks/nn-attention-and-transformers.html. These questions would seem to be a framework for putting aside some of the assumptions made by AI researchers who don't understand their inventions, and examining the capacity of GPTs to manipulate symbols.
So, treat questions of how GPTs do what they do as a set of interesting empirical questions about how they manipulate cultural data (symbols). The fact that in place of a research program working on this, we have a bunch of overcapitalized get-rich-quick schemes run out of Silicon Valley is going to make this harder than it needs to be.
I have become convinced that Charles Peirce is important, and that process philosophy (I've started using Whitehead's term instead of pragmatism, thanks to Kevin Munger), rather than logical positivism and its offspring, provides a better theoretical framework for thinking about all this.
He had a banger of a post on Bergson and Whitehead that really has me drawing lines to connect dots among the North Atlantic writers who were exploring alternatives to Western metaphysics of Descartes and Kant. He is following through on Andrew Pickering's insight that cybernetics, as practiced by the weirdos in the UK like Beer, Pask, Ashby, and their fellow travelers, can be understood as emerging from a tradition that includes pragmatists and Bergson.
Usually, people point to Heraclitus and his river, but I think the tradition really starts with Western philosophy's taking notice of ancient South Asian texts (think Schopenhauer and Emerson), continuing with pragmatists and Bergson, then Whitehead at Harvard in the 1920s, and the later Wittgenstein.
Munger sent me back to reading Whitehead, which I always meant to do when I was living the glory days of the revival of pragmatism.
Interesting stuff, Ben. I appreciate the guide to work so far from my disciplinary home.
It seems to me you are describing a group of researchers who are skeptical about the intelligence of the latest LLMs, yet are nonetheless excited about the fact that transformer-based models are producing such impressive cultural outputs. They seem to think that the jump we say in 2022 may provide insights useful to the four avenues you outline, but not a meaningful step in that direction. Gotta say, #4 seems more dangerously silly than ambitious.
My window into this is Cosma Shalizi's musings, "You Can Do That with Just Kernel Smoothing!?!" and "You Can Do That with Just a Markov Model!?!!?!" at http://bactra.org/notebooks/nn-attention-and-transformers.html. These questions would seem to be a framework for putting aside some of the assumptions made by AI researchers who don't understand their inventions, and examining the capacity of GPTs to manipulate symbols.
So, treat questions of how GPTs do what they do as a set of interesting empirical questions about how they manipulate cultural data (symbols). The fact that in place of a research program working on this, we have a bunch of overcapitalized get-rich-quick schemes run out of Silicon Valley is going to make this harder than it needs to be.
I have become convinced that Charles Peirce is important, and that process philosophy (I've started using Whitehead's term instead of pragmatism, thanks to Kevin Munger), rather than logical positivism and its offspring, provides a better theoretical framework for thinking about all this.
Damn Rob, just seeing this excellent comment from you. Curious what you're referencing re Kevin Munger and process philosophy? I'm intrigued!
He had a banger of a post on Bergson and Whitehead that really has me drawing lines to connect dots among the North Atlantic writers who were exploring alternatives to Western metaphysics of Descartes and Kant. He is following through on Andrew Pickering's insight that cybernetics, as practiced by the weirdos in the UK like Beer, Pask, Ashby, and their fellow travelers, can be understood as emerging from a tradition that includes pragmatists and Bergson.
Usually, people point to Heraclitus and his river, but I think the tradition really starts with Western philosophy's taking notice of ancient South Asian texts (think Schopenhauer and Emerson), continuing with pragmatists and Bergson, then Whitehead at Harvard in the 1920s, and the later Wittgenstein.
Munger sent me back to reading Whitehead, which I always meant to do when I was living the glory days of the revival of pragmatism.
Here is the piece: https://kevinmunger.substack.com/p/the-antimeme-haunting-western-philosophy