1. Without committing to any firm position, this is why “they’re just word prediction machines” is such a weak criticism of LLMs. There are things you can say about them that are both meaner and less reductive, so why turn to that one? But hey, I’m just a next-action machine, I guess.
2. More essays like this, I say. There are a lot of things I disagree with in it, but I never felt that my attention or the complexity of the issue was disrespected.
Actually, I’ll walk that half a step back. I think “just word prediction machines” has some value for introducing people to the idea of an LLM from scratch. But it’s a bit of a “whales are like fish that breastfeed” thing – an orienting statement that is true-ish but really not a good place to stop.
Like, “Keeping orcas in captivity is unethical because they’re basically just hairy sharks” is not a good argument.
