As long as I’m in a listy mood, this language model that backspaces out of dead ends is interesting because:
1. It’s an example of something categorically less interesting than the pro-LLM camp is going to claim, and more interesting than the anti-LLM camp wants to talk about; that gap seems important.
2. It points toward the text diffusion models I keep trying to subtly warn people are probably coming without saying so explicitly, oops, backspace backspace backspace.
Like I can totally see both the “AI now corrects itself, is unstoppable; how does this affect bitcoin?” headlines and the “Yes, I spent all of March calling people credulous fools for not knowing that self-correction is the exclusive purview of ensouled life, but nevertheless this is Just Math” takes and I wish we could just skip that part of everything.
