I was recently asking myself a really obvious but strangely complex question: if all a large language model does is predict the next word, how can it actually reason? At their mathematical core, large language models are simply next token prediction engines. During training, they are shown vast quantities of text and asked over and over again, given everything that came before, what comes next? Then billions of internal parameters are adjusted until they get better and better