Chain of thought

A model has no scratchpad other than its own output: each token gets a fixed amount of computation, and the only way to get more thinking is to write more tokens. Written-out steps are those tokens, and they stay in the context for what follows. Reasoning models are trained to do this before answering, which is what the reasoning effort setting on those models controls. A caution: the visible steps are not always a faithful account of what produced the answer.

• Wei et al.: Chain-of-thought prompting elicits reasoning in LLMs