dayuwang401 / spectral-orthogonal-exploration

What it does

LLMs exhibit a paradox on hard reasoning tasks: repeated stochastic sampling tends to produce lexical variations of the same erroneous logic rather than genuinely new reasoning — a phenomenon we call Reasoning Collapse. We show that failed traces are associated with a low-rank bias manifold in the model's hidden-state geometry, which suppresses exploration toward corrective solution directions.