Event Dime

Revisiting GSM-Symbolic: Do 2026 Frontier Models Still Fail at Confounded Grade School Math?

Family & Kids
When:
June 5, 2026 · 8:54 PM
Where:
A review of Where Is My Flying Car? by J. Storrs Hall — LessWrong
Source:
A review of Where Is My Flying Car? by J. Storrs Hall — LessWrong

TL;DR The GSM-Symbolic paper (ICLR 2025) purported to show that language models rely on pattern matching rather than genuine reasoning by demonstrating that perturbing the questions to make them break the pattern of the original question would catastrophically reduce performance in the model. Running the results again in March 2026 with GPT-4o, Claude Opus 4.6, and Claude Haiku 4.5 shows that we precisely replicate the original findings only when we do not audit out examples that may actually be

Why this event appears here

This listing is shown because it is tied to the current city coverage and carries structured metadata that can be indexed and compared with other events.

  • City coverage

    A review of Where Is My Flying Car? by J. Storrs Hall — LessWrong

  • Primary source

    A review of Where Is My Flying Car? by J. Storrs Hall — LessWrong

  • Categories

    Family & Kids