Event Dime
Two More Methods for Consistency Training and Some New Ways to Apply It
- When:
- June 5, 2026 · 9:06 PM
- Where:
- A review of Where Is My Flying Car? by J. Storrs Hall — LessWrong
- Source:
- A review of Where Is My Flying Car? by J. Storrs Hall — LessWrong
Authors: Sukrati Gautam*, Neil Shah*, Arav Dhoot*, Bryan Maruyama*, Caroline Wei*, Rohan Kapoor, Robert Sidey, Prakhar Gupta, Zi Cheng Huang, David Demitri Africa. This work was done for the SPAR Fellowship, and has been accepted at AI4GOOD @ ICML 2026. It was supervised by David Africa. TL;DR We introduce two new consistency training methods, MLPCT (enforcing consistency on MLP hidden states) and AttCT (enforcing consistency on per-head attention distributions). Consistency training generalizes