New Study Explains How Transformers Learn Low‑Rank Regression Tasks

A new arXiv paper shows linear attention (replacing soft‑max) reveals a sharp phase transition in error for low‑rank regression tasks. Read more: getnews.me/new-study-explains-how-t... #linearattention #lowrank

Sparse State Expansion Boosts Linear Attention for Long-Context AI

Sparse State Expansion (SSE) lets linear‑attention models process long text; a 2 billion‑parameter SSE‑H model scored 64.5 on AIME 24 and 50.2 on AIME 25. Read more: getnews.me/sparse-state-expansion-b... #sse #linearattention