Bottlenecked Transformers Consolidate KV Cache to Improve Reasoning

Bottlenecked Transformer adds a periodic KV‑cache consolidation step, boosting multi‑step reasoning. On math benchmarks it beats a vanilla transformer by up to 6.6 pp. Read more: getnews.me/bottlenecked-transformer... #bottleneckedtransformer #kvcache