Cumulant Expansion Reveals Geometry of Next-Token Prediction in LLMs

Researchers applied cumulant analysis to GPT-2 and Pythia on 10,000 Pile prompts, finding structured inputs cause a rise-and-plateau of variance, skewness and kurtosis across layers. getnews.me/cumulant-expansion-revea... #cumulantanalysis #gpt2