Latent Thought Modeling Improves Data Efficiency in LM Pretraining

A 1B-parameter language model boosted data efficiency via latent-thought inference, gaining improvements after three EM cycles without an external teacher model. Read more: getnews.me/latent-thought-modeling-... #languagemodel #latentthought