Wenbo Gong
Wenbo Gong

Senior Researcher

Hi ๐Ÿ‘‹, Iโ€™m Wenbo Gong โ€” a Senior Researcher at Microsoft. I work on making LLMs more efficient, from better optimizers to smarter architectures and theory.
20

Publications

664

Citations

13

h-index

Welcome ๐Ÿ‘‹

Iโ€™m interested in how LLMs learn, and how we can use that understanding to make them more efficientโ€”through better optimizers, efficient architectures, and a bit of theory. Earlier, I worked on Bayesian inference and generative models (MCMC, variational inference, and their combinations), and later explored sequential decision-making by connecting causality and Bayesian methods. Outside of research, I love the gym ๐Ÿ‹๏ธโ€โ™€๏ธ, sports โ›น๏ธ, and relaxing with music ๐ŸŽง.

Research areas: Learning dynamics, optimization, foundation models, causality, approximate inference.

Recent News

๐ŸŽ‰ Our paper SWAN is accepted at ICML 2025!

๐Ÿฅณ๐Ÿฅ‚ Our paper SWAN:SGD with Normalization and Whitening Enables Stateless LLM Training has been accepted in ICML 2025 conference. This optimizer allows stateless LLM training to maximize the memory efficiency, while achieving on-par or better performance than standard AdamW optimizer. ๐Ÿฅณ๐Ÿฅ‚