🎉 Our paper Gradient Multi-Normalization is accepted at NeurIPS 2025!
🎉🥳 Our paper Gradient Multi-Normalization for Efficient LLM Training has been accepted in NeurIPS 2025 conference. We will be there to present the poster, welcome to our poster for more discussion! 🎉🥳
Oct 1, 2025
🎉 Our paper SWAN is accepted at ICML 2025!
🥳🥂 Our paper SWAN:SGD with Normalization and Whitening Enables Stateless LLM Training has been accepted in ICML 2025 conference. This optimizer allows stateless LLM training to maximize the memory efficiency, while achieving on-par or better performance than standard AdamW optimizer. 🥳🥂
Apr 1, 2025