Email
Google Scholar
GitHub
X
01ICLR 2026Oral
On the Surprising Effectiveness of a Single Global Merging in Decentralized Learning
A single global model merging can be remarkably effective in decentralized learning, challenging conventional wisdom about frequent communication.
Tongtian Zhu
Convince future generations, not just peers.
I am a Ph.D. candidate at the Computer Science Department of Zhejiang University (ZJU), supervised by Professors Can Wang and Chun Chen.
My research focuses on understanding learning mechanisms.
Research area
Language model training
Developing more effective and efficient training methods for language models, including optimizers, looped architectures, and efficient parallelism.
Research area
Decentralized learning
Understanding learning in distributed systems, including training dynamics, generalization, data influence, and communication allocation.
Research area
Science of learning
Studying learning mechanisms, such as training dynamics / implicit bias, symmetries, and emergence of mergeability.
Publications
Selected research
My research is my brand; papers are my products.
02ICLR 2025
DICE: Data Influence Cascade in Decentralized Learning
We study how data influence propagates in decentralized learning and reveal a cascade effect that shapes training dynamics and generalization.
03ICML 2023
Decentralized SGD and Average-direction SAM are Asymptotically Equivalent
We show that decentralized SGD and average-direction SAM are asymptotically equivalent, providing a unified understanding of their implicit regularization.
04ICML 2022Spotlight
Topology-aware Generalization of Decentralized SGD
We establish how the network topology affects the generalization of decentralized SGD, revealing a connection between graph structure and learning performance.
Recent updates
-
Our paper “On the Surprising Effectiveness of a Single Global Merging in Decentralized Learning” has been accepted to ICLR 2026 (Oral Presentation, Top 1.2%).
-
Gave a talk at the Conference on Scientific Machine Learning (CSML 2025).
-
Our DICE paper was accepted to ICLR 2025.
Beyond research
Outside of research, I enjoy drawing and basketball. They help me stay curious and balanced.
