Shenzhen University School of Mathematical Sciences  

Liyuan Scholars Colloquium Session 183

Title: The Staircase Phenomenon and Its Application in Initializing the Training Dynamics of Neural Networks

Speaker: Professor Yang Jiang (Southern University of Science and Technology)

Time: 10:00–11:00, October 21, 2026

Location: Classroom 1, Huixing Building, Yuehai Campus, Shenzhen University

Abstract:Understanding the training dynamics of deep neural networks (DNNs)—particularly how they evolve low-dimensional features from high-dimensional data—is one of the core challenges in deep learning theory. This paper proposes ε-rank as a new metric for quantitatively characterizing the effective features of the terminal hidden layer neurons. Extensive experiments reveal a universal “step phenomenon”: during standard stochastic gradient descent training, the decrease in the loss function is accompanied by an increase in the ε-rank, which occurs in a step-like manner. Theoretically, we rigorously prove the negative correlation between the lower bound of the loss and the ε-rank, indicating that a higher ε-rank is a necessary condition for a significant decrease in loss; Numerical results also indicate that the ε-rank of a subsequent hidden layer in the same network is higher than that of the preceding hidden layer. Based on this, to eliminate the staircase phenomenon, we propose a novel pre-training strategy targeting the initial hidden layers to enhance the ε-rank of the final hidden layer. Numerical experiments validate that this method reduces training time and improves accuracy across a variety of tasks.

Speaker Profile: Yang Jiang is a tenured professor in the Department of Mathematics at Southern University of Science and Technology. He received his bachelor’s degree from Zhejiang University in 2010 and his Ph.D. from Hong Kong Baptist University in 2014. From 2014 to 2017, he conducted postdoctoral research at Pennsylvania State University and Columbia University, respectively, and has been on the faculty at Southern University of Science and Technology since 2017. His research focuses on computational mathematics, with areas of interest including the modeling of phase-field and nonlocal models, numerical methods and their applications, and the design and theory of deep learning algorithms. His work has been published in journals such as SIAM Review, SINUM, Math. Comp., M3AS, SISC, and JCP. He has received the Outstanding Paper Award at the World Congress of Chinese Mathematicians (2024), the “Frontier Science Award” at the International Congress of Basic Sciences (2025), and the First Prize in the Ministry of Education’s Award for Outstanding Achievements in Natural Science Research (2025, ranked 2nd out of 4). He was selected as one of the top 2% of scientists worldwide by Stanford-Elsevier (2025, 2026). He has been selected for the Youth Program of the National High-Level Talent Plan and the Shenzhen Outstanding Young Scholars Program, and has led multiple research grants, including the National Natural Science Foundation of China’s Tianyuan Mathematics Interdisciplinary Key Special Project, General Projects, and the Guangdong Provincial Natural Science Foundation.

All faculty and students are welcome!  

School of Mathematical Sciences  

October 15, 2026