现在位置:首页 > 学术报告
 

 

Academy of Mathematics and Systems Science, CAS
Colloquia & Seminars

Speaker:

杨周旺 教授, 中国科学技术大学

Inviter:  
Title:
Sparse Deep Neural Networks Through L(1,∞)-Weight Normalization
Time & Venue:
2018.10.24 15:30 N205
Abstract:
We study L_{1,\infty}-weight normalization for deep neural networks to achieve the sparse architecture. Empirical evidence suggests that inducing sparsity can relieve overfitting, and weight normalization can accelerate the algorithm convergence. In this paper, we theoretically establish the generalization error bounds for both regression and classification under the L_{1,\infty}-weight normalization. It is shown that the upper bounds are independent of the network width and sqrt(k)-dependence on the network depth k, which are the best available bounds for networks with bias neurons. These results provide theoretical justifications on the usage of such weight normalization. We also develop an easily implemented gradient projection descent algorithm to practically obtain a sparse neural network. We perform various experiments to validate our theory and demonstrate the effectiveness of the resulting approach.
 

 

附件下载:
 
 
【打印本页】【关闭本页】
电子政务平台   |   科技网邮箱   |   ARP系统   |   会议服务平台   |   联系我们   |   友情链接