In this paper, we study zeroth-order algorithms for nonconvex minimax problems with coupled linear constraints under the deterministic and stochastic settings, which have attracted wide attention in machine learning, signal processing and many other fields in recent years, e.g., adversarial attacks in resource allocation problems and network flow problems etc. We propose two single-loop algorithms, namely the zeroth-order primal-dual alternating projected gradient algorithm and the zeroth-order regularized momentum primal-dual projected gradient (ZO-RMPDPG) algorithm, for solving deterministic and stochastic nonconvex-(strongly) concave minimax problems with coupled linear constraints. The iteration complexities of the two proposed algorithms to obtain an epsilon-stationary point are proved to be O(epsilon-2) (resp. O(epsilon-4)) for solving nonconvex-strongly concave (resp. nonconvex-concave) minimax problems with coupled linear constraints under deterministic settings and O(epsilon-3) (resp. O(epsilon-6.5)) under stochastic settings respectively. To the best of our knowledge, they are the first two zeroth-order algorithms with iterative complexity guarantees for solving nonconvex-(strongly) concave minimax problems with coupled linear constraints under the deterministic and stochastic settings. The proposed ZO-RMPDPG algorithm, when specialized to stochastic nonconvex-concave minimax problems without coupled constraints, outperforms all existing zeroth-order algorithms by achieving a better iteration complexity, thus setting a new state-of-the-art.

Publication:

MATHEMATICS OF COMPUTATION

http://dx.doi.org/10.1090/mcom/4196

Author:

HUILING ZHANG

Department of Mathematics, College of Sciences, Shanghai University, Shanghai 200444, People’s Republic of China; and LSEC, ICMSEC, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, People’s Republic of China

Email address: zhanghl1209@shu.edu.cn

ZI XU

Department of Mathematics, College of Sciences, Shanghai University, Shanghai 200444, People’s Republic of China; and Newtouch Center for Mathematics of Shanghai University, Shanghai 200444, People’s Republic of China

Email address: xuzi@shu.edu.cn

YU-HONG DAI

State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, People’s Republic of China; and School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, People’s Republic of China

Email address: dyh@lsec.cc.ac.cn