Chris Junchi Li

Accelerated Fully First-Order Methods for Bilevel and Minimax Optimization

  • Chris Junchi Li
    • Categories Convex Optimization, Nonlinear Optimization Tags ,

    We present in this paper novel accelerated fully first-order methods in Bilevel Optimization (BiO). Firstly, for BiO under the assumption that the lower-level functions admit the typical strong convexity assumption, the \emph{(Perturbed) Restarted Accelerated Fully First-order methods for Bilevel Approximation} (\texttt{(P)RAF${}^2$BA}) algorithm leveraging \emph{fully} first-order oracles is proposed, whereas the algorithm for finding approximate first-order … Read more

    SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator

  • Cong Fang
  • Chris Junchi Li
  • Zhouchen Lin
  • Tong Zhang
    • Categories Nonlinear Optimization, Stochastic Programming Tags , , , , ,

    In this paper, we propose a new technique named \textit{Stochastic Path-Integrated Differential EstimatoR} (SPIDER), which can be used to track many deterministic quantities of interest with significantly reduced computational cost. We apply SPIDER to two tasks, namely the stochastic first-order and zeroth-order methods. For stochastic first-order method, combining SPIDER with normalized gradient descent, we propose … Read more