Bolte, Jérôme, Le, Tam and Pauwels, Edouard (2022) Subgradient sampling for nonsmooth nonconvex minimization. TSE Working Paper, n. 22-1310, Toulouse

[thumbnail of wp_tse_1310.pdf]
Download (492kB) | Preview


Risk minimization for nonsmooth nonconvex problems naturally leads to firstorder sampling or, by an abuse of terminology, to stochastic subgradient descent. We establish the convergence of this method in the path-differentiable case, and describe more precise results under additional geometric assumptions. We recover and improve results from Ermoliev-Norkin [27] by using a different approach: conservative calculus and the ODE method. In the definable case, we show that first-order subgradient sampling avoids artificial critical point with probability one and applies moreover to a large range of risk minimization problems in deep learning, based on the backpropagation oracle. As byproducts of our approach, we obtain several results on integration of independent interest, such as an interchange result for conservative derivatives and integrals, or the definability of set-valued parameterized integrals.

Item Type: Monograph (Working Paper)
Language: English
Date: February 2022
Place of Publication: Toulouse
Uncontrolled Keywords: Subgradient sampling, stochastic gradient, online deep learning, conservative gradient, path-differentiability
Divisions: TSE-R (Toulouse)
Institution: Université Toulouse 1 Capitole
Site: UT1
Date Deposited: 25 Feb 2022 12:12
Last Modified: 03 Feb 2023 14:43
OAI Identifier:
View Item


Downloads per month over past year