Bolte, Jérôme, Le, Tam and Pauwels, Edouard (2022) Subgradient sampling for nonsmooth nonconvex minimization. TSE Working Paper, n. 22-1310, Toulouse

[thumbnail of wp_tse_1310.pdf]
Preview
Text
Download (492kB) | Preview

Abstract

Risk minimization for nonsmooth nonconvex problems naturally leads to firstorder sampling or, by an abuse of terminology, to stochastic subgradient descent. We establish the convergence of this method in the path-differentiable case, and describe more precise results under additional geometric assumptions. We recover and improve results from Ermoliev-Norkin [27] by using a different approach: conservative calculus and the ODE method. In the definable case, we show that first-order subgradient sampling avoids artificial critical point with probability one and applies moreover to a large range of risk minimization problems in deep learning, based on the backpropagation oracle. As byproducts of our approach, we obtain several results on integration of independent interest, such as an interchange result for conservative derivatives and integrals, or the definability of set-valued parameterized integrals.

Item Type: Monograph (Working Paper)
Language: English
Date: February 2022
Place of Publication: Toulouse
Uncontrolled Keywords: Subgradient sampling, stochastic gradient, online deep learning, conservative gradient, path-differentiability
Subjects: B- ECONOMIE ET FINANCE
Divisions: TSE-R (Toulouse)
Institution: Université Toulouse 1 Capitole
Site: UT1
Date Deposited: 25 Feb 2022 12:12
Last Modified: 03 Feb 2023 14:43
OAI Identifier: oai:tse-fr.eu:126674
URI: https://publications.ut-capitole.fr/id/eprint/44587
View Item

Downloads

Downloads per month over past year