From error bounds to the complexity of first-order descent methods for convex functions

Bolte, Jérôme, Nguyen, Trong Phong, Peypouquet, Juan and Suter, Bruce W. (2017) From error bounds to the complexity of first-order descent methods for convex functions. Mathematical Programming, 165 (2). pp. 471-507.

[img]
Preview
Text
Download (375kB) | Preview
Official URL: http://tse-fr.eu/pub/32471

Abstract

This paper shows that error bounds can be used as effective tools for deriving complexity results for first-order descent methods in convex minimization. In a first stage, this objective led us to revisit the interplay between error bounds and the Kurdyka-Lojasiewicz (KL) inequality. One can show the equivalence between the two concepts for convex functions having a moderately flat profile near the set of minimizers (as those of functions with Hölderian growth). A counterexample shows that the equivalence is no longer true for extremely flat functions. This fact reveals the relevance of an approach based on KL inequality. In a second stage, we show how KL inequalities can in turn be employed to compute new complexity bounds for a wealth of descent methods for convex problems. Our approach is completely original and makes use of a one-dimensional worst-case proximal sequence in the spirit of the famous majorant method of Kantorovich. Our result applies to a very simple abstract scheme that covers a wide class of descent methods. As a byproduct of our study, we also provide new results for the globalization of KL inequalities in the convex framework. Our main results inaugurate a simple method: derive an error bound, compute the desingularizing function whenever possible, identify essential constants in the descent method and finally compute the complexity using the one-dimensional worst case proximal sequence. Our method is illustrated through projection methods for feasibility problems, and through the famous iterative shrinkage thresholding algorithm (ISTA), for which we show that the complexity bound is of the form O(qk)O(qk) where the constituents of the bound only depend on error bound constants obtained for an arbitrary least squares objective with l1l1 regularization.

Item Type: Article
Language: English
Date: October 2017
Refereed: Yes
Uncontrolled Keywords: Error bounds, Convex minimization, Forward-backward method, KL inequality, Complexity of first-order methods, LASSO, Compressed sensing
Subjects: B- ECONOMIE ET FINANCE
Divisions: TSE-R (Toulouse)
Site: UT1
Date Deposited: 13 Apr 2018 15:01
Last Modified: 20 Mar 2019 15:29
OAI ID: oai:tse-fr.eu:32471
URI: http://publications.ut-capitole.fr/id/eprint/25851

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year