Stochastic Heavy Ball

Gadat, Sébastien, Panloup, Fabien and Saadane, Sofiane (2018) Stochastic Heavy Ball. Electronic Journal of Statistics, 12 (1). pp. 461-529.

This is the latest version of this item.

[img]
Preview
Text
Download (1MB) | Preview
Official URL: http://tse-fr.eu/pub/32573

Abstract

This paper deals with a natural stochastic optimization procedure derived from the so-called Heavy-ball method differential equation, which was introduced by Polyak in the 1960s with his seminal contribution [Pol64]. The Heavy-ball method is a second-order dynamics that was investigated to minimize convex functions f. The family of second-order methods recently received a large amount of attention, until the famous contribution of Nesterov [Nes83], leading to the explosion of large-scale optimization problems. This work provides an in-depth description of the stochastic heavy-ball method, which is an adaptation of the deterministic one when only unbiased evalutions of the gradient are available and used throughout the iterations of the algorithm. We first describe some almost sure convergence results in the case of general non-convex coercive functions f. We then examine the situation of convex and strongly convex potentials and derive some non-asymptotic results about the stochastic heavy-ball method. We end our study with limit theorems on several rescaled algorithms.

Item Type: Article
Language: English
Date: 2018
Refereed: Yes
Uncontrolled Keywords: Stochastic optimization algorithms, Second-order methods, Random dynamical systems
Subjects: B- ECONOMIE ET FINANCE
Divisions: TSE-R (Toulouse)
Site: UT1
Date Deposited: 22 May 2018 11:34
Last Modified: 24 Jul 2019 09:11
OAI ID: oai:tse-fr.eu:32573
URI: http://publications.ut-capitole.fr/id/eprint/25889

Available Versions of this Item

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year