prove
Mercoledi 25 Maggio
Dipartimento di Matematica, Sala Seminari
- 14:00 Enrico Malatesta (Bocconi): Phase transitions in the landscape of solutions of overparametrized neural networks.
Current deep neural networks are nonlinear devices composed of a number of parameters that far exceed the number of data points. Understanding how these systems can fit the data almost perfectly through variants of gradient descent algorithms and achieve exceptional levels of prediction accuracy without overfitting are key conceptual challenges. In this talk I will show how common techniques used in machine learning (e.g. the choice of the activation function or the loss) deeply affect the loss landscape, tending to mild its roughness. Then we shed light on the role of overparameterization in non-convex neural networks. By analytically studying a non-convex model of random features, we identify a novel (non-equilibrium) phase transition, that we call “Local Entropy” transition, controlled by the degree of overparameterization. In non-convex models this transition is strictly different to the SAT/UNSAT threshold and it coincides with the appearance of highly entropic minima of the error loss function. Those minima, in turn, are found to be highly attractive to the learning algorithms currently used in deep learning.
Link for Attending Seminars Online
All seminars are also broadcasted on Google Meet, join at the following link.The research group
Past members
- V. De Mattei
- G. Di Gesù
- C. Geldhauser
- M. Leocata
- D. Luo
Home
Future seminars
Past seminars
Visitors & guests
Workshops
Stochastic analysis day, March 3, 2015
Stochastic analysis day, July 25, 2016
Stochastic analysis day, February 27, 2017
Workshop for Maurizio, June 29, 2018
Speakers cloud
Acciaio AnnaPaolaTodino Antonelli Aryasova Baglioni Bambi Bernardin Bessaih Bevilacqua Bion-Nadal Blömker Brzezniak Cannizzaro Capasso Cappelletti Caraceni Catellier Cerrai Coghi DarrickLee DeMattei Debussche Delarue EliseoLuongo Erbar Flandoli FrancescaPistolato Gautier GiuliaCarigi Giuliano Glatt-Holtz Grotto Guasoni Gubinelli Hocquet Jara JulesPitcho LangxuanSu Leocata LuisaAndreis Luo Maas Macci Majewski Matetski Maurelli Mueller Nardi Nassif Neklyudov Neves Olivera Orsingher Perkowski Pham Pratelli Priola Proske Röckner Romito RuojunHuang Russo Swiech Tankov Touzi Trevisan Tsagkarogiannis Weber Zambotti Zanco