prove

Mercoledi 25 Maggio

Dipartimento di Matematica, Sala Seminari
  • 14:00 Enrico Malatesta (Bocconi): Phase transitions in the landscape of solutions of overparametrized neural networks.
  • Current deep neural networks are nonlinear devices composed of a number of parameters that far exceed the number of data points. Understanding how these systems can fit the data almost perfectly through variants of gradient descent algorithms and achieve exceptional levels of prediction accuracy without overfitting are key conceptual challenges. In this talk I will show how common techniques used in machine learning (e.g. the choice of the activation function or the loss) deeply affect the loss landscape, tending to mild its roughness. Then we shed light on the role of overparameterization in non-convex neural networks. By analytically studying a non-convex model of random features, we identify a novel (non-equilibrium) phase transition, that we call “Local Entropy” transition, controlled by the degree of overparameterization. In non-convex models this transition is strictly different to the SAT/UNSAT threshold and it coincides with the appearance of highly entropic minima of the error loss function. Those minima, in turn, are found to be highly attractive to the learning algorithms currently used in deep learning.

Link for Attending Seminars Online

All seminars are also broadcasted on Google Meet, join at the following link.

The research group

Past members

Brownian sheet: reload to get a different one

Home

Future seminars

Past seminars

Visitors & guests

Workshops

Stochastic analysis day, March 3, 2015
Stochastic analysis day, July 25, 2016
Stochastic analysis day, February 27, 2017
Workshop for Maurizio, June 29, 2018

 

Speakers cloud

Acciaio AnnaPaolaTodino Antonelli Aryasova Baglioni Bambi Bernardin Bessaih Bevilacqua Bion-Nadal Blömker Brzezniak Cannizzaro Capasso Cappelletti Caraceni Catellier Cerrai Coghi DarrickLee DeMattei Debussche Delarue EliseoLuongo Erbar Flandoli FrancescaPistolato Gautier GiuliaCarigi Giuliano Glatt-Holtz Grotto Guasoni Gubinelli Hocquet Jara JulesPitcho LangxuanSu Leocata LuisaAndreis Luo Maas Macci Majewski Matetski Maurelli Mueller Nardi Nassif Neklyudov Neves Olivera Orsingher Perkowski Pham Pratelli Priola Proske Röckner Romito RuojunHuang Russo Swiech Tankov Touzi Trevisan Tsagkarogiannis Weber Zambotti Zanco