(Deep) Learning Theory, PhD course in mathematics @ UNIPI (2023-2024)

Class times: Aula Riunioni, Mathematics Department, UNIPI, (and streamed online through the teams platform on the class teams channel)
Monday 10:30 - 12:30,
Thursday 14:00 - 16:00 (the THU lecture will approximately take place once every two weeks).

The class will start on MON, DEC 4th 2023 and end before the start of the Spring semester.
If these lecture times have conflicts with your classes, please contact the course instructor.

Class format: Lectures will be in hybrid format: they will be held live and also streamed through the class team channel

Please enroll in the teams channel of the class for announcements and up-to-date information.

Instructor: Andrea Agazzi,

E-mail: andrea.agazzi at unipi.it (please include "DLT" in email title).

Corse summary:
Deep learning has emerged as a powerful approach to solving complex problems in artificial intelligence, and understanding the underlying theory is crucial for practitioners and researchers alike. This postgraduate course offers an introduction to the theory behind deep learning, focusing specifically on mathematically rigorous results on the subject.
The course will start with an overview of the rudiments of statistical learning theory, such as loss functions, empirical risk minimization, kernel methods, generalization, and regularization. We will then thoroughly discuss the fundamentals of neural networks theory, covering topics such as architecture, activation functions, expressivity, approximation theorems, and training through (stochastic) gradient descent. The third part of the course will be devoted to some aspects of the optimization theory of neural networks. In particular, we will discuss the training dynamics of neural networks in the infinitely wide limit in two contrastive regimes: the neural tangent kernel regime and the mean-field regime. Depending on time and interest, other topics that might be covered are: (deep) reinforcement learning, generalization bounds for stochastic gradient descent and time-series learning with recurrent neural networks.

Class requirements: The class is open to advanced master students. While no previous knowledge of machine learning theory is expected, a solid background in analysis and probability will be necessary to reach an in-depth understanding of the topics treated in the course.

Textbooks and references (to be updated during the course):

Lecture notes:
04.12.2023 Lecture 1
07.12.2023 no class
11.12.2023 Lecture 2
14.12.2023 Lecture 3 and sandbox
18.12.2023 Lecture 4
08.01.2024 no class
11.01.2024 Lecture 5
15.01.2024 Lecture 6
19.01.2024 at 10:30 Lecture 7
22.01.2024 no class
25.01.2024 no class
29.01.2024 Lecture 8
01.02.2024 Lecture 9
05.02.2024 no class
08.02.2024 Lecture 10
12.02.2024 Lecture 11
15.02.2024 Lecture 12
19.02.2024 Lecture 13
22.02.2024 (lecture starts at 15:00) Lecture 14

Papers for presentations: