Generalization in Deep Learning
Module Description
| Course | Module Abbreviation | Credit Points |
|---|---|---|
| BA-2010 | AS-CL | 8 LP |
| Master | SS-CL, SS-TAC | 8 LP |
| Lecturer | Mayumi Ohta |
| Module Type | |
| Language | English |
| First Session | 09.11.2020 |
| Time and Place | Monday, |
| Commitment Period | 8th Feb. 2021 - 17th. Feb. 2021 |
[Update on 15. February 2021]
The presentation on Double Descent has been postponed until next week. Instead, we will have a short review-session this week.
[Update on 8. February 2021]
Commitment period: 8th Feb. - 17th Feb.
[Update on 25. January 2021]
We have no session on 1. Feb. 2021 (Mo).
[Update on 14. December 2020]
We have no session on 14. Dec. 2020 (Mo).
[Update on 18. November 2020]
We have no session on 23. Nov. 2020 (Mo).
I recommend to watch the lecture series
Statistical Machine Learning Part 38 - 44 (Prof. Ulrike von Luxburg, University of Tübingen)
It's a very nice introduction to get an overview on Statistical Learning Theory.
[Update on 6. November 2020]
I've sent the registered students an info email with HeiConf link. If you haven't received the email, please contact me.
[Update on 4. November 2020]
If you haven't registered, but still want to participate, please write an email to
the secretary sekretariat [at] cl.uni-heidelberg with the following information:
- your name
- your student ID (``Matrikelnummer'')
- your email address
- your degree program (``Studiengang'')
- which institute you belong to
- which semester you are in (``Fachsemester'', i.e. first semester in MA)
See here for late registration.
[Update on 2. November 2020]
We won't use Moodle in this seminar. I'm now preparing an info email to the registered students including HeiConf link. I'll announce here when I send you the info email.
[Update on 26. October 2020]
The time slot has been changed to 11:15-12:45.
Registration
Students have to register for this course until 26.10.2020. This applies for CL and Non-CL students and for freshmen as well as higher semesters. To enrol, please follow the instructions here: course registration .
If you have any question, please contact the lecturer by email: ohta[at]cl.uni-heidelberg.de.
Prerequisite for Participation
Assessment
Content
Deep neural networks have seen great success in a wide range of applications, but why they generalize well remains still
a mystery. A number of researches have tackled the problem to uncover the generalization mystery in deep learning models
from both theoretical and empirical perspectives. In this seminar, we will study theoretical proofs and experimental
generalization bounds. We will discuss evaluations of complexity measures, such as VC-dimension, PAC-Bayes, Sharpness
measure, Optimization-based or Compression-based measure, and the correlation of different measures to generalization
in image classification experiments.
Students are expected to reimplement one of those complexity measures and extend
to some sequence-to-sequence tasks in their term project.
