DEEP LEARNING

The term Deep Learning identifies a class of machine learning algorithms based on the automatic learning of data representations in a hierarchical way in order to solve a given problem. The recent availability of large amounts of data and appropriate computational resources to analyse them has favoured the emergence of such technologies for solving problems such as image and video analysis, audio signal analysis, text processing, and in general, the analysis of large amounts of data of various kinds. Given the complexity that Deep Learning-based algorithms can easily achieve, specific skills are required in order to correctly design such algorithms, train them, measure their goodness, and understand their limits. The purpose of this course is to provide these skills and in particular:

• Provide an extensive knowledge of the problems that deep learning aims to solve;

• Provide an overview of the methodologies currently available to solve these problems;

• Provide an overview of computational modules useful for solving certain sub-problems within a more complex problem;

• Provide a methodology based on reasoning for computational modules useful for the development of deep learning architectures to address a given problem;

• Provide the tools to correctly evaluate deep learning algorithms;

• Provide the tools to properly train deep learning algorithms. 

General educational objectives of teaching in terms of expected learning outcomes.

1. Knowledge and understanding: the student will acquire the knowledge and concepts underlying the algorithms that exploit the Deep Learning paradigm and in particular the methodologies that allow you to create automatic learning systems to solve different tasks starting from data heterogeneous.

2. Applying knowledge and understanding: the student will acquire practical skills useful for the design and development of Deep Learning algorithms through laboratory activities.

3. Making judgments: Through homeworks proposed by the teacher and the correction in class of the same, the student will be able to independently develop solutions capable of solving the basic problems that could arise in the world of work in which to apply Deep Learning techniques.

4. Communication skills: the student will acquire the necessary communication skills and expressive appropriateness in the use of technical language in the general context of automatic learning systems that exploit the Deep Learning paradigm.

5. Learning skills: The aim of the course is to provide theoretical and practical knowledge on the main Deep Learning techniques. In this regard, neural models and architectures will be presented to solve various problems (Domain Adaptation, Metric Learning, Classification / Regression, Representation Learning, etc.). Methodologies to implement and optimise machine learning algorithms will be discussed, as well as those useful for evaluating the performance of algorithms. Open source software libraries in Python language will be used to put into practice the theoretical notions presented in the course. In particular, the course aims to train students who:

1. understand key concepts underlying Deep Learning techniques

2. know a wide range of Deep Learning algorithms to solve various classic Machine Learning problems

3. understand how to carry out the design and tuning of Deep Learning in order to apply the same to new data sets

4. are able to perform the evaluation of Deep Learning algorithms in order to select the best model

5. know the python libraries useful for the development of Deep Learning algorithms

• Deep Learning - Basic Concepts

• Deep Autoencoders

• Deep Generative Models

• Deep Domain Adaptation and Transfer Learning

• Deep Metric Learning

• Self-Supervised, Unsupervised, Representation Learning

• Deep Sequential Modeling

• Deep Reinforcement Learning

• Video Understanding

• Other Deep Learning Paradigms and State-of-the-Art Architectures

1. Slides and notes provided by the teacher

2. E. Alpaydin, Introduction to Machine Learning

3. M. P. Deisenroth et. al, Mathematics for Machine Learning

4. I. Goodfellow et al., Deep Learning

5. Ovidiu Calin, Deep Learning Architectures

6. Sutton and Barto, Reinforcement Learning

7. David Foster, Generative Deep Learning

8. Gabriela Csurka, Domain Adaptation for Visual Applications: A Comprehensive Survey

9. Mei Wang, Weihong Deng, Deep Visual Domain Adaptation: A Survey
10. Aurélien Bellet, Amaury Habrard, Marc Sebban, Metric Learning

Additional teaching material is available at the following link: https://www.dmi.unict.it/farinella/DL

• Define the Deep Q-learning algorithm formally and by pseudocode.

• Formally define a GAN type architecture.

• Discuss an architecture for generating synthetic data.
• Discuss an architecture for data compression.