Deep Learning

Module Advanced Models and Methods

Knowledge and understanding: At the end of the course, the student will have an in-depth knowledge of advanced Deep Learning models and methods, exploring the frontiers of research and complex applications. It will include techniques such as Metric Learning, Domain Adaptation, Knowledge Distillation, Deep Unsupervised and Self-Supervised Representation Learning, Multimodal Learning, and Reinforcement Learning, developing a critical understanding of their theoretical underpinnings and current challenges.

Applying knowledge and understanding: The student will be able to apply advanced Deep Learning methodologies for complex problems (learning representations from unlabeled data, transferring knowledge across domains, understanding multimodal content and integrating modalities). He/she will have the skills to implement and optimize models for Multimodal Learning, work with large architectures and apply optimization techniques such as Knowledge Distillation. He will also be able to understand and apply the principles of Reinforcement Learning.

Making judgements:  The student will develop a strong autonomy of judgment in the evaluation and selection of advanced Deep Learning models and methods. He will be able to critically analyze the scientific literature, identifying strengths and weaknesses of solutions and proposing innovative approaches. They will acquire the ability to make informed decisions about the applicability and effectiveness of cutting-edge techniques in real-world contexts, considering practical implications.

Communication skills: The student will be able to communicate complex concepts related to advanced Deep Learning models in an expert and appropriate way. He/she will be able to clearly present the results, projects and analyses, using an accurate technical language. Communication skills will include the ability to critically discuss, collaborate in multidisciplinary groups, and draft high-quality technical documentation and scientific reports.

Learning skills: The student will develop a high capacity for autonomous learning, which is essential to stay at the forefront of advanced Deep Learning. He/she will be proactive in identifying and assimilating new theories, algorithms and technologies, autonomously exploiting complex resources such as research articles and tutorials. This expertise will prepare them to undertake advanced research and development paths, addressing the future challenges of artificial intelligence.

The course takes place through lectures and laboratory activities, which will allow the practical application of theoretical concepts with exercises and case studies.

If the course is taught in mixed or distance mode, the necessary changes may be introduced with respect to what was previously stated, in order to comply with the program provided and reported in the syllabus.

Solid foundation of Deep Learning (convolutional and recurrent neural networks, backpropagation, regularization) and familiarity with scientific programming in Python.

Regular participation in the lessons is strongly recommended for an in-depth understanding of the topics and methodologies.

The course "Deep Learning: Advanced Models and Methods" explores the current frontiers of Deep Learning. It starts with advanced techniques to improve the generalization and efficiency of models: Metric Learning (learning semantic representation spaces) and Domain Adaptation (adaptation to data from different distributions). Knowledge Distillation will be introduced to transfer knowledge from complex models to more efficient models. This will be followed by the Unsupervised and Self-Supervised Representation Learning module, which is crucial for exploiting large amounts of unlabeled data to learn meaningful representations. Ample space will be dedicated to Multimodal Learning, including methods capable of managing data that include different modalities, such as images, video, audio, and text. Advanced methods of modeling sequential data, such as state space models, will therefore be discussed. Finally, it will conclude with an introduction to the fundamental principles and algorithms of Reinforcement Learning, with examples of application in decision-making contexts.

1.     Material provided by the teacher and distributed through the teacher's website (http://antoninofurnari.github.io/) and the team with code "cq90ptp".

2.     Bellet, A., Habrard, A., & Sebban, M. (2015). Metric learning. Morgan & Claypool Publishers.

3.     Csurka, Gabriela. "Domain adaptation for visual applications: A comprehensive survey." arXiv preprint arXiv:1702.05374 (2017).

4.     Wang, M., & Deng, W. (2018). Deep visual domain adaptation: A survey. Neurocomputing, 312, 135-153.

5.     Gou, Jianping, et al. "Knowledge distillation: A survey." International Journal of Computer Vision 129.6 (2021): 1789-1819.

6.     Chen, Y., Mancini, M., Zhu, X., & Akata, Z. (2022). Semi-supervised and unsupervised deep visual learning: A survey. IEEE transactions on pattern analysis and machine intelligence, 46(3), 1327-1347.

7.     Gui, J., Chen, T., Zhang, J., Cao, Q., Sun, Z., Luo, H., & Tao, D. (2024). A survey on self-supervised learning: Algorithms, applications, and future trends. IEEE Transactions on Pattern Analysis and Machine Intelligence.

8.     Jabeen, Summaira, et al. "A review on methods and applications in multimodal deep learning." ACM Transactions on Multimedia Computing, Communications and Applications 19.2s (2023): 1-41.

9.     Gao, J., Li, P., Chen, Z., & Zhang, J. (2020). A survey on deep learning for multimodal data fusion. Neural Computation, 32(5), 829-864.

10.   Zhang, D., Yu, Y., Dong, J., Li, C., Su, D., Chu, C., & Yu, D. (2024). Mm-llms: Recent advances in multimodal large language models. arXiv preprint arXiv:2401.13601.

11.   Patro, Badri Narayana, and Vijay Srinivas Agneeswaran. "Mamba-360: Survey of state space models as transformer alternative for long sequence modelling: Methods, applications, and challenges." arXiv preprint arXiv:2404.16112 (2024).

12.   Barto, A. G. (2021). Reinforcement learning: An introduction. by Richard's Sutton. SIAM Rev, 6(2), 423.

The exam of the "Deep Learning" course consists of two individual exams, associated with the two modules. An individual grade is assigned to each exam. The grade of the overall exam is obtained as the average of the marks associated with the two modules "Core models and methods" and "Advanced models and methods". 

The exam of the "Core models and methods" module consists of a written test and an individual or group project, generally consisting of the implementation and analysis of an advanced Deep Learning model on a real dataset. The written test is aimed at verifying the theoretical knowledge of the topics of the course, as well as the expressive skills and language properties of the students. The written test generally consists of three open-ended questions. The project will assess the application of advanced knowledge, autonomy, quality of implementation and analysis. The project must be carried out after the written test. Each of the two tests is assigned a score out of thirty and the final grade assigned to the module is obtained by means of the arithmetic average of the two marks.

The exam of the module "Advanced models and methods" consists of a written test and an individual or group project, generally consisting of the implementation and analysis of an advanced Deep Learning model on a real dataset. The written test is aimed at verifying the theoretical knowledge of the topics of the course, as well as the expressive skills and language properties of the students. The written test generally consists of a set of multiple-choice and open-ended questions. The project will assess the application of advanced knowledge, autonomy, quality of implementation and analysis. The project must be carried out after the written test. Each of the two tests is assigned a score out of thirty and the final grade assigned to the module is obtained by means of the arithmetic average of the two marks.

The tests of the "Core models and methods" module must be taken before the tests of the "Advanced models and methods" module.

The following criteria will normally be followed for the attribution of the final grade:

·       Not approved: the student has not acquired the basic concepts and is not able to carry out the project.

·       18-23: the student demonstrates minimum mastery of advanced concepts; project with significant deficiencies.

·       24-27: the student demonstrates good mastery of advanced content; well-structured project and adequate analysis.

·       28-30 laude: the student has acquired all the advanced contents; Excellent and innovative design, complete, critical and error-free analysis, demonstrating full autonomy.

Students with disabilities and/or SLD must contact the teacher, the CInAP contact person of the DMI (Prof. Daniele) and the CInAP sufficiently in advance to take advantage of the appropriate compensatory measures. Learning assessments may also be conducted electronically, should circumstances require it.

·       Illustrate the concept of Metric Learning and its use to improve image-based localization systems.

·       Explain the difference between Domain Adaptation and traditional Transfer Learning, describing a Domain Adaptation algorithm based on adversarial networks.

·       Describe the concept of Knowledge Distillation and how it can be used to improve the efficiency of a model.

·       Explain the principles of Deep Unsupervised and Self-Supervised Representation Learning, highlighting their differences and benefits for unlabeled data.

·       Explain the importance of mode alignment in multimodal models.

·       Illustrate the main advantages of recurrent sequential models based on state space models.

·       Explain the basic principles of Reinforcement Learning.

It should be noted that these questions are purely indicative: the questions proposed for the examination may diverge, even significantly.