Programmazione e Ottimizzazione HPC

Knowledge and Understanding: The student will acquire an in-depth understanding of advanced parallel programming techniques for high-performance shared-memory and distributed memory systems, with particular emphasis on advanced OpenMP constructs (task parallelism, SIMD directives, and affinity management on NUMA architectures) and MPI features (non-blocking communications, derived datatypes, one-sided communications, and parallel I/O), as well as hybrid MPI+OpenMP programming models. The course will also cover the main tools and methodologies for performance analysis, profiling, and code optimization on modern HPC architectures.

Applying Knowledge and Understanding: The student will be able to design, implement, and optimize complex parallel applications on shared-memory, distributed-memory, and hybrid architectures, using profiling tools to identify performance bottlenecks and apply targeted optimization strategies to improve scalability and computational efficiency.

Making Judgements: The student will develop the ability to critically evaluate different parallelization and optimization strategies for a given computational problem, independently selecting the most appropriate approach based on the target architecture, performance requirements, and application characteristics.

Communication Skills: The student will be able to clearly explain and justify the design choices made when parallelizing and optimizing an application, using appropriate technical terminology and presenting performance analysis results in a clear and structured manner to audiences with different levels of technical expertise.

Learning Skills: The student will develop the ability to independently and continuously expand their knowledge of programming and optimization techniques for HPC systems, keeping up to date with developments in computing architectures, parallel programming standards, and performance analysis tools.

The course is delivered through lectures and laboratory sessions. The lectures introduce the theoretical concepts underlying advanced programming and optimization techniques for HPC systems, while the laboratory sessions provide opportunities for their practical application through hands-on exercises and case studies on high-performance computing clusters.

Should the course be delivered in a blended or fully online format, appropriate adjustments to the teaching methods described above may be introduced to ensure that the course content and learning objectives outlined in the syllabus are fully covered.

The HPC Programming and Optimization course builds upon foundational knowledge of parallel programming and provides an in-depth exploration of advanced techniques for developing and optimizing applications targeting shared-memory and distributed-memory high-performance computing systems.

The course covers advanced constructs of the OpenMP and MPI standards, with particular emphasis on task parallelism, vectorization, thread affinity management on NUMA architectures, non-blocking and one-sided communications, derived datatypes, and parallel I/O, as well as hybrid programming models that combine shared-memory and distributed-memory parallelism.

Significant attention is devoted to methodologies and tools for performance analysis, profiling, and debugging of parallel applications, with the aim of developing the ability to identify computational bottlenecks and apply targeted optimization strategies, such as improving communication patterns, dynamic load balancing, and memory hierarchy optimization. Scalability and computational efficiency are evaluated on real HPC architectures.

Laboratory activities provide hands-on experience with the techniques presented in the course through representative case studies and practical exercises on high-performance computing clusters.

[1] Hager, Georg, and Gerhard Wellein. Introduction to High Performance Computing for Scientists and Engineers. CRC Press, 2010.

[2] Multi-platform Shared-memory Parallel Programming in C/C++ and Fortran - https://www.openmp.org/

[3] Message Passing Interface Forum. MPI: A Message-Passing Interface Standard - https://www.mpi-forum.org/

[4] Pacheco, Peter S., and Matthew Malensek. An Introduction to Parallel Programming, 2nd Edition. Morgan Kaufmann, 2021.

[5] Handouts and teaching materials provided by the instructor, distributed via the Studium platform.

The examination for the course consists of two parts: an initial written examination focusing on the theoretical foundations and advanced programming and optimization techniques covered during the lectures, followed by the assessment of an individual project involving the design, implementation, and optimization of a parallel application.

These assessments may be conducted remotely, should circumstances require it. The oral examination, where applicable, may take place on the same day as the written examination or within a few days thereafter.

The examination aims to thoroughly assess the student's knowledge, analytical and reasoning skills regarding the topics covered during the course, as well as the appropriate use of technical terminology.

The project assessment is an integral component of the examination and contributes inseparably to the determination of the final grade. It does not represent an opportunity to obtain additional marks, but rather a necessary component of the overall evaluation of the student's preparation.

The final grade will generally be assigned according to the following criteria:

• Fail: The student has not acquired the fundamental concepts and is unable to solve the exercises.
• 18–23: The student demonstrates a basic understanding of the fundamental concepts, limited ability to explain and establish connections between topics, and the ability to solve simple exercises.
• 24–27: The student demonstrates a good understanding of the course content, good communication skills and ability to establish connections between topics, and can solve exercises with only minor errors.
• 28–30 with honours: The student demonstrates comprehensive mastery of all course topics, can explain them thoroughly and critically establish connections between concepts, and solves exercises completely and without errors.

Students with disabilities and/or Specific Learning Disorders (SLD) must contact the course instructor, the DMI CInAP representative (Prof. Daniele), and CInAP sufficiently in advance of the examination date to communicate their intention to take the examination with the appropriate accommodations and support measures.

Registration through the SmartEdu portal is mandatory to participate in the final examination. For any technical issues related to examination registration, students should contact the Student Academic Office.

• Describe the main strategies for implementing task parallelism in OpenMP and discuss how dependencies between tasks are managed.
• What are the advantages of non-blocking MPI communications compared to blocking communications, and in which scenarios should they be used?
• Explain the concept of one-sided communication in MPI and compare it with the traditional point-to-point communication model.
• How can dynamic load balancing be addressed in a hybrid MPI+OpenMP parallel application?
• Describe the Roofline model and explain how it can be used to guide application optimization strategies.
• What are the main challenges associated with thread affinity management on NUMA architectures, and how can they be addressed using OpenMP?

Please note that these questions are provided for illustrative purposes only. The actual examination questions may differ, potentially significantly, from those listed above.

During the lectures, exercises similar to those students will be required to complete in their final examination will be presented and discussed. Additional exercises will be made available throughout the course.