TECHNOLOGIES FOR ADVANCED PROGRAMMING
Academic Year 2024/2025 - Teacher: SALVATORE NICOTRAExpected Learning Outcomes
The course will introduce a general overview of technologies useful to define end-to-end real time data analysis solutions.
Using automatic delivery systems (Docker) and orchestration (Kubernetes) the course will introduce a method to search and adopt “on the edge” technologies for data ingestion, pipelines, processing, analysis and visualization.
A practical approach using a class github repository and real problems will be analyzed on all aspects enforcing different soft skills including knowledge and understanding, decision, communication and learning skills.
Course Structure
The course will be structured as a laboratory where each topic will be presented and tested.
The class will also form a community to discuss about these technologies and applications.
Should teaching be carried out in mixed mode or remotely, it may be necessary to introduce changes with respect to previous statements, in line with the programme planned and outlined in the syllabus.
Learning assessment may also be carried out on line, should the conditions require it.
Required Prerequisites
Detailed Course Content
- Stream Processing
- Digital Marketing
- Introduction to Docker and Kubernetes
- Data Ingestion (Logstash, Fluentd)
- Data Streaming (Apache Kafka)
- Data Processing (Spark, Flink)
- Fondamenti di Machine Learning e classificazione, Spark MLlib, Spark NLP
- Data Indexing and Query (Elastic Search/Open Search/Open Table Forma))
- Data Visualization (Kibana/Grafana/Metabase)
- Data Catalog
Textbook Information
1. Slides and link to manual of frameworks will be shared during the course
2. Github Organization https://github.com/tapunict/
Course Planning
| Subjects | Text References | |
|---|---|---|
| 1 | Data Ingestion | Slides e manuali di riferimento dei relativi framework utilizzati durante il corso. |
| 2 | Data Processing | Slides e manuali di riferimento dei relativi framework utilizzati durante il corso. |
| 3 | Data Visualization | Slides e manuali di riferimento dei relativi framework utilizzati durante il corso. |
| 4 | Progetti | Repository di codice su github a supporto. |
Learning Assessment
Learning Assessment Procedures
In the exams students need to:
- present the design of the solution and discuss the problem
- Do a live demo
- Reply to questions
The exam is structured so that each student is assigned a grade according to the following scale:
- Not approved: the student has not acquired the basic concepts and is unable to answer at least 60% of the questions or complete the exercises.
- 18-23: the student demonstrates minimal mastery of basic concepts, has limited ability to connect the content, and is able to solve simple exercises.
- 24-27: the student demonstrates a good command of the course content, has solid content-connection skills, and solves exercises with few errors.
- 28-30 with honors: the student has fully acquired all course content and is able to master it with a critical approach; exercises are completed thoroughly and without errors.
Exams could be eventually be done on line if special conditions will occur
Students with disabilities and/or specific learning disorders (SLDs) must contact the professor and the CInAP representative of the Department of Mathematics and Informatics (DMI) well in advance of the exam date to inform them of their intention to take the exam with appropriate compensatory measures
Examples of frequently asked questions and / or exercises
- Data Ingestion from twitter
- Training of a simple classifier using MLIB
- Stream Processing definition
- Docker and Kubernetes