Parallel Programming
When: Winter semester 2016-17
CAMPUS #: 16ws-23895
Contacts: Prof. Paolo Bientinesi, Dr. Diego Fabregat
Language: English
Office hours
- Prof. Bientinesi: Tuesdays, 11am-1pm, by appointment.
AICES R432 (Rogowski Building - Schinkelstrasse 2) - Diego Fabregat: Wednesdays, 11am-1pm, by appointment.
GRS, room 207 (Schinkelstrasse 2a)
Prerequisites
Knowledge of the C programming language. More specifically, you are supposed to know:
- the basic datatypes; how to define and use data structures (structs)
- how to operate on pointers and memory addresses
- how to allocate and free dynamic memory (malloc, free)
- how to declare and call (recursive) functions
- how to pass arguments to functions "by value" and "by address"
- how to use library functions; this means including headers, using the provided data structures, calling functions, ...
Topics
- Principles of parallel performance
- Parallel architectures
- Patterns of parallel programming
- Processes and threads
- Open MP
- Execution model
- Work sharing
- Loop-level parallelism
- Synchronization
- Message Passing Interface
- Collective communication
- Point-to-Point communication
- Derived data types
Lectures & Exercises
- Mondays 14:15 - 15:45 (lecture) 1100|U101, VT, Wüllnerstraße 5
- Mondays 16:15 - 17:45 (exercises) 1100|U101, VT, Wüllnerstraße 5
- Tuesdays 13:15 - 14:00 (lecture) 1010|213, V, Templergraben 55
Schedule
Lectures
- 17.10 - Presentation; overview of the semester [Slides]
- 18.10 - Computer architecture, pipelining [Slides]
- 24.10 - Pipelining, dependencies [Slides]
- 25.10 - Timings [Slides] [Code]
- 31.10 - Speedup, efficiency, strong & weak scalability [Slides]
- 07.11 - Peak performance, Ahmdal's law [Slides] [Code]
- 14.11 - Processes and threads [Slides]
- 15.11 - OpenMP 1 - Parallel regions [Slides] [Code]
- 21.11 - OpenMP 2 - Synchronization [Slides] [Code]
- 22.11 - OpenMP 2 - Synchronization [Slides] [Code]
- 28.11 - OpenMP 3 - Worksharing [Slides] [Code]
- 29.11 - OpenMP 3 - Worksharing [Slides] [Code]
- 05.12 - OpenMP 4 - SIMD [Slides]
- 06.12 - OpenMP 5 - Nested parallelism [Slides] [Code]
- 12.12 - OpenMP 5 - False sharing [Slides]
- 13.12 - OpenMP 6 - Basics on tasks [Slides]
- 19.12 - MPI 1 & Distributed Memory Computations - Preliminaries [Slides] [Code]
- 20.12 - MPI 2 - Collective Communication (1) [Slides] [Code]
- 09.01 - MPI 3 - Collective Communication (2) [Slides] [Code]
- 10.01 - MPI 4 - Lower Bounds [Slides]
- 16.01 - MPI 5 - Point-to-point Communication [Slides] [Code]
- 17.01 - MPI 6 - Point-to-point Communication [Slides] [Code]
- 23.01 - MPI 7 - Exercises, Datatypes [Slides] [Code]
- 24.01 - MPI 8 - Datatypes, examples [Code]
- 30.01 - MPI 9 - Communicators, misc exercises [Slides] [Code]
- 31.01 -
- 06.02 -
- 07.02 -
Exercises
- Introduction to the cluster [Documentation] [Quick test]
- Performance metrics [Exercises]
- OpenMP [Exercises] [Sample code]
- Collectives, Lower Bounds [Exercises]
- MPI coding [Code]
Graded Homework
- Homework 1. Performance Metrics.
- Homework 2. OpenMP.
- Homework 3. MPI.
Exams
- 13.02.2017, 12:15 - 15:45.
- 03.04.2017, 12:15 - 15:45.