Parallel Programming


When: Winter semester 2016-17

CAMPUS #: 16ws-23895

Contacts: Prof. Paolo Bientinesi, Dr. Diego Fabregat

Language: English


Office hours

  • Prof. Bientinesi: Tuesdays, 11am-1pm, by appointment.
    AICES R432 (Rogowski Building - Schinkelstrasse 2)
  • Diego Fabregat: Wednesdays, 11am-1pm, by appointment.
    GRS, room 207 (Schinkelstrasse 2a)


Prerequisites

Knowledge of the C programming language. More specifically, you are supposed to know:

  • the basic datatypes; how to define and use data structures (structs)
  • how to operate on pointers and memory addresses
  • how to allocate and free dynamic memory (malloc, free)
  • how to declare and call (recursive) functions
  • how to pass arguments to functions "by value" and "by address"
  • how to use library functions; this means including headers, using the provided data structures, calling functions, ...

Topics

  • Principles of parallel performance
  • Parallel architectures
  • Patterns of parallel programming
  • Processes and threads
  • Open MP
    • Execution model
    • Work sharing
    • Loop-level parallelism
    • Synchronization
  • Message Passing Interface
    • Collective communication
    • Point-to-Point communication
    • Derived data types

Lectures & Exercises

  • Mondays 14:15 - 15:45 (lecture) 1100|U101, VT, Wüllnerstraße 5
  • Mondays 16:15 - 17:45 (exercises) 1100|U101, VT, Wüllnerstraße 5
  • Tuesdays 13:15 - 14:00 (lecture) 1010|213, V, Templergraben 55

Schedule

Lectures

  • 17.10 - Presentation; overview of the semester [Slides]
  • 18.10 - Computer architecture, pipelining [Slides]
  • 24.10 - Pipelining, dependencies [Slides]
  • 25.10 - Timings [Slides] [Code]
  • 31.10 - Speedup, efficiency, strong & weak scalability [Slides]
  • 07.11 - Peak performance, Ahmdal's law [Slides] [Code]
  • 14.11 - Processes and threads [Slides]
  • 15.11 - OpenMP 1 - Parallel regions [Slides] [Code]
  • 21.11 - OpenMP 2 - Synchronization [Slides] [Code]
  • 22.11 - OpenMP 2 - Synchronization [Slides] [Code]
  • 28.11 - OpenMP 3 - Worksharing [Slides] [Code]
  • 29.11 - OpenMP 3 - Worksharing [Slides] [Code]
  • 05.12 - OpenMP 4 - SIMD [Slides]
  • 06.12 - OpenMP 5 - Nested parallelism [Slides] [Code]
  • 12.12 - OpenMP 5 - False sharing [Slides]
  • 13.12 - OpenMP 6 - Basics on tasks [Slides]
  • 19.12 - MPI 1 & Distributed Memory Computations - Preliminaries [Slides] [Code]
  • 20.12 - MPI 2 - Collective Communication (1) [Slides] [Code]
  • 09.01 - MPI 3 - Collective Communication (2) [Slides] [Code]
  • 10.01 - MPI 4 - Lower Bounds [Slides]
  • 16.01 - MPI 5 - Point-to-point Communication [Slides] [Code]
  • 17.01 - MPI 6 - Point-to-point Communication [Slides] [Code]
  • 23.01 - MPI 7 - Exercises, Datatypes [Slides] [Code]
  • 24.01 - MPI 8 - Datatypes, examples [Code]
  • 30.01 - MPI 9 - Communicators, misc exercises [Slides] [Code]
  • 31.01 -
  • 06.02 -
  • 07.02 -

Exercises

Graded Homework

  • Homework 1. Performance Metrics.
  • Homework 2. OpenMP.
  • Homework 3. MPI.


Exams

  • 13.02.2016, 12:15 - 15:45.
  • 03.04.2016, 12:15 - 15:45.