Home People Publications Talks Teaching Contact Github


Parallel Programming


When: Winter semester 2018-19

CAMPUS #: 18ws-23895

Contact: Prof. Paolo Bientinesi, Markus Hoehnerbach

Language: English


Office hours

  • Prof. Bientinesi: Tuesdays, 11am-1pm, by appointment.
    AICES R432 (Rogowski Building - Schinkelstrasse 2)


Prerequisites

Knowledge of the C programming language. You are supposed to know

  • the basic datatypes; how to define and use data structures (structs)
  • how to operate on pointers and memory addresses
  • how to allocate and free dynamic memory (malloc, free)
  • how to declare and call (recursive) functions
  • how to pass arguments to functions "by value" and "by address"
  • how to use library functions; this means including headers, using the provided data structures, calling functions, ...

Topics

  • Principles of parallel performance
  • Parallel architectures
  • Patterns of parallel programming
  • Processes and threads
  • Open MP
    • Execution model
    • Work sharing
    • Loop-level parallelism
    • Synchronization
  • Message Passing Interface
    • Collective communication
    • Point-to-Point communication
    • Derived data types

Textbook

  • Parallel Programing in Science and Engineering, by Victor Eijkhout [Repository] [HTML]

Lectures

  • Mondays 16:00 - 16:45 HKW 5 (1132|603)
  • Fridays 16:30 - 18:00 1080|005 (R 5), Schinkelstraße 1, Templergraben 51

Exercises

  • Mondays 16:45 - 18:15 HKW 5 (1132|603)

Extra Lectures

  • Mondays 12:15 - 14:30 GRS 001, Schinkelstraße
  • 29.10 - Performance metrics [PDF]
  • 05.11 - MPI
  • 07.01 - MPI
  • 21.01

Lectures

  • 12.10 - Presentation [PDF]. Introduction [PDF].
  • 15.10 - Double lecture. Architectures [PDF], Dependencies [PDF].
  • 19.10 - Performance metrics [PDF]
  • 22.10 - Guest lecture
  • 26.10 - Scalability, Amdahl's law [PDF] [Files]
  • 29.10 - Double lecture [PDF]
  • 02.11 - Processes & Threads. Shared memory vs. distributed memory [PDF]
  • 05.11 - MPI (pt.1): Intro [PDF]
  • 06.11 - 13:00: Guest lecture @ Rogowski Building, room 115. Heinrich Bockhorst (Intel): "One-Sided Communication, MPI on Threads, Overlap of Communication and Computation. Old MPI topics - New Answers?" [PDF]
  • 09.11 - rescheduled
  • 12.11 - rescheduled
  • 16.11 - rescheduled
  • 19.11 - MPI (pt.2): Collective Communication
  • 23.11 - MPI (pt.3): Collective Communication [PDF][Files]
  • 26.11 - MPI (pt.4): Lower Bounds [PDF]
  • 30.11 - MPI (pt.5): Point-to-point Communication [PDF][Files]
  • 03.12 - OpenMP (pt. 1) Intro [PDF][Files]
  • 07.12 - OpenMP (pt. 2) Synchronization [PDF][Files]
  • 10.12 - OpenMP (pt. 3) Worksharing (for) [PDF][Files]
  • 14.12 - OpenMP (pt. 4) Worksharing (sections, single) [PDF][Files]
  • 17.12 - OpenMP (pt. 5) Nested Parallelism [PDF][Files]
  • 21.12 - OpenMP (pt. 6) Tasks [PDF][Files]
  • 07.01 - MPI (pt.6): Communication-computation Overlap [Exercises]
  • 08.01 - 09:00: Guest Lecture @ Rogowski Building, room 115. Torsten Hoefler (ETH): "MPI Remote Memory Access Programming and Scientific Benchmarking of Parallel Codes" [PDF]
  • 11.01 - MPI (pt.7): Point-to-point Communication [PDF]
  • 14.01 - MPI (pt.8): Datatypes. Communicators [PDF][Files]
  • 18.01
  • 21.01 - Exercises: Implementation of MPI collectives
  • 25.01

Exercise sessions


Exams

  • 06.02.2018, 13:00 - 16:00, HKW 1 (1132|203)
  • 06.03.2018, 10:30 - 13:30, VT (1100|U101)