o Parallel Programming WS 17/18


Parallel Programming


When: Winter semester 2017-18

CAMPUS #: 17ws-23895

Contacts: Prof. Paolo Bientinesi, Dr. William McDoniel

Language: English


Office hours

  • Prof. Bientinesi: Tuesdays, 11am-1pm, by appointment.
    AICES R432 (Rogowski Building - Schinkelstrasse 2)


Prerequisites

Knowledge of the C programming language. You are supposed to know

  • the basic datatypes; how to define and use data structures (structs)
  • how to operate on pointers and memory addresses
  • how to allocate and free dynamic memory (malloc, free)
  • how to declare and call (recursive) functions
  • how to pass arguments to functions "by value" and "by address"
  • how to use library functions; this means including headers, using the provided data structures, calling functions, ...

Topics

  • Principles of parallel performance
  • Parallel architectures
  • Patterns of parallel programming
  • Processes and threads
  • Open MP
    • Execution model
    • Work sharing
    • Loop-level parallelism
    • Synchronization
  • Message Passing Interface
    • Collective communication
    • Point-to-Point communication
    • Derived data types

Lectures & Exercises

  • Mondays 12:15 - 13:00 (lecture) 1080|140 (R 140), Schinkelstraße 1, Templergraben 51
  • Mondays 16:00 - 17:30 (exercises) 2090|120 (FT), Melatener Straße 23
  • Wednesdays 12:15 - 13:45 (lecture) 1080|140 (R 140), Schinkelstraße 1, Templergraben 51

Lectures

  • 11.10 - First lecture. Presentation [PDF].
    Introduction to PP, overview of the semester [PDF].
    Architectures, pipelining, throughput [PDF].
  • 16.10 - Dependencies [PDF].
  • 18.10 - Timings [PDF] [code].
  • 23.10 - OpenMP, part 1. Parallel regions [PDF] [code].
  • 25.10 - Speedup, strong & weak scalability [PDF] [code].
  • 30.10 - OpenMP, part 2 [PDF] [code]. [Exercise for Monday 06.11.17]
  • 06.11 - Ahmdal's law [PDF] [code].
  • 08.11 - OpenMP, part 3. [Synchronization] [Work Sharing 1] [code].
  • 13.11 - OpenMP, part 4: Work-sharing & synchronization cont'd. [slides] [code].
  • 15.11 - OpenMP, part 4: Work-sharing & synchronization cont'd. [slides] [code].
  • 20.11 - OpenMP, part 5: Nested parallelism [slides] [code].
  • 22.11 - Double buffering [Reference: Sec 4.1 & 4.2], Granularity [Reference: Sec 4.3] [NO slides!].
  • 27.11 - OpenMP, part 6: Tasks [slides] [code].
  • 29.11 - MPI, part 1: Introduction, "minimal MPI" [slides] [code].
  • 04.12 - MPI, part 2: first program [code].
  • 06.12 - OpenMP, part 7: Vectorization [slides] [code].
  • 11.12 - MPI, part 3: Collective communication [slides] [code].
  • 13.12 - Sample exam: performance, OpenMP
  • 18.12 - TIS-100
  • 20.12 - MPI, part 4: Collective communication [slides] [code].
  • 08.01 - MPI, part 5: Collective communication [slides].
  • 10.01 - MPI, part 6: Lower bounds [slides] - updated!.
  • 15.01 - MPI, part 7: Point-to-point communication [slides] [code].
  • 17.01 - MPI, part 8: Point-to-point communication [slides] [code].
  • 22.01 - MPI, part 9: Send modes [slides] [code].
  • 24.01 - MPI, part 10: Data types [slides] [code].
  • 29.01 - MPI, part 11: Communicators [slides] [code].
  • 31.01 - Exercises [code].

Exercise sessions


Exams

  • 07.02.2018, 13:00 - 16:00.
  • 26.03.2018, 10:30 - 13:30.
  • [Cheat Sheet]