Module 3 — Parallel coding

The goal of this module is to teach basic concepts of parallel programming:

• launching threads within shared memory on multi-core systems (your own computer or server, or a single cluster node),
• using multiple processes on distributed-memory systems (multiple cluster nodes), and
• hybrid programming with multiple processes and multiple threads per process.

We will discuss different approaches to parallelization: task parallelism and data parallelism. We will talk about processing large arrays in parallel. Equally important, we will discuss parallel scaling, typical parallel performance bottlenecks, race conditions and deadlocks, and other potential issues you may encouter in parallel programming.

These concepts are not unique to a single programming language. We cover most of these topics in both Julia and Chapel courses, so feel free to attend one of them (they are independent of each other), or both. Ultimately it comes down to a language preference, so you might want to compare both.

Both languages are relatively new. Julia was designed primarily for scientific computing on a multi-core desktop, with more advanced parallel features added along the way, and not everything working as expected right out of the box. Julia has a larger user base than Chapel.

Chapel was designed specifically for HPC, with support for shared- and distributed-memory programming built into the language.

Tuesday, June 1
Parallel programming in Chapel

Tuesday, June 8
Parallel computing in Julia

Click here to register for this module.