Parallel Programming vs. Concurrent Programming

Takuya Kitazawa
Sep 1, 2018 · 2 min read

What is the difference between parallel programming and concurrent programming? There is a lot of definitions in the literature.

“Executing simultaneously” vs. “in progress at the same time”

For instance, The Art of Concurrency defines the difference as follows:

A system is said to be concurrent if it can support two or more actions in progress at the same time. A system is said to be parallel if it can support two or more actions executing simultaneously. The key concept and difference between these definitions is the phrase “in progress.”

This definition says that, in concurrent systems, multiple actions can be in progress (may not be executed) at the same time. Meanwhile, multiple actions are simultaneously executed in parallel systems. In fact, concurrency and parallelism are conceptually overlapped to some degree, but “in progress” clearly makes them different.

Even though such definition is concrete and precise, it is not intuitive enough; we cannot easily imagine what “in progress” indicates.

Recently, I am taking an online course Parallel Programming on Coursera, and the course gave me more intuitive and easy-to-understand definition of parallel and concurrent programming as follows.

Parallel: Efficiency is its main concern

When we consider parallel programming, programs use parallel hardwares to execute computation more quickly.

Here, “parallel hardwares” could be:

  • multi-core processors
  • symmetric multiprocessors
  • graphics processing unit (GPU)
  • field-programmable gate arrays (FPGAs)
  • computer clusters

in wide-ranging scale.

More concretely, parallel programming requires us to think about:

  • How does code divide original huge problem into smaller sub-problems?
  • Which is the optimal use of parallel hardware?

Since parallel programming strongly focuses on speeding-up computational time, applications such as matrix multiplication, data analysis, 3D rendering and particle simulation can be discussed in the paradigm.


Keep reading original post at takuti.me

Takuya Kitazawa

Written by

A data science engineer at Arm Treasure Data and committer of Apache Hivemall.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade