Parallel Programming vs. Concurrent Programming
What is the difference between parallel programming and concurrent programming? There is a lot of definitions in the literature.
“Executing simultaneously” vs. “in progress at the same time”
For instance, The Art of Concurrency defines the difference as follows:
A system is said to be concurrent if it can support two or more actions in progress at the same time. A system is said to be parallel if it can support two or more actions executing simultaneously. The key concept and difference between these definitions is the phrase “in progress.”
This definition says that, in concurrent systems, multiple actions can be in progress (may not be executed) at the same time. Meanwhile, multiple actions are simultaneously executed in parallel systems. In fact, concurrency and parallelism are conceptually overlapped to some degree, but “in progress” clearly makes them different.
Even though such definition is concrete and precise, it is not intuitive enough; we cannot easily imagine what “in progress” indicates.
Recently, I am taking an online course Parallel Programming on Coursera, and the course gave me more intuitive and easy-to-understand definition of parallel and concurrent programming as follows.
Parallel: Efficiency is its main concern
When we consider parallel programming, programs use parallel hardwares to execute computation more quickly.
Here, “parallel hardwares” could be:
- multi-core processors
- symmetric multiprocessors
- graphics processing unit (GPU)
- field-programmable gate arrays (FPGAs)
- computer clusters
in wide-ranging scale.
More concretely, parallel programming requires us to think about:
- How does code divide original huge problem into smaller sub-problems?
- Which is the optimal use of parallel hardware?
Since parallel programming strongly focuses on speeding-up computational time, applications such as matrix multiplication, data analysis, 3D rendering and particle simulation can be discussed in the paradigm.
Keep reading original post at takuti.me…
