Parallelism and Concurrency: Two related but distinct concepts
What do we mean when we use the terms parallelism and concurrency? What's the difference and how they are related?
In simple words, concurrency is about dealing with lots of things at once and parallelism is about doing lots of things at once.
In most articles and books you would read that concurrency is about dealing with parallel devices and parallel interactions but concurrency is not all about this. Concurrency has nothing to do with parallelism, these two have distinct concepts.
So lets first understand what is concurrency and what is parallelism.
What is Concurrency?
Do you see a single-stepping world doing one thing at a time(Sequential Process) or do you see a complex world of interacting, independently behaving pieces(Concurrent Process)?
All things in the world happening simultaneously from the lowest level like multi-core machines and up to networking, planets, universe….. We live in a concurrent world not sequential but we use sequential computers to write programs. Our computers are not expressing this world-view. This looks like some mismatch, we can solve this by two approaches:
- Make the world sequential
- Make software concurrent
We will always go through the second approach.
Make the world concurrent !!
and in computer system concept of concurrency comes from two notions:
- How the computer executes a process. An executing process has a program-counter (PC) and stack.
- What is the current state of the program? You can represent the state of the program as (PC, stack)
So you can consider a concurrent program as just having lots of (PC, stack) together and in a concurrent program the state as a linear collection of (PC, stack). Now it looks more powerful and easy to understand.
We can say that concurrency is a composition of independently executing processes.
It is a way to structure software and write clean code that can well interact with the real world.
In the above image, the rabbit wants to put all carrots into a bucket. This will too much time for one rabbit to put all carrots into the bucket, for making the process faster we can increase the number of rabbits.
What is Parallelism?
Consider you are cooking a recipe as well as you are on call with your mom speaking about the recipe. Maybe you don’t know but you are doing both works in parallel and this is called parallelism.
Parallelism is about executing two or more things at once not just having two or more things.
In computer language, parallelism is all about doing multiple calculations simultaneously.
Example:
- Concurrency: In your computer, you use an operating system(OS), it can have a mouse driver, a keyboard driver, display driver…etc. All these things are managed by OS as an independent thing. These are concurrent things, they don’t need to be unnecessary parallel. If there is one processor only of them will run at a time.
- Parallelism: parallel things might be something like vector dot product which can be breakdown into microscopic operations that you can execute in parallel.
Concurrency is a structure for efficiently sharing resources. Resources can be anything like it can be anything like variable, database, CPU or network connection. Parallelism is about how many resources are shared and concurrency is all about how resources are shared. Both are intertwined.
How Parallelism and Concurrency are related?
If there is a word concurrency most people start thinking about parallelism, even when Google announce go (which is a concurrent language and concurrent is the most prominent feature of go), programmers said now we can run kinds of stuff parallel. These two words are related but have distinct concepts.
Concurrency is not parallelism even it enables parallelism. If you have the only process executing, your program is concurrent but not parallel. A concurrent program can be run efficiently in parallel on a multiprocessor.
In programming, concurrency is the composition of independently executing processes, while parallelism is the simultaneous execution of multiple things(possibly related).
Concurrency is about efficient structure and parallelism is about execution. But both are important, concurrency is a way to structure things so that you can use parallelism to do a better job.
Making efficient concurrent structure is not about doing less work, even it is about doing more work but makes it faster because the concurrent composition of better-managed pieces can run faster and that parallelism comes from the better concurrent expression of the problem.
We can improve the performance of the program by adding one or more concurrent processes to the existing design/structure. We are adding more things but the whole thing got faster.
Parallelism vs. Concurrency
- An application can be concurrent — but not parallel, which means that it processes more than one task at the same time, but no two tasks are executing at the same time instant.
- An application can be parallel — but not concurrent, which means that it processes multiple sub-tasks of a task in a multi-core CPU at the same time.
- An application can be neither parallel — nor concurrent, which means that it processes all tasks one at a time, sequentially.
- An application can be both parallel — and concurrent, which means that it processes multiple tasks concurrently in a multi-core CPU at the same time.
Now if you are thinking about, is concurrency is better than parallelism? This would be like, Are apples better than oranges? This is not worth to compare two distinct things.
