What is Message Passing Interface (MPI)

MPI

The message passing interface (MPI) is a standardized interface for exchanging messages between multiple computers running a parallel program across distributed memory. The MPI standard defines the syntax and semantics of library routines helpful in writing portable message-passing programs. MPI’s goals are high performance, scalability, and portability. MPI remains the dominant model used in high-performance computing today

Key Concepts

A communicator defines a group of processes that can communicate with one another. Each process gets assigned a unique rank among this group, and they explicitly communicate with one another by their ranks. Communications that involve one sender and receiver are known as point-to-point communications. The communication which involves all processes is known as collective communication.

The default communicator is MPI_COMM_WORLD which is all processes in the cluster. MPI_Comm_split can be used to create a new communicator, which could be a subset of overall processes in the cluster.

MPI_Comm_split(
 MPI_Comm comm, 
 int color, 
 int key, 
 MPI_Comm* newcomm)

Comm — Base communicator on which you want to create a new one. The old one will still be present. This base communicator could be MPI_COMM_WORLD…