Parallel Processing and Programming

Introduction

Define terminology commonly used in parallel computing

  1. Task start-up time
  2. Synchronizations
  3. Data communications
  4. Software overhead imposed by parallel compilers, libraries, tools, operating systems, etc.
  5. Task termination time
  1. Hardware — particularly memory-CPU bandwidths and network communications
  2. Application algorithm
  3. Parallel overhead related
  4. Characteristics of your specific application and coding

Describe different parallel architectures, interconnect networks, programming models, and algorithms for common operations

What are some different parallel architectures?

Algorithms for common operations

  • Split array into subproblems amongst nodes in a hypercube
  • Sort each subproblem local within a node
  • Use the first node to broadcast the pivot integer
  • Partition each list in each node, then swap the left and right arrays across the nodes
  • Repeat the last two steps you reach the first node
$ node index.js
Getting subtitles…
Number of Cores: 4
Starting…
Processing video 1 of 4
Processing video 2 of 4
Processing video 3 of 4
Processing video 4 of 4
Completed All
Execution time Single Core Processing: 57.16 Seconds
Number of Cores: 4
Processing video 1 of 4
Processing video 2 of 4
Processing video 3 of 4
Processing video 4 of 4
Starting…
Completed: 3
Completed: 1
Completed: 4
Completed: 2
Completed All
Execution time Parallel Processing: 22.24 Seconds

Develop an efficient parallel algorithm to solve a problem

Parallel summation of integer array

Pseudo Code

  • Add all of these numbers would be to store the first value in a variable
  • if the next index in the array contains an integer, add it to the variable.
  • Repeat until there are no indexes left in the array
  • Return the variable
  • Split up a number array into sections based on the number of items in array divided by the number of cores
  • Spin up new threads that iterate over the number of items in each section and return the sum
  • Combine the sum of the works into a value
  • Return variable
// Main.java

import pack.RandomInt;
import pack.Sequential;
import pack.Parallel;

public class Main {
public static void main(String[] args) {

int cores = Runtime.getRuntime().availableProcessors();
RandomInt numbers = new RandomInt(100000000);

long begin = System.currentTimeMillis();
Sequential seq = new Sequential(numbers.get());
long end = System.currentTimeMillis();
System.out.println(“Sequential Total: “ + seq.total() + “ Completed: “ + ((end — begin)) + “ milliseconds”);

long pbegin = System.currentTimeMillis();
Parallel par = new Parallel(cores);
long pend = System.currentTimeMillis();
System.out.println(“Parallel Total: “ + par.total(numbers.get()) + “ Completed: “ + ((pend — pbegin)) + “ milliseconds”);

}
}

Terminal Output

java Main
Sequential Total: 804629830 Completed: 45 milliseconds
Parallel Total: 804629830 Completed: 2 milliseconds

Exhibit Summaries

Bibliography

--

--

--

I am working toward a simpler, more meaningful world. kristofferhebert.com

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

CastGram: An IoT system to show the ancient colors of the statues using holograms

Interoperable Master Format (IMF) and the Cloud: the Future of OTT Platforms and the Streaming…

Different forms of computing in simple words

What happens when you type ls *.c on shell?

Swarm communication has moved to Discord

Making your first app for Google Assistant

Why you need Zenduty when Slack is down

PKS Create a Kubernetes Cluster on vSphere with VMware NSX-T

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Kristoffer Hebert

Kristoffer Hebert

I am working toward a simpler, more meaningful world. kristofferhebert.com

More from Medium

Predefined Functional Interfaces

JAVA 16 And 17: Evolutions and new features — Part 2

Build Simple REST API Using Vert.x in Kotlin

Bean injection using custom annotation (BeanPostProcessor)