Distributed Parallel Lisp Midterm Report

Overview

Easy-ISLisp, compliant with ISLisp standards, plans to add distributed parallel capabilities in ver5.0. We are pleased to report that the basic functionality of these features is now operational in ver4.5.

Purpose

Current computers are increasingly multicore, with high-speed clock operation reaching its limits. To supplement this, parallelism appears to be the way forward. Quantum computers represent a breakthrough, but practical application remains distant. Meanwhile, traditional von Neumann digital computers continue to dominate.

Various forms of parallelization are being attempted in popular programming languages today. Programming languages come and go in popularity, and even current mainstream languages’ future is uncertain. However, Lisp, with over 60 years of history rooted in mathematics, is likely to persist resiliently. Gaining experience in parallel computing with this language can be a valuable experience. Basic experience in parallel computing, not just peripheral knowledge, I believe, will remain valuable in the future.

Mechanism of Distribution

Connect multiple computers using TCP/IP communication. Decompose computations from one parent Lisp to several computers for task distribution.

The parent Lisp simultaneously transfers the program for parallel computation to child Lisps. It compiles and loads them from the parent Lisp. For this purpose, the following extensions are provided:

• eisl -n: Launches child Lisps in network mode with the -n option.
• (dp-create c0 c1 …cn): Provides IP addresses of child machines to establish TCP/IP communication with the parent Lisp.
• (dp-let forms body): Distributed parallel version of the let syntax.
• (dp-transfer fn): Transfers file fn to all child machines from the parent machine.
• (dp-compile fn): Compiles file fn on both parent and child machines.
• (dp-load fn): Loads file fn on both parent and child machines.
• (dp-system n sexp): Evaluates S-expression sexp on the nth child Lisp for testing.
• (dp-close): Sends termination command to child machines and closes communication.

Operational Example

• Start Lisp in network mode on child machines.

• Start the parent machine.

• Establish communication with 2 child machines using (dp-create c0 c1).

• Transfer program files to all child machines using (dp-transfer fn).

• Compile program files using (dp-compile fn).

• Load code on all parent and child machines using (dp-load fn).

• Test distributed parallelism using dp-let to ensure functionality on the parent machine.

• close

Interaction with the parent Lisp is displayed on the terminal in child machines.

Future Outlook

Planned parallel functionalities include:

• (dp-exec x0 x1 … xn): Simple parallel execution.
• (dp-part t x0 x1 … xn): Partial parallel execution.
• (dp-part nil x0 x1 … xn): Partial parallel execution.
• (dp-call fn a0 a1 … an): Parallel syntax.
• (dp-report str):Display message(str) on parent terminal

Future Challenges

Currently considering execution with immutable data. Allowing destructive assignment and references to global variables would complicate matters. Control must be maintained to prevent conflicts when child Lisps rewrite global variables.

Immediate focus on deterministic issues, such as solving puzzles or algorithms like QuickSort, completely separated for parallel computation.

Tasks like fluid dynamics calculations would require mechanisms like MPI. This remains a future challenge for when the need arises.

Implementation

Easy-ISLisp is available on GitHub under the BSD 2-Clause License. Feel free to try it out. sasagawa888/eisl: ISLisp interpreter/compiler (github.com)

Digression

The following are personal opinions. Among Lispers, there are those overly fixated on the past or tools like Emacs. It’s crucial to revisit the past to discover the new. Ancient wisdom might pave the way for the future. Clinging only to the past stifles further development. I hope Lisp will be a language of the future.