Challenging the Future: Building Distributed Parallel Lisp with Easy-ISLisp
Continuing the Challenge
Recently, I released Easy-ISLisp ver4.0 and felt that I had achieved my goal. I planned to relax for a while, enjoying my favorite music and building plastic models. However, I couldn’t stop thinking about Professor Takeuchi’s CELIS. Professor Takeuchi and colleagues were solving puzzles using distributed CELIS, and through their 1990s research, I learned about the Connection Machine CM-1. It was very intriguing, and it inspired me to challenge myself with distributed Lisp.
Distributed Parallel Lisp
Using communication lines, multiple computers are connected. The parent Lisp machine uses threads to decompose calculations and sends them to child Lisp machines for processing.
TCP/IP
TCP/IP is utilized for communication lines, leveraging well-developed C language libraries.
Child Machines
Upon Lisp startup, provide the -n option. The child machine operates like a server, accepting computation requests from the parent Lisp machine, evaluating them, and responding via the communication lines.
Parent Machine
The parent machine must provide the IP addresses of the child machines.Prepare an extension function:
(dp-create ip0 ip1 ip2 ... ipn)This function provides the child machine’s IP addresses and prepares sockets accordingly.
File Transfer
The program to be computed is stored on the parent machine. Using the (load fn) function, it is loaded into the parent machine's memory and transferred to the child machines via the communication lines.
Fully Utilizing Multi-core
Both parent and child machines utilize multi-core processors to their full potential. Even inexpensive computers today typically have around 6 cores each. With 3 machines, this results in 3 * 6 = 18 cores available for use. We maximize the use of multi-core and multi-threading capabilities.
Objective
I aim to achieve these goals in ver5.0. Parallelism is fascinating and my new focus.