Three Lisps on GCP Cloud Functions

Published in

Google Cloud - Community

7 min readJun 11, 2023

I have always been a fan of the power of functional programming. I use it whenever I can, and I find it to be extremely powerful. Unfortunately, functional languages are not mainstream, although they are growing in popularity due to their expressiveness, applicability to multi-core programming and fewer LOC. Still, no functional language has an official SDK from a major cloud provider.

Typical cloud languages include:

Java
Python
Node.js
Ruby
Go
.Net
PHP

In this article, I will show how to run x3 different flavors of Lisp on GCP Cloud Functions by transpiling them to cloud native languages. I will also leverage the Google Functions Framework, a set of libraries which add functionality like an embedded web server, which accelerates development. Finally, for all three languages (Node.js, Python and Java), I will need to create a shim function in the cloud native language, that will call into the Lisp function. The shim will dynamically arrange for the Lisp code to be converted into cloud native code.

NBB on Node.js

nbb uses the Small Clojure Interpreter to convert ClojureScript (Clojure/ClojureScript is the dominant Lisp dialect in use today) into Node.js. While there are other tools that can do the conversion (shadow-cljs is pretty impressive) nbb is undoubtedly the easiest tool for the job. Nbb also supports advanced compilation, fast start-up times, and allows for the importing of native javascript packages. The author Michiel Borkent has also done some amazing educational content on YouTube and several blog posts. His videos focus on nbb, as well as its cousin bb.

My directory structure is flat. Cloud Functions assumes that the executable for Node.js will be located in the index file.

$ ls
hello.cljs   index.mjs   node_modules   package-lock.json   package.json   README.md

A quick look at the package.json file shows that the only dependency is the Google Functions Framework library. This library embeds an Express web server into the code as well. I have nbb already installed on my system. There is a reference to it in my package-lock.json file.

$ cat package.json
{
    "type": "module",
    "scripts": {
        "start": "functions-framework --target=hello"
    },
    "main": "index.mjs",
    "dependencies": {
        "nbb": "~1.2.174",
        "@google-cloud/functions-framework": "~3.2.0"
    }
}

Let’s examine the index.mjs file. You can see that it loads nbb, and then loads the hello.cljs clojurescript file.

$ cat index.mjs
import { loadFile } from 'nbb';

const { hello } = await loadFile('./hello.cljs');

export { hello }

Finally, let’s examine the ClojureScript code. If you look closely, you can see that the code actually mixes both ClojureScript and JavaScript. This is normal, since ClojureScript can call into the JavaScript environment and access native functions and javascript data objects.

$ cat hello.cljs
(ns hello)

(defn hello [req res]
  (js/console.log req)
  (.send res "hello world"))

;; exports
#js {:hello hello}

Let’s deploy this into GCP Cloud Functions:

$ gcloud functions deploy hello \
--runtime nodejs20 \
--trigger-http \
--region us-central1 \
--allow-unauthenticated

Test…

$ curl https://us-central1-testing-355714.cloudfunctions.net/hello
hello world

Hy on Python

Hy (sometimes known as “Hylang”) is a Lisp dialect that is embedded in Python. It’s implemented as a kind of alternative syntax for Python. Compared to Python, Hy offers a variety of extra features, generalizations, and syntactic simplifications, as would be expected of a Lisp. Compared to other Lisps, Hy provides direct access to Python’s built-ins and third-party Python libraries, while allowing you to freely mix imperative, functional, and object-oriented styles of programming [from the Hy website].

Let’s check out the directory structure. It is pretty standard for a Python environment.

$ ls
 app.hy main.py   README.md   requirements.txt

Cloud Functions will look in main.py first. That is where we will put our shim function.

$ cat main.py
import hy
from app import hello

# This shim program starts up python, and then calls the hy lisp program.
# This network indirection allows for app to be run from a Docker container
if __name__ == "__main__":
    app.run(debug=True, host="0.0.0.0", port = 8080)

Let’s examine the hy code. We only need 3 lines of code, which would be about the same for Python. The rest of the file is just comments. The embedded web server is Flask, and the decorator syntax is different from what you might expect, which is why I have two examples of different styles.

(import functions_framework)

(defn [(. functions-framework http)] hello [request]
  "Hello world!")

;; Alternate style also works
;(setv hello
;  (functions-framework.http
;    (fn [request] "Hello world!")))

;; For some reason, I could not get the straightfoward defn to work.
;; (defn [(functions-framework.http)] ...     ; did NOT work

;;   HTTP Cloud Function.
;;    Args:
;;        request (flask.Request): The request object.
;;        <https://flask.palletsprojects.com/en/1.1.x/api/#incoming-request-data>
;;    Returns:
;;        The response text, or any set of values that can be turned into a
;;        Response object using `make_response`
;;        <https://flask.palletsprojects.com/en/1.1.x/api/#flask.make_response>.
;;    Note:
;;        For more information on how Flask integrates with Cloud
;;        Functions, see the `Writing HTTP functions` page.
;;        <https://cloud.google.com/functions/docs/writing/http#http_frameworks>

Deploy and test…

# Deploy
gcloud functions deploy python-http-function \
--gen2 \
--runtime=python311 \
--region=us-east4 \
--source=. \
--entry-point=hello \
--trigger-http \
--allow-unauthenticated

# Test
$ curl https://python-http-function-yspciwmbia-uk.a.run.app
Hello world!

Clojure on Java

Clojure is currently the most mainstream Lisp dialect in use today. It was released in 2007, and is considered battle-tested and mature. It runs on top of the Java VM, and as shown earlier can also target JavaScript via its cousin ClojureScript. Its principal designer Rich Hickey is an excellent speaker, and one of my favorite talks of his discusses simplicity and complexity in software design.

Java continues to be the dominant language in the enterprise. However, it is not dominant in web programming, which tends to favor interpreted and dynamically typed languages like JavaScript, Python and Ruby. Besides Java’s heavyweight nature in terms of strict OOO discipline, it also has a relatively long cold start-up time (several seconds), which can make it unsuitable for GCP Cloud Functions/Cloud Run or AWS Lambda.

The project directory:

$ tree .
.
├── README.md
├── classes
├── deployment
│   └── clojure-gcf-0.1.0-SNAPSHOT-standalone.jar
├── java-function-invoker-1.3.0.jar
├── pom.xml
├── project.clj
├── src
│   └── main
│       ├── clojure
│       │   └── functions
│       │       └── cloudfunction.clj
│       └── java
│           └── functions
│               └── MyCloudFn.java

The Java shim function. This code loads the Clojure namespace, and then calls the Clojure function to handle the HTTP call. Just like the other language specific libraries, the Google Functions Framework uses an embedded web server. In this case it is Jetty.

$ cat src/main/java/functions/MyCloudFn.java
package functions;

import com.google.cloud.functions.HttpRequest;
import com.google.cloud.functions.HttpResponse;
import com.google.cloud.functions.HttpFunction;
import clojure.java.api.Clojure;
import clojure.lang.IFn;

public class MyCloudFn implements HttpFunction {

  static {
    Thread.currentThread().setContextClassLoader(MyCloudFn.class.getClassLoader());
    IFn require = Clojure.var("clojure.core", "require");
    require.invoke(Clojure.read("functions.cloudfunction"));
  }
  private static final IFn service_impl = Clojure.var("functions.cloudfunction", "service");

  @Override
  public void service(HttpRequest request, HttpResponse response)
    throws Exception {
    service_impl.invoke(request, response);
  }
}

Let’s examine the Clojure code.

$ cat src/main/clojure/functions/cloudfunction.clj
(ns functions.cloudfunction
  (:require [cheshire.core :as json])
  (:import (com.google.cloud.functions HttpRequest HttpResponse)))

(defn service
  [^HttpRequest request ^HttpResponse response]
  (let [body (json/parse-stream (.getReader request))
        response-writer (.getWriter response)]
    (println "Received " body)
    (.write response-writer "Hello, World\n")))

Although, I created a pom.xml file, I decided not to use maven directly. I ended up using a Clojure build tool called leiningen. It was able to handle compiling both Java and Clojure, and creating an uberjar as well. I won’t go into all the details, but I will have some references posted below.

# Build Java and Clojure classes and uberjar
$ lein uberjar
Compiling 1 source files to /Users/nbrand/gcp/cloud-functions/clojure-gcf/target/classes

Created /Users/nbrand/gcp/cloud-functions/clojure-gcf/target/clojure-gcf-0.1.0-SNAPSHOT.jar
Created /Users/nbrand/gcp/cloud-functions/clojure-gcf/target/clojure-gcf-0.1.0-SNAPSHOT-standalone.jar

Deploy and test.


# Move uberjar to a dedicated directory for Cloud Functions deployment
mv target/clojure-gcf-0.1.0-SNAPSHOT-standalone.jar ./deployment/

# Deploy
gcloud functions deploy clojureFn \
--gen2 \
--region us-central1 \
--source deployment \
--entry-point functions.MyCloudFn \
--runtime java17 \
--trigger-http \
--memory 512MB \
--allow-unauthenticated

# Test
$ curl https://clojurefn-yspciwmbia-uc.a.run.app
Hello, World

Let’s review the logs. You will see that it took almost 4 seconds for the function to cold-start. While we did not review the other languages, they were all under 1 second.

[httpRequest.requestMethod: GET] [httpRequest.status: 200] [httpRequest.responseSize: 689 B] [httpRequest.latency: 4.215 s] [httpRequest.userAgent: curl 7.88.1] https://clojurefn-yspciwmbia-uc.a.run.app/
INFO::main: Logging initialized @876ms to org.eclipse.jetty.util.log.StdErrLog
INFO:oejs.Server:main: jetty-9.4.49.v20220914; built: 2022-09-14T01:07:36.601Z; git: 4231a3b2e4cb8548a412a789936d640a97b1aa0a; jvm 17.0.7+7-Ubuntu-0ubuntu122.04.2
INFO:oejs.AbstractConnector:main: Started ServerConnector@327471b5{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
INFO:oejs.Server:main: Started @3685ms
Default STARTUP TCP probe succeeded after 1 attempt for container "clojure_fn-1" on port 8080.

I really enjoyed this exercise of bringing my favorite programming language Lisp to GCP Cloud Functions. Both Cloud Functions and Cloud Run are powerful server-less platforms that brings the power of Cloud Infrastructure (auto-scaling, low cost, blue-green deployments, database integration and so on) and ease of deployment to developers.

While Lisp is not considered a cloud native language, the Lisp ecosystem has matured to the point such that it is trivial to compile/transpile into a cloud native language like JavaScript or Python. What I love about all the examples shown in this article, is that these Lisp dialects can take advantage of the hosts language library/package ecosystem, giving me the best of both worlds.

Three Lisps on GCP Cloud Functions

NBB on Node.js

Hy on Python

Clojure on Java

References:

Written by Nick Brandaleone