Running Google Cloud Endpoints Extensible Service Proxy (ESP) locally on macOS

How do you get Cloud Endpoints ESP running locally on a macOS machine?


Since the writing of this post, Google has updated their documentation for deploying local ESP to include macOS specific settings. You should use their updated instructions which can be found here.

Recently, I started working on a backend API project for an app I’m starting, and since I already use Google Cloud Platform (GCP), I began to research Cloud Endpoints. It seemed like a good solution for deploying the API, since it provides built-in authentication and monitoring services.

Google Cloud Endpoints

Google provides Endpoints Framework for AppEngine standard for Java and Python. However, since my API backend is written in Go (my favorite language), my decision was to use the OpenAPI specification with AppEngine Flex.

There is great documentation for Cloud Endpoints on the GCP website. Following the “Getting Started” tutorial, I was easily able to implement a simple echo endpoint that prints out a message that is sent in the body. After deploying the API service, and my Go backend, I was able to successfully test the implementation.

Running the proxy locally

The next step was to figure out how to deploy the Cloud Endpoints proxy locally. This would allow for easy development and testing of the API, and to make sure everything is configured and working as expected. I found another doc describing how to set up a local Extensible Service Proxy (ESP).

The documentation is written for a Linux machine, but it was fairly straightforward to use on a Mac. In essence, I needed to set up a service account and then deploy the ESP in a docker container. I downloaded Docker for Mac in order to accomplish this.

Here is the command from Google’s documentation for running the ESP docker container:

sudo docker run -d — name=”esp” — net=”host” -v ~/Downloads:/esp -s $SERVICE_NAME -v $CONFIG_ID -p 8082 -a localhost:8080 -k /esp/serviceaccount.json

Once the API backend and ESP was running locally, I used the following curl command to send a request to the proxy, expecting the “hello from go api!” to be returned:

curl -d '{"message":"hello from go api!"}' -H "content-type:application/json" http://localhost:8082/echo?key=API_KEY

However, instead of the expected response, I got the following connection error from curl:

curl: (7) Failed to connect to localhost port 8082: Connection refused

After making sure that the docker container was running and all of my config options were correct, I was running out of ideas. After a bit of googling, I stumbled onto the source of the problem in this post from the Docker forums. It turns out it was related to the --net=host setting. According to docker docs:

With the network set to host a container will share the host’s network stack and all interfaces from the host will be available to the container.

However, since Docker for Mac is running docker in a VM, instead of natively, the command is connecting the docker container to the VM that’s running docker, not the host macOS. Once I understood the problem, the solution was pretty easy.

Instead of using--net=host, I used port mapping (-p) to map port 8082 to access the proxy. As opposed to the network setting, port mapping seems to work by mapping to the host machine’s ports. Finally, since the proxy running on port 8082 will connect to the api backend running on port 8080, specified by the -a ESP startup setting, I had to change it from “localhost” to the internal IP address in order to connect correctly. Since I removed the --net=host, the ESP would not recognize localhost. So, using ipconfig getifaddr en0 command to get the internal IP address, I passed that to the proxy with port 8080.

The new command to start up the proxy docker container now became the following:

sudo docker run -d — name=”esp” -p 8082:8082 -v ~/Downloads:/esp -s $SERVICE_NAME -v $CONFIG_ID -p 8082 -a INTERNAL_IP_ADDRESS:8080 -k /esp/serviceaccount.json

Hitting the proxy with the curl command again, the expected response was received. By checking the logs on GCP Stackdriver logging, I was able to confirm that everything was working correctly.

Hopefully this post will help someone with setting up and running local ESP for Cloud Endpoints on macOS. Cheers!

PS: If anyone from Google happens to read this, it would be great if Google provided an Endpoints framework implementation for Go on AppEngine standard!