Golang Default GOPROXY Hidden Networking Requirements
Brief overview of Golang GOPROXY defaults and some not well known domain allowlist requirements.
Background
I have very little experience with Golang. I need to read it occasionally to work out where things are going weird because I operate a lot of CNCF and HashiCorp products.
I was recently running some of the Kubernetes performance tooling (some of which depend on Go) and ran in to an issue that doesn’t affect many use-cases and therefor is not well-known.
GOPROXY
By default Golang modules are downloaded via a proxy — for all the benefits of a proxy.
GOPROXY=https://proxy.golang.org,direct
This means that when downloading modules Go should use the proxy and if the module is not found then go direct to the source.
The default proxy is a closed-source system ran by the Go Team / Google.
You are able to use other proxies and there are other flags to ensure that you can download public and private modules from specific proxies or directly as you require.
Problem
Given this is how things work then you might assume you only need to network with the proxy.
Nope!
It turns out that the proxy sometimes returns 302 redirects to Google Cloud Storage for module archives.
Which means Networking Errors if you can not connect to GCS.
Example: No Redirect
curl --include 'https://proxy.golang.org/github.com/google/go-querystring/@v/v1.0.0.zip'HTTP/2 200
Example: With Redirect
curl --include 'https://proxy.golang.org/google.golang.org/api/@v/v0.4.0.zip'HTTP/2 302access-control-allow-origin: *location: https://storage.googleapis.com/proxy-golang-org-prod/3a4011dc517dc6d3-google.golang.org:api-v0.4.0.zip?Expires=1652615465&GoogleAccessId=gcs-urlsigner-prod%40golang-modproxy.iam.gserviceaccount.com&Signature=qX7%2FNYXKrHJXdwJ8tR7WyMZdHPqw%2FAMvaTPAIVkkizqJ5K2mRXS1cFUmyIYhO%2FF%2Fn4XesbnV7Xk5pPtKvrIWS4%2FNiN1akqKz2UA0EHgimGcANaWEbtSX3mBXnpXG4QPGSh56VphsNzkt3pZnqqgkzPE6chE27XlKABMSTV1vrWwz2OU9mhr3Gs6mTRzS2M0h6aJN1ZOrfNL5%2BfRM7HhM0Nkbif5fUM5PaAX%2FpsinSEY%2Byf04yH4UjFQ0x2cuy3FQsTLR%2FxrQJa3%2BD%2FjO7F3AD1tY%2FoK%2B7%2BJtOXDtzX1dAogkGv6PylDWpLS2HVJtAx%2F4SS047tbOsSAxQpRFu41X9Q%3D%3D
See: https://storage.googleapis.com/proxy-golang-org-prod/* .
Resolution
This may not matter to many users but it is knowledge that’s needed if you are in an environment which utilises domain allowlisting.
# Domain Allowlist## Go Proxy
proxy.golang.org
sum.golang.org
index.golang.org### Go Proxy Storage
storage.googleapis.com
Request
This problem seems to have caught the Go Team themselves off-guard at one time but there is very little other mention of it around:
This issue is no biggie. But having looked around the documentation I can not find any reference to the proxy working like this. Given that this is a technical tooling system and not a general consumer service I think the dependency on the external service should be made explicit somewhere.
