Understanding health checks in GKE & Gateway API

GKE implementation of Gateway API introduces container native load balancing. Among the functionality the traffic is forwarded directly to a pod from the Application Load Balancer. This also means that the service abstraction in Kubernetes is not used to choose the serving pod for the traffic. While this is great, it also requires health checks of load balancer to be performed directly towards pods from the GCLB. The advantage of it is that those health checks actually test the connection from the load balancer and all the way into the pod. However, this also makes it possible to have a deviating “readiness” that Google Cloud sees and that kubernetes sees. To illustrate this, I’ve created a simple http server with 3 different routes:

package main
import "net/http"
func main() {
	http.HandleFunc("/healthy", func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(http.StatusOK)
	})
	http.HandleFunc("/unhealthy", func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(http.StatusInternalServerError)
	})
	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		w.Write([]byte("Works"))
	})
	http.ListenAndServe(":8080", nil)
}