Increasing observability on Istio: The new Kiali health configuration

Alberto Gutierrez Juanes
Oct 29 · 4 min read

Kiali 1.24 comes with a new health configuration [1]. As we know Kiali has health indicators in different pages like overview, graph, lists…and so on. Kiali bases these health indicators in some signals, one of them on the aggregation of inbound/outbound response codes.

Since health is based in response codes, response with error codes like 401 and 404 are considered as errors. Making the health show a degraded state.

But what happens if we consider the 401/404 a legitimate response code?

Let’s think of a client that is making requests to a service, for example in the next image we have 3 clients (b, c & d) making requests to the y-server and Kiali shows a Degraded status for this service because is returning 4xx. However, the real problem is that the clients are not updated and are trying to use an endpoint that is no longer available.

Another use case could be a service handling authentication where 401 (unauthorized) or 403 (forbidden) responses are “normal” if credentials are wrong so you can use the health config to tell that 401/403 responses are good and prevent Kiali flagging as “unhealthy”.

What does that mean? That clients are consuming an endpoint that doesn’t exist anymore in y-server?
Isn’t that a good case to show that a client is not healthy?
Perhaps saying that a/b/c clients aren’t authorized to consume y-server or something like that?

Image for post
Image for post

For these cases we can take advantage of the health configuration feature [2] to show the correct status according to our environment needs.

Configuration

Custom health configuration is specified in the Kiali CR[3].

For each health configuration you need to fill in the following options

Image for post
Image for post
Options Health Configuration

By default Kiali will apply the following configuration if it cannot find a configuration that matches the resource whose health it is checking.

In other words, Kiali will check first for a configuration that matches the specified namespace, then the specified resource type (kind), and finally that matches the resource name (the default).

Image for post
Image for post
Kiali default configuration

This configuration [3] indicates that for all resources with all names in all namespaces:

For example in the next configuration we have a health check for all resources in the namespace alpha :

For namespace beta will have:

Image for post
Image for post

Conclusion

With this new feature we can customize the health of our services to be aligned with our needs, focusing specifically on the errors that matter for our environment.

Community reported that, in the current implementation, it is necessary to reload Kiali every time there is a need to change the health configuration. The team is researching a different approach based in annotations.

For more ideas on how to improve this feature please open an issue in https://github.com/kiali/kiali/issues.

Coming soon

References

[1]https://kiali.io/documentation/latest/features/#_health_configuration

[2]https://kiali.io/documentation/latest/configuration/health/

[3]https://github.com/kiali/kiali-operator/blob/master/deploy/kiali/kiali_cr.yaml

[4]Recorded demo for Kiali 1.24

Kiali

Service Mesh Observability

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store