File Based Dynamic Configuration of Routes in Envoy Proxy

Michael Gfeller
Mar 12, 2019 · 3 min read
Image for post
Image for post

As developers are increasingly involved in the configuration of edge proxies, like Envoy, it is important that these appliances provide a developer experience that supports the modern developers’ workflow. With the dynamic service discovery capabilities, Envoy makes this possible. All aspects of the proxy except a static bootstrap configuration can be dynamically configured. This means that the configuration of an Envoy proxy can be updated without cold or hot restart. This is in many cases the preferred and easiest method of handling frequent configuration changes.

The configuration can either be provided by a configuration server, or by files. There are some open source and commercial implementations of such servers. If desired, projects can implement their own, based on available libraries, implementations and examples (envoyproxy/go-control-plane, envoyproxy/java-control-plane). If the requirements do not necessitate a server based solution, a simple file based approach might be sufficient. A file with the new configuration is created, and in a file system move operation the current file is overwritten with the new configuration. Using inotify to monitor the file for changes, Envoy will load the new configuration, and apply it if it is valid. If it is not valid, the previous state will remain active, and a corresponding message will be written to the log.

The whole family of discovery services is designated as xDS. This post shows how the route discovery service (RDS) can be implemented using files as configuration source instead of servers.

The code accompanying this post is available on GitHub. It is based on the front proxy example code from the Envoy proxy repository, documented in the
envoy docs.

In the Envoy configuration file, find the field route_config the envoy.http_connection_manager filter that you would like to replace by a dynamic configuration:

- name: envoy.http_connection_manager
config:
codec_type: auto
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: backend
domains:
- "*"
routes:
- match:
prefix: "/service/1"
route:
cluster: service1
- match:
prefix: "/service/2"
route:
cluster: service2
http_filters:
- name: envoy.router
config: {}

Replace this field with a field called rds that points to the file containing the corresponding configuration:

- name: envoy.http_connection_manager
config:
codec_type: auto
stat_prefix: ingress_http
rds:
route_config_name: local_route
config_source:
path: /etc/rds/rds.yaml
http_filters:
- name: envoy.router
config: {}

The contents of the file rds.yaml is

version_info: "0"
resources:
- "@type": type.googleapis.com/envoy.api.v2.RouteConfiguration
name: local_route
virtual_hosts:
- name: backend
domains:
- "*"
routes:
- match:
prefix: "/service/1"
route:
cluster: service1
- match:
prefix: "/service/2"
route:
cluster: service2

Now, if the configuration needs to be changed, a file with a new configuration can be created, say rds-new.yaml, and then moved over the existing file:

mv rds-new.yaml rds.yaml

Envoy will pick up the change, validate and activate the new configuration.

Further reading and resources:

Originally published at medium.com.

Grensesnittet

Grensesnittet (DK: Grænseflade, EN: interface) er stedet…

Michael Gfeller

Written by

Software engineer, cloud architect, and DevOps enthousiast. Playing accordion and synth, and singing.

Grensesnittet

Grensesnittet (DK: Grænseflade, EN: interface) er stedet for faglig kommunikasjon mellom ansatte i Computas og andre i bransjen. Mulighetene i teknologi er utallige og vi vet at vi kan bruke vår kunnskap til å gjøre en forskjell i folks liv.

Michael Gfeller

Written by

Software engineer, cloud architect, and DevOps enthousiast. Playing accordion and synth, and singing.

Grensesnittet

Grensesnittet (DK: Grænseflade, EN: interface) er stedet for faglig kommunikasjon mellom ansatte i Computas og andre i bransjen. Mulighetene i teknologi er utallige og vi vet at vi kan bruke vår kunnskap til å gjøre en forskjell i folks liv.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store