Toggle4s - Feature toggle managing service with etcd backend

Happy devOps

(λx.x)eranga

Published in

Effectz.AI

4 min readJan 8, 2020

Feature toggles

Feature toggles(a.k.a Feature flags) is a software development pattern that can be used to dynamically add/remove(toggle on/off) functionalities(features) of software services. It can be widely used in microservices based service applications. Feature toggles can help a teams to deliver new functionality to users rapidly but safely. Concept of feature toggle is simple. Functionality gets enabled or disabled based on the state of some configuration.

Consider this scenario. Assume we are building a service platfrom with microservices architecture. Recently we have introduced new auth service in to the microservices platfrom. There is a service named octopus which gonna use this auth service to authenticate the users. We can dynamically configure this use auth service feature in the octopus service by using use_auth_service feature toggle. The value of use_auth_service can be stored and read from any feature toggle storage. For an example we can use environment variables(12factorapps), etcd, redis etc to store feature toggle values.

If we want to enable the feature, we just need to set the value of use_auth_service=yes in the feature toggle storage and restart the service. To disable the feature we need to set the value to use_auth_service=no. By this way we can change the behaviour of the service without modifying the code. No release need to be done, just change the value of the toggle and restart the service. There are lot of advantages of feature toggles, specially when releasing new features. We can enable the feature and monitor how it’s working, if something went wrong we can simply disable the new feature and revet to old behavior by setting the toggle value. Read more about feature toggles from here.

About Toggle4s

Toggle4s is a feature toggle managing service which is using etcd distributed key/value pair storage as the feature toggle storage. It stores all the features toggles in etcd storage and facilitates to dynamically configure the toggles via HTTP REST API. There is a web app which is built on top of REST API to facilitate the feature toggle create, update, delete, search functions. Once feature toggle created, other microservices in the platfrom can read the feature toggle values via connecting to etcd cluster. All the source code which relates to this post available in gitlab. Please clone the repo and continue the post.

Toggle4s Architecture

Following is the high level architecture of the toggle4s. It mainly contains three components.

Toggle4s API
Etcd cluster
Web app

Toggle4s API

toggle4s-api is the HTTP REST API in toggle4s. It allows to create, update, delete and search toggles from etcd storage. toggle4s-api service built with scala by using http4s functional HTTP library, circe JSON library and cat-effects IO monad stack. Following are the main APIs exposed on toggle4s-api service.

Etcd cluster

Toggle4s stores all the feature toggles in etcd storage(etcd cluster). etcd keys which corresponds to feature toggles starting with toggles/ prefix and follow toggles/<service-name>/<toggle-name> format. Assume we need to create a feature toggle feature_user_auth: no for a service named octopus. When creating this feature toggle, it will add toggles/octopus/feature_user_auth: no key/value pair in etcd storage. Other microservices can read the values of the features toggles via connecting to etcd cluster.

Web app

Web app can be built on top of toggle4s-api(HTTP REST API). It can interact with the APIs on toggle4s-api service to create, update, delete and search the feature toggle from etcd storage. Following are some main functionalities that can be implemented in web app.

Create feature toggle
View feature toggle
Update feature toggle
Dump/Export all feature toggles in a service to JSON file
Dump/Export all feature toggles in all services JSON file

Deploy Toggle4s

Deploy services

There is a separate docker-compose.yml deployment file to deploy the related services in toggle4s. I have configured ETCD_HOST in the toggle4s-api service via environment variable(ETCD_HOST=host.docker.internal). host.docker.internal refer to docker host in MAC-OS. In linux environment you have to replace host.docker.internal with your local machine IP.

Following is the way to deploy etcd and toggle4s-api services. Wait few seconds after deploying etcd before deploying the toggle4s-api service(etcd take few seconds to start the service).

Test services

After deploying the services we can interact with the REST API endpoints which exposed in toggle4s-api service. Following are the request and responses of each and every API endpoint.