Chaos Toolkit LOVES Chaos Monkey for Spring Boot

You can now run Application-Level Chaos Engineering experiments using the Chaos Toolkit and the new Spring driver

Russ Miles
Chaos Toolkit
4 min readJun 7, 2018

--

The first release of the Chaos Toolkit’s support for application-level chaos engineering experiments against Spring applications, the chaostoolkit-spring driver, is now available!

Free and open source, you can inspect the code behind the driver in the new chaostoolkit-spring incubator project.

This release includes support for harnessing the excellent Chaos Monkey for Spring Boot support from codecentric so that you can run a variety of chaos engineering probes and actions against your Spring Boot applications and services including:

  • Enabling and Disabling the Chaos Monkey on a specific service at runtime (useful for turning on the Chaos Monkey support for only the duration of your chaos engineering experiment).
  • Enabling and Configuring chaos assaults on a specific service at runtime.
  • Inspecting and recording the configuration of the Chaos Monkey’s watcher and assaults through probes in your experiment. Useful for capturing this information for further analysis after your experiments conclude.

All this from within your declarative Chaos Toolkit automated chaos experiments! Let’s look at a couple of examples.

NOTE: All the code from the following examples is available here.

Experiment Example 1: “What if a dependent service dies?”

One of the things the Chaos Monkey for Spring Boot offers is a way of telling a Spring Boot application to kill itself some watched facet of the application occurs. We can use this feature to explore the question:

“What happens if the service I am dependent on dies unexpectedly?”

In this example we’ve set up two services, a consumer and a provider. The consumer starts off being fairly naive and directly invokes the provider with no consideration for failure.

Suspecting there may be a weakness here we can use the Chaos Toolkit’s support for Spring, the new chaostoolkit-spring driver, to set up these conditions using the Chaos Monkey for Spring Boot to kill the dependent service when it is invoked using the following experiment:

In this experiment we first ensure that the Chaos Monkey for Spring Boot is enabled on the provider service by invoking the enable_chaosmonkey action (see Line 25).

Next we ensure that the killApplicationActive assault is enabled and that it will be triggered every time a request or service invocation is received (Line 37).

Finally we trigger the killApplicationActive assault (Line 58) before examining the effect against our steady-state hypothesis (Line 9).

Assaults are triggered according to the watcher configuration in your Spring Boot application. This is set up by the Chaos Monkey for Spring Boot and is not amendable at runtime. The default configuration for the service below shows that we’ve turned on the watchers for incoming requests (in addition to the default for @Service invocations) in the provider application’s configuration (Line 13)

Using this experiment we can look at how resilient the consumer service is to potentially catastrophic failures in the consumed provider service.

Experiment Example 2: “What if a dependent service becomes slow?”

The Chaos Monkey for Spring Boot also supports introducing latency into your service. We can trigger this assault using the following experiment to ask the question:

“How would a consumer deal with a provider service that suddenly becomes very slow in its responses?”

The key difference here is that Lines 43–45 enable and configure the parameters for chaos-introduced latency on both service and request invocations. Now you can explore how the consuming services deal with the newly slowly-responding provider service.

These are just two examples of the sorts of application-level weaknesses in your system that you can explore now with the Chaos Toolkit’s new support for Spring. Take a peek through the demo code and see what else you can explore, we’d love to hear where you’re taking things using this new driver.

Open Source and Extendable

One of the reasons the Chaos Toolkit is free and open source is because of how quick and easy it is to extend the project to explore new chaos-inducing and system-probing capabilities as new open source, and commercial, projects come to the chaos engineering marketplace.

The Chaos Toolkit community will be tracking the great work on the Chaos Monkey for Spring Boot project so that Chaos Toolkit users get all the power available from this effort.

Getting Involved in the Chaos Toolkit

The Chaos Toolkit is a community-led open source and free project whose goal is to enable everyone to create and use their own automated chaos engineering experiments. Vendor-neutral and free now and always, it’s a great place learn about chaos engineering, to use as the foundation for your own experiments, and even to bring your own chaos engineering experiment requirements and ideas.

You can get involved today by raising issues and feature requests, grabbing the code, or even just joining us on our community slack.

--

--

Russ Miles
Chaos Toolkit

People, Team and Organizational Developer. Writer, psychologist, speaker and humanistic Head of Engineering. https://twitter.com/russmiles