Automation for everyone with TheHive and WALKOFF
Over the last few years I’ve had a very prominent issue with most Security Incident Response Platforms. They don’t take into account that most security analysts are not developers, which in smaller teams leads to several problems, with the most severe and noticable issue being alert fatigue. To help solve the problem, I’m leveraging two open source systems, namely TheHive (SIRP) and WALKOFF (SOAR) and their integration capabilities.
Update May 2020 — I’ve built a platform — Shuffle — that fixes the issues I had with WALKOFF. Check it out here:
First off, this is not a blogpost about TheHive. Rather, the focus is showing the ease of creating simple playbooks to help incident responders in their day to day tasks. The spotlight in this case, WALKOFF, is SOAR tool, made for security orchestration purposes by NSA. It’s the only real SOAR platform I’ve found which is open source and easy to use. WALKOFF is under heavy development currently, and might not work 100% as seen below, but the essentials should still be right.
If you want a quick and dirty test, here’s a one-button test using docker-compose (creates WALKOFF, TheHive and a webhook with already set up data).
How do you make automation available to everyone? There are four steps to this blogbost:
- Deploy WALKOFF (REALLY straight forward — see their github page)
- Set up the integration between TheHive and Walkoff (webhook)
- Create a sample workflow and add an app
- Do a simple webhook integration test
Step 1: I’m jumping straight to step 2, as the docker-compose setup is real simple. I’ll probably make another post about how to configure WALKOFF and TheHive properly in the future, as there is a lot to be said there, but for now, the defaults are more than enough (use docker-compose).
Step 2: To set up the webhook, add the following to the file /etc/thehive/application.conf on your TheHive endpoint(s). Below is a functional webhook integration for TheHive -> WALKOFF.
webhooks {
myLocalWebHook {
url = "http://webhook:port/webhook"
}
}Step 3: Now to the configuration of WALKOFF! The project, as of this writing is in alpha on their new build which uses Docker Swarm to containerize “apps”, by having workers set up new Docker containers. Apps, in short, contain the backend code used by the WALKOFF frontend to do their task. The point of this post is to make it way easier to try it out, as I’ve used quite some time to get it all to work (not to mention in a locked down environment…). Here again, we’ll do a few steps:
3.1. Add our example TheHive app to WALKOFF
3.2 Add our sample workflow and do a testrun
Step 3.1: I’ve taken the liberty to create a few apps to test out the app framework (which is guaranteed to change). This folder contains a few apps I’ve created to test what is possible. Clone the repository and copy over “thehive” folders to the WALKOFF/apps folder. Restart the walkoff_umpire_1 container to properly reload it (or wait one minute)
git clone https://github.com/frikky/walkoff-apps
cp -r walkoff-apps/thehive ~/YOURWALKOFFLOCATION/apps/
cp -r walkoff-apps/helper ~/YOURWALKOFFLOCATION/apps/
docker restart walkoff_umpire_1Open your browser and go to your WALKOFF endpoint. default: http://localhost:8080. Log in with admin:admin and create a sample workflow as per below.
The app should show up similar to the below picture. This means it’s “installed”.
Drag and drop “show_secret” into the open field, click “Save workflow” in the upper left corner, then “Execute Workflow” further to the right of it. Within 30 to 60 seconds, the docker container for thehive app should be built and executed, and you will see some feedback in the “execution” box in the bottom left corner, as well as with the “show_secret” box becomming yellow and eventually green. “show_secret” is just a way I use to test whether my apps work, as they simply return the URL and TheHive API key used in the backend (don’t keep this in prod 😅).
Step 3.2: As we have our TheHive app in place, we need to do a simple test before moving onto webhooks. As an example, we’ll edit an alert in TheHive by changing it’s title through WALKOFF. I’ve made two usable functions for the app to interact with TheHive, where one is about getting an item from TheHive (alert, case etc.) and another for updating. As we’re about to edit an item, let’s have a look at the function “update_field_string”.
- field_type: This is the field to edit. I write “alert” in the field, as what we want to edit is an alert.
- cur_id: This is the ID of the item (alert) to edit. This can be found by clicking “Preview and Import” on an alert.
- The field to edit “title” in this case, as we want to set the title to something else.
- The actual data to change it to. I’ve set it to “walkoff app automation testing” in this case.
Finally click save and execute again, and you should see the box turn yellow, followed by green again. Below are before an after pictures of an alert that I used this function on. VOILA
Debugging: If at any moment you’re unsure whether anything is happening, open a shell on the machine running WALKOFF and make sure to check the docker logs and containers leveraging these commands. These steps will hopefully be possible through the GUI at some point :)
docker logs -f walkoff_umpire_1 # Logs for umpire(controls workers)
docker logs -f worker (...) # Logs for worker. This might crash.
docker ps # Look for a container named "walkoff_app_thehive..."Step 4: Now that all steps are tested, and you have some general understanding, we can do a proper integrationtest. The webhook repository I linked to earlier is just one example of how this can be done, but the essentials are as follows:
- Alert is made in TheHive (e.g. the testscript in the webhook / thehive4py samples repo)
- TheHive triggers a webhook action for “AlertCreation”
- Webhook looks for a workflow in WALKOFF called “AlertCreation”, then sets the local environment variable “webhook_input” to the webhook value from TheHive
- When we execute, the first action of the WALKOFF workflow will use the input from the previously defined environment variable
- The workflow looks for the word “portscan” in the title of the alert comming from thehive, and subsequently changes the alert title to “walkoff app automation testing”
As we already set up and pointed the webhook from thehive to the webhook in step two, what we need to do is make a workflow. The workflow needs the following set up:
- The name needs to be “AlertCreation”, “CaseUpdate” etc. (full list here)
- It requires an environment variable called “webhook_input”
- A startpoint that can be used to extract the ID, title or whatever else you want
- A box for updating back to TheHive
Example workflow that you can import if you want to skip: gist
As the data looks something like this: {“object”: {“id”: “dcd944869825beea627bcf94e8e04927”}}, we have to retrieve the ID of the alert to be able to update it.
Don’t forget to save, and you should be good to go. Every alert that is created in TheHive should now change name to “walkoff app automation testing”.
If you’re lazy like me, you’d like an example that you can import and try for yourself. Use this script to generate a new Alert in TheHive, and it should within a few minutes (first run, requires Docker build), be up and running. A few features I did not take into account for this issue was “Triggers” and “Conditions”. I’ll probably mention these at a later time.
Again, if you would like an out of the box working solution, use the following link repository (fork with a finished TheHive integration):
I hope I’ve given some hope as for how to attack this issue. I’ll make more integrations for the platform myself, including for Splunk and Carbon Black, as well as some playbooks and hopefully write another post on how to make such an app.
Big ups to both the TheHive team and NSA for making these amazing tools :)
