Alerting
Hello and welcome to our new article which will be covering the alerting part in our SOCaaS solution. As you all know alerts in any SOC play a vital rule in notifying the response team.
So, they can interrupt the cyber kill chain or monitor that attack furthermore depending on the enterprise and team policy. You are probably wondering why we need to cover more alerts. Isn’t the alert modules of Open Distro enough? Well, it is because it lacks in terms of the number of outputs and its integrability with the rest of our solution like Thehive. Thus we will be introducing you to other alternative.
I encourage you all to check the previous article to have a better understanding of what we are going to discuss here.
The article is divided into these sections:
*Installing and configuring ElastAlert, ElastAlert-Server and Praeco
*Creating Rules
1-Installing and configuring ElastAlert, ElastAlert-Server and Praeco :
1.1 Introduction:
A- Definitions
- Praeco: allows for the building of alerts with notification options including Slack, e-mail, Telegram, Jira.
Alerts within Praeco can either be assembled by choosing fields to be alerted on and their relevant operators using the query builder, or manually using the Kibana Query Language (KQL).
- ElastAlert is a simple framework for alerting on anomalies, spikes, or other patterns of interest from data in Elasticsearch. It works by combining Elasticsearch with two types of components, rule types and alerts. Elasticsearch is periodically queried and the data is passed to the rule type, which determines when a match is found. When a match occurs, it is given to one or more alerts, which take action based on the match.
This is configured by a set of rules, each of which defines a query, a rule type, and a set of alerts.
- Sigma Rules: Sigma is a generic and open signature format that allows you to describe relevant log events in a straight forward manner. The rule format is very flexible, easy to write and applicable to any type of log file. The main purpose of this project is to provide a structured form in which researchers or analysts can describe their once developed detection methods and make them shareable with others.
B- Projects cloning:
cd /etc
git clone https://github.com/Yelp/elastalert.git
git clone https://github.com/ServerCentral/elastalert-server.git
You can find more information in this URL : https://github.com/ServerCentral/praeco
1.2-Setting up Elastalert:
cd /etc/elastalert
mkdir rules rule_templates
cp config.yaml.example config.yaml
nano config.yaml
Configure the elastalert config.yaml with:
Your es_host: localhost
A unique writeback_index: elastalert_status
Change the rules_folder to rules
NOTE : If you are running python 2.7 , you have to change it to 3.6
A- Installing python3.6 on Ubuntu :
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt install python3.6
B-Updating python configuration :
sudo update-alternatives — install /usr/bin/python python /usr/bin/python2.7
sudo update-alternatives — install /usr/bin/python python /usr/bin/python3.6
C-Changing the default python:
update-alternatives — config python
And select python3.6
You should now have python3.6 by default
D-Install pip3:
sudo apt install python3-pip
E- You also have to install PyYAML ( example 5.1 ):
pip install PyYAML==5.1
F- Install Requirement and elastalert
cd /etc/elastalert
pip3 install “setuptools>=11.3”
python setup.py install
G- Creating index :
cd /usr/local/bin/
./elastalert-create-index
ES Host : localhost
ES Port : 9200
Use ssl : t
Verify ssl :f
We will use the default ES_username : admin and ES_password : admin
Also keep the default answer for the rest of questions
1.3- Setting up the API server :
Configure the API server /etc/elastalert-server/config/config.json with:
- An absolute path to your elastalert folder for elastalertPath : /etc/elastalert
- The address of your elasticsearch instance for es_host : elasticsearch
- The same writeback_index from the config.yaml : elastalert_status
A- Troubelshooting Alert Logs ( No Data ) :
The metadata handler specific to the alert logging writeback index, is searching for documents with a _type of elastalert. Since 7.x, this returns no results, because all documents have a _type of _doc.
So in your Alert Log ( in Praeco Interface Later ) you will see No Data.
So you have to :
cd /etc/elastalert-server/src/handlers/metadata/
nano get.js
remove the line containing type: ‘elastalert’,
Now you will be able to see your alert logs in your praeco interface
B-Install Elastalert-Server :
sudo npm install
sudo npm run start
You should see this line if it started successfully. This is just a warning because of the insecure connection ( SSL_verify = False ).
1.4- Setting up Praeco :
A- Change Configuration files :
cd /etc/praeco/config
nano api.config.json
nano elastalert.yml
B- Install Praeco :
sudo npm install
export PRAECO_ELASTICSEARCH=localhost
C- Copy BaseRule.cfg :
Before starting the services you have to :
cp /etc/praeco/rules/BaseRule.config /etc/elastalert/rules/
This file contains Settings for Slack , SMTP and Telegram
Here we will add the Slack Webhook URL used in Section 2
cd /etc/elastalert/rules/
nano BaseRule.config
add the Webhook URL
D- Start Praeco :
npm run serve
You should now see the UI running at http://yourServerIP:8080.
Here is your Praeco Interface
2- Creating Rules:
2.1.- Creating Rules with Praeco Interface and sending them to slack webhook :
Navigate to Rules — -> Add Rule:
Now, you can see that creating the rule is very similar to Open Distro Alerting Tool, we will filter the alert and specify the destination.
Click on UNFILTERED and specify the filter manually or using the prebuild tool.
Then Click Close
We will use Slack notifications, with the Same Webhook URL used in Section 2 and same channel ( #test)
Click Save
Your Alert is enabled by default
We can check that the alerts were successfully sent to Slack.
In our slack channel:
2.2- Sending Alerts from ElastAlert to TheHive
Unfortunately, Praeco does not provide an alert output to TheHive, so we will be editing our rules manually and sending them using elastalert-server.
With this workaround, rules will be working well in the background thanks to Elastalert-server, they will be also present in Praeco interface but unfortunately, we will not be able to edit or configuring them using Praeco interface.
A- Creating a rule: “User_creation”:
First, we will create our rule from Praceo Interface like we did before, we will specify any URL in our HTTP output because we will be removing it later on.
When you finish click Save.
B- Shipping alerts to Thehive:
Add TheHive setting , Save and restart Elastalert-Server
Go to /etc/elastalert/rules
nano User_creation.yml
C- Checking alerts in Praeco :
Our Alert was successfully sent to the Hive
Unfortunately, as we mentioned before, the rule will not be editable with Praeco Interface, you will have to edit it manually in /etc/elastalert/rules :
D- Checking alerts in TheHive Interface:
2.3- Get help with Sigma tool to create rules :
As mentioned before, Sigma tool helped us convert sigma rules in multiple formats including Elastalert.
Project URL with detailed steps : https://github.com/Neo23x0/sigma.git
A- Download Sigma tool
cd ~
git clone https://github.com/Neo23x0/sigma.git
B- Create your alert using sigma tool
cd ~/sigma/tools
pip3 install -r requirements.txt
Execute ( example of rule ) :
./sigmac -t elastalert -c winlogbeat ../rules/windows/builtin/win_user_creation.yml
Unfortunately, this rule can’t be directly used by ( Praeco/Elastalert-Server ) because of several missing fields.
So, you can pick the important information from this rule (the query string) and create your own rule in Praeco interface with this information.
This tool is very important because it helps you to gather information about a lot of rules and their query strings.
Note: Sometimes you have to check your logs in (Kibana → Discover Interface) and their fields to make sure that name fields in your logs match with name fields in sigma rules. If your fields show a yellow error, go to index pattern, choose the index to match and click refresh fields.
2.4- Sending Wazuh Alerts to theHive :
We will use the same workaround mentioned before for wazuh alerts , first we will create wazuh-alerts using praeco interface then we will edit the rule file manually to add theHive output :
A- Create a wazuh rule and save it :
We have used to filter the rule with rule.id ( you can pick any other field )
You can get rules id under wazuh → Overview → Security events
B-Edit the rule and restart elastalert-server :
nano /etc/elastalert/wazuh-alert-TEST.yaml
C- Checking alerts :
Thank you for your Time. We hope you liked this article and got a better understanding of the alerting system used in our project. See you in the next article.
