Splunk and using to set up a detection lab

Krishna Sai Marella
Jul 10 · 8 min read

It’s been a long time, right? Many events took place during this period, for starters, I graduated, and from then on I was a bit in a slump but learned a lot, specifically at SOC roles and Malware Analysis. Besides, learning via online resources, I realized getting my hands dirty by performing the tasks and actually learning from the mistakes whilst chasing the target. Like, how not to delete the root on your guest VM machine, the consequences of not having a backup snapshot, and having to do a tedious process from the start.

Until May, roles that I have explored in cybersecurity were Pentesting, Threat Intel, and Malware Analysis roles. So, during the “Hibernation” (xD) period I have decided I might as well explore the SOC roles, and till now I only have theoretical knowledge on Monitoring and Threat hunting but never got to do hands-on. The next thing I did was to get started with Splunk hands-on and first started to install it on my ubuntu machine and then from my Kali machine. I have tried various attacks like brute force, SQL injection, and many more. Honestly, it was kinda cool, you know, how an attack looks from the perspective of a blue teamer and how powerful Splunk really is.

During my first attempt, I didn’t document any of the work, and I seriously mess with my ubuntu machine, and most of the time I am not the root, so if something bad happens, I could work back. But a few days back I messed up real hard while I was on the root account. The worst part of all was I didn’t take the snapshot of the machine which I usually keep for the rest of my 2 machines (Kali, Windows Sandbox).
So, I got an idea of how about we do it in a true real-life scenario, how about I install it on my main windows machine and look for any suspicious actions that could have taken place on my machine. Generally, I think I keep my main Windows OS as safe as possible and honestly, Windows Defender does all the work for us and with any new vulns, I individually patch ‘em as soon as the threat advisory comes off and I don't install or download anything without verifying the sources and the legitimacy of the file. So, I am basically secure unless some huge 0-day pops-up or I am out of my mind and downloading something off the internet without verifying it. So, I installed Splunk windows on my main desktop, and this time I did document it, so one can apply it and, who knows, one can find anything ‘malicious’ that happened on your machine.
So one can download Splunk from their official site and install it and create your own credentials for Splunk login with admin as username and once done with the installation, you can go to http://127.0.0.1:8000/ in your web browser and log in with the credentials that you created during the installation phase. Upon logging in, you can see the home screen

Home screen

Go to “Search & Reporting” then you will be greeted with a search board and this is where the majority of the work happens.

Search Head

Now the Search bar highlighted in yellow is where you put in your search commands and just like SQL Splunk uses Search Processing Language (SPL). Splunk has a few important components, Indexer, Forwarder, and Search head. So, the Forwarder is used to send the logs from a remote machine to Indexer where the logs are processed to events and made human-readable, and these processed logs are accessed via search heads. In our scenario, we don’t have a forwarder as our logs are from the local machine. One can search and filter their searches with time as highlighted in blue.

Before we start any search we first need to feed our Splunk with logs for that go to Settings → Data Inputs → Local log event collection → Select your desired logs.

In our case, I have chosen Applications, Setup, System, and Security logs, and, honestly, there are more important logs that one can look at. But for the sake of simplicity, and as far I am aware, 4 logs contain most of the information. So let’s get back to searching head-to-head to see what events have taken place on our machine. Back to basics:

“Event is something that has taken place which is either normal or abnormal”.

“Incident is an abnormal event that has took place.”

When we search using search heads we have some constraints and functions one can work with to fine-tune their results first we need to find where our logs are indexed at and while fed the data into the Splunk indexer we have chosen the index to be main. So let’s check index=main

index main

We see our search head back with some results and our time constraint to be last 24 hours, and you can change it by clicking on it and selecting the predefined time and on the left, All Fields section, we can see the source and source type in the section. One thing I have got to say is that my commands from here on are going to be different as I have switched the global search index time to “All Time”. It indexed all the logs that I found on the internet to which shows all kinds of sources types.

And you guys don't need it so from now on you can ignore the first part and just use index=main followed by the rest of the search queries.

So my search command sourcetype="WinEventLog:application" OR sourcetype="WinEventLog:Security" OR sourcetype="WinEventLog:SetUp" OR sourcetype="WinEventLog:System" is equivalent to index=main

One thing that I like about Splunk is the data analysis that one can do with it. One can simply look at the number of events without any hassle. Not much but it does help tho.

Now Splunk has an amazing feature for visualizing the data along with finding the statistics, which helps a lot in finding the security posture of an enterprise.
Before we continue further, we need to know that Windows assigns an ID for every event that ever took place, and we fine-tune it, and this EventID helps us find the incidents that took place.
The first search constraint we got is top, which allows finding the most occurred EventID. By default, the top only shows the top 10, and we change it using top limit=100, which shows the top 100 occurrences.

To find the number of occurrences of all the EventID we can use stats count which counts all the occurrences of different EventID’s.

Now we can visualize the data in the Visualization tab where we the graphs and we can change what kind of graph we are dealing with.

We can save these reports what known as dashboards. To save the report to a dashboard.

Select the Save As and choose the ‘New Dashboard’ option you get this pop-up

dashboard pop-up

Now you can fill in the details such as name, layout style, and much more. To see your dashboards you can navigate to the Dashboards tab and select your dashboard and take a look into the saved reports.

Splunk allows integrating multiple stats commands to get more clarity in the events in my case we can take look into the no of unique events that have taken place with the source type.

Now that we have our EventID’s and their occurrences we can match the EventID’s to that of EventID’s that can be considered as an incident. I have put up the list of events one can look in their windows 10 logs to find if any malicious is happening.

And my most common event that has taken was EventID 5379 which is not considered malicious and the ‘abnormal’ event ID was the 4648 which was understandable because my password is kinda f’d up xD.

So, this was entire work was uncalled for, and honestly by this dumb thing I have learned a lot about what different EventID means and how one can co-relate it to triage an event to an incident. Again this helped me get to know what happened on my PC for the last few months and that felt really good tbh xD even though there were few suspicious events most of them are failed login xD I really need to reduce my password length tho xD.

Again, if you find any suspicious EventID you can simply use index=main EventID=<id number> to get more information about the event as Windows Event Logs has something called a message where you can see more information about it.

I am adding the rest of the stats below.

All Windows Event Logs
Individual Windows Event Logs
Windows System EventID
Windows Setup EvnetID
Windows Security EventID
Windows Appluication EventID

This article is totally unstructured but trust this my first and would be my last one to write it this way.

PS: I am always up for correction and please do give some suggestions that I can work on.

Socials to connect: Twitter @krishnasai_456, LinkedIn- Link and GitHub- Link

Stay safe and stay curious!!! Till next time ❤

Thank you ❤

Nerd For Tech

From Confusion to Clarification

Nerd For Tech

NFT is an Educational Media House. Our mission is to bring the invaluable knowledge and experiences of experts from all over the world to the novice. To know more about us, visit https://www.nerdfortech.org/.

Krishna Sai Marella

Written by

Malware Analysis and Forensics ❤|| In love and hate relation with cryptography || N00b Skiddie || ❤You can bait me with a good cup of coffee ❤

Nerd For Tech

NFT is an Educational Media House. Our mission is to bring the invaluable knowledge and experiences of experts from all over the world to the novice. To know more about us, visit https://www.nerdfortech.org/.