Monitoring Browser History using HistoryLogger, NXLog, and ELK

Browsers are used for almost everything these days: Reading PDFs, browsing the web, and for internal web-based applications. The ‘issue’ for most businesses (and perhaps also parents) is the fact, that almost all internet traffic has become encrypted. So, unless you are using some form of MiTM (Man-in-the-Middle) proxy solution with SSL bumping, then you are not seeing the complete picture. Browser history has long been used by IT-Security forensics after an unfortunate event, but why not collect all the information while the activity is occurring?

Introducing HistoryLogger

HistoryLogger is a simple, yet very effective, tool for creating readable log files containing users browsing history. HistoryLogger is a Windows Service that continuously monitors Mozilla Firefox, Google Chrome, and Microsoft Edge (Chromium-based) for new user activity, and keeps a record of it in log files named after each browser. HistoryLogger can be downloaded here: Download.

Image for post
Image for post
Log files created by HistoryLogger

Each log file is created using the same syntax: Time;Username;Title;URL. An example can be seen below from firefox_log.txt:

Image for post
Image for post
Output from HistoryLogger

The field (column 3) is essentially whatever is placed inside the HTML-tagsLine 1 is a Google search for “System32 syswow64The following lines show that the user visited medium.com, what articles the user read, and lastly that the user logged into his protonmail.

Collecting the logs using NXLog

NXLog is an excellent log shipper that, besides being able to collect Windows Security logs, Sysmon logs, can collect custom logs. NXLog can be downloaded here. Below is an example of how NXLog can be configured for the .

Image for post
Image for post
NXLog configuration for collecting HistoryLogger logs for Firefox

The configuration file contains an input, an output, and a route. The input section reads the and keeps track of any new entries. In the output section, it is configured to ship the logs using syslog. The route section describes the relationship between the input and output sections.

Parsing the logs using Logstash

Logstash is an intermediate log processor, and is part of the Elastic suite, alongside Elasticsearch, and Kibana. Logstash might seem like a weird beast at first, however, when you realize the potential of Logstash it can help you turn ugly logs beautiful, and remove any unwanted logs as well. A Logstash configuration consists of three sections: Input, Filter, and Output. Below is an example of how Logstash can be configured for HistoryLogger:

First, we know that NXLog is shipping the to UDP port 1514. Logstash will be configured to listen on this port and logs received on this port will be given a “type” named which will be used later.

Image for post
Image for post
The input filter for firefox_log.txt

Second, the filter section, which is where the magic happens. The csv filter is used first, and it reads the log as being separated, and it creates the columns last_visit, user, title, and url.

Next, is an example of how Logstash can be used to highlight Google searches, which will later be used when visualizing the data in Kibana. Looking at the logs in it can be seen that Google searches all contain the string “- Google-søgning” (That is if your operating system is in Danish like mine). In Logstash we perform a simple string search, and if it is true, then we add a field called containing the value “Google”.

Image for post
Image for post
Filter configuration for firefox_log.txt

A similar configuration could be created to highlight Wikipedia, Medium, and, Github searches, etc.

And finally, the output section is configured to store the logs in the right index in Elasticsearch.

Image for post
Image for post

Visualizing it using Kibana

Lastly, the logs can be visualized in Kibana. Using the “action” field created earlier, it is possible to create visualization only for Google searches as seen in the upper-left corner of the dashboard.

Image for post
Image for post
A Kibana dashboad example

Summing up

Browser logs are incredibly valuable, however, most people just rely on their proxy logs for visibility into user activity, and depending on how SSL/TLS is handled by the proxy those logs do not provide the full picture. HistoryLogger is a simple Windows service that can help you gain insight into otherwise encrypted data. Please do remember, that encryption is there for a reason, and, depending on rules and laws, this type of monitoring might be a no-go.

References

I have utilized a bunch of work contributed by others. Please do check out their stuff for more information:

How to search the relevant sqlite-db’s: https://gist.github.com/dropmeaword/9372cbeb29e8390521c2

How to create a Windows installer using Wix: https://www.youtube.com/watch?v=6Yf-eDsRrnM&t=6593s

How to parse Firefox history using PowerShell: https://laconicwolf.com/2017/12/12/parsing-firefox-history-powershell/

How to disable private browsing: https://www.thewindowsclub.com/disable-private-browsing-internet-explorer-chrome-firefox.

Written by

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store