How to use Logstash along with Sentry?

To pipe logs from any source to Sentry through Logstash, you must install an additional plug-in. Since there is not an official Logstash plug-in for Sentry, we have to use an open-source one. Luckily, Javier Matos has already built a plug-in for Sentry and uploaded it to RubyGems.

First, install the plugin from RubyGems using the Terminal command

After installing the plug-in you should prepare a configuration .conf file to setup Logstash.

Preparing a .conf File

A typical Logstash configuration file consists of three components: input, filter and output. By using these three components, you can input logs from almost any format, filter them by your needs and output them.

.conf File Structure

To configure the output part for Sentry, you should have a Sentry account. After setting up your account at sentry.io, go to Settings -> Client Keys to get your keys. Generally, Client Keys has the format:

[http|https]://[key]:[secret]@[host]/[product_id]

So, you have to fill out the necessary parts of the output part according to your keys you obtained.

• By default, the plugin connects to https://app.getsentry.com/api. Set the url if you have installed Sentry on your own machine:

• If you don’t configure anything else, the necessary fields will be set automatically, i.e., event_id, timestamp (set to @timestamp), logger (set to "logstash") and platform (set to "other"). All the other fields from logstash are going to be put into the extra field in sentry. Additionally, the level is set to "error" and the server_name to the value of host.
• The plugin can write to all the fields that the sentry interface currently supports, i.e., timestamp, message, logger, platform, sdk, level, culprit, server_name, release, tags, environment, modules, extra, fingerprint, exception, sentry.interface.Message, stacktrace, template, breadcrumbs, contexts, request, threads, user, debug_meta, repos, sdk. To set a field, you can either read the value from another field or set it to a constant value by setting the corresponding _value:

As for the Input part, we are reading the logs from a .log file and thereby we need to specify the path of the file to the path variable. Also Logstash uses a .txt file to remember where it was on the file to not to read all along from the beginning every time, although you can change this behavior by changing the starting_position variable. You can specify the path of this file to sincedb_path variable by yourself otherwise Logstash will create a default one for you.

Filter part can be constructed solely for your needs. Here we use a Regex in match variable to extract the necessary information as log level, logdate, logtime and message. Words in capital letters are types of data, and the words in lowercase letters are the variables we assign the data. In the mutate part, we clean up the parsed log, as gsub takes out the “\r” from the end of the message, join converges the jvm variables and merge simply merges the logdate and logtime. Also in the date part, we parse the merged logdate to ISO8601 standard to use it as “@timestamp”. This timestamp uses UTC time and therefore it comes three hours before the Turkish Local Time.

Using the official and unofficial plug-ins, we can construct input, filter and output plugins in many different ways to address our different needs. More information about Logstash can be found at:

Logstash Introduction | Logstash Reference [6.3] | Elastic

Starting Logstash

After preparing the .conf file, you are ready to start the Logstash service. To run with a specific .conf file, use the following command in the logstash home directory:

Now, you can see your logs uploaded to Sentry through Logstash. Well Done!

If you see an exception such as “Could not find or load main class org.logstash.Logstash”, then you may have a space in the name of your logstash home directory. You should remove the spaces to run.

Now, you can view your logs at Sentry. Note that, in order to query them or create visuals to build insights you also need to install ElasticSearch and Kibana.

Note that, if you want to experiment with .log files, you should not add new log lines to the log file by hand using a text editor, since text editors generally do not edit the file but instead they form a new file with the same content and saves it with a different node number. Therefore, Logstash can not remember where it was on the log file.