Integrate Splunk ❤️ BPM
To understand Splunk usage with BPM Products let me take an example of Pega as a BPM product.
Definitions:
Splunk : For layman’s understanding Splunk is a product used to analyse data which is in unstructured format like logs, feed etc or structured data and build presentable dashboards, reports or one stop shop to search across log files. (https://www.splunk.com/)
Pega : BPM tool developed on Java and Object Oriented programming. (https://www.pega.com/bpm)
Why do we need Splunk ?
Lets try to understand the necessity of the Splunk tool using an example. Lets say I have a enterprise application which is built on Pega.
A sample configuration used to cater 1000 users. Lets say we need Pega running on a WebSphere application server. Let’s assume we have like 50 Pega nodes split across 5 App servers residing on a Red Hat Enterprise Linux Server.
Now lets see what logs would we be interested . In the above environment each Pega Node will have following logs:
PegaRULES-ALERTSECURITY.log
BIX → PEGABIX.log
SERVICES-PAL →PegaRULES-SERVICES-PAL.csv
ALERT→ PegaRULES-ALERT.log
ARCHIVE→ PegaRULES-ARCHIVE.log
PEGA→ PegaRULES.log
With above logs being generated by every node(50 PegaRULES.log files). Let’s say I need to lookup for SSL handshake exception on PROD at runtime.
Without Splunk: You reach out to WebAdmin. He fetches logs across the nodes and shares it across. You go over those log files and find the occurrence. Pretty laid back!!
With Splunk: If you are using Splunk and have streamed your logs to it. You could just go to Splunk UI and pass the interested String “SSLHandshakeException” and fetch the results which would look across all the log files and give you the results like below.
Deploying Splunk locally:
In a enterprise application monitoring landscape any organization would use Splunk as a org wide service than procuring license per application. However for learning you can install Splunk locally by following the below link based on kind of OS you have Windows, Mac or linux:
https://docs.splunk.com/Documentation/MSExchange/4.0.0/DeployMSX/InstallaSplunkIndexer
Feeding data to Splunk Engine:
There are couple of ways you can push data to Splunk engine(the below mentioned is what I have used):
- Splunk Universal Forwarder
- Logstash Agent
Splunk Universal Forwarder:
Elaborating on the above shared example where we try deploying the Splunk forwarder which pushes all the logs from the nodes to Splunk for indexing. Basically sending the log stream to a index identified for your application.
Below link helps you with steps for configuring Splunk Forwarder:
https://docs.splunk.com/Documentation/Forwarder/8.0.2/Forwarder/HowtoforwarddatatoSplunkEnterprise
Below pic depicts the splunk forwarder installed on each Server which pushes logs to the Splunk indexer.
Configuring your Splunk Index for Forwarding:
- Splunk saves all logs in a directory based structure called ‘index’ (in splunk’s language). If you did not define any index while configuring your inputs, then splunk will store everything into ‘main’ index by default.
2. You can specify index name in your inputs.conf file. Locate appropriate inputs.conf file usually located under $SPLUNK_HOME/etc/system/local or $SPLUNK_HOME/etc/apps/local directories and define as below:
Under inputs.conf:
[monitor:///custom_path_to_log_files]
index = your_custom_index_name
sourcetype =
your_custom_sourcetype_name ##
Sourcetype is used to differentiate
data types in a single index. Highly
recommended by splunk to use assign
the right sourcetype while configuring
your inputs. More info @
http://docs.splunk.com/Documentation/Splunk/7.1.1/Data/Listofpretrainedsourcetypes
After the above steps you should be able goto the Splunk Search UI /Search Head and start analyzing your logs.
Query Searches based on index:
chart count by host on a index
Modifying time ranges
Splunk extracts properties from logs based on search like below fields:
Quick Query to check disk space on your server:
index=os sourcetype=df host="yourHostName" | table Filesystem, PercentUsedSpace, UsedMBytes, TotalMBytes, _time |Will add another post of logstash.
To be contd…..
#bpm #splunk #pegasplunk #pegasplunkintegration #bpmsplunk #pega #pegasystems