How to create server/machine disk usage dashboard and trigger “delete logs” DevOps action automatically?

WLSDM for WebLogic
Jan 29, 2019 · 4 min read

Author: Haydar Seyfi and Salih Ozdemir

“Oracle WebLogic Server has quite a number of log files that are generated which naturally tend to grow. Without any type of log rotation, archiving, or purging set up, your file system can be filled up rather quickly and eventually pose stability issues to your application environment.”

Fortunately, WLSDM can create “Generic DevOps Mbeans” to prevent this scenario. It is also extremely simple.

INFO: We are going to create a dashboard in WLSDM console and add Generic DevOps MBean Metrics.

1. Creating Generic DevOps Mbean

1.1 Go to WLSDM console.

Image for post
Image for post

1.2 Go to “Configuration > Monitoring & Diagnostics” page.

Image for post
Image for post

Script Types and Usage

(OS Executables: sh, bash, bat, cmd)

Key / value pairs are the expected output format for the OS executable scripts. Sample output(echo) is listed below.

attributeName=attributeValue  
TotalCompletedOrders=2017
TotalWaitingOrders=35

Note: MBean attribute names cannot contain non alphanumeric characters except underscore ( _ ). Non-alphanumeric characters must be replaced.

Sample Shell Script — 1 : Disk usage of a Linux server

df -H | grep -vE '^Filesystem|tmpfs|lv|dev' | awk '{ print $5"="$4}' | sed 's/%//' | sed '1s/\//ROOT/'

Sample Shell Script — 1 Output :

ROOT=85  
data=67
u01=76

1.3 Type “Script” as the following highlighted script.

echo DISK_DATA=$(df /data | grep /data | awk '{print $5}' | tr '%' ' ') ; echo DISK_ROOT=$(df / | grep / | awk '{print $5}' | tr '%' ' ')

This script will generate below attributes; values comes from domain disk usage.

DISK_DATA=57  
DISK_ROOT=4

1.4 Click “Transform: Run & Get MBean Attributes” button. Wizard is going to generate JMX MBean Attributes then click “Next” button.

Image for post
Image for post

1.5 MBean Values can be visualize on WLSDM Console Smart Dashboards.

1.6 Click the Smart Dashboards > Custom Dashboard in the main menu.

Image for post
Image for post

If you are able to see below dashboard on your WLSDM console, it means you have successfully created Generic Disk Usage Mbean.

Image for post
Image for post

2. Adding Action to Generic DevOps Mbean

INFO: We are going to add an action in Generic DevOps MBean.

2.1 Go to “Configuration > Monitoring & Diagnostics” page.

Image for post
Image for post

2.2 Click the “New User Defined Action / Script” button from the right corner in WLSDM console.

Image for post
Image for post

2.3 Type action name ( i.e. DeleteLogFilesAction )

2.4 Choose Type as “Embedded Script” from “Script Source Type” dropdown list.

2.5 Type “Executable Script” as the following highlighted script(Specify your WebLogic path and log type)

find /data/weblogic/domains/$1/servers/. -type f -mtime +3 | grep -E 'server-2019*|access-2019*' | awk '$1' | xargs rm -rf

This script going to delete your rotate log files (as starting with ‘server-2019’, ‘access-2019’ logs) older than 3 days. You can specify your domain path and log type.
‘$1’ parameters will came from WLSDM “System Argument/Parameters” section if you choose one of them.

Sample scripts;

find /data/wlslogs/$1/. -type -f -mtime +5 | grep -E '*.log0*|*.out0*|*.err0*' | awk '$1' | xargs rm -rf  
find /data/wlslogs/sample-dmn/. -type -f -mtime +5 | grep -E '*.log0*|*.out0*|*.err0*' | awk '$1' | xargs rm -rf

2.6 Select Parameters as “$DOMAIN_NAME” from “System Arguments/Parameters” dropdown list, then click “Save”.

Image for post
Image for post

2.7 Now go to “Smart Dashboards > Custom Dashboard” page.

Image for post
Image for post

2.8 Click “Metric & Chart Options” button.

Image for post
Image for post

2.9 Click “Action” button and select “DeleteLogFilesAction” action then click save.

Image for post
Image for post

2.10 Set the alarm “ON”, choose Operator as “>=” then give Threshold value; finally save your configuration.

Let WLSDM reduce your disk usage and optimize your domain stability.

2.11 Check your domain disk usage with “df -h” command.

Image for post
Image for post
Image for post
Image for post

Our Threshold value is : 50
Data disk usage is : 57

DiskUsage Mbean will trigger “DeleteLogFiles” action and domain disk usage will be reduced.
If you are able to see changes like below dashboard, it means you have successfully completed this Generic Disk Usage Mbean Tutorial.
That’s it!
That’s how WLSDM organize your domains…

Image for post
Image for post

Watch disk usage DevOps MBean youtube tutorial video:

WLSDM for WebLogic

WLSDM is a WebLogic console extension which enables…

WLSDM for WebLogic

Written by

WLSDM for WebLogic

Native Oracle WebLogic Monitoring WLSDM is developed by the best WebLogic developers for the best WebLogic administrators to support mission critical HA domains. Supports all plain WebLogic and Fusion Middleware (FMW) WebLogic domains. Download now! https://wlsdm.com/download

WLSDM for WebLogic

Written by

WLSDM for WebLogic

Native Oracle WebLogic Monitoring WLSDM is developed by the best WebLogic developers for the best WebLogic administrators to support mission critical HA domains. Supports all plain WebLogic and Fusion Middleware (FMW) WebLogic domains. Download now! https://wlsdm.com/download

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store