Changing Logging Configurations at Runtime on Wildfly Server
The Problem
If you have ever been asked to investigate an issue that only seems to happen on production, you will quickly realise that being able to enable logs for certain classes or packages at runtime is literally indispensable.
Imagine this…
You have changed the log level of a class/packages in your logging configuration and redeployed just to debug the issue on production… Then, immediately after redeploying the new version of your app with the updated logging configurations to the server, you realise that there is yet another class that you need to reduce the log level to aid your debugging… At this point, I believe that it is clear that this approach is going to be very frustrating and definitely not sustainable.
This is even worse if you work with clients (like my team) who actively monitor downtimes of all their server instances (well everyone should by the way) and any server downtime is quickly detected and would have you drafting a very detailed root cause analysis (RCA) document of what led to the downtime.
Changing logging configurations for debugging would definitely not look pretty on the RCA document. A bulk of the apps my team work on are deployed on Wildfly application server, and when I was assigned tasks to fix a couple of production bugs, it dawned on me that I had to figure out a way to change logging configurations at runtime without any restart or redeployment on the server.
Thankfully, there was a way to do this and the basics of how I achieved it are below. I will split the remainder into 4 sections. Our projects are maven based packaged as Web Archives (WAR) files, hence, this example is targeted at maven projects. Well, who doesn’t build their JavaEE apps with maven these days anyway? So I definitely doubt this will be an issue for most of you reading this.
- Adding the logging profile name to your MANIFEST.MF file of your app.
- Introduction to JBoss CLI (command line interface) tool.
- Disabling the use of in-app logging configurations.
- Adding a simple logging profile.
Adding the logging profile name to your MANIFEST.MF file of your app
To enable runtime logging on your app on Wildfly server, you will need to include a logging-profile property to the MANIFEST.MF file located in the META-INF directory of your application which is usually packaged as a WAR or JAR file. You could do this by adding the logging profile with the desired name to manifestEntries of the maven-war-plugin in your pom.xml as shown in the listing below.
Listing 1
Introduction to the JBoss CLI tool
Wildfly comes with a very useful command tool called JBoss CLI located in the bin directory of the Wildfly home directory. To get started, launch the jboss-cli.bat or jboss-cli.sh script depending on the OS version you are running. It is quite robust and can be used to manipulate just about any configuration at runtime. A lot of resources also exist on JBoss sites showing how to use it as seen in the references below. Once the tool is launched, you proceed by connecting to the Wildfly server using the connect command. You can also specify the IP and management port if necessary. The JBoss CLI tool can be exited using the quit command. This is all shown below…
Disabling the use of in-app logging configurations
To enable the changes to log configuration on Wildfly, you have to disable in-app logging configurations. This basically means that Wildfly should ignore the logging configuration contained in your packaged application (like log4j.properties and so on) and use the configurations in the server. Disabling the use of the in-app logging configurations can be done via CLI by updating the value of the use-deployment-logging-config to false as shown below…
Observe that the success response above comes with some extra info about a reload being required. This basically means that the configuration changes made won’t take effect until the reload command is ran. This can simply be done as shown below…
Adding a simple logging profile
Now that the in-app logging configurations have been disabled as illustrated in the previous section, I will now show the steps to add a simple logging profile to Wildfly at runtime. There are a lot of configurations possible and I have added some valuable links in the reference section just in case you want to explore further.
Since we used “DEMO” as the Logging-Profile value in listing 1 above, we shall be creating a corresponding Logging-Profile with the same name. The steps taken are below…
Add a logging profile with the name “DEMO”
Add a pattern formatter (named demo-log-format) to the logging profile.
Add a size rotating-file-handler.
Add the root logger
Assign the rotating file handler to the root logger
Change the log level of the root logger to ERROR
The commands and responses are shown below
The commands ran in the command line window in the image above are as follows…
/subsystem=logging/logging-profile=DEMO:add
/subsystem=logging/logging-profile=DEMO/pattern-formatter=demo-log-format:add(pattern=”%d{dd-MM-yyyy HH:mm:ss,SSS} %-5p %c{1}:%L — %m%n”
/subsystem=logging/logging-profile=DEMO/size-rotating-file-handler=demo-rotating-handler:add(max-backup-index=40,rotate-size=4M, file={relative-to=”jboss.home.dir”, path=”bin/logs/demo.log”}, suffix=”.yyyy-MM-dd”, named-formatter=demo-log-format)
/subsystem=logging/logging-profile=DEMO/root-logger=ROOT:add
/subsystem=logging/logging-profile=DEMO/root-logger=ROOT:root-logger-assign-handler(name=”demo-rotating-handler”)
/subsystem=logging/logging-profile=DEMO/root-logger=ROOT:change-root-log-level(level=ERROR)
Well, there are a couple of deprecation warnings so the documentation will definitely come in handy to get the updated versions, still, all the commands returned a success response. Now with the logging profile in place, logging configurations for packages/classes can be added, modified and removed at runtime without a server restart or fresh deployment. A sample using a class with the full class name com.github.dawuzi.model.Ping is shown in the image below…
The commands ran in the image above are as follows…
/subsystem=logging/logging-profile=DEMO/logger=com.github.dawuzi.model.Ping:add(level=INFO)
/subsystem=logging/logging-profile=DEMO/logger=com.github.dawuzi.model.Ping:change-log-level(level=DEBUG)
/subsystem=logging/logging-profile=DEMO/logger=com.github.dawuzi.model.Ping:remove()
Enabling change in logging configuration at runtime for applications deployed on Wildfly application server has been very beneficial to my team and is highly recommended. I used Wildfly 13 for this example but it shouldn’t be so different for more recent versions. I intend to write more about things I learn so you can follow me on twitter.
Do you have a different approach to using Wildfly? or something to add to mine? Leave your comments below.
References
[1] https://docs.jboss.org/author/display/WFLY/Command+Line+Interface
[2] https://docs.jboss.org/author/display/AS71/CLI+Recipes
[3] https://developer.jboss.org/wiki/CommandLineInterface
[4] http://docs.wildfly.org/13/Admin_Guide.html
[5] https://wildscribe.github.io/WildFly/13.0/index.html
[6] https://maven.apache.org/shared/maven-archiver/examples/manifestEntries.html
