Centralized Logging - Mobile Test Automation with Appium
Introduction
Mobile app testing has become increasingly complex and challenging in recent years, with the need to support a wide range of mobile devices and operating systems.
This complexity has led to the growing popularity of Appium, an open-source mobile test automation framework that allows you to write and run tests for both iOS and Android apps.
While parallel testing can greatly improve the efficiency and speed of Appium tests, it also comes with its own set of challenges, particularly when it comes to real-time monitoring.
Monitoring Appium parallel test logs can be challenging for several reasons:
- Multiple devices: When running tests in parallel, there are multiple devices being tested simultaneously. This means that there are multiple logs being generated at the same time, which can be difficult to track.
- Time synchronization: Parallel testing also requires time synchronization across all devices. This can result in logs being generated at slightly different times, making it difficult to correlate the logs.
- Distributed environment: When running tests in parallel, the test execution is distributed across multiple devices, which can make it difficult to track the logs in a centralized location.
- Volume of logs: When running tests in parallel, the volume of logs can quickly become overwhelming. This can make it challenging to identify specific issues or errors within the logs.
To overcome these challenges, we decided to develop a centralized logging project. After some research, we preferred to use the graylog.
Graylog
- Graylog is a log management tool which works in integration with mongodb and elasticsearch.
- With Graylog, all logs from different sources can be centralized in one place, providing a single point of access for log analysis and troubleshooting. This allows for faster and more efficient problem resolution, as well as easier compliance with regulatory requirements.
- Graylog is highly scalable, which means it can handle large volumes of log data from multiple sources without compromising performance and it also provides powerful search and analysis capabilities, which allow users to easily find and analyze log data from different sources.
- For set up, you can view this document and you can easily set up graylog using docker.
GELF via HTTP
In our current automation system, we have a mobile device farm and these mobile devices connected to multiple servers.
Since we will get different log types from many different sources, instead of the classical graylog logging logic, we found the option to collect the relevant log data in the project and then send it to graylog via a http request within certain configurations. “Gelf Http Input” makes it possible.
After the installation steps, you can go to the system/inputs section from the graylog interface and add gelf http input from there. You can customize certain parameters (which port, buffer size etc) as you wish from here. The result should look something like this:
Finally we have a running graylog server and Gelf Http input endpoint is ready for requests.
Preparing The Logs
Well, now we need to prepare log data to send graylog server for each test lifecycle. We want to save, track and real time monitor 4 different types of log to get more detailed information about the test.
- Appium server log
- Android logcat log
- Test output log
- Test result log
To get appium server log and android device logcat log is quite easy. We can access it from the appium driver like:
In our project we use Junit5 and Spring Boot. There is an OutputCaptureExtension to get test output log data.
To use with @ExtendWithannotation, inject the CapturedOutput as an argument to your test class constructor, test method, or lifecycle methods. It depends on your project architecture, you can figure out best option for the project. Example usage is like below.
Lastly, we need test result log which we can easily get from TestWatcher api. We can get the stacktrace from Throwable object. You can find the example usage in the next gist.
Sending Log Data To Graylog Server
- So we can get 4 different type of log data for our running appium test.
- In our project, we create a context class and logType variables to store this data.
- We create a new extension class to manipulate and send this data to graylog server.
- We ensured that the tests running in the project take different actions according to their status.
- We used RestAssured to make a post request to graylog server.
- The request must be made according to the criteria specified when creating the configuration gelf http input. (url, port etc)
- The sample coding is:
Monitoring in Graylog
- After the data is sent successfully, we can now view it via graylog interface.
- Based on the data gathered in Graylog, expected outcome/results can be obtained with certain search criteria.
- You can filter according to your buildNumber, testMethodName, testStatus or logType etc. You can customize this data with more parameters by sending from your project.
- We can create new dashboards. We can also add new charts, favorite search queries or integrate new analysis tools to graylog.
In conclusion
- With this project, we have been able to access the detailed explanation of the problems experienced faster and in more detail.
- We do not have to wait for the completion of the executed build to see the detailed reporting.
- Based on the data gathered in Graylog database, we can make detailed analysis.
- I hope you liked this article and it was useful for you. See you in the next posts :)
References
