Using METL to Monitor SymmetricDS
METL’s flexibility as an integration platform allows us to write to many different end point types including SMTP. One way we’ve utilized METL is to monitor our production environment for any potential issues and alert us before these issues escalate.
For one customer we set up a job we called the “System Health Check”. This flow would run every 10 minutes. Multiple systems were monitored when this flow ran. One of those systems was SymmetricDS.
While the modern version of SymmetricDS supports email alerts, this support did not exist at the time. The latest version of SymmetricDS can be configured to send an alert if a batch fails to send. This specific example may not be relevant to you but it should serve as a general example of using the RDBMS reader and email components to monitor another application.
This customer was working in a fast changing environment with schema changes being made by multiple groups across IT. There was a risk that if a schema change was made on the source node and not the destination node, we could run into some type of constraint violation. Without notifications, the sync would be suspended until the logs or the database was reviewed.
We configured METL to monitor both the source and target databases for batches that would go into error. This was done by connecting to both of the databases using the RDBMS reader component.
If an error was detected, the error as well as a description would be sent to the support distribution list to investigate. This allowed us to quickly troubleshoot issues and fix them before the business noticed the interruption.
The initial RDBMS readers perform a simple select from the sym_outgoing_batch table. This table is used by SymmetricDS to group multiple database data changes into groups. These groups or batches have a status in the column ‘status’. The following query was used to query for any batches in error (‘ER’). Note: A custom model was created with an extra column, ‘system_name’. Since we have two source databases, it’s important to record which database or system has the error. Notice the comment hints below are used to map the result set from the query to the METL model.
The next step is to format the email content. Using a groovy script, we can build a message including information about this flow so the recipient has an idea of what sent the message and what the issue might be. The below script gets most of this information from the built in METL parameters. e.g. ‘_flowName’, ‘_agentName’, ‘_host’, etc….
The last component is an ‘Email Writer’. This utilizes a resource of type ‘Mail Session’ which maintains the SMTP server host name.
The Email Writer is set up to reference the parameters created in the previous script component.
The above configuration proved to be very useful limiting downtime and catching issues early. This same process can be used to monitor any other database fields you may need to keep an eye on.
