Advanced Unit Testing with Camel Blueprint

A common integration scenario is when a business object can be received via different transports. And while the business object can go through these different entrances to your enterprise, the goal is to handle the business object in the same way. In this post, I will outline an architecture that routes different transport layers to a business layer using JBoss Fuse, Apache Camel, and Apache ActiveMQ — but that isn’t the main focus. The focus here will be on the subject that is often left to last minute, How do we Unit Test each layer?

Background

This implementation centers on the receiving of a CSV document via the File System, FTP Server, SFTP Server, and Email Attachment transport layers. This CSV document can contain multiple records, and each record is routed to an Orchestration business layer via ActiveMQ and then persisted to a database.

Here is Architecture Diagram:

In Apache Camel, we can construct a listening route that looks like the following:

<route id=”poll.file.receive”>
  <from uri=”file:data/in/flightSegment?delete=true” />
  <log message=”File Received: ${file:name}: ${body}” />
  <split>
    <tokenize token=”\n”></tokenize>
    <log message=”File: FLIGHT SEGMENT RECORD CSV — — ${body}”/>
    <unmarshal>
      <csv delimiter=”,” />
    </unmarshal>
    <to uri=”bean:flightSegmentRecordCsvHandler?method=fromCSV” /
    <marshal ref=”jaxb” />
    <log message=”File: FLIGHT SEGMENT RECORD XML — — ${body}” />
    <to uri=”activemq:{{outbound.jms.dest}}” />
  </split>
</route>

In this route, the CSV Document is split on each row in the file. Each row is then unmarshaled via the Camel CSV data format and passed to a CSV Handler that converts it to a Java Object. This Java Object is then marshaled to a XML Document and sent to an ActiveMQ Destination.

This pattern is repeated for the other transport listeners (FTP, SFTP, and EMail). Each route handles encapsulates their own transport ceremony with the final output a common XML Document. The business orchestration tier accepts this XML document and handles the logic to persist it to a database. With this approach, the business tier does not get into the details of each transport. In fact, it knows nothing of them. Another benefit is that the business tier can be worked on independently of the individual listeners.

So how to unit test this Architecture? There is a need to test against SFTP / FTP / File System / JMS / Database / Email Servers. Wouldn’t it be nice to handle each technology within a unit test, reducing the time to test in the deployed environment where resources are often shared? The following will demonstrate how to embed these servers within the Camel Blueprint Unit Test.

Unit Testing With Blueprint

Apache Camel provides a Camel Blueprint Test Support library which provides a large set utilities/convenience methods that allow for the testing of a OSGi Blueprint context. This framework can be extended to add in dependencies required to test your routes, dependencies such as FTP/SFTP/JMS/EmailServer/Databases.

Creating a FTP Server within your Unit Test

The FTP implementation chosen is Apache FtpServer which is a pure Java FTP server, making it a perfect choice for a Blueprint Unit Test. The details of creating the server involve creating a NativeFileSystemFactory, associating a PropertiesUserManagerFactory, and setting up a ListenerFactory. The code for these steps can be found in my github location.

Once this FTP server is provisioned during the setup of the test, it is then available for the application blueprint context to connect to it. By programmatically placing the CSV payload in the FTP’s server file system, the constructed Camel routes that monitor a FTP location will automatically invoke, thereby causing your Camel Route to execute.

To validate the proper execution, the unit test extends the blueprint context to add an additional route via createRouteBuilder. This Route Builder receives Exchanges from the ActiveMQ destination and routes to a Mock Endpoint. This Mock Endpoint can then be queried for proper execution.

Another benefit of this approach is that it actually embeds an ActiveMQ broker in the Unit Test also, so the code is executing over a similar framework to the deployed environment.

Creating a SFTP Server within your Unit Test

Much in the same way as embedding a FTP Server, a SFTP server can also be provisioned in your unit test. Using the Apache SSHD implementation, the server is constructed during the set up for your application blueprint context to connect to it. The code for these steps can be found in my github location.

The example follows the same pattern as the FTP test. It extends the context to pass the payloads through an ActiveMQ broker and the routed to a Mock Endpoint.

Creating a Mail Server within your Unit Test

The implementation of the mail server that was chosen was IceGreen’s GreenMail server. It provides a pure Java lightweight implementation of SMTP/POP/IMAP including SSL, making it suitable for Unit Tests. The code for the steps to start a Mail server in your unit test can be found in my github location.

Also in the provided code is the steps to construct a MIME message with an attachment (which is the CSV payload) and how to “send” the mail. The application blueprint context is monitoring the email account for new mail and fetches the email when it arrives. This Camel Route then removes each email attachment, converts it to a XML Document and then routes to the Business Layer via a ActiveMQ Destination.

Creating a Database within your Unit Test

There are a number of approaches to embedding a database within your unit test. The example I am providing embeds an Apache Derby database. The application context uses property placeholders to construct the data source, so it is possible to unit test via Derby but the application uses a PostgreSQL Database in the deployed environment.

The unit test approach is to setup a database, construct a database table, and embed an ActiveMQ broker during the unit test setUp. The test then sends XML Documents to the Camel Route that handles incoming XML documents. The Camel Route persists the data to the database and for verification, the database is queried for data.

An example of this approach can be found at my github location.

Putting it all together

I have a working example of this project located here: https://github.com/joelicious/eip-example-polling-service

It contains seven modules which perform the following:

• entities.travel : a module that contains Java Objects with XML Bind Annotations. These objects are used to marshal and unmarshal data between Java and XML.
• services.polling.emaillistener : a module that listens to an Email Account and will pass the attachments of an email to the Orchestration layer. It contains a unit test that embeds a GreenMail server and a JMS broker in order to test the route.
• services.polling.filelistener : a module that watches a file system location for CSV files. It contains a unit test that embeds a JMS broker in order to test the route’s functionality.
• services.polling.ftplistener : a module that monitors a FTP location for CSV files. It contains a unit test that embeds a FTP server and a JMS broker in order to test the route’s functionality.
• services.polling.sftplistener : a module that monitors a SFTP location for CSV files. It contains a unit test that embeds a SFTP server in order to test the route’s functionality.
• orchestration.polling.core : a module that monitors a JMS location for XML Documents and then persists the data to a database. It contains a unit test that embeds a JMS broker and a Derby Database in order to test the route’s functionality.
• features.polling : a module that creates a Karaf Feature file for easier deployments to JBoss Fuse.

To install the application, perform the following:

karaf@root> features:addurl mvn:org.jboss.fuse/features.polling/0.0.1-SNAPSHOT/xml/features
karaf@root> features:install app-polling

Summary

When constructing Camel Routes, it is important to think about how you plan to test them from the very beginning. Different transport protocols are often seen as blockers to unit test, thereby pushing the tests to deployed environment. Being able to work with these transport protocols within a unit test increases the quality of the software as it provides a platform to test those edge cases more easily. The project available in GitHub demonstrates how to embed a FTP/SFTP/JMS/Database/Email Server within your unit tests to bring more of the testing you do back into the unit test.