Apache Camel: Your Open Source Ally for Message-Oriented Middleware

If you’re dealing with multiple data sources and destinations, Apache Camel is worth a try.

We first used and learned Apache Camel for a Banking development project. Although the microservice architecture was not intensive in data processing (just some reorganization, renaming, and grouping in most cases), the challenge was dealing with multiple data sources and destinations. We worked with a cache, two databases (a relational one and a non-relational one), a message queue, and multiple external services integrations with both Rest and SOAP.

By simplifying the communications and the code written for it, being explicit in the information flow, and providing plugins for each integration needed, Camel was the best fit we found for the job.

Apache Camel works really well within architectures built heavily on open source code. This free integration framework supports over 50 data formats and is packed with several hundred components that are used to access databases, message queues, APIs…you name it. As an open-source framework Camel relies on its community for support. Bear in mind that it can require extensive research and a certain level of shared team knowledge to fully take advantage of its benefits.

So let’s dive in!

This post gives you a comprehensive introduction to Camel without requiring any previous knowledge. Understanding the way it handles data will help technical leaders and architects to decide whether or not Camel would be a good choice for their project and team. Also, it will help new developers to start a project with Camel and understand how it structures code and the role of each part.

Anatomy of a Camel-Based Application

Camel Routes

A Camel application consists of a series of routes which are objects that can be configured to move information from one endpoint to another. A camel route starts with from(endpoint) and ends with to(endpoint), and it moves a block of information from the starting endpoint to the finishing endpoint with processing in between, should it be needed. These endpoints can be other camel routes, files, websites, queues or caches, which is particularly interesting for easily integrating multiple kinds of transportation methods in the same model: HTTP, ActiveMQ, JMS, CXF, etc.

Camel routes are configured in the configure() method of a RouteBuilder (org.apache.camel.builder.RouteBuilder). All routes can be included in a single file, or rather in multiple categorized files for better organization.

We will cover this further with examples, later on. But first, let’s talk about REST APIs with Camel.

REST API

Camel also provides the possibility to create a REST API through route definitions just like the ones we’ve been studying. A REST API route starts with the word rest and it requires an http method such as get(), post(), put(), etc.

Below, we show an example of a Camel Route Builder with a REST API definition:

Let’s analyze this route definition step by step.

First of all, the class extends RouteBuilder and overrides the method configure(), where we define all the necessary routes and REST definitions. The @Component annotation is necessary for Spring to recognize and scan the class.

Next, we defined the clause restConfiguration(), which allows us to configure certain aspects of our REST API as needed.

We then defined a GET endpoint with the path “/hello”. As we can see, the reserved keywords used for this purpose are rest and get, specifying the path of the endpoint. We also added logic inside the definition, preceded by the keyword route(). This logic chooses between a name header or the default value “world” depending on whether the header is present. The structure is equivalent to the if-else structure we are all familiar with, being choice() the opening sentence, when() a condition, and otherwise() the default for values that don’t match the aforementioned condition. There can be as many when() clauses as required.

Inside the choice, we use the value of the request header “name” of the request to define what we call properties. For now, let’s just say properties are values that, unlike headers, live throughout the entire course of the application. We will talk further about this later.

Let’s see this endpoint in action:

Now let’s make a few changes to see a bit better how routes work.

As we can see, we may define many endpoints with the same rest() clause, and the path may also be determined in the definition of each endpoint, along with its method.

The logic of each endpoint can be contained in a different route definition to aid for a cleaner code. We use the direct component to define synchronous routes that can be invoked by other Camel routes in the same CamelContext. Other examples of components are File, ActiveMQ, CXF, HTTP, Kafka, MongoDB, etc. As we mentioned earlier, the flow of information that travels through routes is defined with the clauses from(endpoint) and to(endpoint).

Exception handling

Exceptions can be caught in the Router with the clause onException(). Say we are, for instance, calling an external service and it doesn’t respond in time. We’ll probably want to catch a TimeOutException and do something in order to respond properly, for example, processing the error information with a certain processor and formatting it according to our preference. The same goes for any kind of exception. Let’s go back to the “hello world” example and assume that the query parameter “name” is compulsory:

The clause can either be included inside a specific route definition in order to handle exceptions thrown in said route as shown in the example, or on top of all route definitions in order to catch exceptions that could be thrown anywhere in the router.

It’s important to mention the clause handled(), which determines what to do when an exception is thrown. In this case, since we want to stop the execution and handle the exception, we used handled(true). If we used handled(false), the exception would be rethrown and it could either be caught in a different onException clause or crash the application. The default value is false, which means if we don’t include the clause, the exception will continue to crash the application.

This would happen if we use .handled(false) (or none at all) instead:

Exchanges

Now, what kind of structure travels through these routes we’ve been talking about? It is a block of information we call an Exchange. The information is stored in properties, headers and body of the Exchange, some of which we’ve covered in the previous examples.

Headers and bodies are volatile, and they are saved in two different Messages or parts of the Exchange: IN (incoming information) and OUT (outgoing information). We call them volatile because they don’t live throughout the entire course of the application, but rather die after reaching an endpoint or processing information from the exchange. Properties, on the other hand, are constant, which means once they are set they can be accessed and modified throughout the entire flow.

Processors

We previously mentioned that the information that goes through Camel routes can be processed. What does this mean? For example, we might need to set certain properties in the Exchange, validate or transform data, handle exceptions, etc.

The data processing we talked about is performed in a Processor class that implements the Camel Processor interface (org.apache.camel.Processor), which can look a bit like this:

In the example above we see how to set a header in the exchange, which can be later on accessed and used in the route. A body and properties can be set in the exchange as well. Headers and bodies are set similarly, although, properties are a bit different. Since they live throughout the entire flow of the application, they are set in the exchange rather than its “in component” or “out component”, by writing exchange.setProperty(“some_property_name”, “some_property_value”).

As for using the processor in the router, it simply needs to be instantiated in the Router class and called with .process(processorName). See the example below:

Here it is in action:

To better understand this concept, you can think of Camel routes as a graph that has processors as its nodes, joined through lines that connect the output of one processor to the input of another one.

External API call

Working with external APIs is very simple with Camel. For this example project, we will use the Weather Stack API, which exposes an endpoint that returns the current weather information based on a location parameter. Our application will expose an endpoint that provides a bit of advice on what to wear depending on the weather: if it’s cold, it will suggest bringing a sweater, if it rains, an umbrella. If it’s sunny, it will recommend sunscreen.

Let’s see first how to set up the route:

First of all, as a disclaimer, for the purposes of this example and to aid with brevity we left out thorough exception handling and error cases in both the router and the processor for this use case.

The endpoint receives a single query parameter: the city whose weather we want to look up. The Camel endpoint receives it as a header named, which we then convert into a property in order to be able to access it later. Both the property and the header are named “city”, a constant we designed with the name CITY_TAG.

Before performing the API call we remove all headers to make sure we only send the ones that are necessary. We then proceed to set the HTTP_METHOD header (in this case, we will be performing a GET request).

Now in the next line, we come across something we haven’t seen before: the .toD clause. We did see the .to clause, but what does the D stand for? It means that the String sent as a parameter is dynamic, meaning that it will be defined during runtime. What does this imply? First, let’s check out the method that generates the service url:

The url we want this method to produce is, for example:

http://api.weatherstack.com/current?access_key=123&query=buenos+aires&units=m.

However, it’s not a hardcoded url, because it depends on the value of the “query” parameter (which represents the city), which we will not have until the user has provided it. So, if we were to use the method .to() instead of .toD(), the produced url would look like this:

http://api.weatherstack.com/current?access_key=123&query=${property.city}&units=m

Because the url would be defined while compiling the application and generating the routes. The .toD method fixes this issue.

After making the API call, we unmarshal the result from JSON to a POJO we’ve created named WeatherResponse, which contains the fields that we need for our application.

Last but not least, we call a processor that will decide what to recommend based on the weather: WeatherAdviserProcessor. Let’s take a look at how it works:

This processor is not any more complex than the one we programmed earlier, only a bit lengthier. The logic included in this processor is merely anecdotic.

And as for the final result:

Conclusion

In this article, we went over some key points regarding message-oriented development with the Apache Camel framework.

Like we mentioned before, it allows developers to integrate multiple technologies and exchange information with different kinds of endpoints like files, queues, and external services while being easy to learn. It’s also an open source framework which makes it perfect to try out and see how it works for a specific project, with no cost whatsoever.

All that being said, if your project requires lots of data sources and destinations, like the one our team worked on for the Banking industry, Camel framework is definitely worth a try.

If you want to learn more about Apache Camel you can take a look at its official documentation.