Bernard Ligny

About Bernard Ligny

Bernard is a Senior IT Architect working on the Java ecosystem since 1999 in many different functional areas. He feels passionate about building elegant, scalable and effective Java solutions using the most appropriate software and techniques, with these last years a particular interest in E.A.I.

Discovering the power of Apache Camel

These last years, ESB software has been getting more and more popular. If most people usually know what is an ESB, they are fewer to clearly understand the exact role of the different components of such architecture.

For instance, Apache ServiceMix is composed of three major components : Apache Karaf (the OSGI container), Apache ActiveMQ (the message broker) and Apache Camel. By the way, what is exactly Camel ? What is a « routing and mediation engine » ? What is it useful for ?
 
 
 
 
I’ve been working with Camel for about one year now, and I think – although not being at all a Camel guru, that I now have enough hindsight to make you discovering the interest and power of Camel, using some very concrete examples.For the sake of clarity, I will, for the rest of this article, be using the Spring DSL – assuming the reader is familiar with Spring syntax.

The Use Case

Let us imagine we want to implement the following scenario using Camel. Requests for product information are coming as flat files (in CSV format) in a specific folder. Each line of such file contains a single request of a particular customer about a particular car model. We want to send these customers an email about the car they are interested in. To do so, we first need to invoke a web service to get additional customer data (e.g. their email). Then we have to fetch the car characteristics (lets us say a text) from a database. As we want a decent look (ie HTML) for our mails, a small text transformation will also be required.

Of course, we do not want a mere sequential handling of the requests, but would like to introduce some parallelism. Similarly, we do not want to send many times the exact same mail to different customers (but rather a same unique mail to multiple recipients). It would be also nice to exploit the clustering facilities of our back-end to load-balance our calls to web services. And finally, in the event the processing of a request failed, we want to keep trace, in some way or another, of the originating request, so that we can for instance send it by postal mail.
 
A (possible) Camel implementation :

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://www.springframework.org/schema/beans

http://www.springframework.org/schema/beans/spring-beans.xsd


http://camel.apache.org/schema/spring

        http://camel.apache.org/schema/spring/camel-spring.xsd "
>

    <camelContext xmlns="http://camel.apache.org/schema/spring" errorHandlerRef="myDLQ">

        <!-- 2 redeliveries max before failed message is placed into a DLQ   -->
        <errorHandler id="myDLQ" type="DeadLetterChannel" deadLetterUri="activemq:queue:errors" useOriginalMessage="true">
            <redeliveryPolicy maximumRedeliveries="2"/>
        </errorHandler>

        <!-- The polling of a specific folder every 30 sec -->
        <route id="route1">
            <from uri="file:///Users/bli/folderToPoll?delay=30000&delete=true"/>
            <unmarshal>
                <csv/>
            </unmarshal>
            <split>
                <simple>${body}</simple>
                <setHeader headerName="customerId">
                    <simple>${body[1]}</simple>
                </setHeader>
                <setHeader headerName="carModelId">
                    <simple>${body[2]}</simple>
                </setHeader>
                <setBody>
                    <simple>${body[0]}</simple>
                </setBody>
                <to uri="activemq:queue:individualRequests?disableReplyTo=true"/>
            </split>
        </route>

        <!-- The consumption of individual (jms) mailing requests -->
        <route id="route2">
            <from uri="activemq:queue:individualRequests?maxConcurrentConsumers=5"/>
            <pipeline>
                <to uri="direct:getCustomerEmail"/>
                <to uri="direct:sendMail"/>
            </pipeline>
        </route>

        <!-- Obtain customer email by parsing the XML response of a REST web service -->
        <route id="route3">
            <from uri="direct:getCustomerEmail"/>
            <setBody>
                <constant/>
            </setBody>
            <loadBalance>
                <roundRobin/>
                <to uri="http://backend1.mycompany.com/ws/customers?id={customerId}&authMethod=Basic&authUsername=geek&authPassword=secret"/>
                <to uri="http://backend2.mycompany.com/ws/customers?id={customerId}&authMethod=Basic&authUsername=geek&authPassword=secret"/>
            </loadBalance>
            <setBody>
                <xpath resultType="java.lang.String">/customer/general/email</xpath>
            </setBody>
        </route>

        <!-- Group individual sendings by car model -->
        <route id="route4">
            <from uri="direct:sendMail"/>
            <aggregate strategyRef="myAggregator" completionSize="10">
                <correlationExpression>
                    <simple>header.carModelId</simple>
                </correlationExpression>
                <completionTimeout>
                    <constant>60000</constant>
                </completionTimeout>
                <setHeader headerName="recipients">
                    <simple>${body}</simple>
                </setHeader>
                <pipeline>
                    <to uri="direct:prepareMail"/>
                    <to uri="direct:sendMailToMany"/>
                </pipeline>
            </aggregate>
        </route>

        <!-- Prepare the mail content -->
        <route id="route5">
            <from uri="direct:prepareMail"/>
            <setBody>
                <simple>header.carModelId</simple>
            </setBody>
            <pipeline>
                <to uri="sql:SELECT xml_text FROM template WHERE template_id =# ?dataSourceRef=myDS"/>
                <to uri="xslt:META-INF/xsl/email-formatter.xsl"/>
            </pipeline>
        </route>

        <!-- Send a mail to multiple recipients -->
        <route id="route6">
            <from uri="direct:sendMailToMany"/>
            <to uri="smtp://mail.mycompany.com:25?username=geek&password=secret&from=no-reply@mycompany.com&to={recipients}&subject=Your request&contentType=text/html"/>
            <log message="Mail ${body} successfully sent to ${headers.recipients}"/>
        </route>

    </camelContext>

<!-- Pure Spring beans referenced in the various Camel routes -->

    <!-- The ActiveMQ broker -->
    <bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
        <property name="brokerURL" value="tcp://localhost:61616"/>
    </bean>

    <!-- A datasource to our database -->
    <bean id="myDS" class="org.apache.commons.dbcp.BasicDataSource">
        <property name="driverClassName" value="org.h2.Driver"/>
        <property name="url" value="jdbc:h2:file:/Users/bli/db/MyDatabase;AUTO_SERVER=TRUE;TRACE_LEVEL_FILE=0"/>
        <property name="username" value="sa"/>
        <property name="password" value="sa"/>
    </bean>

    <!-- An aggregator implementation -->
    <bean id="myAggregator" class="com.mycompany.camel.ConcatBody"/>

</beans>

And the code of the (only!) Java class :

public class ConcatBody implements AggregationStrategy {

	public static final String SEPARATOR = ", ";

	public Exchange aggregate(Exchange aggregate, Exchange newExchange) {
        if (aggregate == null) {
        	// The aggregation for the very exchange item is the exchange itself
        	return newExchange;
        } else {
        	// Otherwise, we augment the body of current aggregate with new incoming exchange
            	String originalBody = aggregate.getIn().getBody(String.class);
            	String bodyToAdd = newExchange.getIn().getBody(String.class);
            	aggregate.getIn().setBody(originalBody + SEPARATOR + bodyToAdd);
            	return aggregate;
        }	
    }

}

Some explanations

  • The “route1” deals with the processing of incoming flat files. Thee file content is first unmarshalled (using CSV format) and then split into lines/records. Each line will be turned into an individual notification that is sent to a JMS queue.
  • The “route2” is consuming these notifications. Basically, fulfilling a request means doing two things in sequence (“pipeline”) : get the customer email (route3) and send him a mail (route4). Note the ‘maxConcurrentConsumers’ parameter that is used to easily answer our parallelism requirement.
  • The “route3” models how to get the customer email : simply by parsing (using XPath) the XML response of a (secured) REST web service that is available on two back-end nodes.
  • The “route4” contains the logic to send massive mails. Each time 10 similar send requests (that is, in our case, 10 requests on same car model) are collected (and we are not ready to wait more than 1 minute) we want the whole process to be continued with a new message (or « exchange » in Camel terminology) being the concatenation of the 10 assembled messages. Continuing the process means: first prepare the mail body (route5), and then send it to the group (route6).
  • In “route5“, a SQL query is issued in order to get the appropriate text depending on the car model. On that result, we apply a small XSL-T transformation (that will replace the current exchange body with the output of the xsl transformation).
  • When entering “route6“, an exchange contains everything we need. We have the list of recipients (as header), and we also have (in the body) the html text to be sent. Therefore we can now proceed to the real sending using SMTP protocol.
  • In case of errors (for instance, temporary network problems) – anywhere in the whole process, Camel will make maximum two additional attempts before giving up. In this latter case, the originating message will be automatically placed by Camel into a JMS Dead-Letter-Queue.

Conclusion

Camel is really a great framework – not perfect but yet great. You will be surprised to see how few lines of code are needed to model a complex scenario or route. You could also be glad to see how limpid is your code, how quickly your colleagues can understand the logic of your routes.

But it’s certainly not the main advantage. Using Camel primarily invites you to think in terms of Enterprise Integration Patterns (aka “EIP”); it helps you to decompose the original complexity into less complex (possibly concurrent) sub-routes using well-known and proven techniques, thereby leading to more modular, more flexible implementations. In particular, using decoupling techniques facilitates the potential replacement or refactoring of individual parts or components of your solution.
 

Reference: Discovering the power of Apache Camel from our W4G partner Bernard Ligny.

Related Whitepaper:

Functional Programming in Java: Harnessing the Power of Java 8 Lambda Expressions

Get ready to program in a whole new way!

Functional Programming in Java will help you quickly get on top of the new, essential Java 8 language features and the functional style that will change and improve your code. This short, targeted book will help you make the paradigm shift from the old imperative way to a less error-prone, more elegant, and concise coding style that’s also a breeze to parallelize. You’ll explore the syntax and semantics of lambda expressions, method and constructor references, and functional interfaces. You’ll design and write applications better using the new standards in Java 8 and the JDK.

Get it Now!  

Leave a Reply


× two = 10



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.

Sign up for our Newsletter

15,153 insiders are already enjoying weekly updates and complimentary whitepapers! Join them now to gain exclusive access to the latest news in the Java world, as well as insights about Android, Scala, Groovy and other related technologies.

As an extra bonus, by joining you will get our brand new e-books, published by Java Code Geeks and their JCG partners for your reading pleasure! Enter your info and stay on top of things,

  • Fresh trends
  • Cases and examples
  • Research and insights
  • Two complimentary e-books