About Bilgin Ibryam

Bilgin Ibryam is a senior software engineer based in London interested in service-oriented architecture, enterprise application integration and application development. He is also open source enthusiast, Apache Open for Business and Apache Camel committer.

Apache Camel meets Redis

The Lamborghini of Key-Value stores

Camel is the best of bread Integration framework and in this post I’m going to show you how to make it even more powerful by leveraging another great project – Redis.

Camel 2.11 is on its way to be released soon with lots of new features, bug fixes and components. Couple of these new components are authored by me, redis-component being my favourite one. Redis – a ligth key/value store is an amazing piece of Italian software designed for speed (same as Lamborghini – a two-seater Italian car designed for
 
speed). Written in C and having an in-memory closer to the metal nature, Redis performs extremely well (Lamborgini’s motto is ‘Closer to the Road’). Redis is often referred to as a data structure server since keys can contain strings, hashes, lists and sorted sets. A fast and light data structure server is like a super sportscars for software engineers – it just flies. If you want to find out more about Redis’ and Lamborghini’s unique performance characteristics google around and you will see for yourself.

Getting started with Redis is easy: download, make, and start a redis-server. After these steps, you ready to use it from your Camel application. The component uses internally Spring Data which in turn uses Jedis driver, but with possibility to switch to other Redis drivers. Here are few use cases where the camel-redis component is a good fit:

Idempotent Repository

The term idempotent is used in mathematics to describe a function that produces the same result if it is applied to itself. In Messaging this concepts translates into the a message that has the same effect whether it is received once or multiple times. In Camel this pattern is implemented using the IdempotentConsumer class which uses an Expression to calculate a unique message ID string for a given message exchange; this ID can then be looked up in the IdempotentRepository to see if it has been seen before; if it has the message is consumed; if its not then the message is processed and the ID is added to the repository. RedisIdempotentRepository is using a set structure to store and check for existing Ids.

 
<bean id="idempotentRepository" class="org.apache.camel.component.redis.processor.idempotent.RedisIdempotentRepository">
<constructor-arg value="test-repo"/>
</bean>
 
<route>
<from uri="direct:start"/>
<idempotentConsumer messageIdRepositoryRef="idempotentRepository">
<simple>${in.body.id}</simple>
<to uri="mock:result"/>
</idempotentConsumer>
</route>

Caching

One of the main uses of Redis is as LRU cache. It can store data inmemory as Memcached or can be tuned to be durable flushing data to a log file that can be replayed if the node restarts.The various policies when maxmemory is reached allows creating caches for specific needs:

  • volatile-lru remove a key among the ones with an expire set, trying to remove keys not recently used.
  • volatile-ttl remove a key among the ones with an expire set, trying to remove keys with short remaining time to live.
  • volatile-random remove a random key among the ones with an expire set.
  • allkeys-lru like volatile-lru, but will remove every kind of key, both normal keys or keys with an expire set.
  • allkeys-random like volatile-random, but will remove every kind of keys, both normal keys and keys with an expire set.

Once your Redis server is configured with the right policies and running, the operation you need to do are SET and GET:

<?xml version="1.0" encoding="UTF-8"?>
<route>
<from uri="direct:start"/>
<setHeader headerName="CamelRedis.Command">
<constant>SET</constant>
</setHeader>
<setHeader headerName="CamelRedis.Key">
<constant>keyOne</constant>
</setHeader>
<setHeader headerName="CamelRedis.Value">
<constant>valueOne</constant>
</setHeader>
<to uri="redis://localhost:6379"/>
</route>

Interap pub/sub with Redis

Camel has various components for interacting between routes:

direct: provides direct, synchronous invocation in the same camel context.
seda: asynchronous behavior, where messages are exchanged on a BlockingQueue, again in the same camel context.
vm: asynchronous behavior like seda, but also supports communication across CamelContext as long as they are in the same JVM. Complex applications usually consist of more than one standalone Camel instances running on separate machines. For this kind of scenarios, Camel provides jms, activemq, combination of AWS SNS with SQS, for messaging between instances.
Redis has a simpler solution for the Publish/Subscribe messaging paradigm. Subscribers subscribes to one or more channels, by specifying the channel names or using pattern matching for receiving messages from multiple channels. Then the publisher publishes the messages to a channel, and Redis makes sure it reaches all the matching subscribers.

<?xml version="1.0" encoding="UTF-8"?>
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<route startupOrder="1">
<from uri="redis://localhost:6379?command=SUBSCRIBE&channels=testChannel"/>
<to uri="mock:result"/>
</route>
 
<route startupOrder="2">
<from uri="direct:start"/>
<setHeader headerName="CamelRedis.Command">
<constant>PUBLISH</constant>
</setHeader>
<setHeader headerName="CamelRedis.CHANNEL">
<constant>testChannel</constant>
</setHeader>
<setHeader headerName="CamelRedis.MESSAGE">
<constant>Test Message</constant>
</setHeader>
<to uri="redis://localhost:6379"/>
</route>
</camelContext>

Other usages

Guaranteed Delivery: Camel supports this EIP using JMS, File, JPA and few other components. Here Redis can be used as lightweight key-value persistent store with its transaction support.

The Claim Check from the EIP patterns allows you to replace message content with a claim check (a unique key), which can be used to retrieve the message content at a later time. The message content can be stored temporarily in Redis.

Redis is also very popular for implementing counters, leaderboards, tagging systems and many more functionalities. Now, with two swiss army knives under your belt, the integrations to make are limited only by your imagination.
 

Reference: Apache Camel meets Redis from our JCG partner Bilgin Ibryam at the OFBIZian blog.

Related Whitepaper:

Functional Programming in Java: Harnessing the Power of Java 8 Lambda Expressions

Get ready to program in a whole new way!

Functional Programming in Java will help you quickly get on top of the new, essential Java 8 language features and the functional style that will change and improve your code. This short, targeted book will help you make the paradigm shift from the old imperative way to a less error-prone, more elegant, and concise coding style that’s also a breeze to parallelize. You’ll explore the syntax and semantics of lambda expressions, method and constructor references, and functional interfaces. You’ll design and write applications better using the new standards in Java 8 and the JDK.

Get it Now!  

Leave a Reply


2 + eight =



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below:
Close