About Francisco Ribeiro Junior

Francisco is a senior software engineer working in the telecom/banking domain focused on Web Content Management. He has been technical lead on many projects for different Brazilian and worldwide clients.

Simple Message Queue using Redis

In this posts we will use Redis as a simple message queue, using list commands.

Let’s say we have an application that allows the users to upload a photo. Then in the application we display the photo in different sizes like Thumb, Medium and Large.

In a first implementation we could have the task of processing the uploaded image in the same request. As it is an expensive task, it would make our requests slow.

A possible solution for this it to make this processing asynchronous by using a Message Queue(MQ), there are a lot of well known MQs like ActiveMQ, RabbitMQ, IBM MQ and other. In the examples below we will be using Redis as a message queue using the LIST structure.

The idea is to have a LIST where the producer will put the messages to be processed and some consumers that will watch the LIST to process the messages sent.

Basically, the producers will add the messages into the end of the list with “RPUSH queue message” and the consumers will read the messages in the beginning of the list with “LPOP queue” configuring a FIFO processing.

The client will always be looking for a new message, for this purpose we will use the BLPOP command that is a blocking version of the LPOP command. Basically there will be a while loop calling BLPOP to get a new message to process.

Considering the image upload example, let’s say we have a class ImageUploader that is responsible for uploading the image to the server, it will add a new message in the queue, indicating there is an image to be processed, a message could be a JSON string like this:

{“imagePath”:”/path/to/image”, “user”:”userid”}

The ImageUploder class could be like this:

public class ImageUploader {

  public void uploadImage(HttpServletRequest request){

    String imagePath = saveImage(request);
    String jsonPayload = createJsonPayload(request, imagePath);
    jedis.rpush("queue", jsonPayload);
    //... keep with the processing
  }

  //.... other methods in the class
}

It is just an example on how the producer would work. In this case we have decoupled the image processing from the ImageUploader class. We just create a new message in the queue, so the consumers would process them.

The message consumer could be something like this:

package br.com.xicojunior.redistest;

import java.util.List;

import redis.clients.jedis.Jedis;

public class MessageConsumer 
{
    public static void main( String[] args )
    {
        Jedis jedis = new Jedis("localhost");   
        List<String> messages = null;
        while(true){
          System.out.println("Waiting for a message in the queue");
          messages = jedis.blpop(0,"queue");
          System.out.println("Got the message");
          System.out.println("KEY:" + messages.get(0) + " VALUE:" + messages.get(1));
          String payload = messages.get(1);
          //Do some processing with the payload
          System.out.println("Message received:" + payload);
        }

    }
}

This consumer code could be running in a different process or even a different machine. This consumers code is runnable we can compile it and run it using eclipse or java command.

For this code we used the jedis.blpop method, it returns a List with 2 Strings, (0) – The key, (1) – The value that was returned. The method also receives an integer, it indicates the timeout. We passed 0 indicating there will be no timeout.

When we run this code and there is no value in the list, in the console we will see just the message

"Waiting for a message in the queue".

Then if a client adds a element in the list “queue” our consumer class will get its value. We can simulate a test by using the redis-cli or even another class that will add elements in the queue like the one below:

package br.com.xicojunior.redistest;

import redis.clients.jedis.Jedis;

public class MessageProducer {

  public static void main(String[] args) {
    Jedis jedis = new Jedis("localhost");

    jedis.rpush("queue", "Value 1");
    jedis.rpush("queue", "Value 2");
    jedis.rpush("queue", "Value 3");

  }

}

If we run the MessageProducer class after the MessageConsumer class is already running we will see this output in the console:

Waiting for a message in the queue
Got the message
KEY:queue VALUE:Value 1
Message received:Value 1
Waiting for a message in the queue
Got the message
KEY:queue VALUE:Value 2
Message received:Value 2
Waiting for a message in the queue
Got the message
KEY:queue VALUE:Value 3
Message received:Value 3
Waiting for a message in the queue

So, message queue would be another possible use case for Redis. There are some queues built on top of redis like RestMQ, Resque – a Job Queue and other.
 

Reference: Simple Message Queue using Redis from our JCG partner Francisco Ribeiro Junior at the XICO JUNIOR’S WEBLOG blog.
Related Whitepaper:

Functional Programming in Java: Harnessing the Power of Java 8 Lambda Expressions

Get ready to program in a whole new way!

Functional Programming in Java will help you quickly get on top of the new, essential Java 8 language features and the functional style that will change and improve your code. This short, targeted book will help you make the paradigm shift from the old imperative way to a less error-prone, more elegant, and concise coding style that’s also a breeze to parallelize. You’ll explore the syntax and semantics of lambda expressions, method and constructor references, and functional interfaces. You’ll design and write applications better using the new standards in Java 8 and the JDK.

Get it Now!  

Leave a Reply


eight + 4 =



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.

Sign up for our Newsletter

20,709 insiders are already enjoying weekly updates and complimentary whitepapers! Join them now to gain exclusive access to the latest news in the Java world, as well as insights about Android, Scala, Groovy and other related technologies.

As an extra bonus, by joining you will get our brand new e-books, published by Java Code Geeks and their JCG partners for your reading pleasure! Enter your info and stay on top of things,

  • Fresh trends
  • Cases and examples
  • Research and insights
  • Two complimentary e-books