About Ashwini Kuntamukkala

Ashwini is an open source, cloud and mobile development enthusiast. He has over 10 years of experience in leading and implementing several enterprise grade solutions in pharmacy, health care and travel industries.

ActiveMQ – Network of Brokers Explained – Part 5

In the previous part 4 we have seen how to load balance remote consumers on a queue using network connectors.

In this part 5, we will see how the same configuration would work if we had concurrent remote durable subscribers on a topic.  Consider the following configuration….

 
 
 
 
 

part-5-lb-rcc - New Page
Fig 1: Network of Brokers – Load balance subscribers on a topic

 
As shown above, we have Broker-1 which initiates two network connectors to Broker-2 and Broker-3. A producer sends messages to a topic “moo.bar” on Broker-1 while Broker-2 has subscriber C1 and Broker-3 has two subscribers C2 and C3 on the same topic “moo.bar”.

You may observe that this set up is very similar to part 4. The only difference is that here we are dealing with topics while in part 4, we were dealing with queues.

Let’s see this in action

  1. Add the following network connector configuration in Broker-1′s activemq.xml configuration file:
      <networkConnectors>
    <networkConnector
    name="T:broker1->broker2"
    uri="static:(tcp://localhost:61626)"
    duplex="false"
    decreaseNetworkConsumerPriority="false"
    networkTTL="2"
    conduitSubscriptions="false"
    dynamicOnly="true">
    <excludedDestinations>
    <queue physicalName=">" />
    </excludedDestinations>
    </networkConnector>
    <networkConnector
    name="T:broker1->broker3"
    uri="static:(tcp://localhost:61636)"
    duplex="false"
    decreaseNetworkConsumerPriority="false"
    networkTTL="2"
    conduitSubscriptions="false"
    dynamicOnly="true">
    <excludedDestinations>
    <queue physicalName=">" />
    </excludedDestinations>
    </networkConnector>
    </networkConnectors>
  2. Let’s start broker-2, broker-3 and broker-1 in that order.
  3.  akuntamukkala@localhost~/apache-activemq-5.8.0/cluster/broker-2/bin$ ./broker-2 console
  4.  akuntamukkala@localhost~/apache-activemq-5.8.0/cluster/broker-3/bin$ ./broker-3 console
  5.  akuntamukkala@localhost~/apache-activemq-5.8.0/cluster/broker-1/bin$ ./broker-1 console
  6. Broker-1′s admin console connections show that two network connectors have been established as configured from Broker-1 to Broker-2 and Broker-3 respectively:
  7.  

    broker-1-connections
    Broker-1′s Connections @ http://localhost:8161/admin/connections.jsp

     

  8. Let’s start the subscriber C1 on Broker-2 subscribing to messages to topic “moo.bar” and subscribers C2 and C3 on Broker-3 subscribing to messages on same topic “moo.bar”.
  9. Durable Subscribers require unique combination of client id and subscriber name. In order for us to create durable subscribers C2 and C3 we need to enhance the functionality provided in /Users/akuntamukkala/apache-activemq-5.8.0/example/src/ConsumerTool.java where /Users/akuntamukkala/apache-activemq-5.8.0 is the directory where ActiveMQ is installed.
  10. The modified code consists of editing build.xml and ConsumerTool.java to add a new parameter “subscriberName”. The edited files build.xml and ConsumerTool.java can be obtained from here and here respectively.
  11. Let’s start the subscribers now.
  12.  akuntamukkala@localhost~/apache-activemq-5.8.0/example$ ant consumer -Durl=tcp://localhost:61626 -Dtopic=true -Dsubject=moo.bar -DclientId=C1 -Ddurable=true -DsubscriberName=mb.C1
  13.  akuntamukkala@localhost~/apache-activemq-5.8.0/example$ ant consumer -Durl=tcp://localhost:61636 -Dtopic=true -Dsubject=moo.bar -DclientId=C2 -Ddurable=true -DsubscriberName=mb.C2
  14.  akuntamukkala@localhost~/apache-activemq-5.8.0/example$ ant consumer -Durl=tcp://localhost:61636 -Dtopic=true -Dsubject=moo.bar -DclientId=C3 -Ddurable=true -DsubscriberName=mb.C3
  15. Durable subscriber on Broker-2:
     

    broker-2-durable-subscribers
    http://localhost:9161/admin/subscribers.jsp

     

  16. Durable subscribers on Broker-3:
     

    broker-3-durable-subscribers
    http://localhost:10161/admin/subscribers.jsp

     

  17. Durable subscribers on Broker-1 (because of network connectors):
     

    http://localhost:8161/admin/subscribers.jsp

     

  18.  Now let’s send 10 durable messages to topic moo.bar on Broker-1.
  19.  akuntamukkala@localhost~/apache-activemq-5.8.0/example$ ant producer -Durl=tcp://localhost:61616 -Dtopic=true -Dsubject=moo.bar -Dmax=10 -Ddurable=true
  20. See the console on Broker-3:
     

    broker-3-duplicate-msg-warning
    Log file output on Broker-3

     

  21. As you may observe, Broker-3 receives the same message twice, once per each subscription C2 and C3. ActiveMQ by default does not permit processing of duplicate messages.
  22. This happens because both the subscriptions mb.C2 and mb.C3 on Broker-3 are propagated to Broker-1. So when 10 messages are published to moo.bar on Broker-1, those messages are sent over to subscribers mb.C2 and mb.C3 on the same broker: Broker-3. Since the messages have the same ID, duplicate messages are discarded and hence the warning shown in the log messages….(shown in step 19).
  23. Here is the console showing statistics on Broker-1:
     

    broker-1-stats
    http://localhost:8161/admin/subscribers.jsp

     

  24. Here is the console showing statistics on Broker-3:
     

    broker-3-stats
    http://localhost:10161/admin/subscribers.jsp

     

  25. As you can see even though the enqueue counter shows 20, the dequeue counter shows only 10, since the other 10 messages were discarded by the Broker-3. This is a useful feature which helps to ensure that a message gets processed at most once by a broker.

The reason why this is occurring is because both subscriptions C2 and C3 are propagated to upstream broker Broker-1.

broker-3-duplicate-msg-warning1
Duplicate Messages on Broker-3

 
Let’s retry the same scenario using a minor tweak in the network connector settings by making conduitSubscriptions=”true”
on both network connectors from Broker-1 to Broker-2 and Broker-3 respectively. After restarting the brokers, delete the inactive durable subscribers and then repeat the above steps.

  <networkConnectors>
<networkConnector
name="T:broker1->broker2"
uri="static:(tcp://localhost:61626)"
duplex="false"
decreaseNetworkConsumerPriority="false"
networkTTL="2"
conduitSubscriptions="true"
dynamicOnly="true">
<excludedDestinations>
<queue physicalName=">" />
</excludedDestinations>
</networkConnector>
<networkConnector
name="T:broker1->broker3"
uri="static:(tcp://localhost:61636)"
duplex="false"
decreaseNetworkConsumerPriority="false"
networkTTL="2"
conduitSubscriptions="true"
dynamicOnly="true">
<excludedDestinations>
<queue physicalName=">" />
</excludedDestinations>
</networkConnector>
</networkConnectors>


The following screenshot shows that Broker-1 now sees only two durable subscribers, one from each broker,  Broker-2 and Broker-3.

 

broker-1-cs-true
Durable Subscribers in Broker-1 when conduitSubscriptions=”true”

 

Upon publishing 10 durable messages on Broker-1, we find that we don’t have the same issue of duplicate messages this time. 

As expected all the 10 messages are processed by C1, C2 and C3 as shown by screenshots below.

 

broker-1-msg-processed
Broker-1′s Durable Topic Subscribers

 

broker-3-msg-processed
Broker-3′s Durable Topic Subscribers C2 and C3 receive and process 10 messages each

 
Hence we have seen how conduitSubscriptions attribute can help in reducing message traffic by avoiding duplicate messages in a network of brokers.

In the next part 6, we will see how ActiveMQ provides “message replay” capabilities in order to prevent stuck message scenarios.

Related Whitepaper:

Functional Programming in Java: Harnessing the Power of Java 8 Lambda Expressions

Get ready to program in a whole new way!

Functional Programming in Java will help you quickly get on top of the new, essential Java 8 language features and the functional style that will change and improve your code. This short, targeted book will help you make the paradigm shift from the old imperative way to a less error-prone, more elegant, and concise coding style that’s also a breeze to parallelize. You’ll explore the syntax and semantics of lambda expressions, method and constructor references, and functional interfaces. You’ll design and write applications better using the new standards in Java 8 and the JDK.

Get it Now!  

Leave a Reply


× four = 12



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.

Sign up for our Newsletter

20,709 insiders are already enjoying weekly updates and complimentary whitepapers! Join them now to gain exclusive access to the latest news in the Java world, as well as insights about Android, Scala, Groovy and other related technologies.

As an extra bonus, by joining you will get our brand new e-books, published by Java Code Geeks and their JCG partners for your reading pleasure! Enter your info and stay on top of things,

  • Fresh trends
  • Cases and examples
  • Research and insights
  • Two complimentary e-books