About Claus Ibsen

Claus Ibsen is a principal software engineer from Red Hat. Claus is working full time as Apache Camel committer. And is author of the "Camel in Action" book.

Apache Camel 2.12 – Backoff support for less aggressive polling routes

This is another blog post about some upcoming improvements and new features in the next Apache Camel 2.12 release.

In the last blog I spoke about cron expressions directly supported in the routes. This blog post is related as we have another new feature to the polling routes (using scheduled polling consumers).

The scheduled polling routes will by default poll for new messages at a fixed rate (usually every half second). Though in some use-cases new messages does not arrive evenly spread over the day, but may tend to arrive in batches. So during the times where there is no new messages arriving, the scheduled polling consumer, would not pickup new messages, but still keep looking for new messages at the same pace.

So what we have introduced is a backoff that allows you to let the scheduled polling consumer check less aggressive when there is no messages.

To support that we have introduce 3 new options

  1. backoffMultiplier = To let the scheduled polling consumer backoff if there has been a number of subsequent idles/errors in a row. The multiplier is then the number of polls that will be skipped before the next actual attempt is happening again. When this option is in use then backoffIdleThreshold and/or backoffErrorThreshold must also be configured.
  2. backoffIdleThreshold = The number of subsequent idle polls that should happen before the backoffMultipler should kick-in.
  3. backoffErrorThreshold = The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in.

The scheduled Polling Consumer is by default static by using the same poll frequency whether or not there is messages to pickup or not. From Camel 2.12 onwards you can configure the scheduled Polling Consumer to be more dynamic by using backoff. This allows the scheduler to skip N number of polls when it becomes idle, or there has been X number of errors in a row. See more details in the table above for the backoffXXX options.

For example to let a FTP consumer backoff if its becoming idle for a while you can do:

from("ftp://myserver?username=foo&passowrd=secret
      ?delete=true&delay=5s
      &backoffMultiplier=6&backoffIdleThreshold=5")
  .to("bean:processFile");

In this example, the FTP consumer will poll for new FTP files every 5th second. But if it has been idle for 5 attempts in a row, then it will backoff using a multiplier of 6, which means it will now poll every 5 x 6 = 30th second instead. When the consumer eventually pickup a file, then the backoff will reset, and the consumer will go back and poll every 5th second again.

In JMX you can see whether the backoff has kick-in, as the consumer would have the attribute backoffCounter > 0.

You can find more details at

 

Do you want to know how to develop your skillset to become a Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

JPA Mini Book

Learn how to leverage the power of JPA in order to create robust and flexible Java applications. With this Mini Book, you will get introduced to JPA and smoothly transition to more advanced concepts.

JVM Troubleshooting Guide

The Java virtual machine is really the foundation of any Java EE platform. Learn how to master it with this advanced guide!

Given email address is already subscribed, thank you!
Oops. Something went wrong. Please try again later.
Please provide a valid email address.
Thank you, your sign-up request was successful! Please check your e-mail inbox.
Please complete the CAPTCHA.
Please fill in the required fields.

One Response to "Apache Camel 2.12 – Backoff support for less aggressive polling routes"

  1. this says:

    Your mode of describing all in this piece of writing is actually pleasant, every one can without difficulty be aware of it, Thanks a lot.

Leave a Reply


one + = 6



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy | Contact
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below:
Close