High Performance Durable Messaging

Overview

While there are a good number of high performance messaging systems available for Java, most avoid quoting benchmarks which include durable messaging and serialization/deserialization of messages. This is done for a number of reasons; 1) you don’t always need or want durable messages 2) you want the option of using your own serialization. One important reason they are avoided is that both of these slow down messaging by as much as 10x which doesn’t look so good. Most messaging benchmarks will highlight the performance of passing raw bytes around without durability as this gives the highest numbers. Some also quote durable messaging numbers, but these are typically much slower.

What if you need to serialize and deserialize real data efficiently and you would like to record and replay messages even if you have learnt to do without these.

Higher performance serialization and durability

I have written a library which attempt to solve more of the problem, as I see it, to give you a better overall solution. It is not the fastest messaging available but it is durable and includes serialization and de-serialization times. As I have noted already this can be 10x higher than the cost of transporting the serialized data so in real applications this solution can be much faster.

An Example: Sending prices

In this test InProcessChronicleTest.testPricePublishing() I send price events consisting of a long timestamp, a symbol, a bid price / volume and ask price / volume. The time to write the data was 0.4 µS (0.0004 milli-seconds) and the time to receive it over a TCP connection was 1.8 µS.

Note: this connection is durable at both ends so you can see what data has been queue to be sent and what has been received. If a connection is lost, it can continue from the place it was up to, or optionally playing any message the client has already received, even if the server is not available e.g. if the client is restarted.

To send and receive 5 million messages I used the -Xmx32m -verbosegc flags which show that little heap needed, and not a single GC occurs during this test. This means the library will have little impact on the rest of your application.

Comparing with Externalize objects.

To put this in context, I have compared this with the time it takes to serialize and deserialize objects containing the same data in InProcessChronicleTest. testSerializationPerformance(). The PriceUpdate object is Externalizable and this Benchmark for “Comparing various aspects of Serialization libraries on the JVM Platform” shows an Externalizable object can be one of the fastest serializations available. The time this took on the same machine was 2.7 µS to serialize and 7.5 µS to deserialize. Note: this doesn’t include messaging or persistence, just to do the serialization and deserialization.

SerializationtransportTime to writeTime to read
Java ChronicleTCP and persistence 0.4 µS 1.8 µS
Externalizable none 2.7 µS 7.5 µS
Serializable none 3.8 µS 13.2 µS

Conclusion

When benchmarking messaging, you should include how long it takes to send and receive real messages, not just byte[], and include durability if that is desirable.
 

Reference: High Performance Durable Messaging from our JCG partner Peter Lawrey at the Vanilla Java blog.

Related Whitepaper:

Functional Programming in Java: Harnessing the Power of Java 8 Lambda Expressions

Get ready to program in a whole new way!

Functional Programming in Java will help you quickly get on top of the new, essential Java 8 language features and the functional style that will change and improve your code. This short, targeted book will help you make the paradigm shift from the old imperative way to a less error-prone, more elegant, and concise coding style that’s also a breeze to parallelize. You’ll explore the syntax and semantics of lambda expressions, method and constructor references, and functional interfaces. You’ll design and write applications better using the new standards in Java 8 and the JDK.

Get it Now!  

Leave a Reply


3 × four =



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.

Sign up for our Newsletter

15,153 insiders are already enjoying weekly updates and complimentary whitepapers! Join them now to gain exclusive access to the latest news in the Java world, as well as insights about Android, Scala, Groovy and other related technologies.

As an extra bonus, by joining you will get our brand new e-books, published by Java Code Geeks and their JCG partners for your reading pleasure! Enter your info and stay on top of things,

  • Fresh trends
  • Cases and examples
  • Research and insights
  • Two complimentary e-books