A down side of durable messaging

Overview

Durable messaging can be very fast, as fast as non-durable messaging up to a point.

Limitations of durable messaging

Durable messaging is dependant on the size of your main memory and the speed of your hard drive.  If you have a HDD, this can be as low as 20 MB/s and as high as 60 MB/s.  A RAID set of HDD can support between 100 and 300 MB/s. An SATA SSD can support between 100 and 500 MB/s and a PCI SSD can support up to 1.5 GB/s.
 

Case study

Say you have 8 GB of memory, writing two million 100 bytes messages per second and a HDD which support 25 MB/s. This works fine at the speed in bursts but you reach a point where your disk cache is full.  Depending on your OS this can be between 20% and 80% of your main memory size.  In my experience, Windows tends to be closer to 20% even if you have plenty of free memory whereas Linux tends to allow in the region of 30% of your memory in uncommitted writes.

Say you are writing two million 100 byte message per second or 200 MB/s and you have 1600 MB of disk cache. The difference in speed is 175 MB/s between the rate you are writing and the rate you are generating it so in just 9 seconds you have filled the cache.  At this point your performance plummets to the write speed of your disk which is 25 MB/second. With each messaging being 100 bytes, you are now writing 250,000 messages per second or 8x slower.

What is the solution?

  • Keep your micro-bursts to less than you can fit in disk cache e.g. in the above case this would be about 18 million messages.
  • Increase the amount of memory you have.  While memory is cheap and you can buy 32 GB for about £150, all this does in include the duration of the micro-burst you can support.
  • Increase the speed of your drive.  With SSD you can support much higher bandwidths.  SATA SSD drives support up to 500 MB/sec which is higher than Chronicle can typically serialize messages, i.e. more than enough.  The downside of this is it reduces the total number of messages you can write. A 500 MB SSD can store 5 billion 100 byte messages. A 6×4 TB RAID-5 set can support a transfer rate of over 200 MB/s which would be enough for the above case study, and can store 200 billion messages.

Conclusion

If you see any durable messaging solution suddenly slow down under load, you need to look at the size of your buffers and the throughput of your disk sub-system.
 

Reference: A down side of durable messaging from our JCG partner Peter Lawrey at the Vanilla Java blog.

Related Whitepaper:

Software Architecture

This guide will introduce you to the world of Software Architecture!

This 162 page guide will cover topics within the field of software architecture including: software architecture as a solution balancing the concerns of different stakeholders, quality assurance, methods to describe and evaluate architectures, the influence of architecture on reuse, and the life cycle of a system and its architecture. This guide concludes with a comparison between the professions of software architect and software engineer.

Get it Now!  

Leave a Reply


eight + 8 =



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.

Sign up for our Newsletter

20,709 insiders are already enjoying weekly updates and complimentary whitepapers! Join them now to gain exclusive access to the latest news in the Java world, as well as insights about Android, Scala, Groovy and other related technologies.

As an extra bonus, by joining you will get our brand new e-books, published by Java Code Geeks and their JCG partners for your reading pleasure! Enter your info and stay on top of things,

  • Fresh trends
  • Cases and examples
  • Research and insights
  • Two complimentary e-books