Chronicle and the Micro-Cloud

Overview

A common question I face is; how do you scale a Chronicle based system if it is single writer, multiple readers. While there are solutions to this problem, it is far more likely it won’t be a problem at all.

The Micro-Cloud

This is the term I have been using to describe a single thread to do the work currently being done by multiple servers. (Or the opposite of the trend of deploying a single application to multiple machines)

There is an assumption that the only way to scale a system is to partition or have multiple copies.  This doesn’t always scale that well unless you have multiple systems as well. All the adds complexity to the development, deployment and maintenance of the system. A common problem I see is that developers are no longer able to test the end-to-end system on their workstations or in unit/functional tests

Chronicle based processing engines have a different approach.  They are typically designed to handle as much load as you can give it.  i.e. the bottleneck is elsewhere such as the touch points with external systems, gateways and databases.  A Chronicle based processing engine can handle 100,000 to 1,000,000 inbound and out bound events per second with latencies of between 1 and 10 micro-seconds.  This is more than the rate than gateways can feed data in/out of the engine so there is never a good use case for partitioning.  It is an option, but it is one you should not assume you need.

The benefit of this is you have a deterministic system which is very simple architecturally. The system is deterministic because you have a record of every inbound and outbound message which can have micro-second timings and the behaviour of the system is completely reproduce-able (and restart-able).  Note: one of the consequences of this is, just restarting a failed service should not help.  If it failed once a particular way based on particular message, it should fail on restart exactly the same way, every time. This makes reproducing issues and testing easier.

Why is it so fast?

A single thread processing engine in one thread has

  • No locking
  • No TCP delays
  • Can be locked to a core or CPU improving caching behaviour.
    i.e. it never context switches or gives up the CPU.
  • Reusable mutable objects are more practical so you can avoid churning your caches with garbage so you memory accesses can be 5x faster. (With the bonus you no longer have GC pauses)

The real reason to avoid producing garbage

A common problem in most Java application is the GC pause time.  You generally want to keep this to a minimum.  I advocate reducing garbage produced but I am not concerned about GC pauses as I haven’t written a system which pauses while trading for four years now. I do this by having an Eden size which is greater than the amount of garbage I produce in a day and I do a System.gc() in a maintenance window (in the middle of the night or on the weekend as appropriate). The main reason to avoid garbage in low latency applications is creating garbage causes a scrolling or flushing of the caches.  If you have a system (all JVMs on a socket) which is producing 200 MB/s of garbage and you have a 20 MB L3 cache, you are effectively filling the cache with garbage every 0.1 seconds.  Memory accesses to the L3 cache can be 5 times faster than access to main memory.

Note: the L3 cache is shared across all cores in your socket so if you want scalability across all your cores you want to be working your L1 cache which is typically 32 KB and if you have busy thread creating 32 MB/s of garbage it is being filled with garbage every milli-second .

Conclusion

For this reason, I have started using the term micro-cloud. i.e. do the work of a whole cloud in one thread.  I have a couple clients who are doing just that.  What was done by many JVMs across multiple systems can be done by just one, very fast thread. Obviously, one thread is much easier to test on one workstation and takes up far less rack space in your data centre in production.
 

Reference: Chronicle and the Micro-Cloud from our JCG partner Peter Lawrey at the Vanilla Java blog.

Do you want to know how to develop your skillset to become a Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

JPA Mini Book

Learn how to leverage the power of JPA in order to create robust and flexible Java applications. With this Mini Book, you will get introduced to JPA and smoothly transition to more advanced concepts.

JVM Troubleshooting Guide

The Java virtual machine is really the foundation of any Java EE platform. Learn how to master it with this advanced guide!

Given email address is already subscribed, thank you!
Oops. Something went wrong. Please try again later.
Please provide a valid email address.
Thank you, your sign-up request was successful! Please check your e-mail inbox.
Please complete the CAPTCHA.
Please fill in the required fields.

Leave a Reply


five × = 20



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy | Contact
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below:
Close