I read a really, really interesting article on memory management strategies for the Erlang VM. It was written as a thesis by Jesper Wilhelmsson I thought it might be nice to discuss the differences between Erlang's memory setup and Oracle's Java VM.
As a real short introduction for those who have never heard of Erlang; it is a functional language that uses asynchronous message passing as its basis for concurrency. The message passing uses copy semantics, making distribution over more than one Erlang VM, running on more than one machine essentially transparent to the programmer.
Erlang and Java are similar in the sense that both use a virtual machine to abstract the hardware into a portable layer. Both languages employ machine independent byte code. Both run-time systems rely on garbage collection to free the programmer of doing memory management.
Threading overhead in Erlang is very low. I believe that the memory requirements for a thread in Erlang is about 512 bytes. In Java threads typically need about 512 kilobytes, about 1000 times more. For a programmer, the upshot is that creating threads to do some work asynchronously is not something you have to sit down and think about. Typical Erlang systems have thousands or tens of thousands of threads. There is no futzing with threadpools and executors like we do in Java.
From what little I have dabbled with it, I found Erlang to be a pleasant compromise between a functional language, and a language that allows you to write real-world applications. (I know I'll get flak for this) Robust distributed error handling is a pleasant surprise and writing a network server of any kind is actually easy. The state-machine approach to web servers makes rolling back on errors completely natural.
But this post is not about the programming model of Erlang. It is about the way the Erlang VM deals with memory.
The current Java virtual machine uses what an Erlang programmer would call a shared heap topology. There is one big heap that is used by all threads. Most memory is allocated on that heap. In addition to the heap, the JVM uses some specialised data areas like the code cache and the permanent generation. These too are shared between all threads.
By contrast, Erlang uses a private heap topology. Each thread has its own tiny heap that contains all data the thread uses and the thread's stack as well. All data for a thread is on that local heap. It is reserved when the thread is created. When the thread dies, the entire heap is simply returned to the pool of free memory.
Aside from the private heaps, all threads share access to a so-called binary heap and a message heap. These are specialised heaps. The binary heap is for allocating large chunks of arbitrary data that are likely to be shared between threads. This is where for example file input or network buffers live.
The message heap is a heap for data that is used in messages. Messages too are shared between processes. Messages are passed between threads by copying a pointer over from the sending thread to the receiving thread. The data for the message is stored on the message heap.
I was impressed by the Erlang memory model. It just strikes me as a lot more scalable than Java's single heap model. The language semantics and the memory model match beautifully.
For instance; the simple fact that heaps are private to a thread relieves the threads of all forms of lock checking on their own data. Add to that the fact that there are no destructive writes and suddenly there is no need for lock checking for shared data either.
The latest version of the Erlang VM take this yet another step further by having more than one scheduler. One scheduler per physical processor to be precise. This eliminates another entire class of locks to check. Only when a scheduler is bored it needs to go out, gather a lock and get some processes off of another scheduler.
In Java, we have a lot to learn still. That said, we have a few nice things in Java that I miss working with large Erlang systems.
The Erlang VM will reallocate and grow heaps when thread accumulate a lot of data. However, the reallocation algorithm causes heap sizes to grow rapidly. Under high load, we have seen Erlang VM's eat up 16GB of RAM in a matter of minutes. Each release has to be carefully load tested to see if its memory requirements are still sane.
There are no mechanisms in the Erlang VM to curb the growth of the memory. The VM will happily allocate so much memory that the system shoots into swap, or that the virtual memory is exhausted. These may cause the machine to become unresponsive even to KVM console access. In the past we have had to power cycle machines to get access to them again.
The queue-based programming model that makes Erlang so much fun to write code for, is also it Achilles heel in production. Every queue in Erlang is unbounded. The VM will not throw exceptions or limit the number of messages in a queue. Sometimes a process stops processing due to a bug, or a process fails to keep up with the flow of messages being sent to it. In that case, Erlang will simply allow the queue for that process to grow until either the VM is killed or the machine locks up, whichever comes first.
This means that when you run large Erlang VM's in a production environment you need to have OS-level checks that will kill the process if memory use skyrockets. Remote hands for the machine, or remote access cards is a must-have for machines that run large Erlang VM's.
In summary, for every day performance I believe the private heap memory model to be a very powerful tool in Erlang's box. It cuts whole classes of locking mechanisms out of the run-time system and that means it will scale better than Java will for the same purpose. Java's hard limits on memory will save your bacon when your system is being flooded or DDoSed.
Last thoughts,
There is a command line switch for Erlang's VM to switch it from using the private heap topology to using the share heap topology.
I like Erlang and Java. They are hard to compare because the have so little in common for a developer. In general, I would use Java for most systems, though. Tool support is better and the number of libraries available is staggering. I would opt for Erlang in the case where I have a stream oriented messaging system. That's when the Erlang programming model really shines.
References :
Happy Coding! Do not forget to share!
Byron
Related Articles:
As a real short introduction for those who have never heard of Erlang; it is a functional language that uses asynchronous message passing as its basis for concurrency. The message passing uses copy semantics, making distribution over more than one Erlang VM, running on more than one machine essentially transparent to the programmer.
Erlang and Java are similar in the sense that both use a virtual machine to abstract the hardware into a portable layer. Both languages employ machine independent byte code. Both run-time systems rely on garbage collection to free the programmer of doing memory management.
Threading overhead in Erlang is very low. I believe that the memory requirements for a thread in Erlang is about 512 bytes. In Java threads typically need about 512 kilobytes, about 1000 times more. For a programmer, the upshot is that creating threads to do some work asynchronously is not something you have to sit down and think about. Typical Erlang systems have thousands or tens of thousands of threads. There is no futzing with threadpools and executors like we do in Java.
From what little I have dabbled with it, I found Erlang to be a pleasant compromise between a functional language, and a language that allows you to write real-world applications. (I know I'll get flak for this) Robust distributed error handling is a pleasant surprise and writing a network server of any kind is actually easy. The state-machine approach to web servers makes rolling back on errors completely natural.
But this post is not about the programming model of Erlang. It is about the way the Erlang VM deals with memory.
The current Java virtual machine uses what an Erlang programmer would call a shared heap topology. There is one big heap that is used by all threads. Most memory is allocated on that heap. In addition to the heap, the JVM uses some specialised data areas like the code cache and the permanent generation. These too are shared between all threads.
By contrast, Erlang uses a private heap topology. Each thread has its own tiny heap that contains all data the thread uses and the thread's stack as well. All data for a thread is on that local heap. It is reserved when the thread is created. When the thread dies, the entire heap is simply returned to the pool of free memory.
Aside from the private heaps, all threads share access to a so-called binary heap and a message heap. These are specialised heaps. The binary heap is for allocating large chunks of arbitrary data that are likely to be shared between threads. This is where for example file input or network buffers live.
The message heap is a heap for data that is used in messages. Messages too are shared between processes. Messages are passed between threads by copying a pointer over from the sending thread to the receiving thread. The data for the message is stored on the message heap.
I was impressed by the Erlang memory model. It just strikes me as a lot more scalable than Java's single heap model. The language semantics and the memory model match beautifully.
For instance; the simple fact that heaps are private to a thread relieves the threads of all forms of lock checking on their own data. Add to that the fact that there are no destructive writes and suddenly there is no need for lock checking for shared data either.
The latest version of the Erlang VM take this yet another step further by having more than one scheduler. One scheduler per physical processor to be precise. This eliminates another entire class of locks to check. Only when a scheduler is bored it needs to go out, gather a lock and get some processes off of another scheduler.
In Java, we have a lot to learn still. That said, we have a few nice things in Java that I miss working with large Erlang systems.
The Erlang VM will reallocate and grow heaps when thread accumulate a lot of data. However, the reallocation algorithm causes heap sizes to grow rapidly. Under high load, we have seen Erlang VM's eat up 16GB of RAM in a matter of minutes. Each release has to be carefully load tested to see if its memory requirements are still sane.
There are no mechanisms in the Erlang VM to curb the growth of the memory. The VM will happily allocate so much memory that the system shoots into swap, or that the virtual memory is exhausted. These may cause the machine to become unresponsive even to KVM console access. In the past we have had to power cycle machines to get access to them again.
The queue-based programming model that makes Erlang so much fun to write code for, is also it Achilles heel in production. Every queue in Erlang is unbounded. The VM will not throw exceptions or limit the number of messages in a queue. Sometimes a process stops processing due to a bug, or a process fails to keep up with the flow of messages being sent to it. In that case, Erlang will simply allow the queue for that process to grow until either the VM is killed or the machine locks up, whichever comes first.
This means that when you run large Erlang VM's in a production environment you need to have OS-level checks that will kill the process if memory use skyrockets. Remote hands for the machine, or remote access cards is a must-have for machines that run large Erlang VM's.
In summary, for every day performance I believe the private heap memory model to be a very powerful tool in Erlang's box. It cuts whole classes of locking mechanisms out of the run-time system and that means it will scale better than Java will for the same purpose. Java's hard limits on memory will save your bacon when your system is being flooded or DDoSed.
Last thoughts,
There is a command line switch for Erlang's VM to switch it from using the private heap topology to using the share heap topology.
I like Erlang and Java. They are hard to compare because the have so little in common for a developer. In general, I would use Java for most systems, though. Tool support is better and the number of libraries available is staggering. I would opt for Erlang in the case where I have a stream oriented messaging system. That's when the Erlang programming model really shines.
References :
- Erlang memory architecture vs Java memory architecture from our JCG partner Kees Jan Koster at Java-Monitor
Happy Coding! Do not forget to share!
Byron
Related Articles:
"For instance; the simple fact that heaps are private to a thread relieves the threads of all forms of lock checking on their own data. Add to that the fact that there are no destructive writes and suddenly there is no need for lock checking for shared data either."
ReplyDeleteIt is a semi-true statement. Suns JVM incorporates TLAB (Thread Local Allocation Buffer) to get rid of heap locking when instancing any new objects. Short lock is set only when java TLAB is empty, and it must be expanded for further allocation. That lock is necessary because of working on shared heap (as you said), but it allows you to set -Xmx easily.
The private thread heap model also ties in well with the erlang garbage collector, which only works on an erlang process at a time, thus not blocking any other erlang process. Also, garbage collecting such a small per process heap is fast. This is what gives erlang processes near-realtime response times, even during high load.
ReplyDeleteWhen it comes to pure GC performance, I must admit, Sun has failed - only G1 (now under Oracles umbrella) has promising characteristics in big heaps, highly parallel environments. But there are other vendors on the market: Azul Systems claims to have GC which cleans 100G of garbage in under 10ms on their hardware (16cpus X 54cores, with custom, rewritten JVM), IBM introduced realtime-like, deterministic Metronome GC with splitted arrays (called arraylets, for example) many years ago. We have RTSJ (Realtime Java Specification) with immortal space and uninterruptable by GC threads and much, much more magic under the hood...
ReplyDeleteClearly, theres much more in Java, than meets the eyes :)
Very interesting! Not very knowlegable about the Java universe, mostly been swimming in the Erlang pond. :-)
ReplyDeleteYou may want to look up ulimit(3).
ReplyDelete"Messages too are shared between processes. Messages are passed between threads by copying a pointer over from the sending thread to the receiving thread."
ReplyDeleteThis is only true for hybrid heap. (configured with --enable-hybrid-heap)
Message is copied for default heap. Source: https://github.com/erlang/otp/blob/dev/erts/emulator/beam/erl_message.c#L924
We`ve talked about Erlang`s fast per thread`s tiny heap GC, but what about common heap (shared/hybrid) GC? As far as I understand, there should be some kind of "major" GC for this memory area (cant even imagine manual deallocation in cross thread context in faulty environment).
ReplyDeleteI`d like to point out that Java is a blue collar language. It means many choices were made to make language easy to adopt (smooth learning curve) for programmers, administrators etc. It means: unified, common memory model (look at all these Erlang`s types of heap),less sintactics sugar (cant even imagine how badly and unreadable code may be when using complex closures), standarization etc...
In the end of it all, I think Java`s approach is just perfect for common, day-to-day, universal language.
Cheers :)
Hey guys,
ReplyDeleteVery interesting comments from all of you! Thanks for contributing!
Cheers,
Ilias
Great one,, Can you please point me in an direction where i can learn more about java, jvm , memory and all...Thanks in advance bro...
ReplyDeleteIt is worth knowing that Azul has eliminated the need to stop threads with their garbage collector almost completely - much better than even Sun G1. Unfortunately their algorithm needs improvements in the memory management functions of the operating systems, and we have yet to convince the Linux people, Microsoft and so forth to implement this. (See http://www.managedruntime.org/). Thus, even Java might be as good as Erlang wrt. garbage collection in the future.
ReplyDeleteIn erlang you have diagnostic functions to get the current memory consumption and message queue length, so you can control these easily just by writing the small server.
ReplyDelete