Java SE 8 was released yesterday. Traditionally, every new major JRE version comes with a free performance boost. Do we get another free lunch? And how big is the gain this time?
Let’s benchmark it.
- Run the same code with 3 different JRE versions (SunJDK
1.8.0). The code itself was written for Java 6 (both in syntax and JDK API’s usage) and compiled for Java 6 with OpenJDK 1.7.
- Each run takes about 55 minutes.
- VM arguments:
Intel® Xeon® CPU W3550 @ 3.07GHz
- Each run solves 13 planning problems with OptaPlanner. Each planning problem runs for 5 to minutes. Up to 2 planning problems are solved in parallel.
- Solving a planning problem involves no IO (except a few milliseconds during startup to load the input). A single CPU is completely saturated. It constantly creates many short lived objects, and the GC collects them afterwards.
- The benchmarks measure the number of scores that can be calculated per millisecond. Higher is better. Calculating a score for a proposed planning solution is non-trivial: it involves many calculations, including checking for conflicts between every entity and every other entity.
- On the biggest dataset (Machine Reassignment B10), which dwarfs any of the other datasets in size, Java 8 is
20%faster than Java 7, which was already
17%faster than Java 6.
- In some cases, Java 8 is slower than Java 7. Specially for the course scheduling datasets, Java 8 is
6%slower than Java 7. Hopefully new releases of Java 8 will resolve this performance regression soon.
- On average, Java 8 is only
1%faster than Java 7. This while Java 7 is already
16%faster than Java 6.
- Despite that this is the first final release of OpenJDK 8, I did not find any regressions in Java 8. OptaPlanner’s examples are 100% reproducible, so as expected, the different JRE’s give the exact same results at every single iteration.
Raw benchmark numbers
|JDK||Cloud balancing 200c||Cloud balancing 800c||Machine reassignment B1||Machine reassignment B10||Course scheduling c7||Course scheduling c8||Exam scheduling s2||Exam scheduling s3||Nurse rostering m1||Nurse rostering mh1||Sport scheduling nl14|
|6 ⇒ 7||15.54%||10.75%||23.25%||17.72%||12.95%||12.29%||18.54%||21.69%||12.78%||14.55%||23.67%|
|7 ⇒ 8||1.87%||3.67%||15.91%||20.15%||-6.21%||-6.26%||-2.37%||-3.58%||-0.22%||-1.57%||-5.07%|
On the big datasets, Java 8 is clearly faster. And this without changing a line of code. On average, the result is less convincing (with the current release), but a free lunch is always welcome.