Home » Java » Core Java » JMH: How to setup and run a JMH benchmark

About Daniel Shaya

Daniel Shaya
Daniel has been programming in Java since it was in beta. Working predominantly in the finance industry he has created real time trading and margin risk applications. He is currently a director at OpenHFT where we are building next generation Java low latency products.

JMH: How to setup and run a JMH benchmark

Health Warning!

This post describes how to setup and run a simple JMH benchmark.  Micro benchmarks are notoriously difficult to get right and even when you do get them right (by using tools such as JMH) they can still be misleading. Just because your code runs in a certain way in an extremely isolated artificial situation does not mean it will run in the same way inside your production code. To name but a few issues, in a real program the CPU caches will be subject to pressures from other parts of your code, any object creation will have a downstream effect on GC and the JIT may have inlined and compiled code from other parts of your code that conflict with the code you have benchmarked. Nevertheless micro benchmarks do have their place and if you are going to do them you might as well do them properly with JMH.

In a recent post I was asked to execute my tests as a JMH performance benchmark.

JMH is a Java harness for building, running, and analysing nano/micro/milli/macro benchmarks written in Java and other languages targeting the JVM. See full documentation here.

Amongst other things JMH is great, because it takes care of warm up iterations, forking JVM processes so that benchmarks don’t interfere with each other, collating results and presenting then in a uniform manner.  And there’s much much more.

I’d heard a lot about JMH and seen many JMH results but never actually run one myself. It was surprisingly easy! This is how I did it.

Add these dependencies to your Maven pom.xml file:

<dependency>
            <groupId>org.openjdk.jmh</groupId>
            <artifactId>jmh-core</artifactId>
            <version>1.5.1</version>
        </dependency>
        <dependency>
            <groupId>org.openjdk.jmh</groupId>
            <artifactId>jmh-generator-annprocess</artifactId>
            <version>1.5.1</version>
</dependency>

Then decide which methods you want benchmarked and add the annotation @Benchmark to them. If you need any initialisation code add it in a method which should be marked @Setup.

The easiest way to run the benchmark is by adding by adding this implementation into your main method. (See here for other ways to run your tests).

public static void main(String[] args) throws RunnerException {
        Options opt = new OptionsBuilder()
                .include(MyBenchmark.class.getSimpleName())
                .forks(1)
                .build();

        new Runner(opt).run();
}

Then just run as you would any normal program and you will get all the JMH goodness!

As an example to see the format of a JMH benchmark, this is what my results looked like:

Benchmark                                         Mode  Cnt     Score    Error  Units
CompTestBenchmark.bmCustomComparator             thrpt   20  2598.617 ± 67.634  ops/s
CompTestBenchmark.bmJDKComparator                thrpt   20   751.110 ± 14.835  ops/s
CompTestBenchmark.bmNoVTLComparator              thrpt   20  1348.750 ± 30.382  ops/s
CompTestBenchmark.bmNoVTLOrAutoBoxingComparator  thrpt   20  2202.995 ± 43.673  ops/s

There are an enormous number of bells and whistles to fine tune your benchmarks which I’m not going into here but hopefully this will get you up and running.

  • For a full code listing of my test see here.
Reference: JMH: How to setup and run a JMH benchmark from our JCG partner Daniel Shaya at the Rational Java blog.
(0 rating, 0 votes)
You need to be a registered member to rate this.
2 Comments Views Tweet it!
Do you want to know how to develop your skillset to become a Java Rockstar?
Subscribe to our newsletter to start Rocking right now!
To get you started we give you our best selling eBooks for FREE!
1. JPA Mini Book
2. JVM Troubleshooting Guide
3. JUnit Tutorial for Unit Testing
4. Java Annotations Tutorial
5. Java Interview Questions
6. Spring Interview Questions
7. Android UI Design
and many more ....
I agree to the Terms and Privacy Policy

2
Leave a Reply

avatar
2 Comment threads
0 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
2 Comment authors
AshishMithilesh Recent comment authors

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  Subscribe  
newest oldest most voted
Notify of
Mithilesh
Guest

Nice site

Ashish
Guest

Very good !