About Martin Mois

Martin is a Java EE enthusiast and works for an international operating company. He is interested in clean code and the software craftsmanship approach. He also strongly believes in automated testing and continuous integration.

API design and performance

When you design a new API you have to take a lot of decisions. These decisions are based on a number of design principles. Joshua Bloch has summarized some of them in his presentation “How to Design a Good API and Why it Matters”. The main principles he mentions are:
 
 
 
 
 
 

  • Easy to learn
  • Easy to use
  • Hard to misuse
  • Easy to read and maintain code that uses it
  • Sufficiently powerful to satisfy requirements
  • Easy to extend
  • Appropriate to audience

As we see from the list above, Joshua Bloch puts his emphasis on readability and usage. A point that is completely missing from this listing is performance. But can performance impact your design decisions at all?

To answer this question let’s try to design a simple use case in form of an API and measure its performance. Then we can have a look at the results and decide whether performance considerations have an impact on the API or not. As an example we take the classic use case of loading a list of customers from some service/storage. What we also want to consider is the fact that not all users are allowed to perform this operation. Hence we will have to implement some kind of permission check. To implement this check and to return this information back to the caller, we have multiple ways to do so. The first try would look like this one:

List<Customer> loadCustomersWithException() throws PermissionDeniedException

Here we model an explicit exception for the case the caller has not the right to retrieve the list of customers. The method returns a list of Customer objects while we assume that the user can be retrieved from some container or ThreadLocal implementation and has not to be passed to each method.
The method signature above is easy to use and hard to misuse. Code that uses this method is also easy to read:

try {
		List<Customer> customerList = api.loadCustomersWithException();
		doSomething(customerList);
	} catch (PermissionDeniedException e) {
		handleException();
	}

The reader immediately sees that a list of Customers is loaded and that we perform some follow-up action only in case we don’t get a PermissionDeniedException. But in terms of performance exceptions do cost some CPU time as the JVM has to stop the normal code execution and walk up the stack to find the position where the execution has to be continued. This is also extremely hard if we consider the architecture of modern processors with their eager execution of code sequences in pipelines. So would it be better in terms of performance to introduce another way of informing the caller about the missing permission?

The first idea would be to create another method in order to check the permission before calling the method that eventually throws an exception. The caller code would then look like this:

if(api.hasPermissionToLoadCustomers()) {
		List<Customer> customerList = api.loadCustomers();
		doSomething(customerList);
	}

The code is still readable, but we have introduced another method call that also costs runtime. But now we are sure that the exception won’t be thrown; hence we can omit the try/catch block. This code now violates the principle “Easy to use”, as we now have to invoke two methods for one use case instead of one. You have to pay attention not to forget the additional call for each retrieval operation. With regard to the whole project, your code will be cluttered with hundreds of permission checks.

Another idea to overcome the exception is to provide an empty list to the API call and let the implementation fill it. The return value can then be a boolean value indicating if the user has had the permission to execute the operation or if the list is empty because no customers have been found. As this sounds like C or C++ programming where the caller manages the memory of the structures that the callees uses, this approach costs the construction of an empty list even if you don’t have a permission to retrieve the list at all:

List<Customer> customerList = new ArrayList<Customer>();
	boolean hasPermission = api.loadCustomersWithListAsParameter(customerList);
	if(hasPermission) {
		doSomething(customerList);
	}

One last approach to solve the problem to return two pieces of information to the caller would be the introduction of a new class that holds next to the returned list of Customers also a boolean flag indicating if the user has had the permission to perform this operation:

CustomerList customerList = api.loadCustomersWithReturnClass();
	if(customerList.isUserHadPermission()) {
		doSomething(customerList.getCustomerList());
	}

Again we have to create additional objects that cost memory and performance, and we also have to deal with an additional class that has nothing more to do than to serve as a simple data holder to provide the two pieces of information. Although this approach is again easy to use and creates readable code, it creates additional overhead in order to maintain the separate class and has some kind of awkward means to indicate that an empty list is empty because of the missing permission.

After having introduced these different approaches it is now time to measure their performance, one time for the case the caller has the permission and one time for the case the caller does not have the necessary permission. The results in the following table are shown for the first case with 1.000.000 repetitions:

MeasurementTime[ms]
testLoadCustomersWithExceptionWithPermission33
testLoadCustomersWithExceptionAndCheckWithPermission34
testLoadCustomersWithReturnClassWithPermission41
testLoadCustomersWithListAsParameterWithPermission66

As we have expected before, the two approaches that introduce an additional class respectively pass an empty list cost more performance than the approaches that use an exception. Even the approach that uses a dedicated method call to check for the permission is not much slower than the one without it.
The following table now shows the results for the case where the caller does not have the permission to retrieve the list:

MeasurementTime[ms]
testLoadCustomersWithExceptionNoPermission1187
testLoadCustomersWithExceptionAndCheckNoPermission5
testLoadCustomersWithReturnClassNoPermission4
testLoadCustomersWithListAsParameterNoPermission5

Not surprisingly the approach where a dedicated exception is thrown is much slower than the other approaches. The magnitude of this impact is much higher than one would expect before. But from the table above we already know the solution for this case: Just introduce another method that can be used to check for the permission ahead, in case you expect a lot of permission denied use cases. The huge difference in runtime between the with and without permission use cases can be explained by the fact that I have returned an ArrayList with one Customer object in case the caller was in possession of the permission; hence the loadCustomer() calls where a bit more expensive than in case the user did not possess this permission.

Conclusion

When performance is a critical factor, you also have to consider it when designing a new API. As we have seen from the measurements above, this may lead to solutions that violate common principles of API design like “easy to use” and “hard to misuse”.

Reference: API design and performance from our JCG partner Martin Mois at the Martin’s Developer World blog.
Related Whitepaper:

Software Architecture

This guide will introduce you to the world of Software Architecture!

This 162 page guide will cover topics within the field of software architecture including: software architecture as a solution balancing the concerns of different stakeholders, quality assurance, methods to describe and evaluate architectures, the influence of architecture on reuse, and the life cycle of a system and its architecture. This guide concludes with a comparison between the professions of software architect and software engineer.

Get it Now!  

6 Responses to "API design and performance"

  1. Andreas says:

    Why not inverting control?
    api.withCustomers(CustomerHandler)

    CustomerHandler being an interface with one method taking the customers as parameter. The permission check is then done within the API not calling the method if access is denied. If the class implementing the interface is stateless it could even be a Singleton. Even nicer, with Java 8 lamdas/method references there are even more options incoming. Not sure how expensive a method reference is in this case though.

  2. Martin says:

    A good idea! Why not giving this approach a try? The source code for this article can be found in my github repo: https://github.com/siom79/martins-developer-world. Let me know what you have found out!

    • Andreas says:

      OK, I did a quick run with your code and an interface passed into the API:

      Open JDK7:
      testLoadCustomersWithExceptionNoPermission(): 695 ms
      testLoadCustomersWithExceptionWithPermission(): 18 ms
      testLoadCustomersWithExceptionAndCheckNoPermission(): 3 ms
      testLoadCustomersWithExceptionAndCheckWithPermission(): 17 ms
      testLoadCustomersWithReturnClassNoPermission(): 4 ms
      testLoadCustomersWithReturnClassWithPermission(): 24 ms
      testLoadCustomersWithListAsParameterNoPermission(): 3 ms
      testLoadCustomersWithListAsParameterWithPermission(): 54 ms
      testLoadCustomersWithCallbackAndNewInnerClassNoPermission(): 6 ms
      testLoadCustomersWithCallbackAndNewInnerClassWithPermission(): 24 ms
      testLoadCustomersWithCallbackAndSingletonNoPermission(): 3 ms
      testLoadCustomersWithCallbackAndSingletonWithPermission(): 21 ms

      OpenJDK 8:
      testLoadCustomersWithExceptionNoPermission(): 804 ms
      testLoadCustomersWithExceptionWithPermission(): 11 ms
      testLoadCustomersWithExceptionAndCheckNoPermission(): 4 ms
      testLoadCustomersWithExceptionAndCheckWithPermission(): 10 ms
      testLoadCustomersWithReturnClassNoPermission(): 6 ms
      testLoadCustomersWithReturnClassWithPermission(): 25 ms
      testLoadCustomersWithListAsParameterNoPermission(): 3 ms
      testLoadCustomersWithListAsParameterWithPermission(): 21 ms
      testLoadCustomersWithCallbackAndNewInnerClassNoPermission(): 7 ms
      testLoadCustomersWithCallbackAndNewInnerClassWithPermission(): 15 ms
      testLoadCustomersWithCallbackAndSingletonNoPermission(): 3 ms
      testLoadCustomersWithCallbackAndSingletonWithPermission(): 16 ms
      testLoadCustomersWithCallbackAndLambdaNoPermission(): 4 ms
      testLoadCustomersWithCallbackAndLambdaWithPermission(): 20 ms
      testLoadCustomersWithCallbackAndMethodReferenceNoPermission(): 5 ms
      testLoadCustomersWithCallbackAndMethodReferenceWithPermission(): 21 ms

      I have to admit I am kind of surprised that inner classes do quite well. It is clearly visible that they have a creation overhead compared to lambdas/method reference. Calling inner classes on the other hand seems to be more efficient than the indirection of lambdas and method references via invokeDynamic. I can improve both lambda and method reference performance by assigning them to a field before using them but it doesn’t bring it to the level of a singleton. The singleton test seems to improve also a bit if I create your Api object outside of the loop.

      I also wonder if branch prediction has an influence on the tests. What would happen if we run the tests not separate but with alternating true/false responses? In theory it should decrease a the performance for the versions with double checks.

      • Martin says:

        Your additions are really interesting. Especially the result that the approaches using lambdas and method references don’t perform as good as the one with the singleton. This indicates that we should not overstress the usage of lambdas with JDK 8.

        Executing the tests with alternating runs of with and without permission would be interesting. But more hard to measure. It seems to be time for some kind of advance TestRunner that lets you run junit tests with alternating sequences a number of times and measures performance. ;)

      • Andreas says:

        OpenJDK 8 random true/false 50/50:
        testLoadCustomersWithException(): 449 ms
        testLoadCustomersWithExceptionAndCheck(): 18 ms
        testLoadCustomersWithReturnClass(): 21 ms
        testLoadCustomersWithListAsParameter(): 32 ms
        testLoadCustomersWithCallbackAndNewInnerClass(): 19 ms
        testLoadCustomersWithCallbackAndSingleton(): 20 ms
        testLoadCustomersWithCallbackAndLambda(): 22 ms
        testLoadCustomersWithCallbackAndMethodReference(): 22 ms

        So yes, branch prediction makes a difference. Taking this out of the picture all the different options are so close together that the influence of the API becomes minimal. The only thing you clearly should not do is misusing exceptions or passing in a list by default.

  3. Dmitri T says:

    Thanks for great article, Martin.

    In addition I’d like to mention that it’s possible to use Apache JMeter free and open-source tool in order to perform end-to-end performance testing of web service endpoints easily and quickly as unit tests may not give a real picture.

    Check out Testing SOAP/REST Web Services Using JMeter guide for more details.

Leave a Reply


eight − 6 =



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.

Sign up for our Newsletter

20,709 insiders are already enjoying weekly updates and complimentary whitepapers! Join them now to gain exclusive access to the latest news in the Java world, as well as insights about Android, Scala, Groovy and other related technologies.

As an extra bonus, by joining you will get our brand new e-books, published by Java Code Geeks and their JCG partners for your reading pleasure! Enter your info and stay on top of things,

  • Fresh trends
  • Cases and examples
  • Research and insights
  • Two complimentary e-books