About Markus Eisele

Markus is a principal technology consultant working for msg systems ag in Germany. Markus is a software architect, developer and consultant. He also writes for IT magazines. Markus is speaking at different conferences about his favorite topics. He is also part of the Java EE 7 expert group.

Documenting Compliance – About TCKs, Specifications and Testing

Working with software specifications is hard. No matter in which exact filed; you end up with the big question: Does everything ever specified is implemented and tested? Back in the days of waterfall driven methodologies this has been an issue and even today at the time of writing, agility and user-stories still don’t guarantee you the perfect fit. Many of today’s agile approaches combine well with Test Driven Development or even Behavior Driven Development concepts to turn the issue upside down. Instead of asking “Does my code cover every single sentence of written specification?” those simply assume that writing the tests first is a valid way of having the needed coverage. The down-side here is the lack of documentation which easily can happen. Additionally you never find a suitable document workflow to re-factor tests to the one single document. What might work for individual solutions and projects comes to an end if you look at stuff like “Technology Compatibility Kits” (TCK) which by nature are more or less gathered from any kind of document based written specification.

TCKs for the Java platforms

Diving into that kind of topics always is a good candidate to polarize the development community. Especially because documentation is still a topic which tends to be forgotten or delayed completely. To me documentation is key on may levels. On a framework level it assures that your users don’t struggle and you lay a good ground for quick adoption. To me the Arquillian project and team did an amazing job in their first years. Even on a project level this makes sense to quickly swap new team members in and out without losing knowledge. But there is another area which not simply benefits from it but has a strong relation to documentation: The Java TCKs. All Java Platforms define Java Specification Requests (JSRs) as the point for language improvements. A Technology Compatibility Kit (TCK) is a suite of tests that at least nominally checks a particular alleged implementation of a Java Specification Request (JSR) for compliance. Given the fact, that most specifications exist in some Office like documents and are pushed around as PDFs for review and comments it is nearly impossible to say weather a TCK has a defined coverage of the original specification at all. This at best is scary. Most of the time it is annoying because Reference Implementations (RIs) simply forget to cover parts of the spec and the user has to handle the resulting bugs or behaviors in specific ways. If that is possible at all.

Just a short note here regarding the availability of TCKs. Most of them aren’t available as of today but subject to terms of license and financial agreements. Hopefully this is going to change with the upcoming changes to the Java Community Process.

Some JBoss Goddess to cure documentation

But some bright minds came up with a solution. It probably isn’t a big surprise that a great effort came out of a couple of RedHats. A small project which initially was created as part of the hibernate-validator project which is the RI for BeanValidation is here to cure the problems. The unknown and itself mostly undocumented jboss-test-audit project calls itself “Utility classes for TCK Test Coverage Report”. This perfectly nails it. It is a very lightweight but still powerful little addition to any RI which post-processes sources for special annotations to gather a coverage report for any project which has the goal of implementing a specification. It is licensed under the Apache License, Version 2.0 and you only need some very few steps to get this up an running against your own setup. It all begins with the specification. This is a xml document which defines the different sections and required assertions.

<specification>
    <section id="1" title="Chapter 1 - Introduction"/>
    <section id ="2" title="Chapter 2 - What's new">
        <assertion id="a">
            <text>A simple sample test</text>
        </assertion>
</section>
</specification>

This document is the base line for your tests. You now need to go ahead and equip all your tests with relevant section and assertion information. This might look like the following:

SpecVersion(spec = "spectests", version = "1.0.0")
public class AppTest {

    @Test
    @SpecAssertion(section = "2", id = "a")
    public void simpleTestForAssertion() {
        App app = new App();
        assertEquals(app.sayHello("Markus"), "Hello Markus");
    }

Combined with a bit of maven magic (maven-processor-plugin) all the annotations are parsed and a nice report is generated about the overall coverage. If you want to have a look at the complete bootstrap example find it on github.com/myfear.

The Hard Parts

This obviously is a no-brainer. To add some annotations to your tests will not be the hardest thing you ever did. What is really hard is to convert your documentation into that fancy audit xml format. There are plenty of ways to do this. Given the fact, that most of the companies leading a JSR have some kind of hard-core document management in place should make this a once in a lifetime thing to implement. If you’re working with Microsoft Word you could also use the available xml schema to write well formed documents with it (which is a pain! Don’t do it!).

Plenty of Ideas

The little utility classes work comparably well. But there is still plenty of room for improvements. It might be a valid idea to have some supportive information here like issue-numbers or other references. I also would like to be able to use asciidoc in the documentation. But I’m not complaining here because I am not going to change it myself. But if anybody is interested, the complete thing is on github.com and I believe those guys know how community works and accept contributions.

Future Wishes for the JCP

Given that simple and easy approach it would be a good thing to foster adoption along with JSRs. So if you like it approach the EC member you trust and make him/her aware of this and put it as an idea on their list.
 

Related Whitepaper:

Bulletproof Java Code: A Practical Strategy for Developing Functional, Reliable, and Secure Java Code

Use Java? If you do, you know that Java software can be used to drive application logic of Web services or Web applications. Perhaps you use it for desktop applications? Or, embedded devices? Whatever your use of Java code, functional errors are the enemy!

To combat this enemy, your team might already perform functional testing. Even so, you're taking significant risks if you have not yet implemented a comprehensive team-wide quality management strategy. Such a strategy alleviates reliability, security, and performance problems to ensure that your code is free of functionality errors.Read this article to learn about this simple four-step strategy that is proven to make Java code more reliable, more secure, and easier to maintain.

Get it Now!  

Leave a Reply


9 − = seven



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.

Sign up for our Newsletter

20,709 insiders are already enjoying weekly updates and complimentary whitepapers! Join them now to gain exclusive access to the latest news in the Java world, as well as insights about Android, Scala, Groovy and other related technologies.

As an extra bonus, by joining you will get our brand new e-books, published by Java Code Geeks and their JCG partners for your reading pleasure! Enter your info and stay on top of things,

  • Fresh trends
  • Cases and examples
  • Research and insights
  • Two complimentary e-books