Home » Java » Core Java » Extracting JavaDoc documentation from source files using JavaParser

About Federico Tomassetti

Federico Tomassetti
Federico has a PhD in Polyglot Software Development. He is fascinated by all forms of software development with a focus on Model-Driven Development and Domain Specific Languages.

Extracting JavaDoc documentation from source files using JavaParser

A lot of people are using JavaParser for the most different goals. One of these is extracting documentation. In this short post we will see how you can print all the JavaDoc comments associated to classes or interfaces.

Code is available on GitHub: https://github.com/ftomassetti/javadoc-extractor

Getting all the Javadoc comments for classes

We are reusing DirExplorer, a supporting class presented in the introduction to JavaParser. This class permits to process a directory, recursively, parsing all the Java files contained there.

We can start by iterating over all the classes and find the associated Javadoc comments.

/**
 * Iterate over the classes and print their Javadoc.
 */
public class ClassesJavadocExtractor {
 
    public static void main(String[] args) {
        File projectDir = new File("source_to_parse/");
        new DirExplorer((level, path, file) -> path.endsWith(".java"), (level, path, file) -> {
            try {
                new VoidVisitorAdapter<Object>() {
                    @Override
                    public void visit(ClassOrInterfaceDeclaration n, Object arg) {
                        super.visit(n, arg);
                        if (n.getComment() != null && n.getComment() instanceof JavadocComment) {
                            String title = String.format("%s (%s)", n.getName(), path);
                            System.out.println(title);
                            System.out.println(Strings.repeat("=", title.length()));
                            System.out.println(n.getComment());
                        }
                    }
                }.visit(JavaParser.parse(file), null);
            } catch (IOException e) {
                new RuntimeException(e);
            }
        }).explore(projectDir);
    }
 
}

As you can see getting the JavaDoc comments is fairly easy. It produces this result:

ASTParserConstants (/javaparser/javaparser-core/target/generated-sources/javacc/com/github/javaparser/ASTParserConstants.java)
==============================================================================================================================
/**
 * Token literal values and constants.
 * Generated by org.javacc.parser.OtherFilesGen#start()
 */
 
ParseException (/javaparser/javaparser-core/target/generated-sources/javacc/com/github/javaparser/ParseException.java)
======================================================================================================================
/**
 * This exception is thrown when parse errors are encountered.
 * You can explicitly create objects of this exception type by
 * calling the method generateParseException in the generated
 * parser.
 *
 * You can modify this class to customize your error reporting
 * mechanisms so long as you retain the public fields.
 */
 
ASTParser (/javaparser/javaparser-core/target/generated-sources/javacc/com/github/javaparser/ASTParser.java)
============================================================================================================
/**
 *
This class was generated automatically by javacc, do not edit.

*/ ASTParserTokenManager (/javaparser/javaparser-core/target/generated-sources/javacc/com/github/javaparser/ASTParserTokenManager.java) ==================================================================================================================================== /** Token Manager. */

Getting all the Javadoc comments and find the documented elements

In other cases we may want to start collecting all the Javadoc comments and then finding the element which is commented. We can also do that easily with Javaparser:

/**
 * Iterate over all the Javadoc comments and print them together with a description of the commented element.
 */
public class AllJavadocExtractor {
 
    public static void main(String[] args) {
        File projectDir = new File("source_to_parse/");
        new DirExplorer((level, path, file) -> path.endsWith(".java"), (level, path, file) -> {
            try {
                new VoidVisitorAdapter<Object>() {
                    @Override
                    public void visit(JavadocComment comment, Object arg) {
                        super.visit(comment, arg);
                        String title = null;
                        if (comment.getCommentedNode().isPresent()) {
                            title = String.format("%s (%s)", describe(comment.getCommentedNode().get()), path);
                        } else {
                            title = String.format("No element associated (%s)", path);
                        }
                        System.out.println(title);
                        System.out.println(Strings.repeat("=", title.length()));
                        System.out.println(comment);
                    }
                }.visit(JavaParser.parse(file), null);
            } catch (IOException e) {
                new RuntimeException(e);
            }
        }).explore(projectDir);
    }
 
    private static String describe(Node node) {
        if (node instanceof MethodDeclaration) {
            MethodDeclaration methodDeclaration = (MethodDeclaration)node;
            return "Method " + methodDeclaration.getDeclarationAsString();
        }
        if (node instanceof ConstructorDeclaration) {
            ConstructorDeclaration constructorDeclaration = (ConstructorDeclaration)node;
            return "Constructor " + constructorDeclaration.getDeclarationAsString();
        }
        if (node instanceof ClassOrInterfaceDeclaration) {
            ClassOrInterfaceDeclaration classOrInterfaceDeclaration = (ClassOrInterfaceDeclaration)node;
            if (classOrInterfaceDeclaration.isInterface()) {
                return "Interface " + classOrInterfaceDeclaration.getName();
            } else {
                return "Class " + classOrInterfaceDeclaration.getName();
            }
        }
        if (node instanceof EnumDeclaration) {
            EnumDeclaration enumDeclaration = (EnumDeclaration)node;
            return "Enum " + enumDeclaration.getName();
        }
        if (node instanceof FieldDeclaration) {
            FieldDeclaration fieldDeclaration = (FieldDeclaration)node;
            List<String> varNames = fieldDeclaration.getVariables().stream().map(v -> v.getName().getId()).collect(Collectors.toList());
            return "Field " + String.join(", ", varNames);
        }
        return node.toString();
    }
 
}

Here most of the code is about providing a description for the commented node (method describe).

Conclusions

Manipulate the AST and finding the Javadoc comments is quite easy. However one missing feature is the possibility to extract the information contained in the Javadoc in a structured form. For example, you may want to get only the part of the Javadoc associated to a certain parameter or to the return value. Javaparser currently does not have this feature, but I am working on it and it should be merged in the next 1-2 weeks. If you want to follow the development take a look at issue 433.

Thanks for reading and happy parsing!

Do you want to know how to develop your skillset to become a Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you our best selling eBooks for FREE!

 

1. JPA Mini Book

2. JVM Troubleshooting Guide

3. JUnit Tutorial for Unit Testing

4. Java Annotations Tutorial

5. Java Interview Questions

6. Spring Interview Questions

7. Android UI Design

 

and many more ....

 

Receive Java & Developer job alerts in your Area

 

Leave a Reply

Be the First to Comment!

Notify of
avatar
wpDiscuz