Swathi V

About Swathi V

Loves Art and Technology! Would like to blog and share.. Involved in Apache Hadoop and its ecosystem. Eager to be a part of Big Data Revolution.

ANTLR: Getting Started

This post drives you towards the basics of ANTLR. Previously, we had learnt about setting up of ANTLR as an external tool.RECAP! It’s here: ANTLR External Tool
:)
So, here we go….

What is ANTLR?

• ANother Tool for Language Recognition, is a language tool that provides a framework for constructing recognizers, interpreters, compilers, and translators from grammatical descriptions containing actions.

What can be the target languages? 

• Action Script, Ada
• C
• C#; C#2
• C#3
• D
• Emacs ELisp
• Objective C
• Java
• Java Script
• Python
• Ruby
• Perl6
• Perl
• PHP
• Oberon
• Scala
 
What does ANTLR support?

• Tree construction
• Error recovery
• Error handling
• Tree walking
• Translation
 
What environment does it support? 

ANTLRWorks is the IDE for ANTLR. It is the graphical grammar editor and debugger, written by Jean Bovet using Swing.

What for ANTLR can be used?

• “”REAL”" programming languages
• domain-specific languages [DSL]
 
Who is using ANTLR?

• Programming languages :Boo, Groovy, Mantra, Nemerle, XRuby etc.
• Other Tools: HIbernate, Intellij IDEA, Jazillian, JBoss Rules, Keynote(Apple), WebLogic(Oracle) etc.
 
Where is that you can look for ANTLR?

You can always follow here http://www.antlr.org
• to download ANTLR and ANTLRWorks, which are free and open source
• docs,articles,wiki,mailing list,examples…. You can catch everything here!

Row your Boat….
 

  • Basic terms

Lexer : converts a stream of characters to a stream of tokens.
Parser : processes of tokens, possibly creating AST
Abstract Syntax Tree(AST): an intermediate tree representation of the parsed input that is simpler to process than the stream of tokens. It can as well be processed multiple times.
Tree Parser: It processes an AST
String Template: a library that supports using templates with placeholders for outputting text

  • General Steps

• Write Grammar in one or more files
• Write string templates[optional]
• Debug your grammar with ANTLRWorks
• Generate classes from grammar
• Write an application that uses generated classes
• Feed the application text that conforms to the grammar


A Bit Further….

Lets write a simple grammar which consists of
• Lexer
• Parser
Lexer: Breaks the input stream into tokens
Lets take the example of simple declaration type in C of the form “int a,b;” or “int a;” and same with float. As we see we can write lexer as follows:

//TestLexer.g
grammar TestLexer;
ID : (‘a’..’z'|’A’..’Z'|’_') (‘a’..’z'|’A’..’Z'|’0′..’9′|’_'|’.'|’a’..’z'|’A’..’Z')*; COMMA: ‘,’;
SEMICOLON:’;';
DATATYPE: ‘int’ | ‘float’;

As we could see, these were the characters that were to be converted to tokens.
So, now lets write some rules which processes these tokens generated and may it create a parse tree accordingly.

//TestParser.g
grammar TestParser;
options {language : Java;}
decl:DATATYPE ID (‘,’ ID)* ;

Running ANTLR on the grammar just generates the lexer and parser,TestParser and TestLexer. To actually try the grammar on some input, we
need a test rig with a main( ) method as follows:
// Test.java

import org.antlr.runtime.*;
public class Test {

public static void main(String[] args) throws Exception {

// Create an input character stream from standard in
ANTLRFileStream input = new ANTLRFileStream("input"); // give path to the file input
// Create an ExprLexer that feeds from that stream
TestLexer lexer = new TestLexer(input);
// Create a stream of tokens fed by the lexer
CommonTokenStream tokens = new CommonTokenStream(lexer);
// Create a parser that feeds off the token stream
TestParser parser = new TestParser(tokens);
// Begin parsing at rule decl
parser.decl();
}}

We shall see how to create an AST and walk over the tree in the next blog post…
Happy learning….! :)

Reference: Getting Started With ANTLR:Basics from our JCG partner Swathi V at the * Techie(S)pArK * blog.

Related Whitepaper:

Functional Programming in Java: Harnessing the Power of Java 8 Lambda Expressions

Get ready to program in a whole new way!

Functional Programming in Java will help you quickly get on top of the new, essential Java 8 language features and the functional style that will change and improve your code. This short, targeted book will help you make the paradigm shift from the old imperative way to a less error-prone, more elegant, and concise coding style that’s also a breeze to parallelize. You’ll explore the syntax and semantics of lambda expressions, method and constructor references, and functional interfaces. You’ll design and write applications better using the new standards in Java 8 and the JDK.

Get it Now!  

One Response to "ANTLR: Getting Started"

  1. Ramesh says:

    Hey Swathi,

    I’m an engineering student working on a mini project on ANTLR, supposed to use string templates to emit Java code that does CRUD operations on a file, database and the like. I’m stuck with the template machinery, all i get are errors. Is there any way you can help me out ??

    Thank You

Leave a Reply


six × 5 =



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.

Sign up for our Newsletter

15,153 insiders are already enjoying weekly updates and complimentary whitepapers! Join them now to gain exclusive access to the latest news in the Java world, as well as insights about Android, Scala, Groovy and other related technologies.

As an extra bonus, by joining you will get our brand new e-books, published by Java Code Geeks and their JCG partners for your reading pleasure! Enter your info and stay on top of things,

  • Fresh trends
  • Cases and examples
  • Research and insights
  • Two complimentary e-books