ANTLR on the web: why?
I started writing my first programs on MS-DOS. So I am very used to have my tools installed on my machine. However in 2016 the web is ubiquitous and so our languages could be needed also there.
Possible scenarios:
- ANTLR also on the web:
- users could want to access and possibly to minor changes files written in a DSL also from the web, while keeping using their fat-clients for complex tasks.
- ANTLR only on the web:
- you are dealing with domain-experts which are reluctant to install IDEs, so they prefer to have some web application where to write their DSL programs.
- you want to offer a simple DSL to specify queries to be executed directly in the browser.
In the first case you can generate your ANTLR parser using a Java target and a Javascript target, while in the second you could target just JavaScript
A simple example: a Todo list
The DSL we are going to use in this example will be super easy: it will represents a todo list, where each todo item is contained in a separate line and started by an asterisk.
An example of a valid input:
* do this * do that * do something else after an empty line
And this is our grammar:
grammar todo;
elements
: (element|emptyLine)* EOF
;
element
: '*' ( ' ' | '\t' )* CONTENT NL+
;
emptyLine
: NL
;
NL
: '\r' | '\n'
;
CONTENT
: [a-zA-Z0-9_][a-zA-Z0-9_ \t]*
;
Using the ANTLR Javascript target
You would need to install the ANTLR tool to generate the Javascript code for our parser. Instead of manually downloading ANTLR and its dependencies you can use a simple Gradle script. It makes also very straightforward to update the version of ANTLR you are using.
apply plugin: 'java'
repositories {
jcenter()
}
dependencies {
runtime 'org.antlr:antlr4:4.5.2'
}
task generateParser(type:JavaExec) {
main = 'org.antlr.v4.Tool'
classpath = sourceSets.main.runtimeClasspath
args = ['-Dlanguage=JavaScript', 'todo.g4', '-o', 'static/generated-parser']
}
You can now generate your parser by running:
gradle generateParser
Ok, this one was easy.
Invoking the parser
Unfortunately the JS libraries we are using do not work when simply opening local files: it means that also for our little example we need to use HTTP. Our web server will just have to serve a bunch of static files. To do this I chose to write a super simple application in flask. There are millions of alternatives to serve static files so pick the one you prefer. I will not detail how to serve static files through flask here but code is available on GitHub and if you have issues with that you can add a comment to this post to let me know.
Our static files will include:
- the generated parser we got by running gradle generateParser
- the Antlr4 JS runtime
- the JS library require.js
- HTML and CSS
You can get the Antlr4 JS runtime from here. To avoid having to import tens of files manually we will use require.js. You can get the flavor or require.js we need from here.
We are going to add a textarea and a button. When the user clicks on the button we will parse the content of the textarea. Simple, right?
This is the HTML code for this design masterpiece:
<div id="inputs"> <textarea id="code"> * play with antlr4 * write a tutorial </textarea> <br/> <button id="parse">Parse</button> </div>
First thing first, import require.js:
<script type="text/javascript" src="lib/require.js"></script>
By the way, we are not using jquery, I know this could be shocking.
Good, now we have to invoke the parser
<script type="text/javascript">
var antlr4 = require('antlr4/index');
var TodoLexer = require('generated-parser/todoLexer');
var TodoParser = require('generated-parser/todoParser');
document.getElementById("parse").addEventListener("click", function(){
var input = document.getElementById("code").value;
var chars = new antlr4.InputStream(input);
var lexer = new TodoLexer.todoLexer(chars);
var tokens = new antlr4.CommonTokenStream(lexer);
var parser = new TodoParser.todoParser(tokens);
parser.buildParseTrees = true;
var tree = parser.elements();
console.log("Parsed: "+ tree);
});
</script>
Cool, now our code is parsed but we do not do anything with it. Sure we can fire the developer console in the browser and print some information about the tree to verify it is working and to familiarize with the structure of the tree ANTLR is returning.
Display results
If we were building some kind of TODO application we may want to somehow represent the information the user inserted through the DSL.
Let’s get something like this:
To do so we basically need to add the function updateTree which navigate the tree returned by ANTLR and build some DOM nodes to represent its content
<script type="text/javascript">
var updateTree = function(tree, ruleNames) {
var container = document.getElementById("tree");
while (container.hasChildNodes()) {
container.removeChild(container.lastChild);
}
for (var i = 0; i < tree.children.length; i++) {
var child = tree.children[i];
var nodeType = ruleNames[child.ruleIndex];
if (nodeType == "element") {
var newElement = document.createElement("div");
newElement.className = "todoElement";
var newElementText = document.createTextNode(child.children[2].getText());
newElement.appendChild(newElementText);
container.appendChild(newElement);
}
}
};
var antlr4 = require('antlr4/index');
var TodoLexer = require('generated-parser/todoLexer');
var TodoParser = require('generated-parser/todoParser');
document.getElementById("parse").addEventListener("click", function(){
var input = document.getElementById("code").value;
var chars = new antlr4.InputStream(input);
var lexer = new TodoLexer.todoLexer(chars);
var tokens = new antlr4.CommonTokenStream(lexer);
var parser = new TodoParser.todoParser(tokens);
parser.buildParseTrees = true;
var tree = parser.elements();
console.log("Parsed: "+ tree);
updateTree(tree, parser.ruleNames);
});
</script>
Here you go!
Code
If it is not the first time you are reading this blog you will be suspecting that some code is coming. As usual, code is on GitHub: https://github.com/ftomassetti/antlr-web-example
Next steps
The next step is to perform error handling: we want to catch errors and point them to the users. Then we may want to add syntax highlighting by using ACE for example. This seems a good starting point:
I really think that simple textual DSLs could help to make several applications much more powerful. However, it is not straightforward to create a nice editing experience on the web. I would like to spend some more time playing with this.
| Reference: | ANTLR and the web: a simple example from our JCG partner Federico Tomassetti at the Federico Tomassetti blog. |
