Generating, visiting and unit testing grammar using ANTLR4 with Java and Scala
“Quality is free, but only to those who are willing to pay heavily for it” — T. DeMarco
Now unit testing is not that heavy to pay off so lets start the blog.
In this blog we will be continuing the voyage to the kingdom of ANTLR4
RECAP
In previous blog we discussed how to test whether a string is grammatically correct or not using console. To refer the first blog [HIT ME]
OVERVIEW
In this blog we will programmatically check whether a file containing the input string is grammatically correct or not.
STEPS TO TREASURE
Step 1 “Staring up” ->
Create new sbt project in intellij.
Step 2 “Including required dependencies in code” ->
Add following dependencies in build.sbt
a) libraryDependencies += “org.apache.commons” % “commons-io” % “1.3.2”
This dependency is included to do input\output from file. Its an optional dependency if you are using some other mechanism to perform file i/o.
b) libraryDependencies += “org.antlr” % “antlr4-runtime” % “4.5”
This dependency adds the antlr jar to project.
c) libraryDependencies += “org.scalatest” %% “scalatest” % “2.2.4” % “test”
It includes the scalatest jar to project which facilitates code testing.
d) libraryDependencies += “org.mockito” % “mockito-all” % “1.9.5”
It facilitates mocking in unit testing.
e) libraryDependencies += “com.typesafe.scala-logging” % “scala-logging_2.11” % “3.1.0”
Used for logging the errors if any wrong test input is provided.
Also following plugins were used in the project (add in plugins.sbt)
a) addSbtPlugin(“org.scoverage” % “sbt-scoverage” % “1.3.5”)
Used to see the code coverage by unit tests
b) addSbtPlugin(“org.scalastyle” %% “scalastyle-sbt-plugin” % “0.8.0”)
this plugin checks if any scalastyle warning is present or not.
Step 3 “Creating Desired Directory Structure” ->
The following image shows the directory structure of the project
In main, the ‘antlr’ folder contains the grammar file(.g4 extension), ‘java’ folder contains the auto generated grammar corresponding code, ‘scala’ folder has the computing logic. In ‘test’, resources folder has 2 sub folders. ‘expression’ folder contains input files and ‘expression_out’ folder contains corresponding output. The ‘scala’ folder has ‘ComputeArithmeticResultSpec’ which contains all the test case and the ‘util’ folder contains ‘TestHelper’ which facilitates testing.
Step 4 “Writing grammar” (My Favorite) ->
Here the grammar is written by keeping track of following conditions :
1) The grammar is based on performing arithmatic operations of addition, subtraction, division and multiplication on the operands.
2) An operand must be a positive integer including 0.
3) General syntax of a syntactically valid grammar expression is -
operation operand operand;
where,
operation can be ADD,SUB,DIV,MUL (either uppercase or lowercase)
operand must be any positive integer
and the expression must end with a semicolon.
The grammar file ‘ArithmaticGrammar.g4’ has following content
[code language=”scala”];
//grammar to showcase basic arithmetic operations with 2 integer operands only.
grammar ArithmaticGrammar;
startRule : operation space+ operand space+ operand space* semicolon ;
operation : add | sub | div | mul ;
operand : DIGIT + ;
space : ‘ ‘ ;
semicolon : ‘;’ ;
DIGIT : [0–9] ;
add : (‘add’|’ADD’) ;
sub : (‘sub’|’SUB’) ;
mul : (‘mul’|’MUL’) ;
div : (‘div’|’DIV’) ;
WS : [\t\r\n]+ -> skip ; //skip tabs, newlines, \r (Windows)
[/code]
Step 5 “Generating corresponding java files for grammar” (the magical step) ->
Follow steps below to generate java code of parser, lexer and visitor for the grammar file
1) Open the ‘ArithmaticGrammar.g4’ file and keep the control over the file by clicking anywhere on the file.
2) Click TOOLS from File Menu Bar and select ‘Configure ANTLR…’. Follow the steps in the link below to successfully configure the antlr
In my case the the confugration window looks like :
Please remeber to check both the check boxes on bottom left of the ‘Configure Antlr box’ to successfully generate the visitor for the grammar
3) Now from Menu Bar click Run and select ‘Generate ANTLR Recogniser’ (And Hocus Focus The java files are present exactly where we specified)
Step 6 “Making classes to use grammar” ->
Following classes are created :
1) CustomArithmetic.scala
This class parses the string from the input file and generates corresponding parse tree . It has 2 methods :
a) computeArithmeticResult
This method takes path of the input file as input and returns Either[Int,String]; Int is the result of the operation which is returned if the expression in the input file is parsed correctly and String is the error message returned when the expression is not parsed correctly.
This method generates the parse tree and from it generates the context (ctx) of type StartRuleContext, a context is defined for every grammar rule and is like a scope through which one can access all the components of that grammar rule. If the parsing is correct, result of the expression is computed and returned. Else the error message is returned.
b) getTestOutput
It is a private method which reads the content of the error stream and returns the corresponding string. If the returned string is empty, this implies that there were no errors in parsing.
2) CustomVisitor.scala
This class overrides ‘ArithmaticGrammarBaseVisitor’ (AutoGenerated Java file) and gives the custom implementation of ‘visitStartRule’ in which we define the logic of fetching the operands and the operator from the ctx(parameter) and evaluating the corresponding result.
[code language=”scala”]
class CustomVisitor extends ArithmaticGrammarBaseVisitor[Int] {
private def notEqualsNull(ref: AnyRef): Boolean = Option(ref).isDefined
override def visitStartRule(ctx: ArithmaticGrammarParser.StartRuleContext): Int = {
val result: Int = if (notEqualsNull(ctx.operation.add)) {
val operand1: Int = ctx.operand(0).DIGIT.toList.mkString.toInt
val operand2: Int = ctx.operand(1).DIGIT.toList.mkString.toInt
operand1 + operand2
}
else if (notEqualsNull(ctx.operation.sub)) {
val operand1: Int = ctx.operand(0).DIGIT.toList.mkString.toInt
val operand2: Int = ctx.operand(1).DIGIT.toList.mkString.toInt
operand1 — operand2
}
else if (notEqualsNull(ctx.operation.mul)) {
val operand1: Int = ctx.operand(0).DIGIT.toList.mkString.toInt
val operand2: Int = ctx.operand(1).DIGIT.toList.mkString.toInt
operand1 * operand2
}
else {
val operand1: Int = ctx.operand(0).DIGIT.toList.mkString.toInt
val operand2: Int = ctx.operand(1).DIGIT.toList.mkString.toInt
operand1 / operand2
}
result
}
}
[/code]
Step 7 “Testing” (The Climax) ->
‘Funsuite’ is used for testing. ‘TestHelper’ object is created to facilitate testing and it has 2 public methods :
a) testCustomArithmetic
Via this method we test the ‘CustomArithmetic’ class. Mocking is used to mock the ‘visitStartRule’ method of ‘CustomVisitor’ class to purely implement unit testing. This method calls ‘computeArithmeticResult’ method with input file path as parameter and its return value(expected result) is compared with the corresponding output file(actual result). If the content of input file as well as the output file are same, the test case is successful otherwise the mismatching returned data of ‘computeArithmeticResult’(expected result) method and output file text(actual result) are logged. Following is the corresponding code :
[code language=”scala”]
def testCustomArithmetic(packageName: String, folderName: String, fileName: String, computedValue:Int): Boolean = {
val customVisitorMock: CustomVisitor = mock[CustomVisitor]
when(customVisitorMock.visitStartRule(getContext(packageName + “/” + folderName + “/” + fileName))).thenReturn(computedValue)
val customArithmetic = new CustomArithmetic
val parsingResult: Either[Int, String] = customArithmetic.computeArithmeticResult(packageName + “/” + folderName + “/” + fileName)
val expectedResult: String = if (parsingResult.isLeft) parsingResult.left.get.toString else parsingResult.right.get
val actualResult: String = FileUtils.readFileToString(new File(packageName + “/” + folderName + “_out/” + fileName + “.out”), “utf-8”).trim
val result: Boolean = expectedResult == actualResult
if (!result) {
val logger = Logger(LoggerFactory.getLogger(“name”))
logger.info(“\n\nFor “ + fileName + “\nEXPECTED RESULT : “ + expectedResult + “\n\n\tis not equal to\n\n” + “ACTUAL RESULT : “ + actualResult + “\n”)
}
result
}
[/code]
b) testCustomVisitor
This method tests the ‘CustomVisitor’ class. The method take the input file path as parameter and returns the result of evaluation of the input expression. Here initially context is created by calling ‘getContext’ method which takes the path of input file as input and returns its corresponding context. Then a call to ‘visitStartRule’ method of ‘CustomVisitor’ class is made which returns integer value(computed result of arithmetic operation). Following is the code :
[code language=”scala”]
def testCustomVisitor(packageName: String, folderName: String, fileName: String): Int = {
val context = getContext(packageName + “/” + folderName + “/” + fileName)
val customVisitor = new CustomVisitor
customVisitor.visitStartRule(context)
}
private def getContext(filePath: String): StartRuleContext = {
val input: ANTLRFileStream = new ANTLRFileStream(filePath)
val lexer: ArithmaticGrammarLexer = new ArithmaticGrammarLexer(input)
val tokens: CommonTokenStream = new CommonTokenStream(lexer)
val parser: ArithmaticGrammarParser = new ArithmaticGrammarParser(tokens)
val ctx: StartRuleContext = parser.startRule
ctx
}
[/code]
The methods of the ‘TestHelper’ are used by ‘computeArithmeticResultSpec’ which extends ‘FunSuite’. Here the actual result(res) is computed by calling corresponding methods of ‘TestHelper’ and is then checked with the expected result(right side of ===). If actual result is equal to the expected result, the test case pass. Following are some test cases for ‘CustomArithmetic’ and ‘CustomVisitor’ :
[code language=”scala”]
/////////// Testing CustomArithmetic ////////////////
test(“Addition expression should parse correctly”) {
val res: Boolean = testCustomArithmetic(“/home/sahil/blogs/blog 2/ANTLRBlog2/src/test/resources”, “expression”, “addition”,164)
assert(res === true)
}
test(“Invalid expression 1 should not parse correctly”) {
val res: Boolean = testCustomArithmetic(“/home/sahil/blogs/blog 2/ANTLRBlog2/src/test/resources”, “expression”, “invalid1”,-1)
assert(res === true)
}
/////////// Testing CustomVisitor ////////////////
test(“Subtraction should give correct output”) {
val res: Int = testCustomVisitor(“/home/sahil/blogs/blog 2/ANTLRBlog2/src/test/resources”, “expression”, “subtraction”)
assert(res === 78)
}
test(“Multiplication should give correct output”) {
val res: Int = testCustomVisitor(“/home/sahil/blogs/blog 2/ANTLRBlog2/src/test/resources”, “expression”, “multiplication”)
assert(res === 5203)
}
[/code]
The above code snippet successfully conclude this blog. One thing to note here is that we have not written the test cases for the auto generated java code, instead we have covered only the code which was overridden from the auto generated code, example — the ‘visitStartRule’ method of auto generated ‘ArithmaticGrammarBaseVisitor’ class is been overridden in ‘CustomVisitor’ class and thus it is covered in unit testing.
The code is available on git :- https://github.com/sahil-sawhney/ANTLR4GrammarWithUnitTesting
Comments are welcomed.
References
1) https://github.com/scoverage/sbt-scoverage
2) http://alvinalexander.com/scala/how-to-use-mock-objects-with-scalatest
3) The Definitive ANTLR4 Reference by Terence Parr
happy coding…
