Scala Software Development Build with QA Unit Test by Scalatest using sbt Tutorial
George Jen, Jen Tek LLC
This tutorial is for developers or learners who are new to Scala as a programming language and want to get familiar with processes of Scala development build and QA unit testing. If this is of interest to you, read on.
Pre-requisite are JDK, Scala compiler, sbt build tool and some basic computer science knowledge.
In this tutorial, it is assumed you have Java JDK has been installed. You will need to install JDK if it is not installed.
Additionally, sbt, the Scala build tool is needed. If you do not have sbt installed, you can get it at
https://www.scala-sbt.org/download.html
Finally, you need to have Scala compiler and run time installed, if it is not there. You can get scala compiler at
https://www.scala-lang.org/download/
Some basic computer science knowledge
Let’s start with something more challenging than “hello world”. This is to assume having some basic computer science knowledge, such as recursive function calls, or data structure such as stack.
Tail Recursion
In computer science, a tail call is a subroutine call performed as the final action of a procedure. If a tail call might lead to the same subroutine being called again later in the call chain, the subroutine is said to be tail-recursive, which is a special case of recursion. Not all recursion is tail recursion.
The benefit of tail recursion run by program coded in functional programming such as Scala is that a tail recursive function will not build new stack frame after each call, all calls will execute in a single stack frame.
To visualize, here is pseudo code for a tail recursive call, the last line is the to call itself only, nothing is applied to the result of the call:
def thisIsATailRecursion(param:T):U={
if <isExitConditionMet>
//end the recursion
return <U>
thisIsATailRecursion(someOperation(param))
}here is the pseudo code for not a tail recursive call, because the last line does something on the result of call itself:
def thisIsNotATailRecursion(param:T):U={
if <isExitConditionMet>
return <U>
doSomeThing(thisIsNOTATailRecursion(someOperation(param)))
}Here are some simple example of tail recursion applications for the discussion in this writing:
Reverse a List (reinvent wheel because there is a method for List class called reverse)
//To reverse List[Any], reinventing wheel of List().reverse method
//This is for the sake of demo tail recursion, which is the final step of
//inner function doReverse() is the to call itself -- final step of
//recursion, i.e., tail recursion.
def reverseList(l:List[Any]): List[Any]={
var result:List[Any]=List()
//Inner function that is tail recursive
def doReverse(l: List[Any]): List[Any]={
if (l.size<1)
result
else
{
result = l(0) :: result
doReverse(l.drop(1))
}
}
return doReverse(l)
}Run Length Encoding Compression
Here are more example of tail recurse call, doing Run-length encoding
According to Wikipedia:
https://en.wikipedia.org/wiki/Run-length_encoding
Run-length encoding (RLE) is a form of lossless data compression in which runs of data (sequences in which the same data value occurs in many consecutive data elements) are stored as a single data value and count, rather than as the original run.
For example:
For string
Get Uppppppppppppppppppppppppppppppppppppp!
After compressed by RLE, it would be:
Get U37p!
Here is the scala code and using tail recursion on the RLE compression:
def runLevelEncode(l: String): String = {
var stringBuf:ListBuffer[String]=ListBuffer.empty[String]
stringBuf.clear()
//Inner helper function that is tail recursion
def doRLE(l: String): String = {
l.toList match {
case Nil=>scala.collection.mutable.ListBuffer[String]()
case h :: t => {
val ms = l.toList.span(_ == h)
if (ms._1.size>1)
{
stringBuf += ms._1.size.toString+ms._1(0)
}
else
{
stringBuf += ""+ms._1(0)
}
doRLE(ms._2.mkString)
}
}
return stringBuf.mkString
}
return doRLE(l)
}What if you want to restore (to decompress) the string that has been compressed by RLE? Here is the code to restore to original, which means, if the RLE compressed string is:
Get U37p!
It will be restored to:
Get Uppppppppppppppppppppppppppppppppppppp!
Here is the Scala code to do restore to original from compression by RLE
def rleRestore(l:String):String={
var stringBuf:ListBuffer[String]=ListBuffer.empty[String]
var digitArray:ArrayBuffer[Int]=ArrayBuffer[Int]()
stringBuf.clear()
digitArray.clear()
for (i<-l)
{
if (i.isDigit)
{
digitArray += i.toString.toInt
}
else
{
if (!digitArray.isEmpty)
for (_<-0 until digitArray.mkString.toInt)
{
stringBuf += ""+i
}
else
{
stringBuf += ""+i
}
digitArray.clear()
}
}
return stringBuf.mkString
}The above is just snippet, put them together into an Scala singleton object, plus a main method to be self-runnable, store the source code in file tailRecursion.scala
package com.jentekco.scala
//George Jen
//Jen Tek LLC
import scala.collection.mutable.ListBuffer
import scala.collection.mutable.ArrayBufferobject tailRecursion {
//var result=List()//This is to reverse List[Any], reinventing wheel of List().reverse method
def reverseList(l:List[Any]): List[Any]={
var result:List[Any]=List()
//Inner function that is tail recursive
def doReverse(l: List[Any]): List[Any]={
if (l.size<1)
result
else
{
result = l(0) :: result
doReverse(l.drop(1))
}
}
return doReverse(l)
}def runLevelEncode(l: String): String = {
var stringBuf:ListBuffer[String]=ListBuffer.empty[String]
stringBuf.clear()
//Inner helper function that is tail recursion
def doRLE(l: String): String = {
l.toList match {
case Nil=>scala.collection.mutable.ListBuffer[String]()
case h :: t => {
val ms = l.toList.span(_ == h)
if (ms._1.size>1)
{
stringBuf += ms._1.size.toString+ms._1(0)
}
else
{
stringBuf += ""+ms._1(0)
}
doRLE(ms._2.mkString)
}
}
return stringBuf.mkString
}
return doRLE(l)
}
def rleRestore(l:String):String={
var stringBuf:ListBuffer[String]=ListBuffer.empty[String]
var digitArray:ArrayBuffer[Int]=ArrayBuffer[Int]()
stringBuf.clear()
digitArray.clear()
for (i<-l)
{
if (i.isDigit)
{
digitArray += i.toString.toInt
}
else
{
if (!digitArray.isEmpty)
for (_<-0 until digitArray.mkString.toInt)
{
stringBuf += ""+i
}
else
{
stringBuf += ""+i
}
digitArray.clear()
}
}
return stringBuf.mkString
}
def main(args: Array[String]): Unit = {
// Do reverse a list
println("Do List Reverse")
val originalList=List((1,2),3,"abc",3.1416)
val reversedList:List[Any]=reverseList(originalList)
println(s"Original List is: $originalList")
println(s"Reversed List is: $reversedList")
println("")
println("Do String Run Level Encoding")
val x="Get Uppppppppppppppppppppppppppppppppppppp!"
val rleString=runLevelEncode(x)
val originalString=rleRestore(rleString)
println(s"Original String: $x")
println(s"RLE Compressed: $rleString")
println(s"Restored String: $originalString")
}
}
Now the coding is done, before move to unit test by QA, developer would want to test basic sanity:
Compile it:
scalac tailRecursion.scalaThen test compiled tailRecursion.class:
scala com.jentekco.scala.tailRecursionDo List Reverse
Original List is: List((1,2), 3, abc, 3.1416)
Reversed List is: List(3.1416, abc, 3, (1,2))Do String Run Level Encoding
Original String: Get Uppppppppppppppppppppppppppppppppppppp!
RLE Compressed: Get U37p!
Restored String: Get Uppppppppppppppppppppppppppppppppppppp!
QA Unit test by scalatest
After that point, you will need to move to more vigorous tests, that is usually gone through Scala unit tests for all possible scenario to ensure the code developed working in all cases. There is a Scalatest class available for such task: org.scalatest.FunSuite
Notice the above singleton object as fully qualified reverse domain name, that can be imported by scalatest code as:
com.jentekco.scala.tailRecursionHere is the Scatatest unit test code that extends (inherits) from class org.scalatest.FunSuite
For demonstration purpose, following are the test codes to be run under sbt test for automatic unit testing, they are writing for sbt test only.
tailRecursionRLETest.scala
import com.jentekco.scala.tailRecursion
class tailRecursionRLETest extends org.scalatest.FunSuite {
test("tailRecursion.runLevelEncode") {
assert(tailRecursion.runLevelEncode("Get Uppppppppppppppppppppppppppppppppppppp!")=="Get U37p!")
}
}tailRecursionRLERestoreTest.scala
import com.jentekco.scala.tailRecursion
class tailRecursionRLERestoreTest extends org.scalatest.FunSuite {
test("tailRecursion.rleRestore") {
assert(tailRecursion.rleRestore(tailRecursion.runLevelEncode("Get Uppppppppppppppppppppppppppppppppppppp!"))=="Get Uppppppppppppppppppppppppppppppppppppp!")
}
}tailRecursionReverseTest.scala
import com.jentekco.scala.tailRecursion
class tailRecursionReverseTest extends org.scalatest.FunSuite {
test("tailRecursion.reverseList") {
assert(tailRecursion.reverseList(List(1,2,3,4,5)) === List(5,4,3,2,1))
}
}Note:
Each of these test scala file only runs one method inside the object tailRecursion as tailRecursion.<method> by using assert and compare with what the correct result should be. If running the method returns the matching result, that unit test is successful; if not, failed.
Also, each of these test scala file can only run one assert. If you have N number of unit tests, you will need to create N test scala file each with one assert command inside.
How to run these unit test scala file? Read on!
Next step, you need to use sbt to built jar file. As you know, Scala is Java at the end of day, it is to be compiled into Java bytecode and runs on JVM.
Here is the plan:
You are going to use sbt to
Compile
main class code
tailRecursion.scala
test code
tailRecursionRLETest.scala
tailRecursionRLERestoreTest.scala
tailRecursionReverseTest.scala
You are going to use sbt to run your compiled main class code testRecursion compiled jar
You are going to use sbt to unit test
tailRecursionRLETest.scala
tailRecursionRLERestoreTest.scala
tailRecursionReverseTest.scala
To run sbt, you need to create below directory structure:
treeFolder PATH listing for volume New Volume
Volume serial number is 000000A2 5858:2433
E:.
├───project
└───src
├───main
│ └───scala
└───test
└───scala
Files placement:
Under application root directory, place file build.sbt in it, it is important to know your Scala version. You can get version info by:
scala --version
Scala code runner version 2.13.1 -- Copyright 2002-2019, LAMP/EPFL and Lightbend, Inc.My Scala version is 2.13.1, use it to specify the scalaVersion parameter in build.sbt
you can open a text editor, I use
notepad build.sbt
lazy val root = (project in file(".")).
settings(
inThisBuild(List(
organization := "com.jentekco",
scalaVersion := "2.13.1"
)),
name := "scalatest-example"
)
libraryDependencies += "org.scalatest" %% "scalatest" % "3.1.0" % TestUnder the project subfolder, place build.properties file inside, the file has one line:
sbt.version=1.3.10Under src\main\scala, place main code tailRecursion.scala inside
Under src/test/scala, place the 3 test scala files inside
tailRecursionRLETest.scala
tailRecursionRLERestoreTest.scala
tailRecursionReverseTest.scala
Following are sequence of commands by sbt
Deletes all generated files (in the target directory).
sbt clean
sbt clean
[info] [launcher] getting org.scala-sbt sbt 1.3.10 (this may take some time)...
downloading https://repo1.maven.org/maven2/org/scala-sbt/sbt/1.3.10/sbt-1.3.10.jar ...
:: loading settings :: url = jar:file:/C:/Program%20Files%20(x86)/sbt/bin/sbt-launch.jar!/org/apache/ivy/core/settings/ivysettings.xml
…
file:/E:/george%20ML%20stuff/teaching/writing/scala_coding/for_sbt/)
[success] Total time: 1 s, completed Jun 26, 2020 7:20:07 PMCompiles the main sources (in src/main/scala and src/main/java directories).
sbt compile
E:\george ML stuff\teaching\writing\scala_coding\for_sbt>sbt compile
[info] Loading project definition from E:\george ML stuff\teaching\writing\scala_coding\for_sbt\project
[info] Loading settings for project root from build.sbt ...
….
[info] Compilation completed in 49.015s.
[success] Total time: 65 s (01:05), completed Jun 26, 2020 7:22:41 PMRuns the main class for the project in the same virtual machine as sbt.
sbt run
E:\george ML stuff\teaching\writing\scala_coding\for_sbt>sbt run
[info] Loading project definition from E:\george ML stuff\teaching\writing\scala_coding\for_sbt\project
[info] Loading settings for project root from build.sbt ...
[info] Set current project to scalatest-example (in build file:/E:/george%20ML%20stuff/teaching/writing/scala_coding/for_sbt/)
[info] running com.jentekco.scala.tailRecursionDo List Reverse
Original List is: List((1,2), 3, abc, 3.1416)
Reversed List is: List(3.1416, abc, 3, (1,2))Do String Run Level Encoding
Original String: Get Uppppppppppppppppppppppppppppppppppppp!
RLE Compressed: Get U37p!
Restored String: Get Uppppppppppppppppppppppppppppppppppppp!
[success] Total time: 2 s, completed Jun 26, 2020 7:25:47 PM
Compiles and runs all tests.
sbt test
E:\george ML stuff\teaching\writing\scala_coding\for_sbt>sbt test
[info] Loading project definition from E:\george ML stuff\teaching\writing\scala_coding\for_sbt\project
[info] Loading settings for project root from build.sbt ...
[info] Set current project to scalatest-example (in build file:/E:/george%20ML%20stuff/teaching/writing/scala_coding/for_sbt/)
[info] Compiling 3 Scala sources to E:\george ML stuff\teaching\writing\scala_coding\for_sbt\target\scala-2.13\test-classes ...
[warn] there were three deprecation warnings (since 3.1.0); re-run with -deprecation for details
[warn] one warning found
[info] tailRecursionRLERestoreTest:
[info] tailRecursionReverseTest:
[info] - tailRecursion.reverseList
[info] - tailRecursion.rleRestore
[info] tailRecursionRLETest:
[info] - tailRecursion.runLevelEncode
[info] Run completed in 1 second, 43 milliseconds.
[info] Total number of tests run: 3
[info] Suites: completed 3, aborted 0
[info] Tests: succeeded 3, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 12 s, completed Jun 26, 2020 7:26:40 PMIf you go from sbt build directory to the target subfolder and then scala* subfolder, you will see the jar file for the tailRecursion.scala has been generated
Directory of E:\george ML stuff\teaching\writing\scala_coding\for_sbt\target\scala-2.1306/26/2020 07:26 PM <DIR> .
06/26/2020 07:26 PM <DIR> ..
06/26/2020 07:22 PM <DIR> classes
06/26/2020 07:25 PM 5,683 scalatest-example_2.13-0.1.0-SNAPSHOT.jar
06/26/2020 07:26 PM <DIR> test-classes
06/26/2020 07:21 PM <DIR> update
You can run the jar file using scala runtime
E:\george ML stuff\teaching\writing\scala_coding\for_sbt\target\scala-2.13>scala scalatest-example_2.13-0.1.0-SNAPSHOT.jar
Do List Reverse
Original List is: List((1,2), 3, abc, 3.1416)
Reversed List is: List(3.1416, abc, 3, (1,2))Do String Run Level Encoding
Original String: Get Uppppppppppppppppppppppppppppppppppppp!
RLE Compressed: Get U37p!
Restored String: Get Uppppppppppppppppppppppppppppppppppppp!
By the way, all 3 test scala files have been compiled into Java bytecode
Directory of E:\george ML stuff\teaching\writing\scala_coding\for_sbt\target\scala-2.13\test-classes06/26/2020 07:26 PM <DIR> .
06/26/2020 07:26 PM <DIR> ..
06/26/2020 07:26 PM 4,314 tailRecursionReverseTest.class
06/26/2020 07:26 PM 3,702 tailRecursionRLERestoreTest.class
06/26/2020 07:26 PM 3,663 tailRecursionRLETest.class
As always, the code used in this writing is in my GitHub repo:
https://github.com/geyungjen/jentekllc/tree/master/Spark/Scala/tailRecursion
This concludes the tutorial, thank you for your time viewing.