Debugging and Testing Support

Debugger:

A debugger is a program or tool that is used to examine the state of a program while the program is running. The debugger is very useful to find bugs as it allows a programmer to run the program line by line and determine which code is generating the bug.

The process of using the debugger and removing the potential error is “Debugging”.

Like many other programming languages, Erlang also provides a tool for debugging. We can set breakpoints and run our program step by step and examine variables at that state of the program. In Erlang, the debugger is a graphical interface for the interpreter.

Erlang allows the programmer to debug from the Erlang shell itself

Steps to Debug in Erlang:

Step1: First of all you need to compile the module that you wish to debug with the debug_info flag.

In the above example, test1 is the name of the module

Step:2 Type “debugger: start().” in the Erlang shell to start the debugger.

This will pop up a Monitor window. Initially, there are no modules selected, you can browse through the Module-> Interpret and select the module that needs the debugging.

Step:3 Go to Module -> Name of your module -> view

This will allow you to view the source code and set the breakpoints.

Step 4: To set the breakpoints select the module in the left pane of the Monitor window and then goto Break->Select the desired type of break, it will pop another window as below

Enter the line number you wish to set the breakpoints in and then press OK.

Step 5: Run the program that needs debugging as usual from the Erlang Shell.

Step6: You can now select the process from the right pane of the Monitor window and attach the process. To do so, you can just double click the pid and window as below will pop up, where you can step through the the program and control the execution, inspect the variable values and much more.

Breakpoints:

In Erlang debugger, once the breakpoint is set as mentioned above, it can be either active or it can be Inactive. You can Enable(Make active) or Disable( Make Inactive) the breakpoints from the Break menu in the monitor window. Disabled/Inactive breakpoints are ignored by the debugger. You can easily distinguish between active and inactive breakpoints by looking at the color. In my case, active breakpoints were “RED” in color and inactive breakpoints were “Blue ” in color.

Moreover, There are three types of breakpoints you can choose from based on the requirements of the code flow.

1. Line Break
2. Conditional Break
3. Functional Break

Line Break:

A line break is a normal breakpoint that is created at a certain line in the module. The execution stops when the program reaches that line.

Conditional Break:

In a conditional break, a breakpoint is created at a certain line in the module, but the execution will stop only when the condition becomes true. If the condition becomes false, the debugger ignores the breakpoint.

Functional Break:

A functional break is just a set of line breaks, which is placed at the start of each clause in a specific function.

Erlang debugger also provides the stack trace which is useful to trace back in case of errors.

Performance wise, execution of interpreted code is slower than regularly compiled code. Moreover, the debugger creates another process for each debugged process which makes the number of processes twice as much.

Erlang debugger also provides the feature to start the debugger globally or locally. By default, the debugger starts in global mode. If you say “debugger:start(local).”, then the debugger will start in local mode. In global mode, the code is interpreted in all known nodes, whereas in local mode the code is interpreted only in the current node.

Here is the video link of how to debug: https://www.youtube.com/embed/f-sVhuHX3Xk

Testing Support:

Mainly in Erlang we use Common Test Framework for testing. It can be used in two ways:

1. For black-box testing (function and system testing) of target nodes of any type it is used as a portable test server.
2. For white-box testing of OTP applications and Erlang programs it is used as a practical tool.

Regression Testing:

1. Test specification is flexible.
2. The result logs and progress of test are printed to file on HTML format.
3. Support is provided for running multiple independent test sessions in parallel.
4. Automated execution is also provided for test suite programs.

Common Test Implementation Structure:

Supporting libraries are:

1. Erlang/OTP libraries
2. Specific test object libraries (test ports)
3. Common Test support libraries for general protocols (Eg: FTP, SNMP)

Test Case Execution:

The Test Suite:

It is callback module which must comply with a defined test server interface. For Example, a test suite must export a function all/0 that returns a list of all test cases in the module. Also, the header file “ct.hrl” must be included in all test suite files.

Test Suite Callback Module:

Test Case Function:

In the list returned, for each test case from all/0, the test server calls a function with same name and one argument:

TestCaseName(config) — config is the runtime configuration data.

A test case is successful if it returns to the caller and is unsuccessful if it crashes or exits on purpose.

Configuration Files:

It is used to describe data related to system under test or a test. It is possible to change properties without having to modify test suites by test configuration data. For example: identities, program names which need to be executed by test, addresses of the test plan.

A configuration file can contain any number of elements on the form:

{Key,Value}

where Key = atom() , Value = term() | [{Key,Value}]

(Key is the name of the configuration variable).

Configuration File Example:

Example of how to access configuration data inside a test case:

FtpAddr = ct:get_config({ftp,host}),

URL = ct:get_config(url), …

Printouts from test cases:

CT provides the following functions for printing information from a test case:

1. ct:comment/1 % print string in comment field in HTML log file
2. ct:log/[1,2,3] % print to test case log file
3. ct:print/[1,2,3] % print to console
4. ct:pal/[1,2,3] % print both to log and console

Common Test Modules:

Common Test consists of the following interface modules:

1. ct — main user interface for the framework
2. ct_master — support for large scale testing
3. ct_ftp — CT interface to FTP client
4. ct_rpc — CT interface to Erlang/OTP RPC
5. ct_telnet — CT interface to Telnet client
6. unix_telnet — ct_telnet callback for Unix host
7. ct_snmp — CT interface to Erlang/OTP SNMP
8. ct_ssh — CT interface to Erlang/OTP SSH/SFTP

The ct module is providing the main interface for writing test cases which includes:

• Functions for executing test cases.
• Functions for printing & logging.
• Functions for reading configuration data.

Test Execution:

The script run_test can be used for starting tests from a Unix command line:

$ run_test -config <configfilenames> -dir <dirs>

Example: $ run_test –config $CFGDIR/node.cfg –dir $TESTDIR/objX_test

Examples of other useful run_test flags:

1. -logdir , specifies where the HTML log files are to be written.
2. -stylesheet, for installing a CSS file.
3. -refresh_logs, refreshes the top level HTML index files.
4. -cover, for performing code coverage tests.
5. -silent_connections [conn_types], tells CT to suppress printouts for specified connections (e.g. telnet).
6. -include, to add include directories for test suite compilation.

Running from an Erlang shell prompt:

ct: run_test(Opts) — which takes the same input options as the run_test script but as tuples in list. Example:

ct:run_test([{logdir,”/ldisk/logdir”}, {dir,”my_test_obj”}]).

Unit Testing Framework for Erlang:

EUnit is a unit testing framework for Erlang. It is flexible, easy to use, powerful, with little syntactical overhead.EUnit uses techniques more used in functional and concurrent programming and is typically less verbose.

Also, EUnit uses many preprocessor macros, they have been designed to be as non-intefering as possible, and should not cause any conflicts with existing code,

which means adding EUnit tests to a module would not normally require changing the existing code.

Unit Testing: It is a level of software testing in which a program unit is tested. A program unit can be anything from a function, a process, a module.

In order to test a unit, we specify a set of individual tests, set up the smallest necessary environment in order to run the tests, we run the tests and collect the results, and at last, do necessary cleanup so that the test can be run again later.

A Unit Testing Framework helps to perform all the above-required steps of each stage and makes the process of writing the test, running the test and debugging and fixing the failed tests.

Advantages of Unit Testing:

1. It reduces the risk of changing the program: Every change to a working program is a risk of introducing new bugs — or reintroducing bugs that had previously been fixed.

Having a set of unit tests that we can run with very little effort makes it easy to know that the code still works the same as it is expected to work.

2. Speed up the development process: By focusing on getting the code to pass the test cases help the programmer to be effective and write code which is correct from the beginning.

3. Makes component integration easier: Component integration becomes easier as having the confidence on the smaller component built having passed the test cases help improve the confidence on building other higher level component with includes the smaller tested unit.

4. Separates interface from implementation: While writing test cases we come across some dependencies which are not required, this helps remove bad dependencies.

5. Self-documenting: Test can be documented and read, showing both examples of correct and incorrect usage along with their outputs.

Getting started with EUnit:

Including the EUnit header file: In order to use EUnit in an Erlang module we need to add the below line at the start of the module after the module declaration, but before any function definitions.

-include_lib(“eunit/include/eunit.hrl”)

Following will be the effect on adding the above line:

a) Creates an exported function test() that can be used to run all the unit tests defined on the module.

b) Causes all functions whose names match …_test() or …_test_() to get exported automatically from the module.

c) Makes all the macros of the EUnit available for writing the tests.

Writing simple test functions:

The EUnit framework makes it extremely easy to write unit tests in Erlang. A function name prefixed by _test() is recognized as a simple test function by EUnit — it takes no arguments, and its execution ends up either in success or failure by throwing an exception of some kind.

Example of simple test function:

reverse_test() -> lists:reverse([6,7,8]).

Use exceptions to signal failure: In order to write more interesting tests, we need to make them crash (throw an exception) when we don’t get the expected result.Below are some examples:

reverse_nil_test() -> [] = lists:reverse([]).

reverse_one_test() -> [6] = lists:reverse([6]).

reverse_two_test() -> [6,7] = lists:reverse([7,6]).

Here the last test above would throw a bad match error. But the first two will return [] and [6] respectively and hence succeed. If we write a test that returns a value, even if it is the incorrect value, EUnit will consider it as a success.

We must make sure that the test is written in such a way that it causes a crash if the result is not as expected.

Using assert macros: If we are using Boolean operators in our test then the assert macro is useful.

length_test() -> ?assert(length([6,7,8]) =:= 3).

Here the assert macro will evaluate the expression if it evaluates to true then it will show ok, but if it evaluates to true then it will throw an exception.

Running EUnit:

If we have already added the declaration -include_lib(“eunit/include/eunit.hrl”) to our module, as shown above, we only need to compile the module and run the automatically exported function test().

For example, if we have a module named abc, then calling abc:test() will run the EUnit test on all the tests defined in the module. No need to write -export declaration for the test function.

Keeping tests in separate modules:

We can keep our test codes separate from the normal code. We can write all the test functions in the module named abc_tests. Whenever EUnit runs the test on module abc it will look for the module abc_tests and run all the tests.

Writing test generating functions:

The disadvantage of the test function is that we need to write a separate function for each test cases. The more wiser way of writing tests is to write functions that return tests.

A function with a name ending with …_test_() is recognized by EUnit as a test generator function. Test generators return a set of tests to be executed by EUnit.

basic_test_() ->

fun () -> ?assert(2 + 1 =:= 3) end

Below test will have same effect as above:

simple_test() ->

?assert(2 + 1 =:= 3).

Using macros to write tests:

In order to make the tests more readable and compact and add useful information automatically to the source code like it will add information about line numbers where a test occurred.

We can use _test macro as shown below:

basic_test_() ->

?_test(?assert(2 + 1 =:= 3))

The _test macro takes any expression as an argument . The body of the _test macro will be like any other simple test function.

An Example to show how Eunit can be used:

-module(fib).

-export([fib/1]).

-include_lib(“eunit/include/eunit.hrl”).

fib(0) -> 1;

fib(1) -> 1;

fib(N) when N > 1 -> fib(N-1) + fib(N-2).

fib_test_() ->

[?_assert(fib(0) =:= 1),

?_assert(fib(1) =:= 1),

?_assert(fib(2) =:= 2),

?_assert(fib(3) =:= 3),

?_assert(fib(4) =:= 5),

?_assert(fib(5) =:= 8),

?_assertException(error, function_clause, fib(-1)),

?_assert(fib(31) =:= 2178309)

].

Disable Testing:

Testing can be disabled by defining NOTEST macro as an option to erlc while compiling:

erlc -DNOTEST my_prgm.erl

Also, it can be disabled by adding below macro definition before Eunit header file is included:

define(NOTEST, 1)

Authors of Group 13:

Prativa Devkota

Neha Singh

Ashna Shah

Aarya joshi

Nivedita Vattipalli