Karate API automation | Cucumber Reporting | Parallel execution | LogBack

Hi everyone!

In the previous post I have explained assertions in Karate API automation. In today’s post I would be explaining how to integrate cucumber reporting in Karate API framework and run scripts in parallel.

When we run TestRunner file, all the features under classpath would run and feature wise reports would be generated and placed under target/surefire-reports.

JUnit reports

Now, to integrate cucumber reporting, first we need to add cucumber maven dependency in pom.xml file and save.

<dependency>
         <groupId>net.masterthought</groupId>
         <artifactId>cucumber-reporting</artifactId>
         <version>3.8.0</version>
</dependency>

3.8.0 is the latest cucumber reporting version. You may update as and when required.

If you remember, in our previous post, in TestRunner class we have used @RunWith(Karate.class) annotation. Now, to run features and scenarios in parallel, we will remove this annotation as it will be a simple JUnit class. Paste the following code in TestRunner class and save it.

public class TestRunner {

    @Test
    public void testParallel() {
        Results results = Runner.parallel(getClass(), 5);
        generateReport(results.getReportDir());
        assertTrue(results.getErrorMessages(), results.getFailCount() == 0);
    }

    public static void generateReport(String karateOutputPath) {
        Collection<File> jsonFiles = FileUtils.listFiles(new File(karateOutputPath), new String[] { "json" }, true);
        List<String> jsonPaths = new ArrayList<String>(jsonFiles.size());
        jsonFiles.forEach(file -> jsonPaths.add(file.getAbsolutePath()));
        Configuration config = new Configuration(new File("target"), "demo");
        ReportBuilder reportBuilder = new ReportBuilder(jsonPaths, config);
        reportBuilder.generateReports();
    }

}

Here, in, Results results = Runner.parallel(getClass(), 5);

5 represent the number of threads in which script/features will run in parallel. Also, the scenarios under each feature will run in parallel.

So, if you don’t want to run any scenario in parallel you can add a tag as “@parallel=false” before the scenario and if you don’t want to run a feature file in parallel, you should put this tag above feature keyword.

However, as per Peter Thomas, “forcing Scenario-s to run in a particular sequence is an anti-pattern, and should be avoided as far as possible.”

@parallel=false
Scenario: Get Token details of Super Admin login

    Given url url+'/cas/oauth/check_token'
    And def call1 = call read('login.feature@tag1')
    And param token = call1.response.access_token
    When method GET
    Then status 200
    Then response.code == 1000

TestRunner class for cucumber reporting and parallel execution

Right click on the TestRunner class file and run as JUnit Test.

Script running in parallel with Thread count as 5

All the features under classpath would run in parallel and reports would be created under target/cucumber-html-reports.

Refresh the target folder and open overview-features.html report in any browser to view the cucumber report.

Cucumber html report

If we click on any feature link from feature column we would be redirected to feature report. We can expand the background, scenario and step section and the report will look something like this.

Detailed feature report

Now, if you observe you can see lots of verbose logging in console and the actual output whether the features/scenarios are passed/ failed are skipped.

To avoid that, we can create logback-test.xml under classpath.

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
 
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>
  
    <appender name="FILE" class="ch.qos.logback.core.FileAppender">
        <file>target/karate.log</file>
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>    
   
    <logger name="com.intuit.karate" level="DEBUG"/>
   
    <root level="info">
        <!-- <<appender-ref ref="STDOUT" /> -->
        <appender-ref ref="FILE" />
    </root>
  
</configuration>

logback-test.xml

Now, run the TestRunner and observe that you would not find all the verbose logs in console which you were getting before and rather it would be saved in a file karate.log under target folder.

karate.log

Thanks for reading!