Executing Generated Tests

Introduction

Once Diffblue Cover has generated tests for your project there are still a few steps to integrate them back into your own test-suite. These break down into two main tasks which are dependent on your build system and how many of the generated test cases from Diffblue Cover you wish to integrate.

The dependancies for executing the tests generated by Diffblue Cover are

  • Powermock version 1.6.5

  • Mockito 1.10.19

  • Diffblue Deeptestutils 1.8.1 (or newer)

  • JUnit 4.7 (or newer)

1. Getting tests

There are three possible methods for getting generated tests from the Diffblue Cover service:

  1. Copy the body of the individual test using the Copy button inside the test detail view for a given test. This is great for situations where you want to integrate a test case into an existing test file for the given .java file.

  2. Download a test file using the Download Java File button inside the test detail view for a given test. Useful when you want to integrate one test quickly.

  3. Download all of the generated test files using the Download All Tests button on the test list view. Useful for seeing the whole increase in test coverage that Diffblue has generated.

a. Using a copied test

Once a test has been generated, and you have selected the relevant test from the test list you can click Copy in the test detail view to copy the contents of the test to your clipboard.

In this example we will take a test from the java-demo project:

/*
 * Test generated by Diffblue Deeptest.
 * This test covers `int checkTicTacToePosition(int [])' block 1 (line 20)
 * This test covers `int checkTicTacToePosition(int [])' block 6 (line 25)
 * This test covers `int checkTicTacToePosition(int [])' block 7 (line 26)
 * This test covers `int checkTicTacToePosition(int [])' block 8 (line 26)
 * This test covers `int checkTicTacToePosition(int [])' block 9 (line 37)
 * This test covers `int checkTicTacToePosition(int [])' block 10 (line 27)
 * This test covers `int checkTicTacToePosition(int [])' block 11 (line 28)
 * This test covers `int checkTicTacToePosition(int [])' block 12 (line 28)
 * This test covers `int checkTicTacToePosition(int [])' block 13 (line 29)
 * This test covers `int checkTicTacToePosition(int [])' block 14 (line 30)
 * This test covers `int checkTicTacToePosition(int [])' block 15 (line 30)
 * This test covers `int checkTicTacToePosition(int [])' block 16 (line 31)
 * This test covers `int checkTicTacToePosition(int [])' block 20 (line 26)
 * This test covers `int checkTicTacToePosition(int [])' block 21 (line 26)
 * This test covers `int checkTicTacToePosition(int [])' block 22 (line 37)
 * This test covers `int checkTicTacToePosition(int [])' block 23 (line 37)
 * This test covers `int checkTicTacToePosition(int [])' block 27 (line 44)
 * This test covers `int checkTicTacToePosition(int [])' block 28 (line 44)
 * This test covers `int checkTicTacToePosition(int [])' block 30 (line 45)
 * This test covers `int checkTicTacToePosition(int [])' block 31 (line 45)
 * This test covers `int checkTicTacToePosition(int [])' block 36 (line 53)
 * This test covers `int checkTicTacToePosition(int [])' block 37 (line 53)
 * This test covers `int checkTicTacToePosition(int [])' block 38 (line 54)
 * This test covers `int checkTicTacToePosition(int [])' block 39 (line 55)
 * This test covers `int checkTicTacToePosition(int [])' block 42 (line 44)
 * This test covers `int checkTicTacToePosition(int [])' block 43 (line 44)
 */
@org.junit.Test
public void com_diffblue_javademo_TicTacToe_checkTicTacToePosition_004_6c06ae82d9098b40() throws Throwable {
  int retval;
  {
    /* Arrange */
    com.diffblue.javademo.TicTacToe param_3 = (com.diffblue.javademo.TicTacToe) Reflector.getInstance("com.diffblue.javademo.TicTacToe");
    int[] board = { 2, 2, 1, 0, 2, 2, 1, 1, 1 };
    /* Act */
    retval = param_3.checkTicTacToePosition(board);
  }
  {
    /* Assert result */
    Assert.assertEquals(1, retval);
  }
}

You could now take this and integrate it into an existing test file, in this example it would be src/test/java/com/diffblue/javademo/TicTacToeTest.java. Simply paste the test into your existing file and ensure that the following dependencies have been imported in the test file:

import org.junit.Test;
import org.junit.Assert;
import java.lang.reflect.Array;
import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import org.junit.rules.ExpectedException;
import com.diffblue.deeptestutils.CompareWithFieldList;
import com.diffblue.deeptestutils.FieldList;
import com.diffblue.deeptestutils.IterAnswer;
import com.diffblue.deeptestutils.Reflector;

Please merge as is relevant to your project

You can now move to step 2 below.

b. Using a downloaded test file

Once a test has been generated you can click on view test to see it in detail. You can then click Download Java File in the test detail view to get a file that can be integrated to your test suite.

In this example we will take a test from the java-demo project:

TicTacToeTest.java

package com.diffblue.javademo;

import org.junit.Assert;
import org.junit.rules.ExpectedException;
import org.junit.Test;

import java.lang.reflect.Array;
import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;

import com.diffblue.deeptestutils.CompareWithFieldList;
import com.diffblue.deeptestutils.FieldList;
import com.diffblue.deeptestutils.IterAnswer;
import com.diffblue.deeptestutils.Reflector;

public class TicTacToeTest {

  @org.junit.Test
  public void com_diffblue_javademo_TicTacToe_checkTicTacToePosition_004_6c06ae82d9098b40() throws Throwable {
    int retval;
    {
      /* Arrange */
      com.diffblue.javademo.TicTacToe param_3 = (com.diffblue.javademo.TicTacToe) Reflector.getInstance("com.diffblue.javademo.TicTacToe");
      int[] board = { 2, 2, 1, 0, 2, 2, 1, 1, 1 };
      /* Act */
      retval = param_3.checkTicTacToePosition(board);
    }
    {
      /* Assert result */
      Assert.assertEquals(1, retval);
    }
  }
}

You could now add this file to the relevant path in my test suite, in this case it would be src/test/java/com/diffblue/javademo/TicTacToeTest.java.

You can now move to step 2 below.

c. Downloading all tests

On the test list for your project you will find the Download All Tests button which will provide you with a tar archive of all tests required for the next step.

Once downloaded extract into the relevant path for your project. In the example of java-demo you would merge the contents of javademo/test from the archive into the src/test folder of the project.

2. Adding Diffblue Deeptestutils as a dependency

In all tests produced by Diffblue Cover there are a number of dependencies that are provided by the deeptestutilsdependency which is stored in the Maven Central Repository. In order to integrate this you need to follow the guide relevant to your projects build tool, here we provide Maven and Gradle.

Note: if you are unable to access the Maven Central Repository, you can use the generated test cases offline, see the section Offline Diffblue Cover.

Deeptestutils in Maven projects

To add the Diffblue Cover dependencies to your local Maven repository do the following:

Add the following dependency information to the dependencies section in the root pom.xml.

<dependency>
  <groupId>org.powermock</groupId>
  <artifactId>powermock-api-mockito</artifactId>
  <version>1.6.5</version>
</dependency>
<dependency>
  <groupId>org.powermock</groupId>
  <artifactId>powermock-module-junit4</artifactId>
  <version>1.6.5</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.mockito</groupId>
  <artifactId>mockito-all</artifactId>
  <version>1.10.19</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>com.diffblue</groupId>
  <artifactId>deeptestutils</artifactId>
  <version>1.8.1</version>
</dependency>

Deeptestutils in Gradle projects

To add the Diffblue Cover dependencies to a Gradle based project do the following:

  1. Inside your build.gradle file: add/merge the following into the repositories section (or create if necessary)

     repositories {
         mavenCentral()
     }
  2. Inside your build.gradle file: add/merge the following into the dependencies section (or create if necessary)

     dependencies {
         testCompile group: 'junit', name: 'junit', version: '4.8.2'
         testCompile group: 'com.diffblue, name: 'deeptestutils', version: '1.8.1'
     }

Java versions

By default the generated tests work with Java 8. If you are using an older version of Java you must add the version to the CBMC Arguments in the project settings using the following format:

java-version: 6

3. Final steps

At this point the tests are integrated into your suite and will work alongside any tests you may already have had by running your build systems test command.