Reporting
After running a test, the Test Runner will provide a detailed output of the tests you have run. You can also export a report of the test run.
Automation supports exporting test results into CSV or PDF format.
PDF reports have options for the following information:
- Screenshots
- Salesforce IDs
- Summary information
- Test summary based on Tags added to the tests
- Logging levels for output messages
To export a report, choose the 
An Export Test Run is displayed. In this screen you can specify the level of detail you would like to see in the PDF report, or select the CSV option if you would like to import the report into an excel file.
Some additional configuration options are also available:
- Test Case Path Hierarchy: Configure whether to show or hide the full path hierarchy. (This checkbox is selected by default to show the test case path hierarchy).
- Show screenshot thumbnails: Configure whether to include screenshot thumbnails.
- Show full screenshot: Configure whether to include full-size screenshots.
Note: You can also include Skipped Steps in the report if selected and exclude the skipped steps if unchecked.
- The Skipped Steps option is displayed as a checkbox.
- The Skipped Steps checkbox is unchecked by default.
The Skipped Steps option is only applicable to the PDF reporting option in Provar as given below:
- Export Test results in Test Runner
- Run under ANT
- Test Plan PDF
If Test Case Path Hierarchy is enabled, the run report will include the test case folders:
This is the default option. If instead it is disabled, the run report will display only the test case names:
If you click Show screenshot thumbnails (this is the default option), the run report will show the thumbnails.
If instead you click Show full screenshot, the run report will show larger captures.
The default location for the report will be your project folder. To see the report file in the Navigator, right click on the project and choose Refresh.
Customizing your report
This feature allows you to generate customized reports in CSV and PDF.
In order to use this feature, please do the the following:
- Create a folder named reports inside the src folder of the test project.
- Access the TestRunMonitorDemo.java information. The code is listed below for reference.
- Add the .java file to the new reports folder.
package reports;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import com.provar.core.testapi.ITestExecutionContext;
import com.provar.core.testapi.annotations.TestExecutionContext;
import com.provar.core.testapi.annotations.TestRunExporter;
import com.provar.testrunner.AbstractTestRunMonitor;
import com.provar.testrunner.ITestRunExporter;
import com.provar.testrunner.api.IExecutionItem;
import com.provar.testrunner.exporters.TestCaseReportingOptions;
public class TestRunMonitorDemo extends AbstractTestRunMonitor {
@TestRunExporter
private ITestRunExporter pdfExporter;
@TestRunExporter(mimetype="application/csv")
private ITestRunExporter csvExporter;
@TestExecutionContext
private ITestExecutionContext testExec ;
String pdfPath;
FileWriter fileWriter = null;
String fileName = "ProvarTestRunExport";
static int i = 1;
TestCaseReportingOptions testCaseReportingOptions = new TestCaseReportingOptions(true, false, true, true, true, true, true, false, true);
File reportsDir ;
public void initialize() {
reportsDir = new File(testExec.getProjectPath() + File.separator + "Reports");
if (!reportsDir.exists()) {
reportsDir.mkdirs();
}
pdfPath = this.testExec.getProjectPath();
try {
fileWriter = new FileWriter(reportsDir.getAbsoluteFile() + "\\" + fileName + ".csv", false);
fileWriter.write("run id, started, ended, path, name, successful, skipped, failed, total, failure Message\n");
fileWriter.close();
} catch (IOException e) {
e.printStackTrace();
}
}
@Override
public void writeTestCaseExecution(IExecutionItem executionItem) {
try {
pdfExporter.exportExecutionItem(getRuntimeConfiguration(), pdfPath, executionItem, getTestArtifactsPath(), testCaseReportingOptions, null, true, "pdf",null, null, testExec.getRunId());
} catch (Exception e) {
e.printStackTrace();
}
try {
fileWriter = new FileWriter(reportsDir.getAbsoluteFile() + "\\" + fileName + ".csv", true);
csvExporter.exportExecutionItem(null, null, executionItem, null, null, null, false, "csv" , fileWriter, ",", testExec.getRunId());
} catch (Exception e) {
e.printStackTrace();
}
}
@Override
public void writeTestRunSummary(IExecutionItem rootExecutionItem) {
try {
pdfExporter.exportExecutionItem(getRuntimeConfiguration(), (reportsDir.getAbsoluteFile() + "\\" + "Summary.pdf"), rootExecutionItem, getTestArtifactsPath(), testCaseReportingOptions, null, false, "pdf",fileWriter, ",", testExec.getRunId());
} catch (Exception e) {
e.printStackTrace();
}
try {
fileWriter = new FileWriter(reportsDir.getAbsoluteFile() + "\\" + fileName + ".csv", true);
csvExporter.exportExecutionSummary(rootExecutionItem, fileWriter, ",");
} catch (Exception e) {
e.printStackTrace();
}
}
}
- Start time
- End time
- Test location path
- Test case name
- Test status (successful, skipped, failed)
- Any environment variables which have been defined
Report location
The name and path where the report should be generated could be changed to a different path inside or outside the test project.
Data writing
Whether the data should be written at the beginning or should be appended to the csv file after each test run.
For more information, check out this course on University of Provar.