Initiating a test run
To execute a test case, use the Run, Debug or Test Builder icons on the Menubar. These represent the different available run modes.
See below for more information on these different run modes.
Alternatively, to run a group of tests, go to the Navigator view and right-click on one or more tests cases or test folders.
Monitoring the test run
Once a test run is in progress, a new tab will open in the Test Runner view to allow you to follow its progress.
In Run and Debug modes, the test run will end automatically once it has finished running its tests. You can also stop a test run early by clicking the Abort icon ().
Once the test run has ended (whether passed or failed) you can generate a test run report using the Export Test Results icon () in the top-right of the Test Runner. Refer to Reporting for more information.
Stop test run on error
As of Provar version 1.8.11, you can also have the test run abort automatically if any test failure is encountered. This is useful if you are running dependent test cases where if one fails there is no need to run the subsequent ones. You can enable this by clicking the Stop Test Run on Error () icon in the Test Runner.
Note that this is different from Error Handling on an individual test case, which allows you to skip or continue the subsequent steps of a test case if an error is encountered. Refer to error handling below for more information on this option.
As of Provar 1.9, there is a new option in Test Runner to allow re-running a specific set of test cases based on a given test run. This means that, when test failures are encountered, they can be re-run locally without needing to set up test cycles first. This is especially useful for locally testing a change to a callable test, connection, environment or browser setting.
Once a test run has been executed, you can re-run it by navigating to the relevant test run and clicking the button Re-Run the test run.
This presents the following options:
- Rerun all: Runs all the test cases from the original test run again. The same Run Mode will be used as the original run (e.g. Debug, Test Builder).
- Rerun failures (Debug Mode): Runs only the failed test cases from the original test run again. Launches in Debug mode if the original run was in Debug or Run, or in Test Builder if it was Test Builder. In both cases the Variables view will be populated to help troubleshoot the test failures.
Note that the Rerun failures option is only available if there were failures in the original test run.
Use this mode for the official execution required for signing off a release. It can also be executed at the command line for Continuous Integration support.
Run Mode supports multiple browsers and execution will not be interfered with by any Chrome plug-ins or breakpoints. Run Mode does not capture variables, so it not recommended for debugging.
You can change the browser to run in by using the Web Browser dropdown on the Menubar.
Debug Mode can be used for debugging issues with a specific browser. Breakpoints are enabled and variables are captured for debugging. As with Run Mode, individual test cases or full test folders can be run, and the Web Browser can be specified from the Menubar.
Use Test Builder mode for building and debugging UI tests with the Test Builder. If you run a test case in Test Builder, it will execute all test steps and pause after executing the last step to allow you to add more steps. To end the test run, click the stop icon at the top of the Test Builder ().
This mode should not be used for final execution sign-off, but is useful for:
- Creating UI test steps and field locations for test authoring
- Debugging using breakpoints and forwards and backwards stepping
- Capturing variables which are visible during and at the end of the test execution
- Creating Page Objects
Refer to Debugging Tests for more information on the Test Runner.
Run under ANT
Run Under ANT mode allows you to run your test cases under ANT from within Provar. It is also useful for creating your Build.xml file to run your tests under ANT on a different machine, to save creating the Build.xml file manually. Refer to Apache Ant: Generating a Build File for more information.
Headless testing in Chrome
As of Provar version 1.9.8, you can run tests headlessly in Chrome by selecting this in the Web Browser dropdown in Provar Desktop. This was previously available in Provar by configuring an Environment Variable.
Selecting this option will cause Run and Debug modes to run tests without launching the browser. Test Builder mode is unaffected by this option and will always launch a Chrome browser to enable test building.
Chrome headless in build files
The Chrome Headless option can also be selected when generating a build file using Run under ANT:
It appears in the Build.XML as follows:
Chrome headless in _chromeOptions
Previously Provar supported running tests headlessly by configuring a _chromeOptions environment variable. This option is still supported and settings defined under a _chromeOptions variable will continue to work in the same manner as before.
If a situation arises where there is a conflict between the Web Browser setting and the setting defined under _chromeOptions, preference will be given to the _chromeOptions setting.
This means that, if headless is enabled in _chromeOptions but Web Browser is set to standard Chrome (non-headless), Provar will still run Chrome in headless.
In each test case you can specify what Provar should do in the event of an error. In the Skip on Error dropdown, you can select one of three options:
- Skip: This will stop the test Case on failure and skip to the next Test Case to be executed
- Continue: This will continue to the next Test Step. This is useful if you would like to view all the errors
- Inherit: This will have the Test Case inherit its behavior from the Calling Test (applies to Callable Tests only).
- Provar Automation
- Installing Provar Automation
- Updating Provar Automation
- Using Provar Automation
- API Testing
- Behavior-Driven Development
- Creating and Importing Projects
- Creating Test Cases
- Custom Table Mapping
- Debugging Tests
- Defining a Namespace Prefix on a Connection
- Defining Proxy Settings
- Environment Management
- Exporting Test Cases into a PDF
- Exporting Test Projects
- Override Auto-Retry for Test Step
- Managing Test Steps
- Namespace Org Testing
- Provar Desktop
- Provar Test Builder
- Refresh and Recompile
- Reintroduction of CLI License Check
- Reload Org Cache
- Running Tests
- Searching Provar with Find Usages
- Secrets Management and Encryption
- Setup and Teardown Test Cases
- Tags and Service Level Agreements (SLAs)
- Test Cycles
- Test Plans
- Testing Browser Options
- Tooltip Testing
- Using the Test Palette
- Test Palette Introduction
- Control Test Steps
- Generate Test Case
- List Compare
- Page Object Cleaner
- Read Test Step
- String Test Steps
- UI Test Steps
- Using Custom APIs
- Callable Tests
- Data-Driven Testing
- Page Objects
- Block Locator Strategies
- Introduction to XPaths
- Creating an XPath
- Label Locator Strategies
- Maintaining Page Objects
- Mapping Non-Salesforce fields
- Page Object Operations
- Refresh and Reselect Field Locators in Test Builder
- Using Java Method Annotations for Custom Objects
- Applications Testing
- Database Testing
- Document Testing
- Email Testing
- Mobile Testing
- OrchestraCMS Testing
- Guide in Salesforce CPQ Testing in Automation
- Guide in ServiceMax Testing
- Skuid Testing
- Vlocity API Testing
- Webservices Testing
- Provar Manager
- Introduction to Test Scheduling
- Apache Ant
- Configuration for Sending Emails via the Automation Command Line Interface
- Continuous Integration
- AutoRABIT Salesforce DevOps in Provar Test
- Azure DevOps
- Running a Provar CI Task in Azure DevOps Pipelines
- Configuring the Automation secrets password in Microsoft Azure Pipelines
- Parallel Execution in Microsoft Azure Pipelines using Multiple build.xml Files
- Parallel Execution in Microsoft Azure Pipelines using Targets
- Parallel execution in Microsoft Azure Pipelines using Test Plans
- Bitbucket Pipelines
- Gearset DevOps CI/CD
- GitHub Actions
- Integrating GitHub Actions CI to Run Automation CI Task
- Remote Trigger in GitHub Actions
- Parameterization using Environment Variables in GitHub Actions
- Parallel Execution in GitHub Actions using Multiple build.xml Files
- Parallel Execution in GitHub Actions using Targets
- Parallel Execution in GitHub Actions using Test Plan
- Parallel Execution in GitHub Actions using Job Matrix
- GitLab Continuous Integration
- Travis CI
- Execution Environment Security Configuration
- Provar Jenkins Plugin
- Parallel Execution
- Running Provar on Linux
- Salesforce DX
- Team Foundation Server
- Version Control
- Salesforce Testing
- Adding a Salesforce Connection
- Assert Page Error Messages on Add/Edit Product
- Internationalization Support
- List and Table Testing
- Salesforce Console Testing
- Salesforce Lightning Testing
- Salesforce Lightning Web Component (LWC) Locator Support
- Salesforce Release Updates
- Visualforce Testing
- Best Practices
- Configurations and Permissions
- Error Messages
- Licensing, Installation and Firewalls
- Test Builder and Test Cases
- Release Notes