Documentation

Looking for something in particular?

Salesforce DX Integration

Provar and Salesforce DX can be used together to generate a scratch org, push the latest Salesforce customizations, and run Provar tests inside the scratch org to get functional testing insights. This allows immediate feedback during a development cycle or when a scratch org is used as a system integration test environment.

As part of our ongoing commitment as the only Salesforce-first test automation tool, we support the integration of Provar with the Salesforce CLI using ProvarDX™. For information about how you can run Provar tests for Salesforce DX scratch orgs using ANT and ConfigureDX, please see this related article.

ProvarDX™ (Provar Developer eXperience) is a free, open-source extension that uses your local Provar installation licenses. ProvarDX™ is designed to integrate Provar seamlessly into your Salesforce developer environments so tests can be executed supporting local development using scratch orgs. This provides a richer set of tools than ConfigureDX, designed purely for customers running Provar tests on scratch orgs under ANT.

ProvarDX™ consists of the following elements

  • A Salesforce CLI plugin that allows you to execute Provar test cases from a command line without requiring the installation or use of ANT. ProvarDX™ can be used instead of ANT as part of your CI/CD pipeline to run Provar tests.
  • An optional Visual Studio Code Extension that provides access to the most useful ProvarDX™ CLI commands using the Visual Studio Code command autocomplete menu.
  • A Provar Desktop right-click option (Export as ProvarDX™) to allow you to export one or more tests (or test plans) as a property file for execution using ProvarDX™.

Both ANT (via ConfigureDX) and ProvarDX™ support using scratch orgs for test execution. However, ProvarDX™ has several advantages, including:

  • The ability to override Provar Salesforce connections using the DevHub org alias as the connection name instead of requiring you to use the original Salesforce username as the alias. This subtle change lets you be very clear in your scratch org names and use meaningful aliases (such as Sales User, Service Admin, Metadata Admin, etc.) instead of john.doe@crafy-otter-hvfpug.com.
  • ProvarDX™ works on scratch orgs and sandboxes, allowing you to use a single solution regardless of whether you follow a source-driven, hybrid, or org-driven Salesforce application lifecycle.
  • ProvarDX™ does not require you to generate passwords for your scratch orgs. ProvarDX™ does this for you when it needs one.
  • With ProvarDX™, you can override property settings on command line execution using parameters.
  • ProvarDX™ is consistent with the Salesforce CLI, requiring a single set of skills to maintain and operate. Updates are also synchronized with the Salesforce CLI.

Everything in this guide applies to teams following source-driven development using scratch orgs and those following org-driven development using sandbox orgs.

Why ProvarDX™?

Provar is designed to help our customers quickly create robust, reusable tests that are intelligent regarding Salesforce org metadata and reusable across multiple user types, environments, and user interfaces.

Provar can uniquely run across multiple environments in a typical software development lifecycle. To align with Salesforce’s source-driven development approach using scratch orgs, and traditional org-driven development, we’ve delivered ProvarDX™ to make this integration seamless for our users.

  1. Admins and developers use scratch orgs and/or developer sandboxes to make atomic changes, which they want to verify before pushing to continuous integration.
  2. CI tools hooked into a version control system (e.g., Git) trigger publication and run Provar regression tests to validate the changes. Issues notified to the development and QA team for correction or update of test cases to match changes in behavior.
  3. CD pipeline invoked as needed to deploy application changes to each environment. Provar regression and smoke test plans executed as appropriate to validate correct deployment package deployment, plus check for legacy issues or primary business scenarios that are working as expected. 

As you can see, with ProvarDX™, we’re unlocking those test cases already developed using Provar so that Salesforce administrators and developers can run a suite of tests to verify their changes much earlier in the development process. We’ve chosen to do this using the same tooling that Salesforce provides, Visual Source Code and the Salesforce Command Line Interface (CLI). 

Note: This does not eliminate the need for developers to write unit tests for their apex code and Lightning components. 

Getting started 

To utilize ProvarDX™, you must first install the Salesforce CLI (following Salesforce’s installation recommendations) in the environment where ProvarDX™ will be executed.

After installing the Salesforce CLI, the ProvarDX™ package can be installed, which extends the Salesforce CLI command set with four simple commands using the Provar namespace:

sfdx provar:runtests

Runs a specific set of tests using environment overrides just as you would using the ANT build.xml file. Like ANT, we support the same muti-browser and test reporting options. Behavior is controlled by the provardx property file. We recommend having multiple provardx properties to reflect each environment you want to test and the appropriate test plan. 

sfdx provar:validate

The optional command allows you to quickly validate changes you make to your ProvarDX™ property files rather than wait until you try running runtests to execute them.

sfdx provar:metadatacache

Downloads the latest metadata for the named Salesforce connections and stores this locally. This optional command allows you to preprocess this step rather than wait during your runtests command for any changes to download. It is very useful for CI/CD pipelines where you want to execute post-deployment steps before starting a full test run.

sfdx provar:compile

Optional command to allow you to precompile any Provar page objects rather than wait until your runtests command is executed. Useful for ensuring your Provar PageObjects, Custom APIs, and PageControls before starting a test run. Typically used in a CI/CD pipeline after pulling the latest Provar project from your version control system.

Each of these commands has several flags and options you can set to control their operation. Each flag has a longform (e.g. –targetusername) and short form (e.g. -u) for convenience. Please see the ProvarDX™ NPM listing to get the up-to-date list of flags and options. 

Note: If you’re struggling to remember the correct command syntax, just type sfdx provar --help to see a list of the options and parameters. You can do this at any level of the command, e.g.

sfdx provar:runtests --help tells you the parameters of the runtestscommand.

In addition to the Salesforce CLI extension, we have also made available a Visual Studio Code Extension for those who prefer to use the Visual Studio Code command palette window. These commands can be accessed like any other Visual Studio Code command using either Command + P (Mac OS) or Ctrl + P (Windows). 

We currently provide three commands via the Visual Studio Code extension

ProvarDX: Create Properties File

Creates a boilerplate provardx-properties.json file so you have to edit the values you need.

ProvarDX: Run Testcases

Executes sfdx:provar:runtests using a specified provardx property file.

ProvarDX: Validate Properties File

Executes sfdx:provar:validate for the named provardx property file.

We plan to add more commands, so please bookmark this page and watch out for upcoming Provar release notes for announcements on ProvarDX™ enhancements.

Using ProvarDX™

Step 1: Using Provar Desktop, first open your Provar project. 

Step 1: Within the Navigator, right-click on the test plan, test case folder, or the individual test cases you want to run under ProvarDX™.

Step 3: Select the Export as ProvarDX option.

Step 4: Update the filename and any parameters you want to change. This does not impact your ANT execution settings or start running any tests.

Note: You can either create a file from here or use the clipboard option to make a copy and paste it into your property file in Visual Studio Code as you prefer.

Step 5: Using Visual Studio Code, you can open this property file in your Provar projects’ ProvarDX™ subfolder. If you use a VCS integration, you may want to add this folder to your repository to share it with others.

Creating this file from Provar Desktop is recommended as it prepopulates the required environment settings and parameters and is ready to run. There are some properties we want to briefly focus on here as they are extremely useful:

  • environment lets you select a predefined Provar environment. This is important if you want to leverage the QA team’s work to predefine each environment, the screen resolution, the default browser, and, most importantly, a suite of connection overrides.
  • connectionOverride lets you provide name-value pairs of a Provar Salesforce connection and the username you want to use instead during metadata or runtestscommand execution. This is currently how you change your tests to run against a new temporary scratch org. Please refer to the Overriding Salesforce connections and tests section for an example of this in more detail.
  • testCase lets you change the list of tests (including wildcards) that are executed using runtests. This is currently how you can change the tests to be run.
  • testPlan lets you do the same as testCase but allows you to name one or more Provar Test Plans and is more powerful in test case reuse. This requires your QA team to already have predefined Test Plans that you can use.

Step 6: Validate the property file, especially if you have made any changes. You can do this using the Command Palette option, ProvarDX: Validate Properties File, or from the terminal window type.

sfdx provar:validate -p provardx-properties.json

Note: The provardx-properties.json filename is a default, like build.xml is with ANT. If you do not specify the property file, this special file will be selected by default if it is available in the local directory.

Step 7: You should now see the Property file is valid and displayed in the output. 

Whenever you want to execute these tests, you have two options.

From the terminal command line type:

sfdx provar:runtests -p provardx-properties.json

Or, from the Visual Studio Code command palette, select the ProvarDX: Run Testcases option. You will be prompted for your property file; you can accept the default of provardx-properties.json.

Overriding Salesforce connections and tests 

We can use the ProvarDX™ property file to pass a user substitution to run this same set of tests against any org with the same or similar configuration. We can do this using the overrideConnections list in the provardx-properties.json file.

Step 1: Check the Salesforce orgs.

sfdx force:org:list

Step 2: The example above is a scratch org, (username  test-1rrspy7jhvy9@example.com). This reflects configuration changes or developments you may have made to deliver a new feature to your Salesforce users.

If we wanted to run our tests precisely as defined by our Provar project, we could simply use

sfdx provar:runtests -p provardx-properties.json

This is great if you want to run the same tests that your QA team already built in Provar, but it is not recommended for running those tests within your scratch org that you just made changes in.

To run the tests against your scratch org, you need to edit the provardx-properties.json file and add some lines to tell provardx to override the connections in the provardx project with our scratch org user. You can do this using the connectionOverride.

This takes the name of the connection in your Provar project. e.g., Admin, in the case below, reassigns all tests using that connection to use the scratch org instead.

Step 3: Now you can take those tests that you were working on in your admin connection and run them against this scratch org by re-running this command:

sfdx provar:runtests -p provardx-properties.json

This time, you will see the original connection username (trialadm@provartesting.com.trialsbx3 in my case) replaced with our scratch org user. You can do this for multiple user connections; ensure you have added them to your DevHub first.

Step 4: To change the test cases that are executed, you can edit your provardx-properties to make the following changes:

  • Amend the testCase JSON entry to change the test cases to execute. You can use a wildcard character of * to select multiple (e.g. “testCase” : [“/Sales/Acct Management 01.testcase”, “Service/*.testcase”]. This would run all of your tests in the service folder after running the Sales Acct management 01 test case.
  • Add a plan entry to execute one or more Provar Test Plans: e.g.

This will execute both the DevSmoketest plan and the additional test cases.

Known limitations

ProvarDX™ is available as a beta feature and is fully supported. However, there are still some known limitations with its usage, and we’re planning further development to optimize the feature and address these issues. We reserve the right to change or remove features as needed to resolve these limitations and extend the product capability.

Learn MOAR

For a full list of up-to-date commands and parameters, look at the ProvarDX™ npm project listing.

If you’re new to SalesforceDX and the Salesforce CLI in general, take a look at the following resources from Salesforce:

Feedback

Was this article helpful for you?
Documentation library

Trying to raise a case with our support team?

Log into the Success Portal

We use cookies to better understand how our website is used so we can tailor content for you. For more information about the different cookies we use please take a look at our Privacy Policy.

Scroll to Top