Introduction to Provar DevOps
Back in 2019 we shared for Provar Automation users how they could host and connect Automation with Gearset and other CI/CD platforms that were unable to run Automation tests on their existing platform infrastructure. Note that you can follow similar steps to execute your tests from within Slack workflows and many other tools too!
Since then a lot has changed, including the introduction of Provar Manager as a 100% native Salesforce app and our serverless Provar Grid capability. These have enabled us to offer new ways of integration for our customers. We also packaged our Docker support to make it easier for teams without full-time DevOps engineers to integrate test automation into their CI/CD pipelines.
If you’re still using a Jenkins server, then there’s no need to change unless you want to take advantage of the additional features offered by Provar Manager. You can also continue to use your Jenkins server to run test execution jobs from Provar Manager if you do not wish to purchase Provar Grid parallel execution licenses.
There are now several ways to integrate any existing CI/CD or DevOps platform, such as Gearset, with Automation:
- Server hosting of Provar Automation CLI within the 3rd party CI/CD solution. Supported by Autorabit, Azure DevOps, Bitbucket Pipelines, CircleCI, and Gitlab to name a few popular choices. Contact our Customer Service team for assistance.
- Packaged integration to another platform that can execute Automation via Ant, ProvarDX via the Salesforce CLI, or using a Provar Docker image. This includes platforms such as Provar Grid, CircleCI, and Jenkins where we have already packaged solutions on their respective marketplaces, or any other CI/CD platform or custom server infrastructure where you have to develop your scripts from our examples.
- Direct integration via Provar Manager into Salesforce. The Salesforce platform is a powerful solution that supports direct and indirect integrations through its low-code and pro-code development tools. In this article, we will focus on using this solution to integrate Gearset using Provar Grid for the remote execution of the test cases for mass parallel testing. You can also use the other solutions listed in Direct Integration above while initiating the execution of tests via Provar Manager as your quality hub.
- Indirect integration via Git events. When using a different product, but retaining your Git repo for your test cases, Github, Azure DevOps, and Gitlab all allow you to trigger a Provar Automation script via the supported mechanisms when a Git event (such as Commit, Merge, or Pull Request) is triggered. This can be used by tools like Gearset, Opsera, and Salto.io, which do not run Automation natively.
- Indirect integration via Provar Manager. This includes platforms that can live inside the same Salesforce instance with plugins for Salesforce DevOps Center, Copado, and Flosum, and more. Using either Salesforce Flow Triggers and/or custom Apex Triggers you can initiate testing based on observing changes in records in these applications. A sample Flow for triggering tests based on specific metadata changes is included in Provar Manager and its extension packages.
Note that the use of popular cross-browser testing platforms like Lambdatest, Saucelabs, and Browserstack is also compatible with Automation but does not host Provar software. They act in a remote browser capacity, where the test workload runs on another server like the examples above, but the provisioning of different browsers and versions is remote. Unlike Provar Grid, these tend to be much slower and require you to manually divide your test plans to achieve parallel test capability.
We have created a Gearset CI/CD Integration guide that focuses on the following integration from the 3rd category above, Direct Integration via Provar Manager. You can replace this with other solutions but the examples show the configuration of Gearset specifically. A recommended option using Apex and an alternative for no-code deployment using existing Flows are included in this guide. The changes to configure Provar Manager are identical regardless of your source system:
Note: If you do not have Manager but would like to use the integration from Gearset to execute Automation tests please contact your Provar Customer Success Manager. A single complimentary license can be provided free of charge for this purpose, but you will still need to either purchase Grid licenses or set up your own execution solution. A 30-day trial of Grid is also available from the Provar Customer Success team.
Please see the Manager documentation on specifics on how to configure remote execution with Provar Grid, Jenkins, and other supported endpoints. The guide will assume you have already set up the remote execution in Manager and have created an existing working Test Plan Schedule record.
For more information, check out these courses on University of Provar:
DevOps
CI/CD
- Provar Automation
- System Requirements
- Browser and Driver Recommendations
- Installing Provar Automation
- Updating Provar Automation
- Licensing Provar
- Granting Org Permissions to Provar Automation
- Optimizing Org and Connection Metadata Processing in Provar
- Using Provar Automation
- API Testing
- Behavior-Driven Development
- Consolidating Multiple Test Execution Reports
- Creating and Importing Projects
- Creating Test Cases
- Custom Table Mapping
- Functions
- Debugging Tests
- Defining a Namespace Prefix on a Connection
- Defining Proxy Settings
- Environment Management
- Exporting Test Cases into a PDF
- Exporting Test Projects
- Japanese Language Support
- Override Auto-Retry for Test Step
- Mapping and Executing the Lightning Article Editor in Provar
- Managing Test Steps
- Namespace Org Testing
- NitroX
- Provar Automation
- Provar Test Builder
- ProvarDX
- Refresh and Recompile
- Reintroduction of CLI License Check
- Reload Org Cache
- Reporting
- Running Tests
- Searching Provar with Find Usages
- Secrets Management and Encryption
- Setup and Teardown Test Cases
- Tags and Service Level Agreements (SLAs)
- Test Cycles
- Test Plans
- Testing Browser Options
- Tooltip Testing
- Using the Test Palette
- Using Custom APIs
- Callable Tests
- Data-Driven Testing
- Page Objects
- Block Locator Strategies
- Introduction to XPaths
- Creating an XPath
- JavaScript Locator Support
- Label Locator Strategies
- Maintaining Page Objects
- Mapping Non-Salesforce fields
- Page Object Operations
- ProvarX™
- Refresh and Reselect Field Locators in Test Builder
- Using Java Method Annotations for Custom Objects
- Applications Testing
- Provar Manager
- How to Use Provar Manager
- Provar Manager Setup
- Provar Manager Integrations
- Release Management
- Test Management
- Test Operations
- Provar Manager and Provar Automation
- Setting Up a Connection to Provar Manager
- Object Mapping Between Automation and Manager
- How to Upload Test Plans, Test Plan Folders, Test Plan Instances, and Test Cases
- Provar Manager Filters
- Uploading Callable Test Cases in Provar Manager
- Uploading Test Steps in Provar Manager
- How to Know if a File in Automation is Linked in Test Manager
- Test Execution Reporting
- Metadata Coverage with Manager
- Provar Grid
- DevOps
- Introduction to Provar DevOps
- Introduction to Test Scheduling
- Apache Ant
- Configuration for Sending Emails via the Automation Command Line Interface
- Continuous Integration
- AutoRABIT Salesforce DevOps in Provar Test
- Azure DevOps
- Running a Provar CI Task in Azure DevOps Pipelines
- Configuring the Automation secrets password in Microsoft Azure Pipelines
- Parallel Execution in Microsoft Azure Pipelines using Multiple build.xml Files
- Parallel Execution in Microsoft Azure Pipelines using Targets
- Parallel execution in Microsoft Azure Pipelines using Test Plans
- Bitbucket Pipelines
- CircleCI
- Copado
- Docker
- Flosum
- Gearset
- GitHub Actions
- Integrating GitHub Actions CI to Run Automation CI Task
- Remote Trigger in GitHub Actions
- Parameterization using Environment Variables in GitHub Actions
- Parallel Execution in GitHub Actions using Multiple build.xml Files
- Parallel Execution in GitHub Actions using Targets
- Parallel Execution in GitHub Actions using Test Plan
- Parallel Execution in GitHub Actions using Job Matrix
- GitLab Continuous Integration
- Travis CI
- Jenkins
- Execution Environment Security Configuration
- Provar Jenkins Plugin
- Parallel Execution
- Running Provar on Linux
- Reporting
- Salesforce DX
- Git
- Version Control
- Masking Provar Credentials on CI
- Salesforce Testing
- Best Practices
- Improve Your Metadata Performance
- Java 21 Upgrade
- Salesforce Connection Best Practices
- Testing Best Practices
- Automation Planning
- Supported Testing Phases
- Provar Naming Standards
- Test Case Design
- Create records via API
- Avoid using static values
- Abort Unused Test Sessions/Runs
- Avoid Metadata performance issues
- Increase auto-retry waits for steps using a global variable
- Create different page objects for different pages
- The Best Ways to Change Callable Test Case Locations
- Working with the .testProject file and .secrets file
- Best practices for the .provarCaches folder
- Best practices for .pageObject files
- Troubleshooting
- How to Use Keytool Command for Importing Certificates
- Installing Provar After Upgrading to macOS Catalina
- Browsers
- Configurations and Permissions
- Connections
- DevOps
- Error Messages
- Provar Manager 3.0 Install Error Resolution
- Provar Manager Test Case Upload Resolution
- Administrator has Blocked Access to Client
- JavascriptException: Javascript Error
- macOS Big Sur Upgrade
- Resolving Failed to Create ChromeDriver Error
- Resolving Jenkins License Missing Error
- Resolving Metadata Timeout Errors
- Test Execution Fails – Firefox Not Installed
- Selenium 4 Upgrade
- Licensing, Installation and Firewalls
- Memory
- Test Builder and Test Cases
- Release Notes