Gearset DevOps CI/CD via Jenkins
This page describes a solution for setting up integration from Gearset DevOps to execute a Jenkins CI Server Job to execute a suite of Provar tests. Optionally you can write the test results back to a custom object in Salesforce. We’ve listed a free AppExchange package that can be used for that purpose, or you can integrate your test results into an RDBMS connection or via other web services. The process is broadly the same for any other DevOps CI/CD tool that doesn’t run ANT natively but does allow you to configure Webhook or HTTP Callouts.
If you’re using a locally hosted Jenkins instance, please ensure it is within your corporate DMZ and can accept incoming connections from Salesforce. If this is the case, you can skip to the next section. The range of Salesforce IP addresses is long and ever-changing, so we recommend a cloud-hosted instance unless you need to execute end-to-end tests with internal systems.
An AWS instance should be configured following the AWS Set Up a Jenkins Server guide. Note the full public AWS DNS JENKINS_URL must be used, not just the IP address,
After completing your setup, ensure you can access your new Jenkins Admin screen remotely using the JENKINS_URL from your local desktop browser before continuing and not just on the AWS instance using localhost:8080 or localhost:8443. If this fails, you must check your AWS Configure Security Group and ensure it’s been applied to your AWS instance. Do not proceed until this is working.
If installing onto a Windows Server, you must create an Inbound Port Forwarding rule on Windows Firewall for port 8080 or 8443. Do not restrict source IP access unless you plan to whitelist every Salesforce IP address.
You are responsible for locking down this AWS instance and Jenkins to meet your corporate security standards. The instance must be accessible from the Gearset DevOps tool.
Follow the Provar Setting Up Continuous Integration guide.
After configuring the server, the CSRF Protection needs to be disabled in the Manage Jenkins -> Configure Global Security. The default settings for the Access Control should be left below until your integration is working, and then it can be locked down using Matrix-Based Security.
Note: By enabling Read Only Anonymous access, you can allow non-authenticated users to inspect the results of the build action. Disable this if you do not want to let this be publicly visible to anyone with the Jenkins server URL, and set up any other non-admin user access you require instead.
We recommend not using your Jenkins Admin user credentials for triggering remote builds and creating a new user specifically for this purpose, using Manage Jenkins -> Manage Users to add a new user.
For the Jenkins user you want to use to trigger tests remotely, note the Username and API Token to be used. The password is not required for API access and should never be shared externally.
Note: You need to log in as the user to be used and click the Show API token BEFORE restricting access if using the Matrix-based security.
Although you can integrate with the Jenkins Admin user, we strongly recommend that you create a new user identity in the Manage Jenkins -> Manage Users and limit their execution to execute build jobs only once you have your integration working and have captured the API Token as above for the new user:
Deploy your test cases to your Jenkins CI Server, or integrate with your Version Control repository within the Build job you wish to trigger.
Jenkins Remote API Configuration
Depending on your AWS Instance, you must follow the standard install for either Windows or Linus to set up and license Provar for your execution environment.
You should also read carefully and follow all appropriate steps in the Provar DevOps guide.
When configuring your Provar Build Job, ensure the job is made available as a Remote Build and make a note of your Build Authentication Token and the Build URL. For remote invocation, we recommend creating an additional user for this purpose. See section. For simplicity, we recommend using the buildWithParameters version of the Build URL to allow you to pass options for which Provar ANT target and tests to be executed.
You can also add the following optional parameters if you wish to override the Test Case Path and/or the ANT target to be executed. You must ensure you account for these in your corresponding Provar build.xml file.
You can name these parameters whatever you wish and ensure you apply them consistently across Jenkins, the build.xml, and your webhook callout from Gearset DevOps.
If you are using the optional parameters in Process Builder to pass the Test Case Path and/or ANT target, make the following additional changes to your build.xml:
- For Test Path, replace
- For creating alternate ANT targets, copy the entire … block and paste underneath, updating the target name to be unique and changing the test execution parameters as desired.
Gearset DevOps Configuration
Within Gearset, there is an option to configure outgoing webhooks as part of a CI Job. For complete documentation, see the Gearset website: https://gearset.com/blog/outgoing-webhooks-for-ci
For the CI Job you wish to trigger a Provar test run, click on the link to Edit Job and then navigate to the Outgoing webhooks tab:
Complete the following:
- Outgoing webhook URL: This is the Build URL from your Jenkins configuration
- Authentication: Check the Use Basic Auth and provide the Jenkins Username and API Token for the Gearset user (if you created one as advised)
Content Type and Payload are not required if you have used the buildWithParameters Jenkins URL.
NOTE! If you are passing parameters to the Jenkins Provar job, then you will need to include them in your Build URL, for example:
We recommend testing your integration without the additional parameters and then adding them once Gearset triggers your Jenkins job.
We recommend you test each part of the process before running end to end. Ensure each of the following works to diagnose any issues you have:
- The server where Jenkins is deployed can be reached remotely, or at least the Jenkins instance accessed. If this fails, your AWS Security Rules or Server Firewall may block connections. Ensure the Jenkins port (e.g., 8080) is permitted for Inbound Connections and re-test.
- The Provar job on the Jenkins server can be executed within the Jenkins admin console. If this fails to examine the errors, fix the root causes and re-test.
- The Provar job on the Jenkins server can be executed remotely from the command line using curl. If this fails to examine, the error is returned. The most likely cause is Jenkins Global Security. Try opening up access and re-testing to diagnose the issue.
- A Gearset DevOps CI Job triggers a new Jenkins build for the Provar job.
- Add the optional parameters to pass which test cases you wish to execute and/or which ANT target to invoke.
Hopefully, you have a fully working end-to-end process to execute your Gearset-based Continuous Delivery Cycle; well done!
- Provar Automation
- Installing Provar Automation
- Updating Provar Automation
- Using Provar Automation
- API Testing
- Behavior-Driven Development
- Creating and Importing Projects
- Creating Test Cases
- Custom Table Mapping
- Debugging Tests
- Defining a Namespace Prefix on a Connection
- Defining Proxy Settings
- Environment Management
- Exporting Test Cases into a PDF
- Exporting Test Projects
- Override Auto-Retry for Test Step
- Managing Test Steps
- Namespace Org Testing
- Provar Automation
- Provar Test Builder
- Refresh and Recompile
- Reintroduction of CLI License Check
- Reload Org Cache
- Running Tests
- Searching Provar with Find Usages
- Secrets Management and Encryption
- Setup and Teardown Test Cases
- Tags and Service Level Agreements (SLAs)
- Test Cycles
- Test Plans
- Testing Browser Options
- Tooltip Testing
- Using the Test Palette
- Test Palette Introduction
- Control Test Steps
- Generate Test Case
- List Compare
- Page Object Cleaner
- Read Test Step
- String Test Steps
- UI Test Steps
- Using Custom APIs
- Callable Tests
- Data-Driven Testing
- Page Objects
- Block Locator Strategies
- Introduction to XPaths
- Creating an XPath
- Label Locator Strategies
- Maintaining Page Objects
- Mapping Non-Salesforce fields
- Page Object Operations
- Refresh and Reselect Field Locators in Test Builder
- Using Java Method Annotations for Custom Objects
- Applications Testing
- Database Testing
- Document Testing
- Email Testing
- Mobile Testing
- OrchestraCMS Testing
- Salesforce CPQ Testing
- ServiceMax Testing
- Skuid Testing
- Vlocity API Testing
- Webservices Testing
- Provar Manager
- Provar Manager Setup and User Guide
- Installing Provar Manager
- Configuring Provar Manager
- How to Use Provar Manager
- Managing Your Testing Life Cycle
- Provar Manager Test Execution
- Test Executions and Defect Management
- Provar Manager Test Coverage
- How to Integrate Provar Manager
- Setting Up a Connection to Provar Manager
- Object Mapping Between Automation and Manager
- How to Upload Test Plans, Test Plan Folders, Test Plan Instances, and Test Cases
- Provar Manager Filters
- Uploading Callable Test Cases in Provar Manager
- Uploading Test Steps in Provar Manager
- How to Know if a File in Automation is Linked in Test Manager
- Test Execution Reporting
- Provar Manager Plugins
- Uploading Existing Manual Test Cases to Provar Manager with DataLoader.Io
- Metadata Coverage with Manager
- Provar Grid
- Introduction to Provar DevOps
- Introduction to Test Scheduling
- Apache Ant
- Configuration for Sending Emails via the Automation Command Line Interface
- Continuous Integration
- AutoRABIT Salesforce DevOps in Provar Test
- Azure DevOps
- Running a Provar CI Task in Azure DevOps Pipelines
- Configuring the Automation secrets password in Microsoft Azure Pipelines
- Parallel Execution in Microsoft Azure Pipelines using Multiple build.xml Files
- Parallel Execution in Microsoft Azure Pipelines using Targets
- Parallel execution in Microsoft Azure Pipelines using Test Plans
- Bitbucket Pipelines
- GitHub Actions
- Integrating GitHub Actions CI to Run Automation CI Task
- Remote Trigger in GitHub Actions
- Parameterization using Environment Variables in GitHub Actions
- Parallel Execution in GitHub Actions using Multiple build.xml Files
- Parallel Execution in GitHub Actions using Targets
- Parallel Execution in GitHub Actions using Test Plan
- Parallel Execution in GitHub Actions using Job Matrix
- GitLab Continuous Integration
- Travis CI
- Execution Environment Security Configuration
- Provar Jenkins Plugin
- Parallel Execution
- Running Provar on Linux
- Salesforce DX
- Team Foundation Server
- Version Control
- Masking Provar Credentials on CI
- Salesforce Testing
- Adding a Salesforce Connection
- Assert Page Error Messages on Add/Edit Product
- Internationalization Support
- List and Table Testing
- Salesforce Console Testing
- Salesforce Lightning Testing
- Salesforce Lightning Web Component (LWC) Locator Support
- Salesforce Release Updates
- Visualforce Testing
- Best Practices
- Salesforce Connection Best Practices
- Improve Your Metadata Performance
- Testing Best Practices
- Automation Planning
- Supported Testing Phases
- Provar Naming Standards
- Test Case Design
- Create records via API
- Avoid using static values
- Abort Unused Test Sessions/Runs
- Avoid Metadata performance issues
- Increase auto-retry waits for steps using a global variable
- Create different page objects for different pages
- The Best Ways to Change Callable Test Case Locations
- Working with the .testProject file and .secrets file
- Best practices for the .provarCaches folder
- Best practices for .pageObject files
- Configurations and Permissions
- Error Messages
- Provar Manager 3.0 Install Error Resolution
- Provar Manager Test Case Upload Resolution
- Administrator has Blocked Access to Client
- macOS Big Sur Upgrade
- Resolving Failed to Create ChromeDriver Error
- Resolving Jenkins License Missing Error
- Resolving Metadata Timeout Errors
- Test Execution Fails – Firefox Not Installed
- Selenium 4 Upgrade
- Licensing, Installation and Firewalls
- Test Builder and Test Cases
- Release Notes