Running Automation Tests on Jenkins
There are multiple options available for executing your Automation tests on the Jenkins server.
Option 1: Automation ANT libraries on Jenkins server
You should also read carefully and follow all relevant steps in the Introduction to Test Scheduling.
Setting up continuous integration for your Automation project is always recommended. This will help you schedule test runs, get consistent reporting and allow you to execute test cases on a remote server.
There are a few additional steps to keep in mind for use with your RM tool; this is because these tools will be sending build parameters to customize how the Automation tests are run, which tests are run, etc. You can add any additional build parameters as necessary for your use case.
First, create your new job in Jenkins using the type Freestyle Job and add a few build parameters. In order to add build parameters, you must first check the box This project is parameterized.
These are all recommended build parameters as they will be passed from Salesforce to Jenkins to execute the Automation test cases.
TEST_PLAN should be the name of the test plan that you wish to execute in the Automation test project. You can also use TEST_PATH and specify this as a variable/parameter inside of your build.xml. This part is up to the user, and depends on if you are using Automation Test Plans or not.
ANT_TARGET is the name of the ANT target inside of the build.xml file that is used to execute Automation tests via ANT. This is defaulted to runtests and will likely be unchanged unless you have defined multiple targets. The most common use case for this is parallel/distributed execution.
DEPLOYMENT_ID is the RM tool’s deployment ID that is created to execute the URL callout. The deployment phase will need to be built out externally using a RM tool in your Salesforce org. This can be left as the default for now.
BUILD_FILE will be the relative path name of the build file you wish to execute via ANT. This is defaulted to ANT/build.xml, which is the name of the build.xml file Automation generates in the desktop application. You can change this to match your build file.
Now that all of the build parameters have been added, a proper Source Code Management (SCM) can be configured in this Jenkins job. We will be using Git SCM for this guide.
Adjust your settings to match your Git repository that was setup inside of Automation (referenced in the Application Layer of the Architecture section of your RM Tool’s guide).
If you are planning on triggering this Jenkins job from another tool, such as Copado, Flosum, or Gearset, then you will need to add the Trigger builds remotely build trigger.
The next step will be to add an authentication token that will be used to authenticate when calling the Jenkins job. This is only for triggering builds remotely from another tool.
This needs to be set to the NAME of the API token that you generated previously.
Once you set this token, make sure you save the URL that is displayed below the field and the token itself to a local file (using Notepad or any similar software). This URL will be used in a later section of this guide.
You can read more about different build triggers here.
The last mandatory section will be to add the build action itself.
In this section (and in any field inside of the Jenkins job), the build parameters can be utilized using the following syntax: $PARAMETER_NAME. This is the step to execute your Automation tests using the parameters passed from Copado.
If you would like to store the results of test runs and utilize the JUnit plugin in Jenkins (which you should install as a recommended plugin), then you’ll need to add one more step in the Post-build Actions.
In the Post-build Actions section at the bottom of the build, add the step called Publish JUnit test results report.
For the Test report XMLs field, put the following value to find all Automation test results:
Typically you would want to check the box to allow for empty test results, that way the build wouldn’t show as failed if no tests were run. The rest of the settings you can configure however you wish for your use case.
Save the job and test the job by triggering it manually first. If all goes well, you should see a very nice Test Result view as shown below.
You can drill down to the test results even more by clicking the Latest Test Result link.
For a more detailed guide on the Jenkins set up and job configuration, see this support article.
Option 2: Automation on Docker
If you wish to execute your test cases inside of a Docker container, please refer to our Docker guide. It is best to utilize this option for any new Jenkins server setups. This will save both time and maintenance on this server since the Dockerfile is managed by Provar. The last part of the Docker guide has detailed instructions on how to implement this solution in conjunction with Jenkins.
- Provar Automation
- Installing Provar Automation
- Updating Provar Automation
- Using Provar Automation
- API Testing
- Behavior-Driven Development
- Creating and Importing Projects
- Creating Test Cases
- Custom Table Mapping
- Debugging Tests
- Defining a Namespace Prefix on a Connection
- Defining Proxy Settings
- Environment Management
- Exporting Test Cases into a PDF
- Exporting Test Projects
- Override Auto-Retry for Test Step
- Managing Test Steps
- Namespace Org Testing
- Provar Desktop
- Provar Test Builder
- Refresh and Recompile
- Reintroduction of CLI License Check
- Reload Org Cache
- Running Tests
- Searching Provar with Find Usages
- Secrets Management and Encryption
- Setup and Teardown Test Cases
- Tags and Service Level Agreements (SLAs)
- Test Cycles
- Test Plans
- Testing Browser Options
- Tooltip Testing
- Using the Test Palette
- Test Palette Introduction
- Control Test Steps
- Generate Test Case
- List Compare
- Page Object Cleaner
- Read Test Step
- String Test Steps
- UI Test Steps
- Using Custom APIs
- Callable Tests
- Data-Driven Testing
- Page Objects
- Block Locator Strategies
- Introduction to XPaths
- Creating an XPath
- Label Locator Strategies
- Maintaining Page Objects
- Mapping Non-Salesforce fields
- Page Object Operations
- Refresh and Reselect Field Locators in Test Builder
- Using Java Method Annotations for Custom Objects
- Applications Testing
- Database Testing
- Document Testing
- Email Testing
- Mobile Testing
- OrchestraCMS Testing
- Guide in Salesforce CPQ Testing in Automation
- Guide in ServiceMax Testing
- Skuid Testing
- Vlocity API Testing
- Webservices Testing
- Provar Manager
- Introduction to Test Scheduling
- Apache Ant
- Configuration for Sending Emails via the Automation Command Line Interface
- Continuous Integration
- AutoRABIT Salesforce DevOps in Provar Test
- Azure DevOps
- Running a Provar CI Task in Azure DevOps Pipelines
- Configuring the Automation secrets password in Microsoft Azure Pipelines
- Parallel Execution in Microsoft Azure Pipelines using Multiple build.xml Files
- Parallel Execution in Microsoft Azure Pipelines using Targets
- Parallel execution in Microsoft Azure Pipelines using Test Plans
- Bitbucket Pipelines
- Gearset DevOps CI/CD
- GitHub Actions
- Integrating GitHub Actions CI to Run Automation CI Task
- Remote Trigger in GitHub Actions
- Parameterization using Environment Variables in GitHub Actions
- Parallel Execution in GitHub Actions using Multiple build.xml Files
- Parallel Execution in GitHub Actions using Targets
- Parallel Execution in GitHub Actions using Test Plan
- Parallel Execution in GitHub Actions using Job Matrix
- GitLab Continuous Integration
- Travis CI
- Execution Environment Security Configuration
- Provar Jenkins Plugin
- Parallel Execution
- Running Provar on Linux
- Salesforce DX
- Team Foundation Server
- Version Control
- Salesforce Testing
- Adding a Salesforce Connection
- Assert Page Error Messages on Add/Edit Product
- Internationalization Support
- List and Table Testing
- Salesforce Console Testing
- Salesforce Lightning Testing
- Salesforce Lightning Web Component (LWC) Locator Support
- Salesforce Release Updates
- Visualforce Testing
- Best Practices
- Configurations and Permissions
- Error Messages
- Licensing, Installation and Firewalls
- Test Builder and Test Cases
- Release Notes