Running Automation Tests on Jenkins
There are multiple options available for executing your Automation tests on the Jenkins server.
Option 1: Automation ANT libraries on Jenkins server
Depending on your AWS Instance you will either need to follow the standard install for either Windows or Linux to setup and license Automation for your execution environment.
You should also read carefully and follow all relevant steps in the Introduction to Test Scheduling.
Setting up continuous integration for your Automation project is always recommended. This will help you schedule test runs, get consistent reporting and allow you to execute test cases on a remote server.
You can follow this help page to setup Jenkins for Automation using Jenkins build pipelines.
There are a few additional steps to keep in mind for use with your RM tool; this is because these tools will be sending build parameters to customize how the Automation tests are run, which tests are run, etc. You can add any additional build parameters as necessary for your use case.
First, create your new job in Jenkins using the type Freestyle Job and add a few build parameters. In order to add build parameters, you must first check the box This project is parameterized.
These are all recommended build parameters as they will be passed from Salesforce to Jenkins to execute the Automation test cases.
TEST_PLAN should be the name of the test plan that you wish to execute in the Automation test project. You can also use TEST_PATH and specify this as a variable/parameter inside of your build.xml. This part is up to the user, and depends on if you are using Automation Test Plans or not.
ANT_TARGET is the name of the ANT target inside of the build.xml file that is used to execute Automation tests via ANT. This is defaulted to runtests and will likely be unchanged unless you have defined multiple targets. The most common use case for this is parallel/distributed execution.
DEPLOYMENT_ID is the RM tool’s deployment ID that is created to execute the URL callout. The deployment phase will need to be built out externally using a RM tool in your Salesforce org. This can be left as the default for now.
BUILD_FILE will be the relative path name of the build file you wish to execute via ANT. This is defaulted to ANT/build.xml, which is the name of the build.xml file Automation generates in the desktop application. You can change this to match your build file.
Now that all of the build parameters have been added, a proper Source Code Management (SCM) can be configured in this Jenkins job. We will be using Git SCM for this guide.
Adjust your settings to match your Git repository that was setup inside of Automation (referenced in the Application Layer of the Architecture section of your RM Tool’s guide).
If you are planning on triggering this Jenkins job from another tool, such as Copado, Flosum, or Gearset, then you will need to add the Trigger builds remotely build trigger.
The next step will be to add an authentication token that will be used to authenticate when calling the Jenkins job. This is only for triggering builds remotely from another tool.
This needs to be set to the NAME of the API token that you generated previously.
Once you set this token, make sure you save the URL that is displayed below the field and the token itself to a local file (using Notepad or any similar software). This URL will be used in a later section of this guide.
You can read more about different build triggers here.
The last mandatory section will be to add the build action itself.
In this section (and in any field inside of the Jenkins job), the build parameters can be utilized using the following syntax: $PARAMETER_NAME. This is the step to execute your Automation tests using the parameters passed from Copado.
If you would like to store the results of test runs and utilize the JUnit plugin in Jenkins (which you should install as a recommended plugin), then you’ll need to add one more step in the Post-build Actions.
In the Post-build Actions section at the bottom of the build, add the step called Publish JUnit test results report.
For the Test report XMLs field, put the following value to find all Automation test results:
**/ANT/Results/*.xml
Typically you would want to check the box to allow for empty test results, that way the build wouldn’t show as failed if no tests were run. The rest of the settings you can configure however you wish for your use case.
Save the job and test the job by triggering it manually first. If all goes well, you should see a very nice Test Result view as shown below.
You can drill down to the test results even more by clicking the Latest Test Result link.
For a more detailed guide on the Jenkins set up and job configuration, see this support article.
Option 2: Automation on Docker
If you wish to execute your test cases inside of a Docker container, please refer to our Docker guide. It is best to utilize this option for any new Jenkins server setups. This will save both time and maintenance on this server since the Dockerfile is managed by Provar. The last part of the Docker guide has detailed instructions on how to implement this solution in conjunction with Jenkins.