Setting up Continuous Integration
Once the build file is ready and the version control has been set up (if required), tests can be scheduled using a Continuous Integration (CI) Server such as Jenkins.
The role of the CI Server is to run the regression pack (from a local machine or via a version control repository). The CI Server will trigger the Provar automation pack on a defined schedule or based on an external trigger, such as a new build or deployment.
Start by downloading and installing Jenkins.
To download Jenkins, visit the Jenkins website. Click Download and select the file which suits your operating system.
To install Jenkins, first move the downloaded Jenkins.war file to an appropriate directory, e.g. C:\Program Files\Jenkins.
Then, open command prompt in this directory by clicking on the filepath and typing cmd.
Once command prompt has opened, type and run the command java –jar Jenkins.war.
Once the command has run, a password will be generated for logging into Jenkins.
In this example, the password is cd3f375e0baa47f4b4a5591f1e920bfe.
Note: This password can also be checked by referring to the InitialAdminPassword.txt file which has been created in the \.jenkins\secrets folder.
Next, open your browser and navigate to URL: localhost:8080.
The following screen will appear.
Enter your password then click Continue.
On the next screen, choose the option Select plugins to install.
Some plugins will be defaulted already. These can be left as defaulted.
Make sure that the following plugins are ticked:
Ant Plugin (under Build Tools).
JUnit Plugin (under Build Analysis and Reporting).
If you want Jenkins to download the regression pack from your Version Control repository, also tick the relevant plugin under Source Code Management (e.g. the Git plugin).
When this is completed, click Continue as admin on the following screen (instead of filling out Create First Admin User).
Note: The next time you log in, your username will be admin and your password will be the one generated earlier.
On the next screen, click Start using Jenkins to complete the setup.
Scheduling a Run
To schedule test in Jenkins, start by clicking the New Item option on the left-hand side.
Provide a name for the new project, e.g. ProvarProject, then select the Freestyle project option and click OK.
If you are using version control, navigate to Source Code Management and select Git. (If you are not using version control, skip ahead to Build Triggers).
Enter the Repository URL, which can be copied by clicking the Clone or Download button against the repository.
Then, provide Credentials (if applicable – this is only needed for private repositories).
To define the execution schedule, navigate to Build Triggers. Here, you can define when you want Provar tests to be run. Choose Build periodically if you want to set up a schedule.
(Alternatively, you can also select Trigger builds remotely or Build after other projects are built if you want to trigger the execution using a script or following completion of other project builds.)
If you choose the Build periodically option, you will then need to enter an expression in Schedule to define when the tests should be run. This is done using a CRON expression.
To create this expression, you can write your own from scratch or copy or adapt one of the examples below.
H H(3-4) * * *
Schedule: Every Day of the week, every Month in a year, every Day in a month at a time between 3 – 4 am in the morning at any minute.
H (30 -45) 3 * * *
Schedule: every Day of the week, every Month in a year, every Day in a month at 3 am in the morning between 30 -45 minutes.
*/5 * * * * :
Schedule: every 5 minutes.
0 8 * * *
Schedule: every day at 8h 00.
Enter your preferred syntax in Schedule, then check the validation message underneath to confirm the syntax is correct and the schedule has been defined as expected.
The ‘Would last have run at…’ information will show the most recent (past) run and next upcoming run, based on your syntax.
Finally, navigate to Build and click the Add build step dropdown, then choose Invoke ANT.
Leave Targets blank, unless you know that your build file has multiple targets. (If your build file was generated using the default settings supplied in Run Under ANT, leaving Targets blank will work fine.)
If you have modified the Build File so that there are multiple targets, enter the name of the desired target, e.g. runtests.
Then, choose the Advanced… option and provide the build.xml path.
If the project is fetched from git, it should use the JENKINS_HOME variable (see screenshot above). If the project is local, it can be a filepath, e.g. C:\Users\Heather\Documents\GitHub\dev\DevOps\ANT\build.xml.
Test your build by clicking the Build Now option in the left-hand sidebar.
Build progress will be shown within Jenkins. If the build is successful, results will also be emailed as defined in the original build file.
Configuring test reports
Once your build generates test results, Jenkins’ JUnit plugin can be used to display these results graphically.
When Provar tests are executed via CI, Provar automatically generates a JUnit.xml file alongside its HTML and PDF reports. Using the JUnit plugin, this JUnit.xml file can be converted into a graphical visualization of the test results. The plugin also provides a web UI for viewing test reports and tracking failures.
Since Provar generates the JUnit.xml file automatically, very little further set up is required. One thing to keep in mind is that the Results folder, which contains this JUnit.xml, should be generated or placed in Jenkins’ workspace folder.
For example, if the project is inside Jenkins’ workspace, define the results path as shown below.
Follow the steps below to get the JUnit plugin up and running.
In Jenkins, click on the dropdown arrow on your Job Name and select Configure.
Navigate to ‘Post-build Actions’, then click the Add post-build action dropdown and select Publish JUnit test result report.
In Test report XMLs, set the path of the JUnit.xml file (note that this is generated inside your Results folder). This XML file should also be inside the workspace of the job of Jenkins. Enter the relative path of the results folder generated after execution:
Note: Before the first execution of your build, this field will show an error, since the file is not generated until after the execution.
Then, save and trigger the job.
Once execution completes, you can view JUnit results by clicking Test Result on the Execution View screen.
On the next screen, a History link is available to show the historical view or trend of execution.
Note: The historical data is visible until two or more execution cycles.
This completes the CI set up.
- Provar Automation
- Installing Provar Automation
- Updating Provar Automation
- Using Provar Automation
- API Testing
- Behavior-Driven Development
- Creating and Importing Projects
- Creating Test Cases
- Custom Table Mapping
- Debugging Tests
- Defining a Namespace Prefix on a Connection
- Defining Proxy Settings
- Environment Management
- Exporting Test Cases into a PDF
- Exporting Test Projects
- Override Auto-Retry for Test Step
- Managing Test Steps
- Namespace Org Testing
- Provar Automation
- Provar Test Builder
- Refresh and Recompile
- Reintroduction of CLI License Check
- Reload Org Cache
- Running Tests
- Searching Provar with Find Usages
- Secrets Management and Encryption
- Setup and Teardown Test Cases
- Tags and Service Level Agreements (SLAs)
- Test Cycles
- Test Plans
- Testing Browser Options
- Tooltip Testing
- Using the Test Palette
- Test Palette Introduction
- Control Test Steps
- Generate Test Case
- List Compare
- Page Object Cleaner
- Read Test Step
- String Test Steps
- UI Test Steps
- Using Custom APIs
- Callable Tests
- Data-Driven Testing
- Page Objects
- Block Locator Strategies
- Introduction to XPaths
- Creating an XPath
- Label Locator Strategies
- Maintaining Page Objects
- Mapping Non-Salesforce fields
- Page Object Operations
- Refresh and Reselect Field Locators in Test Builder
- Using Java Method Annotations for Custom Objects
- Applications Testing
- Database Testing
- Document Testing
- Email Testing
- Mobile Testing
- OrchestraCMS Testing
- Salesforce CPQ Testing
- ServiceMax Testing
- Skuid Testing
- Vlocity API Testing
- Webservices Testing
- Provar Manager
- How to Use Provar Manager
- Provar Manager Setup
- Provar Manager Integrations
- Release Management
- Test Management
- Test Operations
- Provar Manager and Provar Automation
- Setting Up a Connection to Provar Manager
- Object Mapping Between Automation and Manager
- How to Upload Test Plans, Test Plan Folders, Test Plan Instances, and Test Cases
- Provar Manager Filters
- Uploading Callable Test Cases in Provar Manager
- Uploading Test Steps in Provar Manager
- How to Know if a File in Automation is Linked in Test Manager
- Test Execution Reporting
- Metadata Coverage with Manager
- Provar Grid
- Introduction to Provar DevOps
- Introduction to Test Scheduling
- Apache Ant
- Configuration for Sending Emails via the Automation Command Line Interface
- Continuous Integration
- AutoRABIT Salesforce DevOps in Provar Test
- Azure DevOps
- Running a Provar CI Task in Azure DevOps Pipelines
- Configuring the Automation secrets password in Microsoft Azure Pipelines
- Parallel Execution in Microsoft Azure Pipelines using Multiple build.xml Files
- Parallel Execution in Microsoft Azure Pipelines using Targets
- Parallel execution in Microsoft Azure Pipelines using Test Plans
- Bitbucket Pipelines
- GitHub Actions
- Integrating GitHub Actions CI to Run Automation CI Task
- Remote Trigger in GitHub Actions
- Parameterization using Environment Variables in GitHub Actions
- Parallel Execution in GitHub Actions using Multiple build.xml Files
- Parallel Execution in GitHub Actions using Targets
- Parallel Execution in GitHub Actions using Test Plan
- Parallel Execution in GitHub Actions using Job Matrix
- GitLab Continuous Integration
- Travis CI
- Execution Environment Security Configuration
- Provar Jenkins Plugin
- Parallel Execution
- Running Provar on Linux
- Salesforce DX
- Team Foundation Server
- Version Control
- Masking Provar Credentials on CI
- Salesforce Testing
- Adding a Salesforce Connection
- Assert Page Error Messages on Add/Edit Product
- Internationalization Support
- List and Table Testing
- Salesforce Console Testing
- Salesforce Lightning Testing
- Salesforce Lightning Web Component (LWC) Locator Support
- Salesforce Release Updates
- Visualforce Testing
- Best Practices
- Salesforce Connection Best Practices
- Improve Your Metadata Performance
- Testing Best Practices
- Automation Planning
- Supported Testing Phases
- Provar Naming Standards
- Test Case Design
- Create records via API
- Avoid using static values
- Abort Unused Test Sessions/Runs
- Avoid Metadata performance issues
- Increase auto-retry waits for steps using a global variable
- Create different page objects for different pages
- The Best Ways to Change Callable Test Case Locations
- Working with the .testProject file and .secrets file
- Best practices for the .provarCaches folder
- Best practices for .pageObject files
- Configurations and Permissions
- Error Messages
- Provar Manager 3.0 Install Error Resolution
- Provar Manager Test Case Upload Resolution
- Administrator has Blocked Access to Client
- macOS Big Sur Upgrade
- Resolving Failed to Create ChromeDriver Error
- Resolving Jenkins License Missing Error
- Resolving Metadata Timeout Errors
- Test Execution Fails – Firefox Not Installed
- Selenium 4 Upgrade
- Licensing, Installation and Firewalls
- Test Builder and Test Cases
- Release Notes