GitLab Continuous Integration
This support article provides step-by-step instructions to execute Automation test cases in GitLab Continuous Integration using its built-in pipeline.
Continuous Integration (CI) works to integrate code from your team in a shared repository. Developers share their new code in a merge (pull) request, which triggers a pipeline to build, test and validate the new code before merging the changes in your repository.
Continuous Delivery delivers CI-validated code to your application.
Prerequisites in GitLab Continuous Integration
- Provar ANT zip file.
- Provar project with at least one test case and associated ANT build.xml file.
- The Provar project is pushed in Gitlab as a public or private repository.
- Provar Execution Only (or Floating) license key pushed into the .license folder of the Github repository. Please only upload license keys on private repositories to avoid unlicensed usage by third parties. Unlicensed access will be tracked and can cause you to be locked out of Provar until a new license key is re-issued to you.
Steps to Create a Project in GitLab
The following steps are completed using a free as well as paid GitLab account:
Step 1: Login into your GitLab account.
Step 2: Create a new project and mention the project name, description, and visibility level.
Step 3: Clone this new repository into your local machine.
Step 4: Add three folders in this repository:
- ProvarProject: It contains provar test cases and a build.xml file to execute those tests.
- Provar: It contains .license folder.
- Provar_Home: This folder is created by extracting Provar_Ant_<Version>.zip file from the latest version of Provar, which can be downloaded from the Provar community. This folder contains ant and lib folders.
Step 5: Add .gitlab-ci.yml file in the base location of the repository.
Screenshot of folder structure:
Steps to Create .gitlab-ci.yml File
For Gitlab CI to build your project, you will need to add .gitlab-ci.yml configuration file to the root directory of your repository.
If .gitlab-ci.yml is not in your repository or is not valid YAML, GitLab CI will ignore it.
Here is an example file:
image: "ubuntu:latest" before_script: - apt-get update && apt-get install -y xvfb wget curl unzip gnupg -qq - if [ ! -d Provar_Home ]; then curl -O https://download.provartesting.com/latest/Provar_ANT_latest.zip && unzip Provar_ANT_latest.zip -d Provar_Home && rm Provar_ANT_latest.zip; else echo "Found Provar_Home"; fi - apt-get install -y openjdk-8-jdk - apt-get install -y ant - java -version - ant -version - wget -q -O - https://dl.google.com/linux/linux_signing_key.pub | apt-key add - - echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" | tee /etc/apt/sources.list.d/chrome.list - apt-get update -y - apt-cache search chrome - apt-get install google-chrome-stable -y - export DISPLAY=:99.0 - wget -O /etc/init.d/xvfb https://gist.githubusercontent.com/axilleas/3fc13e0c90ad9f58bee903a41e8a6d48/raw/169a60010635e05eaa902c5f3b4393321f2452f0/xvfb - chmod 0755 /etc/init.d/xvfb - sh -e /etc/init.d/xvfb start build: variables: PROVAR_HOME: "$CI_PROJECT_DIR/Provar_Home" LICENSE_PATH: "$CI_PROJECT_DIR/ProvarLicense" script: - cd demo/ANT - xvfb-run ant -Dprovar.home=$PROVAR_HOME -Dlicense.path=$LICENSE_PATH -f build.xml cache: paths: - $PROVAR_HOME/.provarCaches - $PROVAR_HOME artifacts: paths: - $CI_PROJECT_DIR/demo/ANT/Results reports: junit: - $CI_PROJECT_DIR/demo/ANT/Results/JUnit.xml
Just walking through the example script above:
Firstly, we must mention the base docker image, as we support Java 8, and test case execution requires ANT. So, we have taken an official image of ant frekele/ant:1.10.3-jdk8.
Describe before_script:
As we need to execute our UI test cases on a browser, that is why the chrome installation is included.
To execute test cases headlessly, we also need to install xvfb, so before performing the actual test script section, we are installing xvfb and running the xvfb service.
The build step contains variables like PROVAR_HOME and the script required to execute test cases using the command xvfb-run ant -Dprovar.home=$PROVAR_HOME -Dlicense.path=$LICENSE_PATH -f ProvarProject/ANT/build.xml.
Parameter Changes in build.xml
Edit the build.xml file.
- provar.home: value is ${env.PROVAR_HOME}
- testproject.home: value is Project name i.e. ProvarProject
- testproject.results: It’s an It’slt path ProvarProject/Results
- license.path: It’s where the .license folder is located. (The path is provided in.gitlab-ci.yml.)
See the Example build.xml
<project default="runtests"> <property environment="env"/> <property name="provar.home" value="${env.PROVAR_HOME}"/> <property name="testproject.home" value="ProvarProject"/> <property name="testproject.results" value="ProvarProject/Results"/> <property name="license.path" value=""/> <echo message="provar home is:${provar.home}"/> <taskdef name="Provar-Compile" classname="com.provar.testrunner.ant.CompileTask" classpath="${provar.home}/ant/ant-provar.jar"/> <taskdef name="Run-Test-Case" classname="com.provar.testrunner.ant.RunnerTask" classpath="${provar.home}/ant/ant-provar.jar;${provar.home}/ant/ant-provar-bundled.jar;${provar.home}/ant/ant-provar-sf.jar"/> <taskdef name="Test-Cycle-Report" classname="com.provar.testrunner.ant.TestCycleReportTask" classpath="${provar.home}/ant/ant-provar.jar;${provar.home}/ant/ant-provar-bundled.jar;${provar.home}/ant/ant-provar-sf.jar"/> <target name="runtests"> <Provar-Compile provarHome="${provar.home}" projectPath="${testproject.home}"/> <Run-Test-Case provarHome="${provar.home}" projectPath="${testproject.home}" resultsPath="${testproject.results}" resultsPathDisposition="Replace" testEnvironment="" webBrowser="Chrome_Headless" webBrowserConfiguration="Full Screen" excludeCallableTestCases="true" salesforceMetadataCache="Reload" projectCachePath="${provar.home}/.provarCaches" licensePath="${license.path}/.licenses" testOutputlevel="WARNING" pluginOutputlevel="WARNING" stopTestRunOnError="false" invokeTestRunMonitor="true" > <fileset dir="../tests/sample"/> </Run-Test-Case> </target> </project>
Trigger the First Build
Once you push all changes into GitLab Continuous Integration, the process automatically gets triggered.
The status of the execution can be checked on CI/CD -> Pipelines.
Caching in GitLab CI/CD
GitLab CI/CD provides a caching mechanism that saves time when your jobs run. For more information, check this link.
https://docs.gitlab.com/ee/ci/caching/
.provarCaches is used to speed up a given job in subsequent pipelines. Shortly, include it in the cache, Provar will not download the metadata during each test execution, reducing execution time.
Including the Provar_Home directory in the cache will eliminate the need to download and unzip the ANT zip file between jobs.
image: "frekele/ant:1.10.3-jdk8" before_script: - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - - echo "deb http://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google.list - apt-get update -yqqq - apt-get install -qq google-chrome-stable - apt-get install -y xvfb -qq - if [ ! -d Provar_Home ]; then curl -O https://download.provartesting.com/latest/Provar_ANT_latest.zip && unzip Provar_ANT_latest.zip -d Provar_Home; else echo "Found Provar_Home"; fi build: variables: PROVAR_HOME: "$CI_PROJECT_DIR/Provar_Home" LICENSE_PATH: "$CI_PROJECT_DIR/Provar" script: - cd demo/ANT - xvfb-run ant -Dprovar.home=$PROVAR_HOME -Dlicense.path=$LICENSE_PATH -f build.xml cache: paths: - $PROVAR_HOME/.provarCaches - $PROVAR_HOME artifacts: paths: - demo/ANT/Results
Test Results
After successful execution, generate test reports in the location mentioned in the build.xml testproject.results parameter.
<property name=”testproject.results” value=”ProvarProject/Results”/>
Refer to this link for help customizing the reporting options in Provar.
To get the reports folder as artifacts in the GitLab pipeline, add the below code in your .gitlab-ci.yml file.
artifacts: paths: - ProvarProject/Results
Download artifacts from the button on the right side of the screenshot below.
If you don’t place Provar_Home in GIT, add these lines of code in the before_script tag.
- curl -O https://download.provartesting.com/latest/Provar_ANT_latest.zip - unzip -o Provar_ANT_latest.zip -d Provar_Home - rm Provar_ANT_latest.zip
- Provar Automation
- Installing Provar Automation
- Updating Provar Automation
- Using Provar Automation
- API Testing
- Behavior-Driven Development
- Creating and Importing Projects
- Creating Test Cases
- Custom Table Mapping
- Functions
- Debugging Tests
- Defining a Namespace Prefix on a Connection
- Defining Proxy Settings
- Environment Management
- Exporting Test Cases into a PDF
- Exporting Test Projects
- Override Auto-Retry for Test Step
- Managing Test Steps
- Namespace Org Testing
- NitroX
- Provar Automation
- Provar Test Builder
- Refresh and Recompile
- Reintroduction of CLI License Check
- Reload Org Cache
- Reporting
- Running Tests
- Searching Provar with Find Usages
- Secrets Management and Encryption
- Setup and Teardown Test Cases
- Tags and Service Level Agreements (SLAs)
- Test Cycles
- Test Plans
- Testing Browser Options
- Tooltip Testing
- Using the Test Palette
- Using Custom APIs
- Callable Tests
- Data-Driven Testing
- Page Objects
- Block Locator Strategies
- Introduction to XPaths
- Creating an XPath
- JavaScript Locator Support
- Label Locator Strategies
- Maintaining Page Objects
- Mapping Non-Salesforce fields
- Page Object Operations
- ProvarX™
- Refresh and Reselect Field Locators in Test Builder
- Using Java Method Annotations for Custom Objects
- Applications Testing
- Provar Manager
- Provar Manager Setup and User Guide
- Installing Provar Manager
- Configuring Provar Manager
- How to Use Provar Manager
- Managing Your Testing Life Cycle
- Provar Manager Test Execution
- Test Executions and Defect Management
- Provar Manager Test Coverage
- How to Integrate Provar Manager
- Setting Up a Connection to Provar Manager
- Object Mapping Between Automation and Manager
- How to Upload Test Plans, Test Plan Folders, Test Plan Instances, and Test Cases
- Provar Manager Filters
- Uploading Callable Test Cases in Provar Manager
- Uploading Test Steps in Provar Manager
- How to Know if a File in Automation is Linked in Test Manager
- Test Execution Reporting
- Provar Manager Plugins
- Uploading Existing Manual Test Cases to Provar Manager with DataLoader.Io
- Provar Grid
- DevOps
- Introduction to Test Scheduling
- Apache Ant
- Configuration for Sending Emails via the Automation Command Line Interface
- Continuous Integration
- AutoRABIT Salesforce DevOps in Provar Test
- Azure DevOps
- Running a Provar CI Task in Azure DevOps Pipelines
- Configuring the Automation secrets password in Microsoft Azure Pipelines
- Parallel Execution in Microsoft Azure Pipelines using Multiple build.xml Files
- Parallel Execution in Microsoft Azure Pipelines using Targets
- Parallel execution in Microsoft Azure Pipelines using Test Plans
- Bitbucket Pipelines
- CircleCI
- Copado
- Docker
- Flosum
- Gearset DevOps CI/CD
- GitHub Actions
- Integrating GitHub Actions CI to Run Automation CI Task
- Remote Trigger in GitHub Actions
- Parameterization using Environment Variables in GitHub Actions
- Parallel Execution in GitHub Actions using Multiple build.xml Files
- Parallel Execution in GitHub Actions using Targets
- Parallel Execution in GitHub Actions using Test Plan
- Parallel Execution in GitHub Actions using Job Matrix
- GitLab Continuous Integration
- Travis CI
- Jenkins
- Execution Environment Security Configuration
- Provar Jenkins Plugin
- Parallel Execution
- Running Provar on Linux
- Reporting
- Salesforce DX
- Git
- Team Foundation Server
- Version Control
- Salesforce Testing
- Best Practices
- Troubleshooting
- Release Notes