Automation V3: CircleCI Orbs

Overview of CircleCI Orbs

CircleCI orbs are open-source and shareable packages of configuration elements, including jobs, commands, and executors. Using orbs makes customizing CircleCI configuration simple.

The key steps to set up the CircleCI Orb integration are as follows –

Set up the project repository.
Customize the Automation build.xml file.
Configure the CircleCI config file.
How to build a CircleCI Pipeline.

Prerequisites in CircleCI Orbs Git Repository

This support article assumes you’ve already integrated your Automation project with a Git repository, as documented here. The details provided in this support article include which artifacts to commit to your Git repository.

The steps in this support article can be completed using a free CircleCI account. Sign-up is available here, or you can use an existing account.

Note: Currently, the CircleCI can only be used with GitHub or Bitbucket repositories.

The code repositories supported by CircleCI can be found on their integrations page.

Steps to Set up the CircleCI Orb Integration

The detailed steps to set up the CircleCI integration are given below.

In the example, we have used the following tools to create a Continuous Integration (CI) pipeline as in the screenshot given below. You can adapt this to your environment.

Step 1: Set up the project repository.

We will configure the Version Control System (VCS) repository for this step. This public repository contains an example of a CircleCI configuration file and an Automation project. The minimum requirements are as follows (looking at the repository home).

-ProvarProject (containing .testproject, tests folder, and build files)
   -.settings
   -ANT
      -build_circleci.xml   // can be named anything, just use this one as an example.
  -META-INFS
      -MANIFEST.MF
  -plans	// not required, but our main example uses a build file with a test plan.
      -Any test plan folders or test plans
  -tests
      -Any tests or test folders
  -templates
      -Any Test data
  -src
      -pageobjects
      -customapis
  -.secrets     // Make sure your project is encrypted prior to checking in this file.
  -.testproject
  -.classpath
  -build.properties
  -.project
-.circleci
  -config.yml

Step 2: Customize the Automation build.xml file.

Example build_circleci.xml file (using test plans):

Example build_circleci.xml file (without test plans):

Step 3: Configure the CircleCI config file.

Example of config.yml file:

We suggest that you begin with our build.xml and config.yml files as templates. However, you don’t need to match our example build.xml and config.yml files exactly.

There are some common environment variables and parameters referenced to make things easy. You can modify the following parameters according to your project:

-test_plan: The name of the test plan in the plans/ folder of your Automation project. You can leave this parameter blank if you are not using a test plan. For information on test plans, please refer to this page.

-email_target: The email address of the person to send test results/reports.

-build_file: The name of the build file in the ANT/ folder of your Automation Project you want to execute.

-environment: The name of the test environment you want to override your connections/variables within Automation.

-project_name: The name of the folder in your repository home that contains the Automation project.

-test_folder: The name of the test folder in your project that you want to execute (This can be specific or general as you prefer). If you use a test plan, you can leave this parameter blank.

Make sure your project is checked into the VCS of your choice before you continue.

Step 4: How to build a CircleCI Pipeline.

Please refer to this Orb link to build a CircleCI pipeline. For this article, refer to the run_provar_command example as a base. The steps to develop your CircleCI pipeline to run Automation tests are below.

Step 4a: Set up the CircleCI Project.

Navigate to the CircleCI project page to set up your project. CircleCI will prompt you to log in or create a new account if you haven’t already. Then, select your organization through GitHub or BitBucket to continue. Find the repository that contains your test project (as configured in the above section) and choose Set Up Project.

Step 4b: Build the CircleCI Pipeline.

Select Use Existing Config and click Start Building to initialize the pipeline. If you don’t have a config.yml file in the “.circleci” folder of the root of your repository, you’ll need to add one before you can begin building your pipeline.

Note: This will kick off an initial build immediately if you already have a config.yml file in your repository. Make sure to add the example config.yml listed above if you do not already have it checked in. The build may fail on the first run, but we still have a few more steps to configure in the pipeline.

Step 4c: Add the Environment Variables.

Navigate to Project Settings > Environment Variables. You can add project or job-specific environment variables here. In this case, we created an environment variable for the Automation Secrets password we used to encrypt the project. We can access environment variables created here in the build file of our Automation project. In the example repository above, the build_circleci.xml refers to an environment variable, as given below.

We have created a CircleCI Project Environment Variable named ProvarSecretsPassword. We can set the encryption secrets password here. You can also make other optional environment variables for –

Test plan
Project name
Build file
Test environment
Email target

Step 4d: Store the Test results.

The last step, store_test_results, will store the test results in the pipeline details. If your job succeeds, you’ll see the results below.

And if all the tests pass, you will see the results below.

All these files are displayed as hyperlinks and can be opened directly in CircleCI. For example, you can examine the PDF report by clicking on ProvarProject/ANT/Results/Test_Run_Report.pdf.