Setting up Continuous Integration
Once the build file is ready and the version control has been set up (if required), tests can be scheduled using a Continuous Integration (CI) Server such as Jenkins.
The role of the CI Server is to run the regression pack (from a local machine or via a version control repository). The CI Server will trigger the Provar automation pack on a defined schedule or based on an external trigger, such as a new build or deployment.
Installing Jenkins
Start by downloading and installing Jenkins.
To download Jenkins, visit the Jenkins website. Click Download and select the file which suits your operating system.
To install Jenkins, first move the downloaded Jenkins.war file to an appropriate directory, e.g. C:\Program Files\Jenkins.
Then, open command prompt in this directory by clicking on the filepath and typing cmd.
Once command prompt has opened, type and run the command java –jar Jenkins.war.
Once the command has run, a password will be generated for logging into Jenkins.
In this example, the password is cd3f375e0baa47f4b4a5591f1e920bfe.
Note: This password can also be checked by referring to the InitialAdminPassword.txt file which has been created in the \.jenkins\secrets folder.
Jenkins Setup
Next, open your browser and navigate to URL: localhost:8080.
The following screen will appear.
Enter your password then click Continue.
On the next screen, choose the option Select plugins to install.
Some plugins will be defaulted already. These can be left as defaulted.
Make sure that the following plugins are ticked:
Ant Plugin (under Build Tools).
JUnit Plugin (under Build Analysis and Reporting).
If you want Jenkins to download the regression pack from your Version Control repository, also tick the relevant plugin under Source Code Management (e.g. the Git plugin).
Click Install.
When this is completed, click Continue as admin on the following screen (instead of filling out Create First Admin User).
Note: The next time you log in, your username will be admin and your password will be the one generated earlier.
On the next screen, click Start using Jenkins to complete the setup.
Scheduling a Run
To schedule test in Jenkins, start by clicking the New Item option on the left-hand side.
Provide a name for the new project, e.g. ProvarProject, then select the Freestyle project option and click OK.
If you are using version control, navigate to Source Code Management and select Git. (If you are not using version control, skip ahead to Build Triggers).
Enter the Repository URL, which can be copied by clicking the Clone or Download button against the repository.
Then, provide Credentials (if applicable – this is only needed for private repositories).
To define the execution schedule, navigate to Build Triggers. Here, you can define when you want Provar tests to be run. Choose Build periodically if you want to set up a schedule.
(Alternatively, you can also select Trigger builds remotely or Build after other projects are built if you want to trigger the execution using a script or following completion of other project builds.)
If you choose the Build periodically option, you will then need to enter an expression in Schedule to define when the tests should be run. This is done using a CRON expression.
To create this expression, you can write your own from scratch or copy or adapt one of the examples below.
Example 1:
H H(3-4) * * *
Schedule: Every Day of the week, every Month in a year, every Day in a month at a time between 3 – 4 am in the morning at any minute.
Example 2:
H (30 -45) 3 * * *
Schedule: every Day of the week, every Month in a year, every Day in a month at 3 am in the morning between 30 -45 minutes.
Example 3:
*/5 * * * * :
Schedule: every 5 minutes.
Example 4:
0 8 * * *
Schedule: every day at 8h 00.
Enter your preferred syntax in Schedule, then check the validation message underneath to confirm the syntax is correct and the schedule has been defined as expected.
The ‘Would last have run at…’ information will show the most recent (past) run and next upcoming run, based on your syntax.
Finally, navigate to Build and click the Add build step dropdown, then choose Invoke ANT.
Leave Targets blank, unless you know that your build file has multiple targets. (If your build file was generated using the default settings supplied in Run Under ANT, leaving Targets blank will work fine.)
If you have modified the Build File so that there are multiple targets, enter the name of the desired target, e.g. runtests.
Then, choose the Advanced… option and provide the build.xml path.
If the project is fetched from git, it should use the JENKINS_HOME variable (see screenshot above). If the project is local, it can be a filepath, e.g. C:\Users\Heather\Documents\GitHub\dev\DevOps\ANT\build.xml.
Click Save.
Test your build by clicking the Build Now option in the left-hand sidebar.
Build progress will be shown within Jenkins. If the build is successful, results will also be emailed as defined in the original build file.
Configuring test reports
Once your build generates test results, Jenkins’ JUnit plugin can be used to display these results graphically.
When Provar tests are executed via CI, Provar automatically generates a JUnit.xml file alongside its HTML and PDF reports. Using the JUnit plugin, this JUnit.xml file can be converted into a graphical visualization of the test results. The plugin also provides a web UI for viewing test reports and tracking failures.
Since Provar generates the JUnit.xml file automatically, very little further set up is required. One thing to keep in mind is that the Results folder, which contains this JUnit.xml, should be generated or placed in Jenkins’ workspace folder.
For example, if the project is inside Jenkins’ workspace, define the results path as shown below.
Follow the steps below to get the JUnit plugin up and running.
In Jenkins, click on the dropdown arrow on your Job Name and select Configure.
Navigate to ‘Post-build Actions’, then click the Add post-build action dropdown and select Publish JUnit test result report.
In Test report XMLs, set the path of the JUnit.xml file (note that this is generated inside your Results folder). This XML file should also be inside the workspace of the job of Jenkins. Enter the relative path of the results folder generated after execution:
Note: Before the first execution of your build, this field will show an error, since the file is not generated until after the execution.
Then, save and trigger the job.
Once execution completes, you can view JUnit results by clicking Test Result on the Execution View screen.
On the next screen, a History link is available to show the historical view or trend of execution.
Note: The historical data is visible until two or more execution cycles.
This completes the CI set up.