Override Auto-Retry for Test Step

Provar users can now add a global variable to override the auto-retry properties for every test step. You can now override the auto-retry properties in a single place using a global variable rather than updating test steps individually. When executing a test case, the global variable will override the test step auto-retry configuration.

You can override the auto retry value at test step through different modes like using  a command line argument, using a system environment variable, or using a global variable.

It will be helpful where Provar customers have different environments. And, it is possible in different environments that they may have slow loading issues or any other issues like that. So, if they have to run test cases  in different environments  and they want to enable variable overrides only for a particular environment, they can give the environment overrides and need not change the values every time.

The Auto-Retry variable names are listed below:

• Auto RetryTimeout:
  • Global Variable: provar_autoretry_timeout
  • Environment Variable: PROVAR_AUTORETRY_TIMEOUT
  • Command Line: com.provar.autoretry.timeout
• Auto Retry Override:
  • Environment Variable: PROVAR_AUTORETRY_OVERRIDE
  • Command Line: com.provar.autoretry.override

Order of Precedence

The test step auto retry configuration can be overridden using with the order of precedence as given below:

1. Command-line variable
2. Environment variable
3. Global variable

Background

Provar has a functionality that the users can select on the test steps i.e. on the interaction steps like UI Assert step they can set the Auto Retry, along with the Timeout for the auto-retry. And, users can also enable or disable this at each step.

When Provar users executed their test cases in bulk, they sometimes saw failures in their test cases because of the sync issues. The auto-retry at specific steps isn’t a feasible solution to apply at every step, when the number of test cases are high.  

How to override auto-retry properties in Provar

To override the auto-retry value at test step we have used two variables as shown in the screenshot below:

1. Auto-retry Timeout
2. Auto-retry Override

In Auto-retry timeout, we can provide a timeout value through 3 different modes:

• through a Provar global variable
• through a system environment variable  
• through a command line argument

And, same with auto-retry override. The auto-retry override is used to enable and disable the auto-retry value at test step through these 3 different modes given above.

We have used two variables here: provar_autoretry_override and  provar_autoretry_timeout.

• The  provar_autoretry_override variable is to enable or disable the auto-retry on each step . 
• The provar_autoretry_timeout variable is to set the time-out value i.e. to override the timeout value.

For example, we have created an environment variable provar_autoretry_override.

This variable is case sensitive and has 3 allowed values as listed below:

• ENABLE_ALL – it will enable the auto-retry for each and every step, irrespective of whatever is given in the test step.
• DISABLE_ALL – it will disable the auto-retry for each and every step, irrespective of whatever is given in the test step.
• NONE –  it will not be overriding the enabling or disabling of auto-retry but it will just be overriding the time that is given in the provar_autoretry_timeout variable. 

Note: A warning is displayed if any other values are used. For example, by mistake if the user gives the value as DISABLE_ALLS then it will give us a warning that it is an incorrect value for that particular variable and the accepted values are ENABLE_ALL, DISABLE_ALL and NONE. Also, if a user enters a negative value say (-60), a warning message is shown to enter a positive integer.

If the value is ENABLE_ALL, it will enable the auto-retry for each and every step, irrespective of whatever is given in the test step. For example, we have given provar_autoretry_override as ENABLE_ALL and as you can see in this test case on this particular step, we don’t have the auto-retry enabled. Let’s try and execute this. Click Run. 

Since we have given ENABLE_ALL, in the overrides, let’s first see the logs. On the very first connect step, we will get the logs for auto-retry, if the overrides are there.

You can see the logs for auto-retry overrides.  These are the basic logs, for details we have to click on Show Details which will then give the overrides that have been applied. So, you can see there are two auto-retry’s applied.

One is the system environment variable (as we can see in the logs.) which is provar_autoretry_timeout i.e. 60.

And, the other one is the Provar global variable for the auto-rety override; provar_autoretry_override which is ENABLE_ALL.

So, now you can see since we have given this as ENABLE_ALL, even though on this particular step; the auto-retry is disabled but still it is doing the auto-retry when it is not able to find that particular field ; that is because we have enabled the auto-retry here. And, it will continue doing it until the time-out and then it will just fail because the particular screen is not matching with this.

And, we can also give it in the system variables.

In the environment variables, we  have already set up and already given this auto-retry PROVAR_AUTORETRY_TIMEOUT which is the corresponding environment variable for provar_autoretry_timeout.

Note: There is a precedence that has to be followed in case these overrides have been given at more than one place.

For example, if we have given an override in the global variable as well as the environment variable, then the environment variable will take precedence. And, similarly, if we have also given it via command line, i.e. if we are running  with ANT, then it will take precedence above all.

So, that’s why you can see even in this test run, we had a provar_autoretry_timeout as “30” in the Provar global variable, still while running the test it took the time as “60” and it took it from the environment variable. So, we can get to know from where Provar picked these particular overrides by looking at the variables names. 

If we are running from ANT, the variable which we’ll be providing will be com.provar.autoretry.timeout.

But, if we are making a Provar global variable for the same, we will be making a variable with name provar_autoretry_timeout. Similarly, for the system environment variables, if we want to give it in the system environment variables or in CI/CD, we have to give the variable as PROVAR_AUTORETRY_TIMEOUT.