Override Auto-Retry for Test Step
Provar users can now add a global variable to override the auto-retry properties for every test step. You can now override the auto-retry properties in a single place using a global variable rather than updating test steps individually. When executing a test case, the global variable will override the test step auto-retry configuration.
You can override the auto retry value at test step through different modes like using a command line argument, using a system environment variable, or using a global variable.
It will be helpful where Provar customers have different environments. And, it is possible in different environments that they may have slow loading issues or any other issues like that. So, if they have to run test cases in different environments and they want to enable variable overrides only for a particular environment, they can give the environment overrides and need not change the values every time.
The Auto-Retry variable names are listed below:
- Auto RetryTimeout:
- Global Variable: provar_autoretry_timeout
- Environment Variable: PROVAR_AUTORETRY_TIMEOUT
- Command Line: com.provar.autoretry.timeout
- Auto Retry Override:
- Environment Variable: PROVAR_AUTORETRY_OVERRIDE
- Command Line: com.provar.autoretry.override
Order of Precedence
The test step auto retry configuration can be overridden using with the order of precedence as given below:
- Command-line variable
- Environment variable
- Global variable
Provar has a functionality that the users can select on the test steps i.e. on the interaction steps like UI Assert step they can set the Auto Retry, along with the Timeout for the auto-retry. And, users can also enable or disable this at each step.
When Provar users executed their test cases in bulk, they sometimes saw failures in their test cases because of the sync issues. The auto-retry at specific steps isn’t a feasible solution to apply at every step, when the number of test cases are high.
How to override auto-retry properties in Provar
To override the auto-retry value at test step we have used two variables as shown in the screenshot below:
- Auto-retry Timeout
- Auto-retry Override
In Auto-retry timeout, we can provide a timeout value through 3 different modes:
- through a Provar global variable
- through a system environment variable
- through a command line argument
And, same with auto-retry override. The auto-retry override is used to enable and disable the auto-retry value at test step through these 3 different modes given above.
We have used two variables here: provar_autoretry_override and provar_autoretry_timeout.
- The provar_autoretry_override variable is to enable or disable the auto-retry on each step .
- The provar_autoretry_timeout variable is to set the time-out value i.e. to override the timeout value.
For example, we have created an environment variable provar_autoretry_override.
This variable is case sensitive and has 3 allowed values as listed below:
- ENABLE_ALL – it will enable the auto-retry for each and every step, irrespective of whatever is given in the test step.
- DISABLE_ALL – it will disable the auto-retry for each and every step, irrespective of whatever is given in the test step.
- NONE – it will not be overriding the enabling or disabling of auto-retry but it will just be overriding the time that is given in the provar_autoretry_timeout variable.
Note: A warning is displayed if any other values are used. For example, by mistake if the user gives the value as DISABLE_ALLS then it will give us a warning that it is an incorrect value for that particular variable and the accepted values are ENABLE_ALL, DISABLE_ALL and NONE. Also, if a user enters a negative value say (-60), a warning message is shown to enter a positive integer.
If the value is ENABLE_ALL, it will enable the auto-retry for each and every step, irrespective of whatever is given in the test step. For example, we have given provar_autoretry_override as ENABLE_ALL and as you can see in this test case on this particular step, we don’t have the auto-retry enabled. Let’s try and execute this. Click Run.
Since we have given ENABLE_ALL, in the overrides, let’s first see the logs. On the very first connect step, we will get the logs for auto-retry, if the overrides are there.
You can see the logs for auto-retry overrides. These are the basic logs, for details we have to click on Show Details which will then give the overrides that have been applied. So, you can see there are two auto-retry’s applied.
One is the system environment variable (as we can see in the logs.) which is provar_autoretry_timeout i.e. 60.
And, the other one is the Provar global variable for the auto-rety override; provar_autoretry_override which is ENABLE_ALL.
So, now you can see since we have given this as ENABLE_ALL, even though on this particular step; the auto-retry is disabled but still it is doing the auto-retry when it is not able to find that particular field ; that is because we have enabled the auto-retry here. And, it will continue doing it until the time-out and then it will just fail because the particular screen is not matching with this.
And, we can also give it in the system variables.
In the environment variables, we have already set up and already given this auto-retry PROVAR_AUTORETRY_TIMEOUT which is the corresponding environment variable for provar_autoretry_timeout.
Note: There is a precedence that has to be followed in case these overrides have been given at more than one place.
For example, if we have given an override in the global variable as well as the environment variable, then the environment variable will take precedence. And, similarly, if we have also given it via command line, i.e. if we are running with ANT, then it will take precedence above all.
So, that’s why you can see even in this test run, we had a provar_autoretry_timeout as “30” in the Provar global variable, still while running the test it took the time as “60” and it took it from the environment variable. So, we can get to know from where Provar picked these particular overrides by looking at the variables names.
If we are running from ANT, the variable which we’ll be providing will be com.provar.autoretry.timeout.
But, if we are making a Provar global variable for the same, we will be making a variable with name provar_autoretry_timeout. Similarly, for the system environment variables, if we want to give it in the system environment variables or in CI/CD, we have to give the variable as PROVAR_AUTORETRY_TIMEOUT.
- Provar Automation
- Installing Provar Automation
- Updating Provar Automation
- Using Provar Automation
- API Testing
- Behavior-Driven Development
- Creating and Importing Projects
- Creating Test Cases
- Custom Table Mapping
- Debugging Tests
- Defining a Namespace Prefix on a Connection
- Defining Proxy Settings
- Environment Management
- Exporting Test Cases into a PDF
- Exporting Test Projects
- Override Auto-Retry for Test Step
- Managing Test Steps
- Namespace Org Testing
- Provar Automation
- Provar Test Builder
- Refresh and Recompile
- Reintroduction of CLI License Check
- Reload Org Cache
- Running Tests
- Searching Provar with Find Usages
- Secrets Management and Encryption
- Setup and Teardown Test Cases
- Tags and Service Level Agreements (SLAs)
- Test Cycles
- Test Plans
- Testing Browser Options
- Tooltip Testing
- Using the Test Palette
- Test Palette Introduction
- Control Test Steps
- Generate Test Case
- List Compare
- Page Object Cleaner
- Read Test Step
- String Test Steps
- UI Test Steps
- Using Custom APIs
- Callable Tests
- Data-Driven Testing
- Page Objects
- Block Locator Strategies
- Introduction to XPaths
- Creating an XPath
- Label Locator Strategies
- Maintaining Page Objects
- Mapping Non-Salesforce fields
- Page Object Operations
- Refresh and Reselect Field Locators in Test Builder
- Using Java Method Annotations for Custom Objects
- Applications Testing
- Database Testing
- Document Testing
- Email Testing
- Mobile Testing
- OrchestraCMS Testing
- Salesforce CPQ Testing
- ServiceMax Testing
- Skuid Testing
- Vlocity API Testing
- Webservices Testing
- Provar Manager
- Provar Manager Setup and User Guide
- Installing Provar Manager
- Configuring Provar Manager
- How to Use Provar Manager
- Managing Your Testing Life Cycle
- Provar Manager Test Execution
- Test Executions and Defect Management
- Provar Manager Test Coverage
- How to Integrate Provar Manager
- Setting Up a Connection to Provar Manager
- Object Mapping Between Automation and Manager
- How to Upload Test Plans, Test Plan Folders, Test Plan Instances, and Test Cases
- Provar Manager Filters
- Uploading Callable Test Cases in Provar Manager
- Uploading Test Steps in Provar Manager
- How to Know if a File in Automation is Linked in Test Manager
- Test Execution Reporting
- Provar Manager Plugins
- Uploading Existing Manual Test Cases to Provar Manager with DataLoader.Io
- Metadata Coverage with Manager
- Provar Grid
- Introduction to Provar DevOps
- Introduction to Test Scheduling
- Apache Ant
- Configuration for Sending Emails via the Automation Command Line Interface
- Continuous Integration
- AutoRABIT Salesforce DevOps in Provar Test
- Azure DevOps
- Running a Provar CI Task in Azure DevOps Pipelines
- Configuring the Automation secrets password in Microsoft Azure Pipelines
- Parallel Execution in Microsoft Azure Pipelines using Multiple build.xml Files
- Parallel Execution in Microsoft Azure Pipelines using Targets
- Parallel execution in Microsoft Azure Pipelines using Test Plans
- Bitbucket Pipelines
- GitHub Actions
- Integrating GitHub Actions CI to Run Automation CI Task
- Remote Trigger in GitHub Actions
- Parameterization using Environment Variables in GitHub Actions
- Parallel Execution in GitHub Actions using Multiple build.xml Files
- Parallel Execution in GitHub Actions using Targets
- Parallel Execution in GitHub Actions using Test Plan
- Parallel Execution in GitHub Actions using Job Matrix
- GitLab Continuous Integration
- Travis CI
- Execution Environment Security Configuration
- Provar Jenkins Plugin
- Parallel Execution
- Running Provar on Linux
- Salesforce DX
- Team Foundation Server
- Version Control
- Masking Provar Credentials on CI
- Salesforce Testing
- Adding a Salesforce Connection
- Assert Page Error Messages on Add/Edit Product
- Internationalization Support
- List and Table Testing
- Salesforce Console Testing
- Salesforce Lightning Testing
- Salesforce Lightning Web Component (LWC) Locator Support
- Salesforce Release Updates
- Visualforce Testing
- Best Practices
- Salesforce Connection Best Practices
- Improve Your Metadata Performance
- Testing Best Practices
- Automation Planning
- Supported Testing Phases
- Provar Naming Standards
- Test Case Design
- Create records via API
- Avoid using static values
- Abort Unused Test Sessions/Runs
- Avoid Metadata performance issues
- Increase auto-retry waits for steps using a global variable
- Create different page objects for different pages
- The Best Ways to Change Callable Test Case Locations
- Working with the .testProject file and .secrets file
- Best practices for the .provarCaches folder
- Best practices for .pageObject files
- Configurations and Permissions
- Error Messages
- Provar Manager 3.0 Install Error Resolution
- Provar Manager Test Case Upload Resolution
- Administrator has Blocked Access to Client
- macOS Big Sur Upgrade
- Resolving Failed to Create ChromeDriver Error
- Resolving Jenkins License Missing Error
- Resolving Metadata Timeout Errors
- Test Execution Fails – Firefox Not Installed
- Licensing, Installation and Firewalls
- Test Builder and Test Cases
- Release Notes