Automation V3: Test Management
This section will teach you how to leverage Provar Manager to structure your testing activities and track relevant information.
Requirement Analysis
The Requirement Analysis phase is essential for understanding and defining the needs and expectations of stakeholders for the software being developed. In this phase, the development team works closely with the clients or end-users to gather, document, and analyze their requirements. This includes functional requirements that outline what the software should do and non-functional requirements that address performance, security, usability, and other quality attributes. Properly defining the requirements upfront helps ensure that everyone involved in the project understands what needs to be built and reduces the chances of misunderstandings or miscommunications down the line. This results in a more successful software development project with fewer revisions and changes, ultimately saving time and resources.
After analyzing the requirements, the first step is to create a Test Project by opening the Test Projects tab and clicking the New button. Give your test project a name and a key to identify, and don’t forget to fill in the Description field so everyone can understand what the test project is about.
The Test Project record page is divided into two main sections:
- Test Project Navigation shows related test plans, suites, and test cases.
- Test Project Details is where the main information can be found.
Test Planning
The Test Planning phase ensures that effective and efficient testing activities are carried out throughout development. A test strategy is created in this phase to guide the overall testing effort. This includes determining which types of testing (e.g., functional, performance, security) will be required, setting testing objectives and priorities, defining test cases and test data, allocating resources, and establishing a schedule for testing activities. Proper planning also involves selecting appropriate testing tools and environments and identifying the roles and responsibilities of the testing team members. By creating a comprehensive test plan, organizations can ensure that they can identify and address any potential issues early in the development process, which ultimately leads to higher-quality software and greater customer satisfaction.
Provar Manager comes with a comprehensive but customizable Test Plan template that can be used to track all the information you need.
To create a Test Plan, go to your Test Project record, click the dropdown arrow on the Test Project Navigation component, and select Create Test Plan. Enter a name for it and click Create.
The Test Plan record page is divided into three main sections:
- Documentation: This section includes space to define objectives, an overview of the system, definitions and acronyms that may be used across the plan, the scope and features to be tested, the environments under test, any assumptions or constraints, and the overall approach to testing, among other things.
- Execution: This section includes Test Plan Schedules and Test Cycles.
- Analytics: This section includes some charts related to daily executions and risk coverage.
At the top of the test plan, you will also find the Progress Tracker, which can help you quickly see if basic information like scope, testing types, test tools, or coverage has already been defined in the test plan.
Risk Management
Defining and managing risks is part of planning your testing activities, which can be done from the test plan record under the Documentation > Risks section.
Every risk has a description, an owner, and an impact assessment (likelihood and impact), which will produce a Risk Level. You can also record the mitigation strategy, approach, and proposed solution.
Finally, you can link test cases to risks to track your risk coverage.
Test Case Design
The Test Case Design phase focuses on developing detailed test cases used during testing activities to verify a software application’s functionality, performance, and reliability. In this phase, the testing team works closely with developers, business analysts, and other stakeholders to gather information about the system under test and its requirements.
Test cases are designed based on the requirements specifications, user stories, use cases, and other relevant documentation. Each test case should cover a specific functional or non-functional requirement and include clear and concise steps to reproduce the test, expected results, and acceptance criteria. The design of test cases also considers various testing techniques, such as boundary value analysis, equivalence partitioning, and decision table testing, to ensure comprehensive coverage of the software’s functionality.
Effective test case design requires a solid understanding of the software and the testing process. The resulting test cases will provide a foundation for executing successful tests throughout the development lifecycle and help to identify any defects or issues early in the development process, ultimately leading to higher-quality software and greater customer satisfaction.
Using Provar Manager, testers can create new test cases with ease from multiple places:
- Test Cases tab > New button
- Test Project Navigation component > Down arrow on Test Cases Not Linked To Test Suites > New Test Case option
- Test Project Navigation component > Down arrow on a test suite item > Add New Test Case option.
The Test Case record page is divided into four main sections:
- Details: This section allows you to define type, status, owner, and priority.
- Steps: This section allows you to define all test steps.
- Executions: This section lets you see the latest result per environment, average execution times, and historical test executions.
- Related: This section lets you see related test suites, defects, issues, and risks.
You can relate Test Cases to risks to track Risk Coverage by clicking the New button on the Covered Risks related list under the Related section.
You can also link Test Cases to Metadata Components by clicking on the Link to Metadata Components option in the menu.
BDD Test Case Generation
Provar Manager can leverage AI to generate Behaviour-Driven Development (BDD) test cases off the back of a user story, which can help you develop possible test scenarios.
| To use this feature, make sure you have saved your OpenAI API Key in Provar Manager’s Setup page (see the Setup section of this document). |
To generate BDD test cases, navigate to an Issue record representing a user story, ensure it has a description following the classic convention “as a… I want to… so that…” and click the Generate Test Cases button.
The pop-up will show the issue description used to generate the test cases. Click the Generate Test Cases button and wait for approximately 5 seconds.
Once the list of generated BDD test cases is shown, you can click the refresh icon to re-generate them if you don’t like them, or select the ones you want and click Save to create the test cases in Manager.
If you refresh the page, you will see that the test cases have been created and linked to the user story.
Managing Test Steps
You can easily add, remove, and reorder test steps using the Test Steps Manager on the test case record page.
- The Cancel button cancels any changes not saved.
- The Add Step button adds a new step at the bottom of the list.
- The Save button saves any changes made to the list.
- The Up and Down arrow buttons can be used to change the order of the step within the same hierarchical level.
- The Bin button deletes the step.
- The Clip button shows/hides the section where you can manage the step’s attachments.
- The Down/Right arrow buttons expand/collapse the step’s child steps.
- The Level Down button creates a child step.
Test Case Approvals
Teams may want to configure an approval process so that each test case requires approval to progress from one stage to another (e.g., Design to Pending Implementation).
Your Salesforce admin should be able to implement a Salesforce approval process that fits your requirements as a team.
Test Execution
The Test Execution phase focuses on running the test cases designed during the Test Case Design phase to validate a software application’s functionality, performance, reliability, and other quality attributes. In this phase, the testing team executes the planned tests using appropriate tools and environments to identify any defects or issues in the software under test.
Test execution can be performed manually, where a tester follows the steps outlined in each test case and records the results, or automated, where specialized tools (e.g., Provar Automation) execute predefined test scripts and report the results. Manual testing is typically used for exploratory, usability tests requiring human judgment or interaction. Automated testing, on the other hand, is ideal for regression testing, performance testing, and other repetitive tests.
Test execution should be organized with clear documentation of test results and any identified defects or issues. The testing team must follow the established test plan and cover all the defined test cases to cover the software’s functionality comprehensively. This phase is crucial for identifying any issues or bugs early in the development process and providing feedback to the development team, allowing them to make improvements before the software is released to customers.
Effective test execution contributes to the overall success of a software project by ensuring that the software is functioning as intended, meeting its requirements, and providing an excellent user experience. This results in increased customer satisfaction, reduced time to market, and ultimately leads to higher quality software.
Provar Manager helps you record the execution (manual or automated) and the results of your tests.
To start, go to a test plan record. Under its Execution section, you will find a list of test cycles where you can create new Test Cycles.
The Test Cycle record page is split into multiple sections:
- Details: This section lets you define the test environment status and track the start/end date times.
- Test Executions: This section allows you to manage test executions.
- Configuration: This section allows you to define the testing tool and test environment settings to be used.
- External Test Results: This section allows you to track imported test results.
Managing Test Cycles
The Test Executions Manager component found on every test cycle record’s Test Executions section allows you to do the following:
- Add test suites, automatically creating a test execution record for each test case in the test suite.
- Add individual test executions.
- Perform mass status changes on the selected test executions.
- Delete the selected test executions.
- Search test executions.
- Retry all executions, just the failed ones, or only the selected executions.
