Quality and Efficiency Value Driver

Test Execution Success Rate

This metric tracks the overall success rate for all test executions for a given time period. This includes any Apex Tests, Manual Tests, Provar Tests, and other tests as well.

The metric card here shows the current maturity tier, as defined by the KPI for this metric.

The small circle with the % (e.g. 90%) displays the percentage of the overall KPI achieved for this time range. Test execution success is not prorated, so the % will remain the same for any given filter size, assuming the success rate is the same.

The larger value displays the overall success rate for the given time range for any test executions executed in that time.

If you click into the drill-down for this metric, you’ll see the current value, a trend chart over time, key insights for your given organization, a breakdown of the test executions, and more maturity tier information, such as the gaps between the tiers and steps to improve.

These insights are highly contextualized and Provar specific in many cases. We’ve incorporated our best practices into the algorithm and ensure the data is relevant for your environments.

Automation Coverage

This metric tracks the total number of automated test cases / the total number of tests in the org to give you a way to track automation coverage improvements over time. For the sake of calculation, it ignores Apex tests.

The metric card here shows the maturity tier, the % of test automation coverage across all tests in the org, and the total number of tests as well.

The small circle with the % shows the overall goal towards maturity tier 4 for the defined automation coverage KPI.

If you navigate to the metric drilldown, you’ll see a trend of your automation coverage over time, key insights on your tests, a breakdown of your tests and their types, and steps to improve your overall coverage for the org.

The How to Improve section will include current maturity tier, % above previous tier, and % gap to the next tier, giving you a focused objective for coverage.

Additionally, you’ll see actionable insights here on how to improve your automation coverage, with a preview of each maturity tier’s threshold.

Manual Hours Saved

This metric tracks the total number of manual hours saved through the strategic usage of test automation. It takes into account total test execution time and uses that value.

The metric card shows the total number of hours saved, the maturity tier info, the % goal to maturity tier 4 (prorated), and a highlighted insight with the time in weeks, as well as estimated labor costs savings over manual test execution.

Navigating into the metric drilldown shows you the manual hours saved over time (cumulative), key insights on these savings, a breakdown of the data, which includes types of testing activities saved estimated across common testing areas, and the maturity tier with steps to improve or stabilize.

Provar Grid ROI

If you are using Provar Grid, this metric helps to track the relative return-on-investment from your Grid licenses. This helps to justify to stakeholders Grid usage, as well as providing a pathway to success on scaling your parallel testing efforts. For more information on Provar Grid and its benefits, see this page here!

The metric card shows the same information as other metrics, with an additional highlighted insight on the amount of manual testing hours saved and value generated from Grid based on parallelization time savings.

The metric drilldown shows even more information on your Grid usage, including an ROI trend over time, the value generation, net ROI, and savings breakdown by type (labor infrastructure, testing time).