This section describes how to create requirement coverage for flows and business components in addition to test configurations.
This task is part of a higher-level task. For details, see Plan business process tests and flows.
When defining the criteria that ALM should use to calculate coverage for a test configuration, keep in mind that the same criteria are used for all configurations of the test. It is not possible to define different criteria for different configurations for the same test.
Requirement coverage overview
You can create coverage between requirements and business process tests. When working with business process testing, you can define coverage by criteria, such as flows or business components. This enables you to determine if a test passed or failed on a more granular level.
For example, a test can be considered passed even if just one of its business components passed, such as the most critical one. Less critical business components have no effect on the overall status of the test.
The logic used for calculating coverage by criteria is similar to the logic used for calculating coverage by tests and test configurations. For conceptual details on coverage analysis for tests and test configurations, see Create test coverage.
When defining coverage for criteria, consider the following:
|Coverage is calculated only for each instance of the selected criteria||
If a business process test contains three components, Component1, Component2, and Component3, and Component2 is the only criterion selected for coverage, each instance of Component1 and Component3 has no affect on coverage, regardless of whether the instances pass or fail. When all the instances of Component2 run, the requirement for the entire configuration is considered covered.
|How coverage for flows is calculated||
Coverage for flows is calculated as follows:
|When working with more than one configuration and multiple iterations||
When working with more than one configuration and multiple iterations, coverage for a criterion is calculated for each iteration of each configuration. If one of the criteria for an iteration of a configuration fails, the requirement coverage for the associated test fails.
Requirement coverage example
Business process test OrderFlights has two business components, CheckFlights and ReserveFlights. Only ReserveFlights is set as a criterion for covering the test requirements, perhaps because it is assumed that if the user was able to reserve a flight, the user must have been able to check the flight. The test has two different configurations, Domestic and International, representing different types of flights. Each configuration iterates three times, testing the reservation of three domestic flights and three international flights.
Using this example, only the following component instances are examined when calculating coverage:
- Domestic configuration, Iteration 1, ReserveFlights
- Domestic configuration, Iteration 2, ReserveFlights
- Domestic configuration, Iteration 3, ReserveFlights
- International configuration, Iteration 1, ReserveFlights
- International configuration, Iteration 2, ReserveFlights
- International configuration, Iteration 3, ReserveFlights
If just one of these six component instances fails, the associated tests fails when requirement coverage is calculated.
Create coverage by criteria
- Make sure that requirements are defined in the project.
- Make sure that the test has defined parameters.
You can create coverage from either the Test Plan module, or from the Requirements module.
From the Test Plan module
In the Test Plan module, select the Test Plan Tree view.
Select the business process test or flow, and click the Req Coverage tab.
Click the Select Req button to display the requirements tree in the right pane.
Select the requirements to add and click the Add to Coverage button.
From the Requirements module
In the Requirements module, select the Requirement Details view.
Select a requirement and click the Test Coverage tab.
Click the Select button to display the test plan tree in the right pane.
Select the flows or tests to add and click the Add to Coverage button.
ALM does not create criteria for a flow's business components when selecting a flow from the test plan tree. To add business components as criteria, add coverage by selecting the corresponding business process test from the test plan tree.
You can modify requirement coverage by criteria in the Criterion Settings tab of the Req Coverage tab.
Setting criteria for coverage enables you to analyze requirement coverage on a detailed level, such as by business component and flow, and not only on a general level, such as by test. The following analysis methods are available:
Analysis method Description Access
Coverage Analysis View
When working with Business Process Testing, this view enables you to examine the status of requirements according to criteria coverage.
For details, see
Create test coverage.
Criterion Results Tab
Displays results of the last test criteria run for the selected business process test.
Perform one of the following:
- In the Test Lab module > Test Sets tab, select a test set and click the Execution Grid tab. Then select a business process test. The tab is displayed in the lower pane.
- In the Test Runs module > Test Runs tab, select a business process test run. The tab is displayed in the lower pane.
- In the Test Lab module > Run Details dialog box sidebar select Criterion Results.
Criteria Status dialog box
Displays the status of each criterion for the last test criteria run used for the coverage of the selected requirement.
In the Test Configuration Status tab, in the Status column, click the hypertext link for the business process test.
If there is no hypertext link to click, it means no components or flows have been added to the test, and therefore there are no criteria to check. Add components to the test in the Test Plan module and refresh the display in the Requirements module.
Create Test Criteria Reports
You can create a template-based report for tests that includes test criteria coverage.
For task details, see Create a custom report template.