Analyze release quality

This topic describes ways to analyze release quality.

Associate tests and defects with features

To enable optimal release quality analysis, ensure that you associate tests and defects with features, user stories, requirements, and defects, using:

  • The Backlog Coverage field in a test

  • The Covered requirement field in a test
  • The Feature field in a defect

Analyze release quality using Backlog columns

Within the Backlog module, use different grid columns including Defect count, Risky commit count, Test coverage, and Automation coverage.

The following table provides additional details:

Test Coverage

The Test Coverage widget summarizes the last unique native run statuses for each test. If you run a test with the same configuration as a previous test run, ALM Octane updates the new result.

Hover over the widget to open the detailed run results or click View runs link to view a filtered list of the last runs.

Automation Coverage

The Automation Coverage widget tooltip shows you how much code of a backlog item (feature, user story, quality story, and defect) is covered by automated tests in different pipelines.

This enables you to understand the number of code lines developed as part of each item, and to quickly see which areas of your development require additional automated test coverage. As a result, you can better estimate the quality and the risk of your newly developed features.

ALM Octane tracks the lines correlated to each of the backlog items by analyzing both commit messages of users, and coverage information injected as part of each pipeline.

Note: To enable this feature, in the pipeline's Details tab select the Track backlog items automation coverage checkbox.

The widget tooltip shows the following:

  • Coverable lines: Lines that are tracked by a supported coverage tool (currently we support JaCoCo).

  • Covered lines: Lines that were executed by at least one automation test in the last pipeline run.

  • Not coverable: Lines that are not executable (such as comments), or lines that are not being tracked by a supported coverage tool.

  • Total lines: The sum of all lines correlated with this backlog item, including both coverable and not coverable lines.

Back to top

Analyze test run results

In the Backlog module, Overview tab and Dashboard module, view test run results by adding different widgets, including:

  • Feature quality status

  • Open defects by feature

  • Defect daily injection

To see a specific epic or feature, select the node in the backlog tree. In the Dashboard, add a filter to display items relevant to your release.

Note: Charts and graphs based on test runs display only the following summarized run statuses: Passed, Failed, Planned, Skipped, and Blocked. Each individual, native test run status falls into one of these categories. This is for clarity of analysis.

Back to top

Identify features at risk

In the Backlog module > Features tab, ALM Octane displays a risk icon for features associated with risky commits. You may want to increase testing on these features or postpone their release.

For more details, see Identify risky commits and features at risk.

Back to top