Quality

Monitor and measure product quality using the Quality module.

Overview

The Quality module is based on an application module tree.

The application module tree is a logical depiction of the product, reflecting the product's functional areas. It is typically defined by the business analyst. The application module tree shows how the application is broken down, and allows you to track quality by logical business areas.

Application module tree showing break down of the application.

The Quality module includes tests, defects, and features, all of which can be associated with application modules to help you analyze quality by business areas, locate pain-points in your product, and concentrate your efforts in these areas. For details, see Assign items to application modules.

The contents of the Overview, Tests, Runs, Features, and Defects tabs reflect the application module selected in the Application Modules tree. Select a different application module to view its associated items. Tree filters and any additional filters applied to a tab can further narrow the displayed content.

When deciding how to structure your application modules, focus on areas of the product for which you want to track quality, how you structure the application, and how users use the application.

Build the application module tree

Define your product areas as application modules.

To build the application modules tree:

  1. Open the global menu and select Quality Management > Quality.

  2. In the Application Modules tree, click the Add Application Module button .

    Select Shared Application Module to create an application module that is available across all the workspaces in a shared space. For details, see Shared application modules.

  3. In the Add Application Module dialog box, enter the required information.

    If you selected an item in the tree, it is used as the parent item. You can drag items to rearrange the tree.

  4. Repeat as necessary to create the full tree.

    Building the application module is typically a process that requires a number of iterations. Drag and drop items to restructure the tree.

  5. To display a subset of the application modules in the tree, click the Filter button . Choose whether to display only shared application modules, or selected application modules from the tree.

    The root Application Modules node now aggregates only those items remaining in the filtered tree. This is reflected in the content included in the Overview, Tests, Features and Defects tabs when you select the root Application Modules node.

Shared application modules

In workspaces that are members of a shared space, space admins can create shared application modules. Shared application modules are inherited by all the workspaces in a shared space.

You can create shared application modules as top-level nodes, or as children of other shared application modules.

Each workspace can add its own application modules to a shared application module.

In shared application modules you can only define shared predefined and custom fields. Team and user-type system fields are not available either. At the workspace level, members of the workspace can set values for workspace-specific fields.

Tip: In the Dashboard, you can create cross-workspace widgets to report on shared application modules.

Sort and filter the application module tree

You can use the filter to narrow down the application modules in the tree.

To filter the application module tree:

  1. Open the global menu and select Quality Management > Quality.

  2. Click the Filter button .

  3. Use the following options to filter the application module tree.

    Option Details
    Show only shared application modules Turn on to display only those modules that are inherited by all workspaces in the shared space.
    Show only selected application modules Turn on to display only the modules you select from the list.

To sort the application module tree:

  1. Click the Sort button and select a field by which to sort the tree.
  2. By default, the field is sorted in ascending order. Click the Ascending button to switch to descending order.

Assign items to application modules

Assign features, tests, and defects to application modules to help you assess product quality.

To find unassigned items, click Unassigned in the application module tree.

To assign items:

  1. In the Backlog or Quality module, open the relevant items' tab and select the items to assign.
  2. In the toolbar, click the Assign to Application Module button .
  3. In the Assign to Application Module dialog box, select the relevant application modules, and then click OK.

If you are assigning automated tests, click Assignment Rule to create a rule that assigns automated tests. For details, see To create test assignment rules from the Tests tab:.

Note: To assign a single item, set the Application modules field in the item.

Quality by application module

Use the Quality by Application Module widget in the dashboard to see an overview of the application modules, and identify those that need attention. You can choose application modules, filter by application module fields, and configure the threshold for switching an area from On track to Needs attention.

In the following example, the size of the application module in the widget is determined by the number of story points developed in the release, and Needs attention is determined by critical open defects, low automation, or risky commits.

Quality by Application Module widget displaying application modules that require attention.

Trace tests to application modules

The following flow is a typical example of a manual test traceability:

  1. The backlog owner creates a new feature and assigns it to an application module.
  2. A tester creates a manual acceptance test for the feature (or one of its user stories) in the Backlog module. The test automatically inherits the feature’s application module.
  3. Access the test from the feature to understand feature coverage, and from the application module to understand the area coverage.

Expert users

The Overview tab in the Quality module contains an Expert users widget. The widget ranks users according to their level of expertise in the selected application module.

The expertise score is evaluated based on the amount of story points users have been assigned in backlog items linked to the application module.

Note:  

  • User stories are weighted higher than defects.

  • As backlog items grow older, based on their close date, their weight in the score decreases.

  • Subtype weight: Defect = 2 points, User story (US) = 4 points, Quality story (QS) = 1 point.

  • Base experience is calculated as: (Story Points x 1) + Subtype Weight.

  • Degradation factor by item age: Within 1 month = 1.0, 1-2 months = 0.9, 2-3 months = 0.8, 3-4 months = 0.7, over 4 months = 0.5.

To display the expert users widget:

  1. Open the global menu and select Quality Management > Quality.

  2. Select the Overview tab.

  3. Click the Add Widget button .

  4. Select the Expert users widget.

Track anomalies

It is useful to see anomalies to prevent delays in the development cycle. Examples of anomalies include defects stuck in one phase for an extended period of time, and a defect reassigned to many users.

Admins can define rules that flag anomalies within entities. For details, see Anomaly flags.

After your admin has set up the anomaly flags, add the Flag rules column to your items grid. Anomalies are displayed in this column. The tooltip for an anomaly shows details about the anomaly's threshold.

Tests and Runs grids

The Tests and Runs tabs provide central views of the workspace's test repository and runs across all tests. The Tests grid lists tests, and the Runs grid lists both planned and executed runs.

The selected application module determines which tests and runs are displayed. To view items across application modules, select the root Application Modules node and clear any tree filters. Additional grid filters and your data access permissions can restrict the items displayed.

Use these grids to filter tests and runs and perform bulk updates. For details on access to the Runs grid and its display limits, see Test runs grid.

You can export tests and runs from these grids to Excel or CSV. The export reflects the current filters. To export the entire filtered grid, leave all rows unselected. For export procedures, permissions, and limits, see Export data.

Measure quality items waste

You can track and measure waste time for the following quality items: manual and Gherkin tests, MBT tests, test suites, test and test suite runs, and BDD scenario runs.

For details, see Track waste.

Next steps