Manual test run options

This topic contains additional actions you can in relation to test runs.

Modify a run's test type

By default, a run inherits its Test Type value from the test or suite's test type.

You can modify this value for the run, so that, for example, if a test is End to End and you execute it as part of a regression cycle, you can choose Regression as the run's test type.

Note: The ability to modify a run's test type is defined by the site parameter EDITABLE_TEST_TYPE_FOR_RUN (default true). For details, see Configuration parameters.

Back to top

Link test runs to existing defects

You can link test runs to existing defects at any time from the grid view.

To link test runs to existing defects:

  1. In any module, open the Tests tab.
  2. Select a test in the grid, and open the test's Runs tab
  3. Select the test runs that you want to link to existing defects, and from Link <item> list , select Link Defect.

  4. In the Add Defects dialog box, select the defects to link to the runs you selected and click Add.

    The number of defects that are linked to each run is displayed in the Linked defects column.

Back to top

Run timer

You can measure the duration of manual or Gherkin test runs automatically. The timer starts when the Manual Runner window is launched, and stops when the run ends of the Manual Runner window is closed.

The timing is stored in the test run's Duration field.

The automatic timing option depends on a site or shared space parameter: AUTOMATIC_TIMING_OF_MANUAL_RUN_DURATIONS_TOGGLE. For details, see Configuration parameters.

Back to top

Edit a completed run

Depending on your role, you have different options for editing completed manual test runs.

Roles Details
Roles with the Modify Completed Run permission

You can edit a completed run at any stage.
Roles without the Modify Completed Run permission

You can edit a completed run while the Manual Runner is still open. One of the following conditions needs to be met:

  • After completing the run, you remain signed in to the system, and the Manual Runner is open.

  • After completing the run, you have signed out and then signed back in again with the same credentials. The Manual Runner remains open.

    In this case, you can edit the run within 30 minutes after it was completed. Your admin can change this period by modifying the CONTINUE_RUN_WITHOUT_PERMISSION_GRACE_PERIOD parameter. For details, see Configuration parameters.

Note:

  • The parameter applies only when you sign out after completing the run. If you remain signed in, and the Manual Runner is still open, you can modify a completed run regardless of the grace period.

  • If the Manual Runner is closed, you cannot reopen it to modify the run, regardless of the grace period.

  • You cannot modify a completed run neither from the Runs grid, nor from an individual run's document view. The run fields become read-only.

Back to top

Iteration-level reports

You can set up dashboard widgets to report on manual runs at the iteration level, so that each iteration with parameters is counted separately as a run. Iteration-level reporting is available for the following widgets.

Widget Runs reported
Summary graphs

Test run history (manual)

Last test runs

Backlog's last test runs
Traceability graph

Backlog's last test runs

Test’s last test runs

Gherkin tests: Iteration-level reporting is not available for Gherkin tests, because a Gherkin test can have multiple scenario outlines with different sets of iterations. For iteration-level reporting, use BDD specifications instead. For details, see Create BDD scenarios.

To enable iteration-level reporting:

  • When configuring a supported widget, select the Count iterations as separate runs check box.

The status of an iteration is updated automatically based on the status you assign to the steps in that iteration when running the test. If you do not assign a status to the steps, the iteration status remains as Planned. For details, see Manual test run options.

Version compatibility

  • Runs created in versions earlier than 16.1.200 have an iteration count of 1.

  • Planned runs created in versions earlier than 23.4.24 have an iteration count of 1.

Back to top

Content field

Manual tests, Gherkin tests, and test suites include a field called Content. The meaning of this field depends on its context.

Test entity Content field indicates
Manual test or manual test run Number of steps (regardless of examples or iterations)
Gherkin test or BDD scenario Number of scenarios and scenario outlines
Test suite Number of tests
Suite run Number of runs

Back to top

Unified test runs grid

The Runs tab in the Quality module provides a unified grid of runs across all tests, both planned and run. Testing managers can filter the grid, and do bulk updates on runs across different tests.

Note: By default, the Runs tab is accessible to workspace administrators only. For details on allowing access to other roles, see Roles and permissions.

Display limits

The Runs grid shows the first 2,000 runs in the filter, according to the sort-by field. If the filter definition results in a larger number of matching runs, refine the filter to make sure that all the runs you are interested in are included.

For grids with over 2,000 runs:

  • You can sort the grid according to the following fields: ID, Started, Creation time, and Latest pipeline run.

  • If you sort by any other field, the grid does not display any results.

  • No results are displayed if the grid is grouped.

Back to top

See also: