Share this page

# Test suites

A test suite is a collection of tests. Use test suites to test an area of your application. You can use test suites for planned testing events, such as regression testing.

A test suite can contain manual tests, BDD scenarios, Gherkin tests, and automated tests. For details, see Testing overview.

In this topic:

## About test suites

A test suite is a collection of tests. Test suites do not contain any content of their own.

Use test suites in a variety of ways:

- Group tests of a particular page, module, or section of your application together. This lets you test that section of the application.
- Keep tests of a particular theme grouped together. You can reuse the same test suite across versions, and compare the suite run from the current version with those from previous versions. In addition, you can run the same suite on different environments and compare suite runs to check for any regressions.
- Enforce the order of a series of tests. You can ensure that a series of manual tests are run in a specific order, so that a test cannot be run until the previous tests in the test suite have been completed.
- Different people run the tests. The owner of the test suite receives the compiled results for the test suite's test. This method is particularly helpful for regression testing.

Tip: Create maintainable and reusable test suites with a relatively small number of tests. You can organize your tests in suites based on one of the suggestions above, and then analyze the results across suites at a higher level, such as release, milestone, or environment.

Back to top

## Create a test suite

Create a test suite to contain the tests you want to group together.

To create a test suite:

1. In the Quality or Backlog module, open the Tests page.

2. If you want to add tests directly to the new test suite, select those tests in the grid.
3. Right-click in the tests grid, and select Create Test Suite.

4. Set the test suite attributes. In particular, provide a value for the Backlog coverage and Application modules attributes.

5. Click Add. The test suite is created.

   Tip: To add tests to the new test suite immediately, click Add & Edit. For details on adding tests, see Add tests to a test suite.

Back to top

## Add tests to a test suite

You can add tests to a test suite either from within the test suite, or from the Tests page.

You can add the same test to a suite multiple times, so for example you could run an automated test twice using different environments or execution parameters.

To add tests to a test suite from within the test suite:

1. In the Quality or Backlog module, open the Tests page.

2. Open a test suite and select its Planning tab.

3. To add existing tests to the test suite:

   1. Click the Add Existing Tests button .
   2. Select all the tests you want to include.
   3. Click Add.

4. To add new tests to the test suite:

   1. Click + Manual Test.
   2. Set values for the test fields. For details, see Create manual tests.
   3. Click Add. The new test is created and automatically added to the test suite.

To add tests to a test suite from the Tests page:

1. In the Quality or Backlog module, open the Tests page.

2. Select the tests you want to add to a test suite.

3. Right-click and select Add to Test Suite.

4. In the Add to Test Suite dialog box, select all the test suites to which you want to add the tests, and click Add.

Note: After you add tests to a suite, any changes you make to the Backlog coverage, Test runner, or Data set field apply only within that particular test suite. The changes do not affect the test outside of the test suite.

Back to top

## Parameterize tests

You can parameterize your test runs using data tables.

Test type | Description
--- | ---
Manual | If you add manual tests to a test suite, you can parameterize them as described in Use parameters in tests.
Automated | If you add executable OpenText Functional Testing tests to a test suite and your OpenText Functional Testing Git repository contains data tables, parameterize your test runs using data tables. For details, see Add tests from an SCM repository. Tip: For the data tables to be located in Git, store them in an entirely separate folder from your tests. To parameterize your test runs using data tables: In the test suite, open the Tests tab and select the grid view. Add the Data table column to the grid. In the OpenText Functional Testing test's row, select a data table to use for the test run. The list of data tables includes all of the data tables that are discovered with your tests. When the suite runs this test, it runs with the data table you selected. To run a test with different data tables, add the test to different test suites. In each suite, you can specify a different data table for this test.

Back to top

## Plan a test suite run

After you create a test suite, prepare the test suite to run. Assign owners and environment tags to each of the tests.

The test suite run process follows these steps:

Tip: To apply the same plan to multiple test suites, in the tests grid, select the necessary test suites and click the Plan Run button .

To plan a test suite run:

1. On the Tests tab, open a test suite.

2. On the test suite's Planning tab, manage the tests you want to run.

   Action | Steps
   --- | ---
   Add new or existing tests | Use the toolbar buttons to add new or existing tests. Tip: You can add the same test multiple times if, for example, you want to run the same test with different environment tags.
   Remove unwanted tests | Select the test's checkbox and select More > Remove from test suite.
   Select tests to include in the next run | Indicate which tests you want to include in the next run by turning on or off the switch in the Include in next run column. By default, all tests are set to run. Note: The checkboxes next to the test ID do not indicate whether to run the test. Selecting these checkboxes applies to other features, such as exporting, deleting, and reporting.
   Set the test run order | Drag and drop the tests to set the order in which they should be run. To send a test to the top or bottom of the list, right-click the test and select Rank Highest or Rank Lowest. The test run order indicates the priority of the tests in the suite, or the recommended order in which to run the tests. By default, the run order is not enforced. You can configure the suite to enforce the run order for manual tests. For details, see Enforce manual test run order.
   Add a test to a planned suite run | To add a test to a planned suite run, right-click the test and select Add to suite run. In the Add to Suite Run dialog box, select the suite runs to which you want to add the test, and click Add & Close. You can also add a run that already exists. This option is available for both manual and automated tests.

3. In the Planning tab, click the Plan Suite Run button . The Plan Suite Run dialog box lets you define the following suite's attributes.

   Field | Details
   --- | ---
   Release | Select a release that you want to test. After you select a release, you can select a milestone or sprint: Milestone: When you select a milestone, it means that the run contributes to the quality status of the milestone. A unique last run is created and assigned to the milestone for coverage and tracking purposes, similar to release. Sprint: When you select a sprint, it means that the run is planned to be performed in the time frame of the sprint. Sprints do not have their own unique run results, so filtering Last Runs by Sprint returns only runs that were not overridden in a later sprint. To see full run status and progress reports based on sprints, use the Test run history (manual) item type instead of using Last Runs in the Dashboard widgets. These values are propagated to all test runs created from the suite.
   Program | (Optional) If you are working with programs, you can select a program to associate it with the suite. All of the runs in the suite are assigned to this program, which is recorded in each run’s history. For details, see Programs.
   Default environment tags and Default run by | Specify the Default environment tags and Default run by values to be used by all test suite runs. The test suite run's Environment tags and Run by fields inherit the default values. If the Default run by field is left empty, test runs are not assigned to any user. You need to manually assign each of the planned test runs to a user. Data access: If a test belongs to a data category that falls outside of the default tester's role, the run is not assigned to the default user. Manually assign the test run to an accredited user. For details, see Data access control. Note: If you modify either of these fields after planning a test suite, the changes only affect new runs. The existing runs continue to inherit the original default values.
   Enforce order for manual | When running a suite containing manual tests, set this field to Yes to enforce the run order of the tests in the suite. For details, see Enforce manual test run order. You can change the value of this field for a suite run only when creating the suite run. After a suite run has been created, this field becomes read-only and cannot be changed. Tip: This field might not be visible by default. To show it, click the Customize Fields button and select Enforce order for manual from the list.

   Data propagation: Besides the above test suite fields that are copied to the resulting test runs, other fields are copied too:

   - Custom field values are copied from the suite run to the test runs, if both the Suite Run and the test run entity have custom fields with matching labels. This behavior is configurable by the COPY_RUN_SUITE_UDFS_TO_RUNS parameter. For details, see Configuration parameters.

   - Custom fields defined on the test run entity are visible and editable on tests within the suite. When a test in the suite is run, these custom field values are automatically copied to the corresponding run item.

4. In the Plan Suite Run box, click Plan.

   The suite run and test runs are created or updated.

5. Users can run the planned tests from one of the following areas.

Area | Details
--- | ---
My Work module | Test runs are displayed in the user's item list in the My Work module. Each assigned run is labeled with the name of the suite run and a link to the individual test. Select the test and click the Run button to start the test run. If all tests in the suite are assigned to the same user, they appear in My Work as a suite run. Select the suite run and click the Run button .
Team Backlog or Quality modules | On the Details tab of the suite run, click the Run/Run Suite button . If you are running the entire test suite, and some tests are not assigned, indicate whether to run your assigned tests with the unassigned tests or all the tests in the test suite.

1. (Optional) You can export the test suite information in the Excel or CSV format.

   The export saves all fields, with the exception of UDFs (user-defined fields), and the following built-in fields: Author, Backlog coverage, BDD Spec, Builds, Component, Covered automated test, Covered manual test, Covered requirement, Data table, External test ID, Flag rules, Flags, Followed by me, In my work, Is draft, Is in latest version, Manual, Owner, Quality stories, and Test level.

Note: After you plan a suite run, the planned test runs are created asynchronously and may be available after a delay.

Back to top

## Run a test suite

A test suite runs as a collection of test runs.

By default, all the tests included in the test suite are run unless certain tests were excluded in the planning stage.

Note: When a test suite runs, the included test runs are created in draft mode. The Alert User business rules that govern test runs creation are ignored.

You can prevent creating test runs in draft mode by setting the IGNORE_ALERT_RULE_WHEN_CREATE_IN_DRAFT_MODE parameter to false. As a result, the Alert User business rules are triggered, and the runs cannot be created as drafts. When a test suite runs, an error occurs.

For details, see Configuration parameters.

To run a test suite:

1. On the test suite Planning tab, verify that the necessary tests are included in the run.

   To change the tests included in the run, in the Include in next run column, select the tests to run. Note that checkboxes next to the test ID do not mark the tests to run.

2. Click the Run Suite button . The Run <test suite name> dialog box opens.

3. Verify the settings and click Let's Run.

   The Manual Runner window opens. For details, see Plan and run manual tests.

The status of the test suite run is set according to the status of the test runs.

Note: Business rules govern how the final suite run's status is determined. If the conditions for a number of rules are met, the last rule applies. For example, the first predefined rule for suite runs states that a suite run is set to Passed if all its tests are either Passed or Skipped. To address the case where all tests are skipped, a later rule states that if all the tests are Skipped, the suite run is set to Skipped. The admin can modify the predefined rules and their order to match their expectations. For details, see Business rules.

Back to top

## Manage suite runs

The suite run's Runs tab lets you manage the runs by changing their rank or other attributes. Use the right-click menu to add items to My Work or to send them as an email.

Tip: To update details for multiple test runs, do a multiple selection and choose Bulk Update from the context menu.

To manage the suite runs:

1. Select the Suite Runs tab. A list of all suite runs is displayed.

2. Click a suite run. The Runs tab opens and displays a list of runs.

3. To change the status of a run, modify the Native status in the Details tab and click Save. For example, if a run was previously blocked, you can change its status to Planned.

4. If you want to run a test twice as part of a suite run, make a copy of the run by selecting it and clicking Duplicate Run. If you want to redo a run from scratch, duplicate the run, make sure the copy's status is set to Planned (this is the default), and delete the old one.

5. Rank and reorder the runs by dragging them to the desired location.

   You cannot change the order of a run which is currently running.

   Tip: To ensure that the results stay aligned with their corresponding runs, keep the runs in the same order as they appear in the suite run, and do not run tests in parallel.

6. If you need to change the test assignment or an environment tag, modify the values in the Run by and Environment tag columns. The assignees receive a notification in their My Work area.

   Note: The Environment tag is an information label. The environment on which the test runs is not selected or detected.

7. After you assign all the tests, the Runs tab displays the updated list.

Back to top

## Enforce manual test run order

When planning a run of a test suite containing manual tests, you can configure the suite to enforce the run order of the manual tests in the suite. This means that a manual test run cannot be run until the preceding manual test runs in the suite have been completed.

Note: Enforcing test run order is not supported for automated tests. In a suite containing both automated and manual tests, the automated tests can be run at any time, and their run status is ignored when enforcing the run order of manual tests in the suite.

To enforce test run order for a suite run, do one of the following:

- Before creating the suite run, select the Details tab of the test suite and set the Enforce order for manual field to Yes.

  The value you set at the test suite level is the default for all future suite runs planned for the suite.

- When creating the suite run, set the Enforce order for manual field for the suite run to Yes. For details, see Plan a test suite run.

  Note: After a suite run has been created, the Enforce order for manual field becomes read-only and cannot be changed.

Test run completion status

By default, a test run is considered completed when its status is Passed, Failed, or Skipped. The administrator can choose which statuses are considered completed, by modifying the relevant business rule for the Manual Run entity. For details, see Business rules.

Tip: The administrator can define a business rule to automatically notify the assigned tester when the next test is ready to run. For details, see Notify the assigned tester when a test is ready to run.

If a test run in the suite was completed, and its run status was then changed back to Planned after subsequent tests were already allowed to run, the updated status affects all subsequent test runs. This means that all test runs after the run marked Planned cannot be run.

Example: A suite contains 5 manual test runs. Test runs 1, 2, and 3 passed. Test run 4 can currently be run.

Run order | Status | Blocked by previous run | Can be run
--- | --- | --- | ---
Test run 1 | Passed | No | Yes
Test run 2 | Passed | No | Yes
Test run 3 | Passed | No | Yes
Test run 4 | Planned | No | Yes
Test run 5 | Planned | Yes | No

Then, Test run 2 is updated from Passed to Planned. Now, Test runs 3 and 4 cannot be run.

Run order | Status | Blocked by previous run | Can be run
--- | --- | --- | ---
Test run 1 | Passed | No | Yes
Test run 2 | Planned | No | Yes
Test run 3 | Passed | Yes | No
Test run 4 | Planned | Yes | No
Test run 5 | Planned | Yes | No

To override this restriction and allow subsequent tests to run, you can manually change the Is completed field to Yes for each of the test runs before the run you want to run.

Back to top

## View the suite run results

When the suite's tests run, the run results are compiled into a single report.

To view the suite run results:

1. In the test suite, select the Suite Runs tab.

2. In the list of suite runs, click the link for the suite run you want to view.

3. In the suite run instance, select the Runs tab. The list of all runs opens.

4. In the suite run, select the Report tab. The suite run report opens.

   Tip: To view the test run details, click the test run ID.

5. In the View by box, select how to view the results:

   - Tests: to show the results per tests
   - Order: to show the results in the order of the runs

   Scroll or click the links to view each test in the suite, as well as the specific steps in each test.

   The report page also shows the phase each test was in when the run was created.

Back to top

## View additional details on suite runs

For automated tests, the Suite Run > Runs tab provides additional information.

Tab | Description
--- | ---
Preview | Shows general information on the test run: Name, description, release assignment, and comments
Report | Provides a test run report after the run is executed. If there is an error, you will see the related stack trace information.
Test Runners | Shows details of the related test runners: Framework, CI server, job name, and build number. Click a test runner to filter its related test runs. When you run a suite, the test runner's status changes to Initializing, In Progress, and Finished. Click the toolbar refresh button to see the status changes. The tab also provides a link to access the test runner execution log. For details on configuring test runners, see Trigger on-demand automated test runs.

Back to top

See also:

- Plan and run manual tests
- BDD testing framework
- Tags

https://admhelp.microfocus.com Software Delivery Management 8942
Tips to improve your search results... OpenText™ Software Delivery Management Help Center OpenText™ Software Delivery Management Help 26.1 All View Local Help Center View Online Help Center Version: