Plan and run test suites

After you create a test suite, you can plan the test suite run, assign the test runs to different users, and view the results after the tests are completed.

Plan a test suite run

Prepare the test suite to be run. Assign owners and environments to each of the tests.

To plan a test suite:

  1. In the Quality module, select the Tests tab.

  2. In the grid, select a test suite, and then in the toolbar, click Plan Run .

  3. In the Plan Suite Run dialog box, define the suite run attributes.

    ALM Octane assigns the values to all test runs for the tests in the suite.

    Tip: ALM Octane also assigns the values of user-defined fields, if the user-defined fields have the same name and type in both the test suite run entity and its test run entities. This is available if the COPY_RUN_SUITE_UDFS_TO_RUNS configuration parameter is set to true.

  4. Click Plan. ALM Octane creates the suite run and the test runs for the suite's tests. ALM Octane displays the test runs in the Default owner's item list in the My Work module.

    If the Default run by field is left empty, test runs are not assigned to any user. You need to manually assign each of the planned test runs to a user.

Tip: To add a test to a planned suite run, first add the test to the test suite, then right-click the test in the Tests tabs and select Add to Suite Run.

After you plan a test suite run, ALM Octane adds a Planned tag to the available Run Status tags in the Filters tab.

Back to top

Run a test suite

When you run a test suite, the test suite runs as a collection of test runs. Assign test runs to different owners and environments.

The test suite run process follows these steps:

To run a test suite:

  1. Plan the test suite run. In the Tests tab of the Quality module or the Tests tab of a feature or story in the Backlog module, select the test suite to run.

  2. In the toolbar, click the Plan Entire Test Suite button.

    Tip: If you want to run all the tests as a one-time run, click the Run button and enter the run attributes. The tests are run in order.

    If the suite contains executable automated tests, ALM Octane triggers the run on the CI server in parallel to the manual run.

  3. Assign suite run attributes. In the Plan Suite Run dialog, provide the suite run attributes.

    ALM Octane assigns the values to all test runs for the tests in the suite.

    Tip: ALM Octane also assigns the values of user-defined fields, if the user-defined fields have the same name and type in both the test suite run entity and its test run entities. This is available if the COPY_RUN_SUITE_UDFS_TO_RUNS configuration parameter is set to true.

    When you have entered these details, click the Plan button to continue.

  4. In the test suite, select the Suite Runs tab. ALM Octane displays a list of all suite runs.

  5. In the list of suite runs, click on the link for your planned run. ALM Octane displays the suite run details.

  6. In the suite run, click the Runs tab. ALM Octane displays a list of all tests included in the suite.

  7. Assign owners and attributes. In the list of test runs, in the Run by and Environment fields, assign the test to the relevant user and specify the environment on which to run the test.

    Environment is an information label. ALM Octane does not select or detect the environment on which the test runs.

    Tip: Update multiple test runs with the Bulk Update context menu option.

    After you assign all the tests, the Runs tab displays the updated list:

    The owner of the test run receives a notification of the assignment in the My Work area of ALM Octane.

    Tip: If you want to run a test twice as part of a suite run, in the Runs tab, select the test and click the Duplicate Run button . ALM Octane adds the same test to the list of tests.

  8. Run the tests. Each assigned owner runs the tests as normal. For details, see Run manual and Gherkin tests.

    The status of the test suite is automatically updated according to the status of the tests runs.

    Owners run the tests from one of the following:

    My Work module

    ALM Octane labels each assigned run with the name of the suite run and a link to the individual test:

    Select the test, and in the right pane of the My Work module, click Run to start the test run.

    Tip: If all tests in the suite are assigned to the same user, they appear in My Work as a suite run. Select the suite and, in the right pane, click Run.

    From the Backlog or Quality modules

    In the Details tab or Runs tab of the suite run, click the Plan Entire Suite Run button .

    If some tests are not assigned, select whether to run your assigned tests with the unassigned tests or all tests in the test suite.

    If someone must pause a test run, click the Continue Run button in the toolbar of the Details tab or Runs tab of the Suite Runs tab in the test suite to resume the test run.

Back to top

View the suite run results

When the suite's tests run, ALM Octane compiles the run results into a single report.

To view the suite run results:

  1. In the Test Suite, click the Suite Runs tab.

  2. In the list of suite runs, click on the link for the suite run you want to view.

  3. In the suite run instance, click on the Runs tab. ALM Octane displays the list of all runs.

  4. In the Runs tab, click the Report tab. The suite run report is then displayed:

    Scroll or click the links to view each test in the suite, as well as the specific steps in each test.

    The report also shows which phase each test was in when the run was created.

Back to top

View additional details on suite runs

In the right pane of the Suite Run > Runs tab, you can view additional information on your test runs:

  • The Preview tab shows general information on the test run: Name, description, release assignment, description, and comments. This can be viewed before or after execution.

  • The Report tab provides a test run report after the run is executed. If there is an error you will see the related stack trace information.

  • If your suite includes automated tests, the Test Runners tab shows details of the related test runners: Framework, CI server, job name, and build number. Click a test runner to filter its related test runs.

    You can also track test execution. When you run a suite, the test runner's status changes to Initializing, In Progress, and Finished. Click the toolbar refresh button to see the status changes. You can also click the link to access the test runner execution log.

    For details on configuring test runners, see Trigger automated test runs in your testing framework.

Back to top

Stop automated tests in a suite run

Note: This feature is currently only supported on Jenkins test runners.

If you are running a test suite with automated tests and you need to stop them before the suite is done, perform the following:

  1. In the Test Suite > Suite Runs tab, select the specific suite run that you want to stop.

  2. Within the suite run, select the Runs tab.

  3. In the Test Runners pane (on the right side of the screen), in each of the test runners whose activity you want to stop, click Stop.

    This stops the specific instance of the test runner's execution, and the test runs related to this runner's instance are therefore stopped. When you refresh the view, the status of the related test runs changes to Skipped.

Back to top

See also: