Define test sets and run tests

The Test Lab tab of Web Runner enables you to define test sets, select tests to include in each test set, and run tests.

Create test set folders

To define a hierarchical framework for your test sets, create test set folders and then add test sets to each folder. You can create folders under the Root folder or an existing folder.

To create a new test set folder:

  1. Select the folder under which you want to create the new folder.

  2. Click New Folder (), and enter a name.

    Alternatively, hover over the folder and select More Actions > New Folder.

To create a test set folder by copy/cut:

  1. Select the folder you want to copy or cut.

  2. Click Copy () or Cut ().

    Alternatively, you can hover over the folder and select More Actions > Copy or Cut.

  3. Select the folder under which you want to add the new folder, and click Paste ().

    Alternatively, you can hover over the folder and select More Actions > Paste.

To rename a test set folder:

  1. Hover over the folder you want to rename, and select More Actions > Rename.

  2. Enter the new name.

To delete a test set folder:

  1. Select the folder you want to delete and click Delete.

    Alternatively, you can hover over the folder and select More Actions > Delete.

  2. In the confirmation dialog box, select one of the following options and click Yes.

    Delete folders only Deletes the folder and moves all test sets under the folder to the Unattached folder.
    Delete folders and test sets Deletes the test set folder, its sub-folders, and all test sets under the folders. All test scripts are also deleted from the file system.

Back to top

Create test sets

You can create test sets under existing test set folders, except the Unattached and Root folders.

To create a test set:

  1. Select the folder under which you want to create the test set.

  2. Click New Test Set ().

    Alternatively, you can hover over the folder and select More Actions > New Test Set.

    Tip: You can also create a test set by copying or cutting an existing test set. To copy or cut a test set, select the test set, click Copy or Cut, select the test set folder where you want to add the new test set, and click Paste.

  3. Enter all required information for the test set, and click Submit.

  4. In the Details tab of the test set, provide other relevant information.

  5. To add tests to the test set, in the Execution Grid tab, click Select Tests.

    In the Select Tests panel, select

    UI Element Description
    Select Tests

    Opens the Select Tests pane that enables you to select tests to add to the test set.

    • Add Tests to Test Set. Adds an instance of the selected test to the test set, including any test configurations associated with the selected test.

    • Add Test Configuration to Test Set. Adds a test instance that uses the selected test configuration to the test set.
    Filter

    Opens the Filter pane that enables you to filter test instances by specified field values.

    For details, see Filter dialog box (for Web Runner and Quality Insight).
    Select Columns Enables you to show or hide columns for the test instance table.

  6. In the Attachments tab, add attachments to the test set.

    To add an attachment to the test, select the Attachments tab, and click Add.

    To select multiple attachments, use the CTRL or SHIFT key.

Back to top

Run tests manually

When you run tests, the run status is automatically synchronized with ALM. After ending the test run, the test status is automatically synchronized with ALM.

To run a test manually:

  1. Open the Test Lab tab. From the test set tree, select the target test set.

    Alternatively, if you know the test set ID, click Go to Test Set and provide the ID to directly open the test set.

  1. Start a test run.

    1. Go to the Execution Grid tab.
    2. You can use the filter options at the top of the Test Sets tab to filter according to the test instance status.

    1. Select a test instance and choose one of the following options:

      Run with Manual Runner. Starts a new manual test run.
      Continue Manual Run. Resumes the previous manual test run.

  1. Run the test steps.

    1. Select a step row.
    2. Follow the instructions provided in the test step description.
    3. Compare the actual result with the expected result.

    4. To associate attachments to a step, from the Attach list, select one of the following options:

      Option Description
      Attach to Step Adds an attachment to the step.
      Attach to Run Adds an attachment to the test run.

    5. Set step status.

      To set step status for a single step, double-click the Status column of the step, and select one of the following options:

      Option Description
      Failed

      The step failed.
      Blocked

      The step cannot run.

      For example, you are unable to perform the step due to an environmental failure, such as network problems or hardware failure.
      N/A The current status of the step is not applicable.
      Not Completed The step was not completed.
      No Run The step has not been run.
      Passed The step was passed.
      Customized Status A user-defined step status.

      To set step status for multiple steps, use the Passed or Failed option above the step table.

      Option Description
      Passed

      Includes the following options:

      • Pass Selected. Sets the status of selected steps to Passed.

      • Pass All. Sets the status of all steps to Passed.
      Failed

      Includes the following options:

      • Fail Selected. Sets the status of selected steps to Failed.

      • Fail All. Sets the status of all steps to Failed.

    6. In the Actual text area, enter the actual step result.

    7. When an expected step result and an actual step result do not match, you can submit or link a defect for the step.

      From the New Defect drop-down list, select one of the following:

      Option Description
      New Defect to Step

      Opens the Add Defect to Test Run window, enabling you to create a defect for the test step. Provide the defect details and click Submit.
      Link Defects to Step

      Opens the Link to Defect window, enabling you to link an existing defect to the test step. Enter the defect ID and click Link.

    8. Continue with the next steps.

      To move to the next step, use the down arrow "↓" shortcut key. To move to the previous step, use the up arrow "↑" shortcut key.

  2. Add or link a defect to the test run.

    If a defect is detected when running the test instance, you can add a defect or link an existing defect to the test run. From the New Defect list, select one of the following:

    Option Description
    New Defect to Run Opens the Add Defect to Test Run window, enabling you to create a defect for the test run. Provide the defect details and click Submit.
    Link Defects to Run Opens the Link to Defect window, enabling you to link an existing defect to the test run. Enter the defect ID and click Link.

  1. To end the test run, click End Run. To cancel the run, click Cancel Run.

    To change the test run status directly to a status, go back to the Execution Grid tab, double-click the Status column of the test, and select a status.

  1. Open the Test Lab tab. From the test set tree, select the target test set.

    Alternatively, if you know the test set ID, click Go to Test Set and provide the ID to directly open the test set.

  1. Start a test run.

    1. Go to the Execution Grid tab.
    2. You can use the filter options at the top of the Test Sets tab to filter according to the test instance status.

    1. Select a test instance and choose one of the following options:

      Run with Manual Runner. Starts a new manual test run.
      Continue Manual Run. Resumes the previous manual test run.

  1. Run the test steps.

    1. Select a run view.

      Displays the steps in the Grid View
      Displays the steps in the Details View
    2. Select a step row.
    3. Follow the instructions provided in the test step description.
    4. Compare the actual result with the expected result.

    5. To associate attachments to a step:

      Grid View
      • Add an attachment. In the Actions column, select Add Attachment to Step .
      • Delete an attachment. Select next to the Status column. The associated attachments are displayed. Select the icon of the attachment and then select .
      Details View
      • Add an attachment. On the right side of the window, select Add Attachment to Step .
      • Delete an attachment. Under the Expected Results section, select and then select .

    6. Select the relevant step status. Options include:

      Passed The step was successful.
      Failed The step failed.
      Blocked

      The step cannot run.

      For example, you are unable to perform the step due to an environmental failure, such as network problems or hardware failure.
      N/A The current status of the step is not applicable.
      Not Completed The step was not completed.
      No Run The step has not been run.
      Customized Status A user-defined step status.

    7. Select the Actual Results text area and enter text that describes the actual step result.

    8. When an expected step result and an actual step result do not match, you can submit a defect for the step.

      Grid View In the Actions column, select Add Defect to Step.
      Details View On the right side of the window, select Add Defect to Step.

      In the Add Defect to Step dialog box, fill in details as necessary.

    9. Continue with the next steps.

      To move to the next step, use the down arrow "↓" shortcut key. To move to the previous step, use the up arrow "↑" shortcut key.

  2. Add or link a defect to the test run.

    If a defect is detected when running the test instance, you can add a defect or link an existing defect to the test run. Click the menu icon and select one of the following:

    Add defect Opens the Add Defect to Test Run window that enable you to create a defect for the test run. Provide the defect details and click Submit.
    Link defect Opens the Link to Defect window that enables you to link an existing defect to the test run. Enter the defect ID and click Link.

  1. End the test run.

    Select the relevant option. The options include:

    Changes the run status directly to the selected status.

    Ends a test run.
    Cancels a test run.

    Ends the current test run and starts running the next test listed in your test set

    Ends the current test run and starts running the previous test listed in your test set.

Back to top

Run tests automatically

You can run test sets automatically.

Prerequisites:

Make sure your test machine meets the following requirements:

  • The TEA agent is installed. For details, see Test Execution Agent (TEA).

  • You can use the test machine.

  • The test machine supports running the type of tests you selected.

To start an automation test run:

  1. Open the Test Lab tab. From the test set tree, select the target test set.

  1. Start a test run.

    1. You can use the filter options at the top of the Test Sets tab to filter according to the test instance status.

    1. Select a test instance and click the Run Automation Test button .

      If you want to run all automation tests in a single test run, click Run All Automation Tests.

  1. End a test run.

    The test run ends automatically and the run status will be updated. You can click the status link to open the test run in ALM.

Back to top

Last Run Report

The Last Run Report shows the results of the last test run of the selected test instance, such as test steps, status, execution time, and step details.

To display the Last Run Report, in the Execution Grid tab, select a test instance. The Last Run Report is shown at the bottom.

Back to top

Link defects to test sets

You can link existing defects to a test set.

To link an existing defect to a test set:

  1. In the Test Lab tab, from the test sets tree, select the target test set.

  2. In the Linked Defects tab, click Link Existing Defect > By ID.

  3. Enter the defect ID and click Link. The defect must already exist in ALM.

    Click the defect's link to open it in the Defects tab.

Back to top

Track test set details changes

You can track the changes made to the details of a test set.

To track test set details changes:

  1. In the Test Lab tab, from the test set tree, select the target test set.

  2. Open the History tab. By default, it displays all the changes to the test details, grouped by change date and changer.

  3. To view changes made to a specific field, from the Field drop-down list, select the field.

Back to top

Keyboard shortcuts

To see the keyboard shortcuts supported for running tests, in the top right corner, click Settings > Keyboard Shortcuts.

Back to top

See also: