Run tests in a Functional test set

You can run specific test instances or an entire Functional test set in ALM. These server-side tests run on remote testing hosts.

In this topic:

Functional test execution overview

The following workflow diagram illustrates how functional test set execution occurs in ALM:

Schedule functional test execution	You can schedule the execution of Functional tests or Functional test sets in the Timeslots module. If there are currently available hosts for your test, you can also use the Execution Grid to arrange for tests to run immediately. For details, see Timeslots.
Testing Host	Test instances added to a functional test set have a Testing Host property field in the Execution Grid. You can use this field to require that the testing host used to execute the selected test instance has certain properties. ALM enables you to define the following properties: Purpose, Attributes, and Location. For details, see Manage testing hosts.
Requested Hosts	When ALM executes a server-side test or test set, it uses the testing hosts that are requested by the test set. You can view, modify, add, or delete the testing hosts reserved for the test set. For details, see Manage host requests for Functional test sets.
ALM Lab Service	When executing a server-side test, ALM works with the ALM Lab Service agent installed on the assigned testing host and requests that the test be run. The Lab Service agent exports the test results back to ALM. For details, see ALM Lab Service.
Execution Report	When you run a functional or performance test set, the Execution Report page opens, keeping you updated with the status of your test execution. You can also view the Execution Report page from the Test Runs module. For details, see View Functional test execution results.

Back to top

Run server-side functional tests and test sets

This task describes how to run server-side functional tests and test sets.

Prerequisite:

Set up testing hosts for server-side test execution	Before running server-side tests, you must set up and register testing host machines for the remote execution of the tests included in your Functional test sets. For details about setting up testing hosts, see Manage testing hosts. For details about registering testing hosts, see ALM Lab Service.
Create automated tests and Functional test sets	Use the Test Plan module to create automated tests and the Test Lab module to define and fill Functional test sets with your automated tests. For details, see Create test sets.

To run tests in a Functional test set immediately:

You can arrange for the tests in a Functional test set to run immediately on a remote testing host.

1. In the Test Lab module > Test Sets tab, do one of the following:

   • To run selected tests, from the test set tree, select a test set. Click the Execution Grid tab, select one or more tests and click Run.

   • To run all the tests in a Functional test set, select a Functional test set and click Run Test Set.

2. To run the tests using a new timeslot, in the Run <Entity> dialog box, select New for Select timeslot, and specify the following:

   UI Element	Description
   Duration	The amount of time, in hours and minutes, that you need the resources reserved. To get a suggested duration from recorded executions whose state is Finished for your current run, click Suggested Duration : Longest previous duration. Enables you to use the duration of the longest recorded execution of the selected run. Select this option if it is essential for the selected run to complete and there are not many other important tests in need of resources. Most frequent duration. Enables you to use the duration used by the 90th percentile of recorded executions of the selected run. Only 10 percent of runs had longer durations. Select this option to maximize efficient usage of testing resources. Suggested Duration. The duration ALM suggests based on the option you selected. Click OK to apply this duration to your run. Shortest Duration. Displays the shortest duration possible for this run type. Note: Suggested Duration is available for test set execution and not available for individual test instance runs.
   Automatically Extend Timeslot	If enabled, the testing timeslot will be automatically extended if tests are still running when the timeslot is about to end. For the timeslot to extend successfully, the necessary testing resources must be available in the timeslot extension.
   Requested Resources	The resources being requested for this test.
   AUT Env. Configuration	Select an AUT environment configuration to be used for the test.
   Calculate Availability	Calculates the availability of the resources required for running the test and checks whether a new timeslot can be created.

3. To run the tests using a reserved timeslot, in the Run <Entity> dialog box, select Reserved for Select timeslot, and specify the following:

   UI Element	Description
   Reserved Timeslots grid	This grid includes the details of manual non-autostart timeslots which are currently reserved by you or another user, but are not yet assigned to another test. Select a timeslot to run the tests. The Status field in the grid can have the following values: Recommended. This timeslot is recommended because it was originally created by you and it is linked to the selected test entity. OK. You can run your test in this timeslot. Update needed. This timeslot is available, but it does not have sufficient resources for your run. If you choose to run your test in that timeslot, ALM automatically allocates the necessary resources. Note: Even if you reserved a timeslot in the Timeslots module, it may not appear in the grid. Some possible reasons include: The timeslot is not open yet. The timeslot is running (it was configured to start automatically instead of manually). The timeslot does not have the required resources, and due to concurrent timeslots, it cannot be updated with more resources. The timeslot was not configured with a duration long enough to run the selected test entity, and it cannot be prolonged. The timeslot was linked to another test by mistake.
   Selected Timeslot Details	Shows the details of the reserved timeslot you selected.

4. Click Run. The selected or all tests in the Functional test set run, using the hosts reserved by the selected timeslot.

In addition, if your system is integrated with Continuous Delivery Automation (CDA), you can instruct ALM to provision machines and deploy your build packages on a dynamic environment. For details on the integration between ALM and CDA, see Lab Resources.

To schedule Functional test execution for the future:

You can arrange for the tests in a Functional test set to run in the future using timeslots. By scheduling a timeslot, you reserve the testing resources needed to run your test in advance.

For details, see Reserve timeslots for running tests.

Back to top

View Functional test execution results

For Functional tests and test set runs, ALM generates an Execution Report. The Execution Report page provides general details about the run, overall information per test, as well as drilled-down run information for each individual test within the test set. For user interface details, see Execution Report Page.

After the test execution is complete, you can view a summary of test results in the Test Runs module. When you run a test set, the test set run is immediately displayed in the Test Set Runs tab. When a test instance completes, it is displayed in the Test Runs tab.

For more details on viewing test results, see Test Runs.

Back to top

Troubleshoot Functional testing

• If the testing hosts being requested for a test set exceed the limits of the available testing hosts in the host pool of your project, the test set becomes invalid. A message will be displayed in the Messages panel (found in the Requested Hosts tab) indicating that the test set cannot be executed due to lack of testing resources.

  Workaround: You can add more testing hosts to your host pool, or combine requested hosts in the Requested Hosts tab.

• In the Requested Hosts tab, if you remove a host request that is required for the test set, the test set becomes invalid. A message is displayed in the Messages panel indicating that the test set cannot be executed due to lack of testing resources.

  Workaround:

  • Click Resolve Missing Hosts. ALM automatically adds the required testing hosts to the grid in the Requested Hosts tab.
  • Click Restore Defaults. ALM automatically resets the test set to auto mode, and adds the required testing hosts to the grid in the Requested Hosts tab.

Back to top