Create test sets

This task describes how to create and define test sets in the Test Lab module.

In this topic:

Test sets overview

After you design tests in the Test Plan module, you organize test execution by creating test sets in the Test Lab module. A test sets tree enables you to organize your application management process by grouping test sets in folders and organizing them in different hierarchical levels.

Test sets, tests, and test instances

A test set contains a subset of the tests in your project designed to achieve specific test goals.

A test instance is an occurrence of a test in a test set. When you add tests to a test set, ALM adds instances of your selected tests to the test set. Each test instance contains a defined test configuration. A test configuration enables you to run the same test under different scenarios. For more details on test configurations, see Test configurations.

Supported test set types

ALM supports different types of test sets. The following table describes each test set type:

Test Set Type	Description
Performance	For running unattended remote performance tests. You can add only performance tests to this test set. You arrange for a performance test to be executed from the server by scheduling a timeslot. A timeslot contains a test set, the details of the remote hosts on which the test set will run, and a time and duration for running the test set. For more details about how to run a Performance test, see Test Lab. For more details on timeslots, see Timeslot reservations overview. ALM Editions:  Performance test execution is available only for ALM Edition and LoadRunner Enterprise. For information about ALM editions and their functionality, see ALM editions. To find out which edition of ALM you are using, ask your ALM site administrator.
Default	For running client-side, locally controlled and launched functional tests. You can add both manual and automated functional tests to this test set. You can arrange for default tests to be run in an ad hoc manner. For more details on how to run default test sets, see Test Lab.
Functional	For running server-side, unattended functional tests. You arrange for a functional test set to be executed from the server by scheduling a timeslot. A timeslot contains a test set, the details of the testing hosts on which the test set runs, and a time and duration for running the test set. For more details about how to run a Functional test set, see Test Lab Build verification suites are a key component in ALM's Continuous Delivery solution. They facilitate an automated, end-to-end deployment and testing framework that makes application development more efficient, reliable, and quick. For details about how build verification suites can be used as part of this process, see Deploy and test your application in ALM. For more details on timeslots, see Timeslot reservations overview. ALM Editions: Functional test sets available only for ALM Edition. For information about ALM editions and their functionality, see ALM editions. To find out which edition of ALM you are using, ask your ALM site administrator.
External	Contains external type tests (EXTERNAL-TEST ). External type test sets are read-only. You cannot create, modify, or run test sets directly from ALM. For more details on external testing, see Integrate with external tests.

Test set example

To decide which test sets to create, consider the goals you defined at the beginning of the application management process. Consider issues such as the current state of the application, and the addition or modification of new features.

Here are examples of general categories of test sets you can create:

Test Set	Description
Sanity	Checks the entire application at a basic level—focusing on breadth, rather than depth—to verify that the application is functional and stable. This set includes fundamental tests that contain positive checks, validating that the application is functioning properly. For example, in the Mercury Tours application, you could test whether the application opens, and enables you to log in.
Regression	Tests the system in a more in-depth manner than a sanity set. This set can include both positive and negative checks. Negative tests attempt to fail an application to demonstrate that the application is not functioning properly.
Advanced	Tests both breadth and depth. This set covers the entire application, and also tests the application's advanced options. You can run this set when there is ample time for testing.
Function	Tests a subsystem of an application. This could be a single feature or a group of features. For example, in the Mercury Tours application, a function set could test all activities related to booking a flight.

Back to top

Define test sets

Define a hierarchical framework for your test sets by creating a test sets tree that contains folders and subfolders.

To create a test set:

1. On the ALM sidebar, under Testing, select Test Lab.

2. Right-click the Root folder and select New Folder. To create a sub-folder, right-click a folder and select New Folder.

3. Right-click the test set folder and select Assign to Cycle. Select a cycle from the releases tree.

4. Right-click a folder and select New Test Set.

   UI Element	Description
   Name	Give the test set a unique name. Syntax exceptions: A test set name cannot include the following characters: \ ^ , " *
   Type	Select the test set type that corresponds to the types of tests you are grouping together. For details, see Supported test set types.
   Details	Lists test set fields.
   Attachments	Enables you to add attachments that provide additional information about the test set. For details, see Work with attachments.
   Test Set Folder	Displays the preselected folder name.

Example:  

Suppose you want to test new features added to Release 10.5 of the Mercury Tours application. Release 10.5 consists of four cycles, and you want to test the new features for the release in Cycle 1, Cycle 2, and Cycle 4. Because the tests you need to run to test the new features are the same for each of these cycles, you want to use the same test set as a basis for testing new features in each of the cycles.

In the test sets tree, you can create the folder, Release 10.5, for the release. Under this folder, you create the folder, Cycle 1, for the first cycle of the release. In the Cycle 1 folder, you create a test set, New Features, containing the tests necessary to test new features for the release. After you create this test set and add its tests, you can copy and paste the Cycle 1 folder and use it as a basis for the other cycles that test new features.



5. To copy test sets from another project:

   Note:  

   • Test run information is not copied.

   • The source and target projects must use the same ALM version and patch level.

   1. In the source project, right-click a test set or a test set folder and select Copy.

   2. Open the target project in a separate browser window.

   3. In the target project, right-click a test set folder below which you want to insert the copied test set or test set folder, and select Paste.

      UI Element	Description
      Copy Test Set Folders and link to existing related entities	ALM copies the test sets or test set folders and pastes them into the target project. The copied test sets or test set folders are linked to existing test resources and called tests with the same name and path. If a related test resource or a test does not exist in the target project, ALM copies it to the target project.
      Copy Test Set Folders and related entities	ALM copies the test sets or test set folders along with the related test resources and called tests, and pastes them into the target project. If a related test resource or a called test already exists in the target project, the copied related test resource or called test is renamed to resolve the duplicate name.
      Copy Test Set Folders without copying related entities	ALM copies the test sets or test set folders without copying the related test resources or called tests, and pastes them into the target project. The copied items are not linked to any related entities.

6. Add tests to the test set.

   1. Click the Execution Grid tab or the Execution Flow tab, click Select Tests.

   2. Add tests from the following tabs in the test selection pane:

      • Test Plan Tree. Add tests or test configurations from the test plan tree to the test set.

      • Requirements Tree. Add tests covering requirements from the requirements tree to the test set.

      Test types that can be added to each test set type:

      Test Set Type	Tests to include
      Functional test set	You can add tests of the following test types to a Functional test set: BUSINESS-PROCESS SERVICE-TEST QUICKTEST_TEST SYSTEM-TEST VAPI-XP-TEST
      Default test set	You can add tests of all test types, except the PERFORMANCE-TEST type, to a Default test set.
      Performance test set	You can only add tests of the the PERFORMANCE-TEST type to a Performance test set.

      For details, see Test types.

Back to top

Manage host requests for Functional test sets

You can manage which hosts are requested for a Functional test set execution.

Host requests management modes

ALM manages testing host requests in two different modes:

Mode	Description
Auto mode	In ALM's default auto mode, each time you add a test instance to a test set, ALM checks if the host requested by this new test instance (whether it is a specific host or an automatch host) was already requested for the selected test set. If the new test instance contains a host request which does not already exist in the Requested Hosts grid, ALM automatically adds a new testing host to the test set. If the grid already contains the host being requested for the new test instance, no new hosts are added. Example: The test set already contains a host request for a host in London with a VAPI-XP purpose, and you add a new test instance which requires an automatch host with a VAPI-XP purpose. ALM adds a new host request to the grid for the automatch host. Even though the London host request technically covers the new automatch host, they are not the same, so ALM adds a new automatch host request to the grid.
Manual mode	You may decide to manually edit the hosts being requested for a selected test set. For example, you may have only one testing host available in your pool and it can cover a few different test instances. In that case, you modify the hosts in the Requested Hosts tab and arrange for only a single host. Once you make a change in the grid, ALM puts the test set into manual mode. In custom mode, ALM automatically adds new host requests for additional test instances only when it is necessary to maintain the validity of the test set. This is the case when the added test instance cannot possibly be satisfied by the already existing host requests for that test set. Example: The test set already contains a host request for an automatch host with a VAPI-XP purpose, and you add a new test instance which requires a host in London with a VAPI-XP purpose. ALM adds a host request for the new test instance because the first automatch host request does not necessarily cover the London host request. But, if your test set already contains a host request for a host in London with a VAPI-XP purpose, and you add a new test instance which requires an automatch host with a VAPI-XP purpose, ALM does not add a new host request to the grid. The London host request suffices for the new test instance, therefore ALM does not add a new host request.

To manage hosts requests for a Functional test set:

1. From the test set tree, select the target Functional test set, and click the Requested Hosts tab.

2. Add or edit host requests.

   UI Element	Description
   Add Automatch Host	Opens the Select Automatch Host dialog box, enabling you to add a new host request based on criteria you specify. ALM automatically allocates and reserves a testing host which fits the criteria. You can block hosts from being included in the automatch selection. For details, see Exclude From Automatch.
   Add Specific Host	Opens the Select Specific Testing Host dialog box, enabling you to request a specific testing host. You can select from the remaining available hosts in the host pool of the project. You can make specific hosts available only to specific users. For details, see Reserved for User.
   Edit Host Request	Enables you to edit the selected host request.
   Remove	Removes the selected host request. If you remove a host request that is required for a specific test instance, the test will not be able to run.
   Restore Default	Resets the host requests to their default state. ALM removes all current host requests and create a default set of requests. One host request will be created for each type of test in the test set. Restores the selected test set to auto mode.
   Resolve Missing Hosts	Enables you to automatically generate the remaining host requests if the current host requests are insufficient for executing the selected test set. The requested hosts will be determined based on the test instances in the Execution Grid.
   Testing Host	Displays the name of the host specified in the host request. If the value Automatch is displayed in this field, it indicates that no particular host was specified. ALM automatically allocates and reserves a host based on the criteria in the Purposes, Location, Amount, and Attributes fields. For details about how ALM allocates and reserves testing hosts, see Host Allocation.
   Amount	Displays the number of hosts specified in the host request.
   Purposes	Lists the purposes specified in the host request.
   Location	Displays the host location specified in the host request.
   Attributes	Lists the host attributes specified in the host request.
   Messages	Displays all messages related to host request validation. Example: If a test set cannot run because the requested hosts have not been reserved, ALM informs you that the tests cannot be run and provide the reason.

Back to top

Pin a test set to a baseline

When you pin a test set to a baseline, you associate the tests in the test set with the versions of the tests stored in a baseline you select. When you run a test set that has been pinned to a baseline, ALM runs the versions of the tests that are stored in the specified baseline.

Before you pin a test set to a baseline

Consider the following before you pin a test set to a baseline:

• A pinned test set can include only tests that are included in the baseline. When you pin a test set, ALM removes tests that are not included in the baseline from the test set.

• A pinned test set is displayed with the pinned test set icon in the test set tree.

• When you pin a test set, ALM deletes all test runs from the test set.

To pin a test set to a baseline:

1. From the test set tree, right-click the target test set, and select Pin to Baseline.

2. From the libraries tree, select the target baseline.

To clear a pinned baseline, from the test set tree, right-click the target test set, and select Clear Pinned Baseline. When you clear a pinned test set:

• The tests in the test set are then associated with the latest version of the tests in the Test Plan module.
• All test runs in that test set are deleted

For concept details on pinned test sets, see Pinned Test Sets.

Back to top

Link defects to test sets

You can view, add, and remove defect links for test sets.

To add linked defects:

1. From the test set tree, select the target test set, and go to the Linked Defects tab.

2. To add and link to a new defect, click Add and Link Defect. Provide the defect details.

3. To link to an existing defect, click Link Existing Defect and select one of the following options:

   • By ID. Enter the defect ID.

   • Select. Select the defect to link.

Back to top