Add automated tests

In ALM Octane, automated test entities represent tests from an external tool. Add these tests and view the tests with other manual tests.

Overview

Automated tests are usually run by CI or automation servers, such as Jenkins or TeamCity, or by ALM.

If you use UFT One for automated tests, ALM Octane can create executable automated test entities based directly on UFT One tests. For details, see Add UFT One tests from an SCM repository.

The other way to add automated tests to ALM Octane is based on test run results. For details, see Create automated tests.

For ALM Octane to receive the test results, do one of the following, as described in the sections below:

Back to top

Create automated tests

ALM Octane receives automated test run results from pipelines, test result collection tools, or the API. ALM Octane creates automated test entities associated with the test run results, and adds the results to the tests as test run entities.

If the relevant automated test already exists, ALM Octane adds the results as a test run to that test.

For each test, ALM Octane stores the results of one run per test environment, the Last run. If you run the test again with the same environment configuration, the new results overwrite the old.

If the test ran with different settings this time, ALM Octane stores the results in a new test run entity for the test.

ALM Octane maintains and displays basic information about the previous runs in a test run's Previous Runs tab, in dashboard trend widgets, and in a pipeline run's Tests tab.

Previous runs are saved for each unique test configuration. The following sections explain how ALM Octane identifies unique configurations of the various test types:

ALM Octane identifies each test by a unique combination of the following fields: Component, Package, Class, Name.

When ALM Octane receives a test run, it checks these attributes to associate the run with a specific test. If no existing test matches the run, ALM Octane creates a new automated test entity.

Each test can have more than one run, each uniquely identified by Test, Release, and Environment. These are the test's Last Runs.

ALM Octane stores the details of a single Last Run for each unique combination of these attributes. Each last run accumulates its own history of Previous Runs.

For example, if a test was run on Chrome and Firefox in two different releases, the test would have four last runs.

ALM Octane identifies each Gherkin test in ALM Octane by its script feature's tag, if one exists. The tag exists if the Gherkin test was downloaded from ALM Octane at some point. The script has a tag that contains the test ID and revision in a format like this: @TID4002REV0.2.0.

When ALM Octane receives a Gherkin test run, it checks these attributes to associate the run with a specific Gherkin test. If no existing Gherkin test matches the run, ALM Octane creates a new automated Gherkin test entity.

Each Gherkin test can have more than one run, each uniquely identified by Test (which was uniquely identified by its tag or path), Release, Environment, and job name. These are the Gherkin test's Last Runs.

ALM Octane stores the details of a single Last Run for each unique combination of these attributes. Each last run accumulates its own history of Previous Runs.

For example, if a Gherkin test was run on Chrome and Firefox in two different releases, the Gherkin test would have four last runs.

Tip: When working with pipelines, you configure a test's environment by configuring the pipeline step that runs it. For details, see Create and configure pipelines.

Back to top

Add UFT One tests from an SCM repository

If you use UFT One for automated tests, you can store your tests in Git or SVN and set up UFT One integration. This lets you see your UFT One tests reflected in ALM Octane as soon as you save them, regardless of test runs. For details, see Set up UFT One integration.

ALM Octane creates executable automated test entities based on the UFT One tests in the SCM repository. You can then run these UFT One tests from ALM Octane as part of a test suite. For details, see Run automated tests from ValueEdge and ALM Octane.

Back to top

Add automated tests from runs in a pipeline

To add automated tests from test runs in an ALM Octane pipeline, perform the following steps:

  1. Set up CI/CD integration

  2. Create and configure pipelines

  3. Run the pipeline that contains the tests.

The test run results are sent to ALM Octane. ALM Octane creates or updates the automated test entities associated with these results.

To enable the ALM Octane CI plugin to send the results to ALM Octane, make sure that the jobs running the tests publish them to the CI server. For example:

  • For LoadRunner Cloud tests run by Jenkins, create a Publish JUnit test result report post build step.
  • For tests run by GoCD, declare the result xml-report-files as artifacts of your build.

Back to top

Add automated tests from external test runs

If you run automated tests on ALM, or on a CI server not integrated with ALM Octane, you can use the following to send your automated test run results to ALM Octane:

ALM Octane creates or updates the automated test entities associated with these results.

For details, see Push results to ALM Octane.

Back to top

Assign tests to application modules and backlog items

Assigning automated tests to application modules and backlog items in ALM Octane lets you view the test results in context. You can then use these results to analyze the progress and quality of your release and product.

Assigning owners to automated tests helps accelerate problem resolution. You can configure a pipeline to notify test owners when their test runs fail.

For details on defining application modules or creating your product backlog, see Assign items to application modules and Build the product backlog.

To manually assign tests:

  1. In ALM Octane, in the Tests tab (Quality or items of a Backlog module), click a test's link to open it.

  2. In the Backlog coverage field, select the relevant backlog items.

  3. In the Application modules field, select the relevant application modules.

  4. In Owner, select the relevant user.

To manually assign multiple tests:

  1. In ALM Octane, in the Tests tab, select the relevant tests.

  2. Do one of the following:

    • Click Assign to Application Module and select the relevant application modules.

    • Right-click and select Bulk Update. Click Select field, select Application modules or Owner, and apply the relevant values.

To create assignment rules for automated tests:

  1. In Settings > Test Assignment Rules, set up rules to assign automated tests that match a filter you specify to specific application modules.

  2. When you save a rule, ValueEdge and ALM Octane assigns all current and future automated tests that match the filter to the selected application modules and test owner. This includes tests currently in the system, as well as new automated tests discovered in the future.

    For details, see Create and manage test assignment rules.

Tip: Use the same rules to assign owners to automated tests, which can help accelerate problem resolution.

To create test assignment rules from the Tests tab

You can add assignment rules to tests when you assign tests to application modules.

  1. Select the tests for which you want to assign rules from the Tests tab in the Quality or Backlog modules.

  2. Click Assign to Application Modules .

  3. Optional. Select one or more application modules.

  4. Click Assignment Rule.

  5. In the Create or Edit Assignment Rule dialog box, add a new rule, or locate an existing one that you can adjust or copy.

    To create a new rule based on the one you selected, change the Rule name.

    To update the selected rule, leave the Rule name unchanged.

  6. Optional. You can also add a Test owner to assign, or one or more test fields: Test framework, Test type, Test level, and Testing tool.

  7. If you selected many tests without opening an existing rule, ALM Octane creates an initial filter. ALM Octane bases this filter on the properties of the selected tests.

    Click Add filter to add conditions to the filter, or edit the existing conditions. Use these fields to create the conditions: Class name, Component, Name, and Package.

    Filters can be multipart and complex and can include '*' as a wildcard. If necessary, click the Add filter button again to add more items to the filter.

    You must meet all conditions for the rule to assign the selected values to an automated test.

  8. Select if the rule assigns application modules and a test owner if those fields already have values, or only if they are empty.

  9. As you edit the rule, view how many currently existing tests you will change with the new rule.

  10. Click Create Rule. The rule affects all current and future tests that match the filter. This includes tests currently in the system, as well as new automated tests added in the future.

To manage test assignment rules:

  1. In the Tests tab select one or more tests, and click Assign to Application Modules .

  2. Select one or more application modules and click Assignment Rule. The Create or Edit Assignment Rule dialog box opens.

  3. Click the Manage Rules link in the lower left corner.

  4. To edit a rule, click the rule ID link and modify its definitions as described above.

    To sort, filter, group, delete, and export rules, use the toolbar options similar to other entities in grids.

Tip: You can also edit rules in Settings > Test Assignment Rules. For details, see Create and manage test assignment rules.

Back to top

Next steps: