Trigger automated test runs in your testing framework

This topic explains how to integrate ALM Octane with your testing framework. You can then select automated tests from your testing framework in ALM Octane, and run them in the context of a test suite.

Testing framework integration overview

This integration enables ALM Octane to run tests in your testing framework via your CI server.

This is useful in a number of scenarios. For example, suppose your CI server ran automated nightly tests and the ALM Octane pipeline shows that a specific test failed. After pushing a fix you can run the specific test directly from ALM Octane, rather than running the full pipeline which can be time-consuming.

The ALM Octane-testing framework integration includes the following steps:

  1. Create a job in your CI server. To set up the integration, create a job on your CI server to run tests on your testing framework.

  2. Create a test runner in ALM Octane. In ALM Octane settings, create a test runner entity that corresponds to this job on your CI server.

  3. Run tests. In ALM Octane, create a test suite that includes tests that you want to run in your testing framework, and run the suite. ALM Octane triggers the test runs via the job in the CI server.

  4. Analyze release and product quality. Track the test results as part of the overall data in the backlog, quality, and dashboard modules.

Back to top

Prerequisites

  1. You must have a CI server connected to ALM Octane. For details, see Set up CI servers.

  2. Within ALM Octane, create a pipeline to inject tests from your testing framework via your CI server. For details, see Create and configure pipelines.

  3. Out-of-the-box, the integration supports the following frameworks via Jenkins or Bamboo:

    • JUnit and TestNG tests over Maven

    • Protractor

    • Gradle

    For other test frameworks or CI servers, contact ALM Octane Support.

Back to top

Create a job on your CI server to run tests in your testing framework

On your CI server, create a job that runs specific tests on your testing framework.

In addition to the command to run specific tests on your framework, the job should also include the following:

  • testsToRun parameter. This parameter receives a lists of tests from ALM Octane that the job will then run. The parameter uses the following syntax: v1:package name|class name|test name;package name|class name|test name.

    Note that v1 refers to the current version of the testsToRun format, which may change in the future.

  • Repository URL and credentials. This is not mandatory, but you will usually need the repository URL and credentials so that your job can get the latest version of the tests to the CI local file system.

  • Micro Focus ALM Octane testing framework converter build step/task, that converts the testToRun raw data to your relevant framework format. To help you do this, you can use the Micro Focus ALM Octane testing framework converter in the Jenkins plugin version 5.5.2 or later, or in the Bamboo plugin version 1.6 or later.

    Note that the test converter step must be run before the test execution step.

  • For Jenkins only: Publish JUnit test result report post-build action.

Back to top

Examples of testsToRun usage in Jenkins

The following are examples of how to use the testsToRun parameter in your testing framework, via Jenkins.

Back to top

Create a test runner in ALM Octane

  1. In ALM Octane, click Settings > Spaces and select a workspace.

  2. Select the DevOps > Test Runners tab.

  3. Click + Test Runner.

  4. Define a name for the test runner, for example JUnit Runner.

  5. Select your testing framework, for example JUnit.

  6. Select your CI server.

  7. Select the job that you created in the CI server to run the tests in your testing framework. The list displays the jobs that have the testsToRun parameter.

  8. Click Save.

ALM Octane creates a test runner to trigger the job that runs your tests.

Note: Each job can only be used for one test runner.

Back to top

Run tests from ALM Octane

Assign test runners to tests in the Quality module. You can then add the tests to test suites, and run them from ALM Octane.

ALM Octane triggers test runs via the job in the CI server that is specified in the test runner. After adding tests to a suite you can select a different test runner for a test in the suite. In addition, after planning a suite you can select a different test runner for a test run.

A suite can include automated tests using multiple test runners. When a suite includes both manual tests and automated tests, the automated tests run in the background while you perform the manual tests.

When you run a test, its status in ALM Octane is Planned until the test runner starts to run the tests in the CI server. The test's status changes to In Progress, until the test results are returned to ALM Octane.

Back to top

Manage test runners

After you associate a test with a test runner, you might want to make changes at a later stage.

For example, suppose you delete a CI server or the test runner's related job, and you want to keep the association between the tests and the test runner. In this case, edit the test runner within DevOps > Test Runners, and specify a different CI server or job for the tests associated with this test runner.

Similarly, if you delete a CI server in ALM Octane, the test runner associated with this CI server is not deleted, and its tests remain associated with the test runner. However, you will not be able to run any of the tests associated with the test runner.

Back to top

Working with parameters in test runners

If your CI server job is parameterized, you can define different sets of parameter values that can be used when running tests using a test runner. Each test runner can have multiple sets of parameter values.

For example, suppose you want to run one test suite on one deployed environment, and another on a different one. You can create one parameter set (Set 1) using the default URL and port, and another (Set 2) with a different URL and port. You can then run the same suite a few times using different settings.

To configure parameters in the test runner:

  1. When creating a test runner, you select a job to run tests. If this job is parameterized, a table appears showing the parameters configured in the CI server, with their default values.

  2. In the column headings, add columns to define additional sets of parameter values.

  3. You can rename each column heading as needed.

  4. You can edit parameter values in each set of parameters as needed.

  5. To delete a column or restore its default values, select the relevant menu item in the upper right corner of the column.

  6. Click Save.

Note:

  • ALM Octane currently supports boolean and string parameters only. If a parameter contains no value or an incorrect value (for example if the parameter is boolean but the value entered is string), it is skipped during the test run.

  • Parameters are not supported for UFT test runners.

To select parameters when running a test:

You define a parameter set for suite run execution when tests are in Planned state. All of the tests using a specific runner will run using the selected parameter values.

  1. In a test suite with automated tests, select Plan Suite > Suite Run > Runs.

  2. In the Test Runners tab, select the Settings icon on a test runner.

  3. From the dropdown list of parameter sets that have been defined for this test runner, select a parameter set to use in the test execution.

    The Data Set field can be added to the Runs grid to see the parameter set that was selected for this suite run.

  4. Run the test suite.

    The test suite runs using one or more sets of parameter values that you selected.

Back to top

Stop automated tests in a suite run

Note: This feature is currently supported on Jenkins and Bamboo test runners.

If you are running a test suite with automated tests you can stop them before the suite is done.

Back to top

Rerun automated tests

If you ran a test suite in an automated run, and you want to run it again (for example, if a test failed) you can rerun it without having to create a new suite run.

Note: A rerun is only possible after the previous run has finished.

Back to top

Next steps: