Automated testing

The following flow describes how to bring automated test run results into ALM Octane using CI server pipelines and include them in the overall analysis of your product.

Overview

Automated tests, as well as load and performance tests, are managed in external testing tools. You can set up a CI server to run those automated tests, and then set up ALM Octane to collect their results. This enables you to see a comprehensive picture of your application's quality and coverage.

This topic describes working with a Jenkins CI server, but you can create a similar set up with other CI servers as well.

Common flow:

  • Set up your automated tool to run tests.
  • Configure your CI server to trigger the runs, publishing the results to the CI server.
  • Integrate ALM Octane with your CI server using an ALM Octane plugin.
  • Set up and run pipelines that run the tests.
  • The ALM Octane plugin collects the results and sends them to ALM Octane.

Note: If you are working with UFT One tests, you can set up a straightforward integration with UFT One without using pipelines. For details, see Set up UFT One integration.

Back to top

Prepare a testing tool environment

Prepare a machine with your testing tool installed, which can access and run automated tests.

Back to top

Set up Jenkins to trigger test runs

Although the tests are run via the CI server, you do not need to configure a full CI system. To run OpenText automated tests on Jenkins and send the run results to ALM Octane, only the steps below are required.

For other automated testing tools, set up whatever those tools require on Jenkins. Make sure that the job running your tests publishes the test run results to Jenkins.

To set up Jenkins for OpenText automated tests:

  1. Install the Jenkins server.

  2. Install the Application Automation Tools plugin.

    This plugin lets you:

    • Run Jenkins jobs specifically designed to run automation tests, such as UFT One or LoadRunner Enterprise tests.

    • Prepare the test run results as required by ALM Octane.

    • Send the results to ALM Octane, enabling you to view the results as part of your overall quality analysis.

    For details on downloading, installing, and configuring this plugin, see the Application Automation Tools wiki page.

    Note: To support LoadRunner Enterprise tests, use version 5.2 or later or the plugin.

  3. For UFT One tests running in your pipeline:

    • Configure Jenkins to locate your tests. If the tests are stored in ALM, provide the ALM location. If they are in an SCM repository, check them out to the Jenkins workspace on your UFT One machine.

    • Define the UFT One machine as an execution node on Jenkins. For details, see the section on execution nodes in the Application Automation Tools wiki page.

Back to top

Set up a Jenkins job to run tests

Create a Jenkins project to run automated tests and publish the results.

For LoadRunner Cloud tests, set up Jenkins to run your tests as described in the LoadRunner Cloud Help Center. Add a post-build action to Publish JUnit test result report.

For UFT One or LoadRunner Enterprise tests, follow the instructions in Application Automation Tools wiki page to do the following:

  1. Build a free-style software project and add build steps that execute the tests. 

  2. Add the relevant post-build action to publish the test results. This ensures that the results are sent to ALM Octane in the format it recognizes.

    For the Report archive mode select to always archive the test reports.

Additional setup requirements:

Back to top

Create a pipeline with Jenkins jobs

To send automated test run results to ALM Octane, the Jenkins jobs that runs the tests must be included in a pipeline run.

Create a pipeline from ALM Octane or from the Jenkins project:

Back to top

Run the pipeline

When a pipeline runs, the results are sent to ALM Octane.

To run a pipeline, do one of the following:

  • In the ALM Octane Pipelines module, select the pipeline and click the Run Pipeline button.

  • In Jenkins, manually launch the build project you created.

  • In Jenkins, schedule the build project to run at a pre-determined time.

After the pipeline and test runs are finished, the plugin sends the test results to ALM Octane.

ALM Octane creates automated test and test run entities based on the test results it receives. If a relevant automated test already exists, the results are associated with that test. For details, see Create automated tests.

Back to top

Connect results to ALM Octane items

Assigning automated tests to application modules and backlog items in ALM Octane lets you view the test results in context. You can then use these results to analyze the progress and quality of your release and product.

Assigning owners to automated tests helps accelerate problem resolution. You can configure a pipeline to notify test owners when their test runs fail.

For details, see Assign tests to application modules and backlog items.

Back to top

Analyze test results

The test run results are available in pipeline runs and in automated test runs.

Open a specific test run for detailed results. To open the detailed report provided by the testing tool, click Testing tool report in the upper right corner of the test run .

Note: LoadRunner Enterprise provides detailed reports only for tests that pass successfully.

View the pipeline run results in the Pipelines module. For details, see Run pipelines.

If you assigned your tests to backlog items and application modules, the results are included in your release and product tracking dashboard widgets.

Back to top

See also: