Automated testing flow (pipelines)
This flow describes how to bring automated test run results into ALM Octane using CI server pipelines and include them in the overall analysis of your product.
Automated tests, as well as load and performance tests are managed in external testing tools. You can set up a CI server to run those automated tests, and then set up ALM Octane to collect those results. This enables you to see a comprehensive picture of your application's quality and coverage.
This topic describes working with a Jenkins CI server, but you can create a similar set up with other CI servers as well.
- Set up your automated tool to run tests.
- Configure your CI server to trigger the runs, publishing the results (JUNit, NUnit, etc.) to the CI server.
- Integrate ALM Octane with your CI server using an ALM Octane plugin.
- Set up and run pipelines that run the tests.
- The ALM Octane plugin collects the results and sends them to ALM Octane.
Note: If you are working with UFT tests, you can set up a straightforward integration with UFT without using pipelines. For details, see Set up UFT integration.
Prepare a machine with your testing tool installed, which can access and run the automated tests.
Although the tests are run via the CI server, you do not need to configure a full CI system. To run Micro Focus automated tests on Jenkins and send the run results to ALM Octane, only the steps below are required.
For other automated testing tools, set up whatever those tools require on Jenkins. Make sure that the job running your tests publishes the test run results to Jenkins.
Jenkins setup for Micro Focus automated tests:
Install the Jenkins server.
Install the Application Automation Tools plugin.
This plugin lets you:
Run Jenkins jobs specifically designed to run UFT, StormRunner Functional, or Performance Center tests.
Prepare the test run results as required by ALM Octane.
Send the results to ALM Octane, enabling you to view the results as part of your overall quality analysis.
For details on downloading, installing, and configuring this plugin, see the Application Automation Tools wiki page.
Note: To support Performance Center tests, use version 5.2 or later or the plugin.
For UFT tests running in your pipeline:
Configure Jenkins to locate your tests. If the tests are stored in ALM, provide the ALM location. If they are in an SCM repository, check them out to the Jenkins workspace on your UFT machine.
- Define the UFT machine as an execution node on Jenkins. For details, see the section on execution nodes in the Application Automation Tools wiki page.
Create a Jenkins project to run the automated tests and publish the results.
For StormRunner Load tests, set up Jenkins to run your tests as described in the StormRunner Load Help Center. Add a post-build action to Publish JUnit test result report.
For UFT, StormRunner Functional, or Performance Center tests, follow the instructions in Application Automation Tools wiki page to do the following:
Build a free-style software project and add build steps that execute the tests.
Add the relevant post-build action to publish the test results. This ensures that the results are sent to ALM Octane in the format it recognizes.
For the Report archive mode select to always archive the test reports.
Additional setup requirements
For UFT tests, make sure to limit the project to run on the machine that you defined as the execution node.
For Performance Center tests, configure the connection to the Performance Center machine.
To send automated test run results to ALM Octane, the Jenkins jobs that runs the tests must be included in a pipeline run.
Create a pipeline from ALM Octane or from the Jenkins project.
From ALM Octane
When a pipeline runs, the results are sent to ALM Octane.
To run a pipeline, do one of the following:
In the ALM Octane Pipelines module, select the pipeline and click the Run Pipeline button.
In Jenkins, manually launch the build project you created.
In Jenkins, schedule the build project to run at a pre-determined time.
After the pipeline and test runs are finished, the plugin sends the test results to ALM Octane.
ALM Octane creates automated test and test run entities based on the test results it receives. If a relevant automated test already exists, the results are associated with that test. For details, see Creating automated test entities.
Assigning automated tests to application modules and backlog items in ALM Octane lets you view the test results in context. You can then use these results to analyze the progress and quality of your release and product.
Assigning owners to automated tests helps accelerate problem resolution. You can configure a pipeline to notify test owners when their test runs fail.
For details, see Assign tests to application modules and backlog items.
The test run results are available in pipeline runs and in automated test runs.
Open a specific test run for detailed results. To open the detailed report provided by the testing tool, click Testing tool report in the upper right corner of the test run .
Note: Performance Center provides detailed reports only for tests that pass successfully.
View the pipeline run results in the Pipelines module. For details, see Run pipelines.
If you assigned your tests to backlog items and application modules, the results are included in your release and product tracking dashboard widgets.