Run and analyze pipelines

After creating and configuring pipelines in ALM Octane, run the pipelines on your CI server. You can then view and analyze the pipeline run results, and begin tracking your build quality.

What happens when a pipeline runs?

When a pipeline runs, the following occurs:

  • ALM Octane collects the results of the pipeline step runs. These are the builds that comprise the run.

    The pipeline run's number is the build number of the root step in the pipeline.

    The pipeline run's status is determined by the result of the root job. Therefore, for the pipeline run status to reflect the results of other jobs in the flow, make sure that on your CI server, the results of those jobs are aggregated up to the root job.

  • ALM Octane collects the results of the automated tests that run as part of the pipeline. ALM Octane creates automated test and test run entities associated with the test run results it collects. If a relevant automated test already exists, the results are associated with that test. For more details, see How ALM Octane creates automated test and test run entities from test run results.

    View the automated tests alongside the manual tests in ALM Octane. View the test run results when tracking the quality of your products and releases.

  • ALM Octane collects any available SCM commit information.

  • If you set up security testing integration, a pipeline run triggers a security assessment of your application's code, and ALM Octane displays the newly found vulnerabilities in the pipeline run.

    This enables you to quickly identify and correct security vulnerabilities introduced into the code.

    For details, see Set up security testing integration (technical preview).

  • If you set up code coverage tracking, your JaCoCo and LCOV reports are sent from Jenkins to ALM Octane. For details, see Track code coverage in pipeline runs.

  • After the pipeline run ends, ALM Octane displays detailed information about the pipeline status, about any builds or tests that failed as part of the pipeline run, commits that may be related to those failures, code coverage information, and more.

    This information helps you analyze the quality and status of your build, product, and release.

Back to top

Trigger a pipeline run from ALM Octane (Optional)

DevOps admin permissions are required.

  1. In the ALM Octane Pipelines module, open the Pipelines tab and select a pipeline.

  2. Click Run Pipeline.

ALM Octane triggers the CI server to run the pipeline's root job.

On Jenkins and Bamboo, the pipeline steps run under the CI Server user that you specified when configuring the ALM Octane CI plugin on the CI server. The pipeline run is limited by the permissions assigned to this user.

Back to top

View pipeline run results and analyze your build quality

After a pipeline runs ALM Octane displays information about the pipeline run status, its run history, related code changes, affected application modules, and more. You can also find analytic information about failed tests and tools to help you analyze failures.

In ALM Octane, go to the Pipelines module and click Pipelines.

Tip: You can filter the list to see only the pipeline that interest you. See Customize your pipeline display.

Use the options below to track and analyze the quality of your build.

Check the pipeline status (in the list of pipelines on the left)

For each pipeline that ran after it was added to ALM Octane, you can see details about the last pipeline run. The expanded view shows the run number, date, status, and duration.

You can also see the status of automated tests that ran as part of the build. The pipeline run results are aggregated (combined) at the highest level, the pipeline root.

A progress bar above the pipeline box indicates that a new run of the pipeline is currently in progress. The color of the progress bar indicates the status of the previous run.

  • The estimated percentage of completion in the progress bar is based on the duration of previous runs, if available. Otherwise, a striped bar is displayed.

  • Hover over the progress bar to see which pipeline step is currently running.

  • Pipeline run results are updated automatically when the run is finished.

Example:  

A pipeline fails on the CI server. This pipeline runs 255 automated tests. Of these tests, 252 passed, 2 failed, and 1 was skipped.

A new run of this pipeline is in progress, currently running the QA-Provision-Env step.

Tip: Click the number of test runs to open the tests in the Quality module. The total number of tests may be different than the number of test runs, as the same test may run more than once in a pipeline.

To filter the list of pipelines, click the filter icon and select a release.

Pipeline run overview: Get a quick comprehensive view of your pipeline's latest run

Use the Pipelines > Pipelines tab to quickly review the overall status of your pipeline run, see a break down of pipeline runs by status and duration, and gain insights into what might be causing failures.

Select a pipeline from the list of pipelines on the left.

What's in this view?

  • In the upper section you can see the most critical problem that ALM Octane identified as the source of the pipeline failure: Compilation Issue, Environment Issue, or Quality Issue.

  • You can modify the pipeline's name (on top) or release (bottom right, click the pencil to edit).

    If you modify a pipeline's release, future pipeline runs are associated with this release. Existing runs are not modified.

  • Click Failure Analysis to open the latest run's failure analysis tab and further investigate problems.

  • recent pipeline history timeline shows the status (by color) and duration (by size) of each run.

    Hover over a specific column in the timeline to see detailed information about that run.

    Click on a column to open a run and investigate it.

  • The Failure Analysis Insight cards show you the most critical problems in the selected pipeline so you know what to fix first. ALM Octane analyzes information regarding failed tests, problematic tests, build failure classification, related committers, and more. This data is then used to give CI owners actionable information, to fix failed pipelines quickly. For details, see Pipeline run failure analysis insights.

  • The Dashboard shows summary information about the selected pipeline's latest or most recent pipeline runs. By default, the dashboard shows how many failed tests and builds are currently being investigated, which tests are problematic, and which application areas are affected.

    For more details about the Application Modules with Failed Test Runs and Failed Builds widgets, see Additional information on pipeline overview widgets, below.

    Press + to add other widgets from the DevOps & Analytics widget gallery or create your own widgets.

    For example, you could add widgets that show code coverage per package and file, as well as code coverage trends over time, based on code coverage reports from Jenkins. For details, see Track code coverage in pipeline runs.

    Tip: To remove a widget, click the widget menu button in the right corner of the widget, and select Remove.

  • The panel on the right shows details about the selected pipeline's latest run: Time, duration, release, how many commits were included in this run, which application modules were affected, and so on.

    If the pipeline is set up to collect Fortify on Demand security testing results, then when the scan is complete, you can see how many new vulnerabilities were found in the scan triggered by the latest pipeline run. Click the number to open the Vulnerabilities tab for that pipeline run. For details, see Track security vulnerabilities (technical preview).

Drill down into the pipeline for more information

Click the pipeline's ID to open a pipeline and view more details about the pipeline and its runs:

  • The Details tab displays information about the pipeline's definitions, as well as the Pipeline run success rate and Average success duration. These are based on all pipeline runs, since the pipeline was created.

  • The Follow button enables you to receive notifications in email and in MyWork, according to your preferences. You can receive notifications if your commit was involved in a failed run, if your test fails, or if the pipeline run fails.

  • In the Runs tab, you can see a list of all the pipeline runs, dating back to when you added the pipeline to ALM Octane.

    For each run, you can see its status, duration, and more.

Learn more about a specific pipeline run

Open a specific pipeline run to view more details and to analyze test failures.

  1. In the Pipelines module, open the Pipelines tab and select a pipeline.

  2. Open a specific pipeline run:

    • To open the last run, click the pipeline's Run number.

    • To open a previous run, click the pipeline's ID to open the pipeline. In the Runs tab, click the ID of a specific run.

In the Details tab, you can see an overview of the pipeline run information. Time, status, number of changed files included in commits on this run, pipeline run history and more.

Tip: In the Changed Files widget, the Unassigned bar shows changed files that were not associated with any defects or user stories via the commit message.

  • Add tags to label the run. The tags are displayed in the pipeline's list of runs, and can be used to group pipeline runs in dashboard widgets.

  • Click the build number to open the specific run on the CI server.

  • Track code changes using the Commits and Backlog Items tabs.

  • Analyze pipeline failures using the Failure Analysis tab.

Back to top

Analyze pipeline failures

In a specific pipeline run, use the Failure Analysis tab to learn more about failed builds and automated tests that were part of this run.

Based on your analysis, you might decide to edit an automated test, roll back a committed change, create a defect for an area in your application, or even extend you project's timeline or reduce the planned content.

For details, see Analyze failures in a pipeline run.

Back to top

Additional information on pipeline overview widgets

  • Application modules with failed test runs: 

    See which application modules have the most failing tests.

    Lists the application modules currently associated with automated tests that failed in this pipeline run. The widget shows how many of the tests assigned to the application module failed.

    Note: To use this widget, automated tests must be assigned to application modules. For details on assigning items to application modules, see Work with application modules.

  • Failed builds classification:

    Displays the number of failed builds included in this pipeline run, broken down by type of failure. You can also see how many of the failures are currently assigned to someone for investigation.

    To use this widget, failed builds must be classified in the Failure Analysis tab. Classification can be done automatically by ALM Octane Help or manually, based on your own failure analysis. For details, see Classify build failures.

Tip: Some builds might not be included.

DevOps admins can configure hiding builds failures resulting from specific pipeline steps. For details, see Configure steps: Ignore or hide results of specific steps.

To check whether a step's builds are hidden, look at the step in the pipeline's Topology tab:

Back to top

Track code changes

If your CI server is set up to work with an SCM system, open a pipeline run to see commit changes and backlog items associated with that run. 

Based on the SCM data, ALM Octane helps you analyze the effects of the changes on your product, and identify Hotspots in your code. Hotspots are sensitive areas of code that are risky to change.

For details, see Track changes committed to your Source Control Management system.

Back to top

View and share a live summary of your pipelines

In the Pipelines module, click Live Summary to see a summary of your pipeline runs. If you have multiple pipelines running, you can click Choose Pipelines to view a subset of them, or to search for a specific one.

The summary provides information about the pipeline's history, its status, and progress. You can see: 

  • The status of the last few runs.

  • What triggered the last run (system or commit).

  • If triggered by a commit, you can see who committed and what files changed.

  • The progress of the current run.

  • After the tests are completed, an aggregated summary of the test run results.

This view is suitable, for example, for displaying on a group plasma screen. This enables the whole group to view the overall build status and quality.

Note: This information is obtained from the CI server and may include information from runs that occurred before the pipeline was added to ALM Octane.

Back to top

Next steps: