View and analyze the pipeline overview

After a pipeline runs, ALM Octane displays information about the pipeline run status, its run history, related code changes, affected application modules, and more. You can also find analytic information about failed tests and tools to help you analyze pipeline run results.

View pipeline run results

In ALM Octane, go to the Pipelines module and click Pipelines.

Tip: You can filter the list to see only the pipeline that interest you. See Customize your pipeline display.

Use the options below to track and analyze the quality of your build.

Check the pipeline status (in the list of pipelines on the left)

For each pipeline that ran after it was added to ALM Octane, you can see details about the last pipeline run. The expanded view shows the run number, date, status, and duration.

You can also see the status of automated tests that ran as part of the build. The pipeline run results are aggregated (combined) at the highest level, the pipeline root.

A progress bar above the pipeline box indicates that a new run of the pipeline is currently in progress. The color of the progress bar indicates the status of the previous run.

  • The estimated percentage of completion in the progress bar is based on the duration of previous runs, if available. Otherwise, a striped bar is displayed.

  • Hover over the progress bar to see which pipeline step is currently running.

  • Pipeline run results are updated automatically when the run is finished.

Example:  

A pipeline fails on the CI server. This pipeline runs 255 automated tests. Of these tests, 252 passed, 2 failed, and 1 was skipped.

A new run of this pipeline is in progress, currently running the QA-Provision-Env step.

Tip: Click the number of test runs to open the tests in the Quality module. The total number of tests may be different than the number of test runs, as the same test may run more than once in a pipeline.

To filter the list of pipelines, click the filter icon and select a release.

Overview tab: Get a quick comprehensive view of your pipeline's latest run

Use the Pipelines > Pipelines tab to quickly review the overall status and trend of your pipeline run, see a break down of pipeline runs by status and duration, and gain insights into what might be causing failures.

Select a pipeline from the list of pipelines in the left pane to see its overview data on the right.

What's in this view?

  • In the upper section you can see the most critical problem that ALM Octane identified as the source of the pipeline failure: Compilation Issue, Environment Issue, or Quality Issue.

  • You can modify the pipeline's name (on top) or release (bottom right, click the pencil to edit).

    If you modify a pipeline's release, future pipeline runs are associated with this release. Existing runs are not modified.

  • Click Builds to open the latest run's Builds tab and further investigate build failures. For details, see Analyze builds.

  • Click Tests to open the latest run's Tests tab and further investigate problems and analyze the run results. For details, see Analyze tests.

  • recent pipeline history timeline shows the status (by color) and duration (by size) of each run.

    Hover over a specific column in the timeline to see detailed information about that run.

    Click on a column to open a run and investigate it.

  • The Failure Analysis Insights cards show you the most critical problems in the selected pipeline so you know what to fix first. ALM Octane analyzes information regarding failed tests, problematic tests, build failure classification, related committers, and more. This data is then used to give CI owners actionable information, to fix failed pipelines quickly. For details, see Failure Analysis Insight cards, below.

  • The Dashboard shows summary information about the selected pipeline's latest or most recent pipeline runs. By default, the dashboard shows how many failed tests and builds are currently being investigated, which tests are problematic, and which application areas are affected.

    For more details about the Application Modules with Failed Test Runs and Failed Builds widgets, see Additional information on pipeline overview widgets, below.

    Press + to add other widgets from the DevOps & Analytics widget gallery or create your own widgets.

    For example, you could add widgets that show code coverage per package and file, as well as code coverage trends over time, based on code coverage reports from Jenkins. For details, see Track code coverage in pipeline runs.

    Tip: To remove a widget, click the widget menu button in the right corner of the widget, and select Remove.

  • The panel on the right shows details about the selected pipeline's latest run: Time, duration, release, how many commits were included in this run, which application modules were affected, and so on.

    If the pipeline is set up to collect Fortify on Demand security testing results, then when the scan is complete, you can see how many new vulnerabilities were found in the scan triggered by the latest pipeline run. Click the number to open the Vulnerabilities tab for that pipeline run. For details, see Track security vulnerabilities.

Drill down into the pipeline for more information

Click the pipeline's ID to open a pipeline and view more details about the pipeline and its runs:

  • The Details tab displays information about the pipeline's definitions, as well as the Pipeline run success rate and Average success duration. These are based on all pipeline runs, since the pipeline was created.

  • The Follow button enables you to receive notifications in email and in MyWork, according to your preferences. You can receive notifications if your commit was involved in a failed run, if your test fails, or if the pipeline run fails.

  • In the Runs tab, you can see a list of all the pipeline runs, dating back to when you added the pipeline to ALM Octane.

    For each run, you can see its status, duration, and more.

Learn more about a specific pipeline run

Open a specific pipeline run to view more details and to analyze test failures.

  1. In the Pipelines module, open the Pipelines tab and select a pipeline.

  2. Open a specific pipeline run:

    • To open the last run, click the pipeline's Run number.

    • To open a previous run, click the pipeline's ID to open the pipeline. In the Runs tab, click the ID of a specific run.

In the Details tab, you can see an overview of the pipeline run information. Time, status, number of changed files included in commits on this run, pipeline run history and more.

  • Add tags to label the run. The tags are displayed in the pipeline's list of runs, and can be used to group pipeline runs in dashboard widgets.

  • Click the build number to open the specific run on the CI server.

  • Track code changes using the Commits and Backlog Items tabs.

  • Analyze pipeline run results using the Builds and Tests tabs.

Back to top

Analyze pipeline run results

In a specific pipeline run, use the Builds and Tests tabs to learn more about builds and automated tests that were part of this run.

You can see how many of your tests ran successfully, how many were skipped or unstable, and how many failed. You can find the reasons that tests are failing.

For details, see Analyze builds and Analyze tests.

Back to top

Additional information on pipeline overview widgets

  • Application modules with failed test runs: 

    See which application modules have the most failing tests.

    Lists the application modules currently associated with automated tests that failed in this pipeline run. The widget shows how many of the tests assigned to the application module failed.

    Note: To use this widget, automated tests must be assigned to application modules. For details on assigning items to application modules, see Work with application modules.

  • Failed builds classification:

    Displays the number of failed builds included in this pipeline run, broken down by type of failure. You can also see how many of the failures are currently assigned to someone for investigation.

    To use this widget, failed builds must be classified in the Builds tab. Classification can be done automatically by ALM Octane, or manually, based on your own failure analysis. For details, see Analyze builds.

Tip: Some builds might not be included.

DevOps admins can configure hiding builds failures resulting from specific pipeline steps. For details, see Configure steps: Ignore or hide results of specific steps.

To check whether a step's builds are hidden, look at the step in the pipeline's Topology tab:

Back to top

Track code changes

If your CI server is set up to work with an SCM system, open a pipeline run to see commit changes and backlog items associated with that run. 

Based on the SCM data, ALM Octane helps you analyze the effects of the changes on your product, and identify Hotspots in your code. Hotspots are sensitive areas of code that are risky to change.

For details, see Track changes committed to your Source Control Management system.

Back to top

View and share a live summary of your pipelines

In the Pipelines module, click Live Summary to see a summary of your pipeline runs. If you have multiple pipelines running, you can click Choose Pipelines to view a subset of them, or to search for a specific one.

The summary provides information about the pipeline's history, its status, and progress. You can see: 

  • The status of the last few runs.

  • What triggered the last run (system or commit).

  • If triggered by a commit, you can see who committed and what files changed.

  • The progress of the current run.

  • After the tests are completed, an aggregated summary of the test run results.

This view is suitable, for example, for displaying on a group plasma screen. This enables the whole group to view the overall build status and quality.

Note: This information is obtained from the CI server and may include information from runs that occurred before the pipeline was added to ALM Octane.

Back to top

Failure Analysis Insight cards

ALM Octane analyzes information regarding failed tests, problematic tests, build failure classification, related committers, and more. The Failure Analysis Insights cards give CI owners actionable information to fix failed pipelines quickly.

ALM Octane categorizes pipeline failures into three types: Compilation, Environment, or Quality issues. The cards for each of these are displayed in order of urgency. Each card shows you the problem and its root cause, the users directly related to the problem, and the percentage of issues being handled. This information tells you if a problem needs immediate attention and whom to contact in order to deal with it.

Click the right and left arrows to navigate between the cards.

Note: To provide data for Failure Analysis Insights, you need to manually classify failed builds. For details, see Build failure classification.

Compilation Issues

Compilation issues include any failure with at least one build which failed due to compilation issues. Related users includes anyone who may have caused the issue (generally committers in the pipeline run), as well as anyone assigned in the compilation build's Who's on it field.

For each build that was classified as a compilation issue, the card shows if it failed in an earlier run, so you know when the failure started. The bottom of the card indicates the percentage of builds that have someone assigned in the Who's on it? field, indicating that the problem is being handled.

You can click on the number of builds, or on a specific build name, to drill to the relevant details.

Environment Issues

Environment Issues include any build failures classified as Environment issues. The cards are divided into subgroups based on the out-of-the-box tags (such as Environment - Database, and Environment - Network), or custom tags you added. Each card shows all the problematic jobs in the relevant category, with the oldest jobs listed first.

Related users includes anyone assigned in the relevant builds' Who's on it? field. The bottom of the card indicates the percentage of builds that have someone assigned in the Who's on it? field.

You can click on the number of builds, or on a specific build name, to drill to the relevant details.

Quality Issues

If any tests failed in a pipeline run, and the build is not classified as a compilation or environment issue, you will see a Quality Issue card. The cards are divided into two types: Quality - Builds with most failed tests, and Quality - Committers related to most test failures.

Tip: Admins can define a percentage threshold that determines which test failures are displayed on the card. For details, see the FAILURE_ANALYSIS_INSIGHT_TEST_FAILURE_THRESHOLD configuration parameter under FAILURE_ANALYSIS_INSIGHT_TEST_FAILURE_THRESHOLD.

You want to focus your attention on test failures that are stable and indicate a regression that needs to be fixed quickly. The Quality - Builds with most failed tests card therefore shows you the number of test failures that are stable, out of the total test failures. These are either tests that are failing consistently or failed for the first time in this run.

Related users include any committers who may have caused the issue or anyone assigned in the test runs' Who's on it? field. The bottom of the card indicates the percentage of tests in these builds that have someone assigned in the Who's on it field.

You can click on the number of builds, or on a specific build name, to drill to the details of stable test failures.

The Quality - Committers related to most test failures card shows you which committers are related to the most test failures. Committers are listed in order of the number of stable test failures that they impacted, so you know whom to contact to resolve problems.

The bottom of the card indicates the percentage of tests that are being handled, regardless of who is handling them.

You can click on a line showing a number of tests to drill to the details on test failures related to the selected committer.

Back to top

See also: