View and analyze run results

In the Scheduled Runs tab you can track test execution and view run results in a central location, as well as manage future runs.

View detailed run results

The run entity generated when a test runs contains information about the run. Details differ, depending on the type of run.

Automated Run results

In an Automated Run entity, click Download Report to download a detailed report of the test run results.

Tip: In some cases, the Download Report option is in a Download drop-down list.

OpenText Functional Testing and Selenium tests or units that run on the cloud are recorded in .MP4 video files, which are stored for 30 days. In the Automated Run entity, click Download > Download Recording to download the video file. You can use these videos to obtain a full picture of what occurred during the test run and improve debugging and collaboration.

For additional details on run result analysis, see the OpenText Core Software Delivery Platform Help Center.

Manual Run results

When tests are run manually, the amount of detail depends on the information that was added manually during the run.

Runs that were run by Autonomous Tester provide detailed run evidence:

In the Details tab, you can download screen captures and a screen recording of the entire run.

In the Report tab:

  • The run is labeled Ran by Autonomous Tester and includes detailed information, such as each step's status, actual outcome, and agent-generated details (AI Reasoning).

    For MBT tests, the report includes expandable rows for each unit. You can see the overall status of each unit, or expand units to see more details about the steps used to perform them.

    Tip: The Run by field in the Details tab reflects the user who triggered the run and not the Autonomous Tester.

  • If the test run used knowledge sources, a list of those Knowledge sources is available.

    In addition, the steps display any Additional guidelines that were gleaned from the sources used for the run. For details, see Leveraging knowledge sources (early access).

Analyze test run results with Aviator

Requires an Aviator license

Aviator can analyze your test run results, provide an overview of the run and a failure analysis, suggest potential fixes, and report on historical runs and trends. For autonomous runs, Aviator can suggest potential test improvements that will improve future autonomous runs.

Aviator analysis is available for automated runs and for manual runs run by the Autonomous Tester.

Depending on the data available, and what you select to include, Aviator can retrieve information from sources such as the following:

  • Error messages in the results

  • The full run results report

  • A recording of the test run
  • Data from previous runs (up to 5 previous runs)

The more information included in the analysis and the better its quality, the better the quality of the analysis provided. The best analysis is achieved by including a recording of the test run. Test run recordings are available for Selenium or OpenText Functional Testing tests or units that ran on the OpenText cloud, and for manual runs that were run by Autonomous Tester.

The basic analysis is based on data returned directly by the run. You can select to include the following additional context in the analysis, if available: Backlog items covered by this run, test run history, and a recording of the test run.

The following table explains which runs Aviator can analyze:

Type of runner Analysis available
Test Runner

A run analysis can be provided only for failed tests.

The analysis is based on the information you can see in the Error info section of the Automated Run.
Cloud Runner

A run analysis can be provided for failed and passed tests, based on the cloud run data.

For example, for tests that passed successfully, Aviator may still suggest potential fixes to make the test run more efficient.

Note: The Autonomous Tester is a type of Cloud Runner.

To generate a test or suite run result analysis:

  1. Open one of the following entities: An Automated Run, a Manual Run that was run by the Autonomous Tester, or a Suite Run that contains those types of runs.

  2. Open the Aviator Smart Assistant pane and click Analyze test run results or Analyze suite run results.

    Note: To generate a Suite Run analysis, the suite must contain at least 2 automated runs, and all runs must be run by the same runner (Automated or Autonomous Tester).

  3. (Optional) In an Automated Run, select resources to include in the analysis:

    • Coverage. Includes any backlog items listed in the Automated Run's Backlog Coverage field.

    • Run History. Includes up to 5 previous runs.

      If you select this option, an additional section is added to the analysis, providing information such pass/fail rate, common failures, and duration anomalies.

      Note: Duration anomalies are provided only for passed runs. Failed run durations are expected to vary in length as failures may happen at different parts of a run. However, if a passing run does not always take the same amount of time, you may want to investigate.

    • Recording. Includes a video recording of the test run.

      Selecting this option provides Aviator with a complete picture of the test and application's behavior during the run, resulting in a higher quality analysis.

      Recordings are only available for 30 days after the run.

      This option is supported only for Selenium or OpenText Functional Testing tests or units that ran on the OpenText cloud.

  4. Click Generate.

See also