Autonomous Tester

Requires an Aviator license

Autonomous Tester runs tests written in natural language at scale on the OpenText cloud, and provides detailed run result assets.  

Overview

Autonomous Tester can run the following types of test: 

  • Manual

  • MBT

  • Gherkin

  • BDD (behavior-driven development) scenarios

To run your tests autonomously, add them to a test suite and configure them to run by Autonomous Tester. You can then run the suite on demand, or add it to a scheduled run.

Autonomous Tester runs tests using Aviator to interpret the natural language steps, and determine how to perform them on the application. This enables you to continue using tests written in natural language, without creating scripts or converting them to automated tests.

The run results provide detailed information about the run, including step outcomes and agent actions, as well as screen captures, and a screen recording of the whole run. This detailed information provides test run evidence, as well as the foundation for in‑depth analysis.

Note: You can use Cloudflare tunneling to access and test applications hosted on your local or private networks. For details, see To test local or private web applications.

Providing step information to Autonomous Tester

The Autonomous Tester carries out test steps described in natural language:

Test type Test instructions
Manual tests The test's Steps tab contains the steps to run.
Gherkin tests The test's Script tab contains the steps to run.
BDD scenarios The steps to run are described in the Script tab of the BDD Specification related to the scenario.
MBT tests

Unit names describe the action represented by the unit. If no other details are provided, these are the instructions that the Autonomous Tester runs.

  • If a unit's Steps tab contains steps, these are performed when running the unit.

  • If no steps are described within the unit, Autonomous Tester checks for a linked manual test and performs the steps it describes.

For all types of tests, you can provide additional information by referring the Autonomous Tester to additional knowledge sources in the system. For details, see Leveraging knowledge sources. Knowledge sources are consulted only if the information in the step is not enough to perform it.

Prepare tests for Autonomous Tester

Prepare your suite and tests in the Execution module so they can run autonomously.

To prepare tests for autonomous runs:

  1. Create or open a test suite and add your tests.

  2. (Optional) In the test suite Details pane, select Use Knowledge Source. This instructs the Autonomous Tester to use relevant knowledge sources defined in the system when necessary.

    Note: Knowledge sources are used for the run only if the information in the test was insufficient and additional context was required.

  3. In the test suite Planning tab, specify which tests to run autonomously and then click Assign to Autonomous Tester.

    • If you select specific tests to run autonomously, make sure to turn off the Include in next run option for all other tests in the suite. When running tests on the cloud, all the tests in a suite run must be using the same runner.

    • If you do not select any tests, the assignment affects all of the tests in this suite.

  4. In the Assign to Autonomous Tester dialog box that opens:

    1. If multiple Autonomous Tester runners are available, select the runner you want to use.

      If no runner is available or if you want additional runners with other names, your space administrator can create new runners. For details, see Create an autonomous testing runner.

    2. Click Configure.

    The following run-related fields are updated in the selected tests:

    Field Value
    Execution status Ready
    Run mode Automatically
    Test runner Autonomous Tester (or another Autonomous Tester you select)

  5. Click Run suite if you want to trigger the suite run now, or click Close to close the dialog box.

Run, track, and analyze autonomous test runs

After setting up your tests to run autonomously, you can run the suite by triggering it manually (Run Suite), planning a run (Plan Suite Run), or adding it to a scheduled run.

For details on running test suites, see the OpenText Core Software Delivery Platform Help Center.

To track and analyze the test runs:

  • During and after a run, you can track run progress in a test suite, in the Suite Runs tab, or in the Execution module, in the Scheduled Runs tab.

  • After the run ends, detailed run evidence and results are provided in a Manual Run entity:

    In the Details tab, you can download screen captures and a screen recording of the entire run.

    In the Report tab:

    • The run is labeled Ran by Autonomous Tester and displays each step's status, a step summary, and agent-generated details (AI Reasoning).

      For MBT tests, the report includes expandable rows for each unit. You can see the overall status of each unit, or expand units to see more details about the steps used to perform them.

      Tip: The Run by field in the Details tab reflects the user who triggered the run and not the Autonomous Tester.

    • If the test run used knowledge sources, a list of those Knowledge sources is available.

      In addition, the steps display any Additional guidelines that were gleaned from the sources used for the run. For details, see Leveraging knowledge sources.

  • Autonomous run results are incorporated into test and suite run results like any other test runs, enabling you to use all of the system's run analysis options.

    For example, you can analyze autonomous run results with Aviator at the test run and suite run levels.

For details, see View and analyze run results.

License usage

Each workspace is allowed up to 200 Autonomous Tester runs per month.

You can track the number of runs already used this month in the Current License Usage widget in the Execution Overview tab.

See also