Use Azure DevOps Server or Services to trigger a test saved in ALM

This topic describes how to run a UFT One test saved in ALM.

Note: This topic describes working with UFT One Azure DevOps extension version 5.*. To benefit from the latest functionality, we recommend updating existing tasks' versions to 5.*. When using this extension version, make sure that you installed the corresponding UFT.zip file from the ADM-TFS-Extension GitHub repository.

Prerequisite

Before you can run UFT One tests from ALM, you must first set up the integration with ALM.

Perform these steps once on the machine that the Azure DevOps Server will use to run the UFT One tests:

  1. Install the ALM Connectivity tool.

  2. Run ALM Client Registration.

For information on performing these steps, see the section on installing ALM tools and add-ins in the ALM Help Center.

Back to top

Run a test saved in ALM

In Azure DevOps Server, do the following:

  1. Create a pipeline. (Note: If you are using TFS, skip this step)

    1. Create a build pipeline or a release pipeline, with the Empty job template.

      Note: A build pipeline is the pipeline type created when you do not explicitly create a release pipeline.

    2. Select the agent pool that includes the agent on which you want to run your tests.

    3. In the pipeline variables, add a UFT_LAUNCHER variable. The variable's value should be the full path to the UFTWorking folder.

    For more details, see the Microsoft Azure documentation.

  2. Add a UFT One task or build step, in the correct place in the build order:

    1. In the Task catalog, select the Test tab to view a list of all available test tasks.

    2. Select the UFT One ALM Run task, and click Add. A new, empty task is added as part of your build plan.

  3. Provide the following details for your task:

    Display name (optional)

    By default, the Azure DevOps Server CI system uses a preset descriptor for the task.

    Provide a more meaningful name for your step.
    ALM server

    The ALM server where your test is saved. Use the following syntax:

    http://<ALMserver name> or IP:port/qcbin

    You may also need to connect to ALM using common registration mode.

    If this is the case, enter the URL format as http://<ALMserver name> or IP:port/qcbin/start_a.jsp?common=true.
    ALM Credentials

    Depending on the authentication type required by your ALM server, credentials can be a username and password or an API key for SSO authentication.

    • Username and password:

      User name: The ALM user name to use to access the server.

      Password: The password for the ALM user specified above.

    • SSO authentication:

      Select SSO enabled.

      Enter the Client ID and API key secret obtained from your ALM site administrator.

    Ensure that the ALM user or client specified in this task has the correct permissions to open and run the UFT One tests.

    For details on permissions, see the ALM Help Center.
    Domain The domain in the ALM server where the project containing the tests is stored.
    Project The project containing the tests.
    Test sets

    The test sets to run with this build task. Provide a full ALM path to these test sets.

    Each line in this field can contain a test set or folder.

    Note: When entering the test set's path, it does not matter if you specify the Root folder or not.

    Timeout

    The amount of time (in seconds) to wait if there is a problem opening or running the test.

    If the field is left blank, there is no timeout.
    Timestamp pattern

    Define the value format of the timestamp field used for reporting in the Extensions tab.

    The default value is yyyy-MM-dd HH:mm:ss.
    Run mode

    The machine on which to run the test sets. Options include:

    • Run locally (default) - Run tests on the agent machine.
    • Run remotely - Run tests on the host you specify in Testing tool host.
    • Run on a planned host - Run tests on a planned host, as defined in ALM.
    Testing tool host

    The name of the host selected to run the test sets.

    Note: Required for Run remotely run mode. Otherwise irrelevant.

    To hide information such as the ALM password or the SSO credentials, define Azure task variables for this information and lock them:

    1. Create the variable: 

      In the task's Variables tab add a variable for each field whose value you want to hide.

      Enter the value for the variable, select the lock option.

      Tip: Optionally, select Settable at queue time too, to enable editing this value when queueing the build.

      For example, create a MyPassword variable and enter your ALM password as its value.

    2. Link your variable to the relevant field in the task:

      In the Tasks tab, click Link settings.

      In Setting to link, select the field to link to the variable, and in Value, enter $(<variable name>). Click Link to complete the process.

      For example, select the Password field and enter $(MyPassword).

  4. (Optional) From the Test tab, add a Publish Test Results task or build step to publish test results to Azure Pipelines.

    Note: This option is supported when working with the UFT One Azure DevOps extension version 2.0.3 or later.

    Provide the following information for the step, to create a JUnit report based on UFT One's XML run result report:

    Display name Specify a name for the step.
    Test result format JUnit
    Test results files **/*.xml
    Search folder

    Provide the path to the folder that contains the test result XML files.

    Use the following format: 

    In a build pipeline: <Path to UFTWorking\res folder>\Report_$(Build.BuildNumber)

    In a release pipeline: <Path to UFTWorking\res folder>\Report_$(Release.ReleaseID)
    Test run title

    Specify a title for the report.

    Use environment variables in the title as follows, to base the report name on the pipeline name and the build or release number:

    In a build pipeline: $(System.DefinitionName)_$(Build.BuildNumber)

    In a release pipeline: $(System.DefinitionName)_$(Release.ReleaseID)

    This helps retrieve the report on Azure DevOps.

  5. Configure the CI system control options for the step, including:

    Enabled Specify whether the step should be run as part of this build.
    Continue on error Instructs the CI system to stop or continue the build if there is an error on this step.
    Run this task Indicates to the CI system when to run this step. To publish the results after running the tests failed, select Even if a previous task has failed, unless the build was canceled.

  6. Set your pipeline to run: 

    Build pipeline: Save and queue the pipeline.

    Release pipeline: Create a release and deploy the pipeline.

    When the pipeline runs, the UFT One tests run as part of the task you added.

    Note: If you manually stop the run from the Azure DevOps server, the UFT One process remains in use and you are unable to continue running the test by connecting to ALM.

    To fix this, manually stop the UFT.exe and UFTRemoteAgent.exe process and resume the build from the Azure DevOps server.

Back to top

View the test results

After the test run, you can view the run results in the following ways: 

A visual report

In the Extensions tab of the run results, you can see a report including the following parts: 

  • The UFT Report section shows the test run status.

  • The Run Summary shows the number of tests that ran and the percentage of each status.

Note: If you are working with a release pipeline, these results are available on the Stage level.

A JUnit test result report

If you added a Publish Test Results step, you can view the JUnit results on the Tests tab of the run results.

A link to the run results that are saved on ALM (if you run the tests from TFS)

UFT One automatically saves a link to the run results on ALM as an artifact with the build.

View this link on the Summary tab of the run results.

For example:

Back to top

See also: