Use Azure DevOps Server or Services to run a local test

This topic describes how to run a UFT One test stored on your local machine from Azure DevOps Server (formerly known as TFS).

Note: This topic describes working with UFT One Azure DevOps extension version 24.2.*. To benefit from the latest functionality, we recommend updating existing tasks' versions to 24.2.*. When using this extension version, make sure that you installed the corresponding UFT.zip file from the ADM-TFS-Extension GitHub repository.

Run a test saved on your local machine

In Azure DevOps Server:

  1. Create a pipeline. (Note: If you are using TFS, skip this step)

    1. Create a build pipeline or a release pipeline, with the Empty job template.

      Note: A build pipeline is the pipeline type created when you do not explicitly create a release pipeline.

    2. Select the agent pool that includes the agent on which you want to run your tests.

    3. In the pipeline variables, add a UFT_LAUNCHER variable. The variable's value should be the full path to the UFTWorking folder.

    For more details, see the Microsoft Azure documentation.

  2. To run tests using the open UFT One visible instance, if it exists, instead of opening a new UFT One instance for each run:

    Define a new pipeline variable: LEAVE_UFT_OPEN_IF_VISIBLE, and set its value to true (or yes or 1).

    UFT One remains open at the end of the run as well.

  3. Add a UFT One task and place the step in the correct place in the build order.

    Note: If you are working with a release pipeline, add the task in the relevant stage, then place the step in the build.

    1. From the Task catalog, select the Test tab. A list of all available test tasks is displayed.

    2. From the Test tab, select the UFT One File System Run task and click Add. A new empty task is added as part of your build pipeline.

      Note: If you are working with a release pipeline, the task is added as part of your deployment process.

  4. Provide the following information for your build step.

    Option Description
    Display name

    By default, the Azure DevOps Server CI system uses a preset descriptor for the task. Provide a more meaningful name for your step.

    Tests The test, test batch file, or folder containing the tests to run. For multiple tests, the test path, name, or test result path should be separated by a comma.
    Timestamp pattern

    Define the value format of the timestamp field used for reporting in the Extensions tab.

    The default value is yyyy-MM-dd HH:mm:ss.

    Timeout

    The amount of time (in seconds) to wait if there is a problem opening or running a test. If the field is left blank, there is no timeout.

    Cancel run on first failure

    Select whether to cancel the entire job if one test in your test run failed.

    Generate 'Failed Tests' report

    Select whether to generate a report which provides information on your tests' failed steps. After the run, the report is available in the Extensions tab.

  5. (Optional) Configure the run step to upload UFT One's test result information to an Azure Storage location.

    Uploading the UFT One results to Azure Storage lets you to access the results from the Azure DevOps portal after the test run.

    Make sure you performed the steps described in Set up Azure Storage for your UFT One test results.

    Then enter the following options in your UFT One File System Run build step:

    Option Description
    Do you want to upload the UFT report to the storage account?

    Select Yes.

    Artifacts to upload

    Select whether to upload only the html run results report, an archive of all the UFT One result files, or both.

    Note: When running an API test we recommend uploading the archive, as the standalone html run results report does not contain any information on the captured data. The content for RequestBody and HttpRawRequest steps is available in the html file contained in the archive.

    Report file name

    Accept the default file name, which is based on the pipeline name and build number, or enter a name of your choice.

  6. To run a local test on a mobile device or cloud browser, configure the Digital Lab options.

    Settings you define in the Azure task override the ones defined in the UFT One Record and Run settings. For anything you don't specify in Azure, the Record and Run settings are used.

    When upgrading a task from a version earlier than 24.2.0: The Digital Lab settings in the Azure tasks are now grouped under the Use Digital Lab option. When you upgrade your Azure tasks to the new extension, you must select this option so the Digital Lab settings continue to take effect.

    Configure the connection to Digital Lab:

    Option Description
    Server Provide the address of your Digital Lab server, in the format of http[s]://<server name>:<port>.
    Authentication type

    Select the authentication mode to use for connecting to Digital Lab:

    • Basic authentication. Authenticate using user name and password.

    • Access key authentication. Authenticate using an access key you receive from Digital Lab.

    User name and Password. If you selected Basic authentication, enter your login credentials for the Digital Lab server.

    Access key. If you selected Access key authentication, enter the access key you received from Digital Lab.

    Use proxy settings

    Provide the following information if you selected to connect using a proxy:

    Proxy Server: The address of your proxy server, in the format of <proxy server name>:<port>.

    Use proxy credentials. Enables you to connect to the Digital Lab server using a specific account.

    Proxy user name and password. The credentials used to connect to the Digital Lab server, if Use proxy credentials is selected.

    Select Use Device Lab to specify details about the device and application you want to test:

    Option Description
    Device

    Specify the device information of your mobile test, including DeviceID, Manufacturer, Model, OSType, and OSVersion.

    You can retrieve the values of these fields by running the Get Digital Lab Resources task (formerly named UFT Mobile Get Resources).

    If the value of DeviceID is provided, the other fields will be ignored.

    Examples:

    DeviceID: "123456789"

    Manufacturer: "Samsung", Model: "SM-G920F"

    OSType: "ANDROID", OSVersion: "7.0"

    Launch on start

    Select which app you want to launch automatically at the start of your test.

    • Home Screen: The device home screen.

    • System App: Pre-installed system application on your device, such as camera.

    • Digital Lab App: An application hosted on Digital Lab.

    System Application. Specify the system application that you want to test on your device.

    Main Digital Lab Application. Specify the main application that you want to test if you select Digital Lab App.

    The value format must be Identifier: "<app identifier>", Packaged: "<Yes/No>", where Identifier is required and Packaged is optional. You can get the app identifier by running the Get Digital Lab Resources task (formerly named UFT Mobile Get Resources).

    Additional Digital Lab Applications

    Specify additional applications that you want to test.

    The value format must be Identifier: "<app identifier>", Packaged: "<Yes/No>", where Identifier is required and Packaged is optional. You can get the app identifier by running the Get Digital Lab Resources task (formerly named UFT Mobile Get Resources).

    Install Select to install your application at the start of your test.
    Restart Select to restart your application at the start of your test.
    Uninstall Select to uninstall your application at the end of your test.

    Select Use Cloud Browser Lab to describe the browser to open for this test run:

    • Specify the URL of the application you want to test. The browser opens to this web page.

    • Specify the browser information, including it's geographic Location, the Operating System to run it on, the Browser type, and its Version.

      You can retrieve the values to use for these fields by running the Get Digital Lab Resources task.

      You can specify a specific version or one of the following:

      • latest: the most recent version supported by your Digital Lab server.

      • latest-1: the second most recent version supported by your Digital Lab server.

      • latest-2: the third most recent version supported by your Digital Lab server.

      For the list of supported versions, see the Digital Lab Support Matrix.

  7. Configure the CI system control options for the step, including:

    Enabled Specify whether the step should be run as part of this build.
    Continue on error Instruct the CI system to stop or continue the build if there is an error on this step.
  8. Set your pipeline to run: 

    Build pipeline: Save and queue the pipeline.

    Release pipeline: Create a release and deploy the pipeline.

    When the pipeline runs, the UFT One tests run as part of the task you added.

Back to top

View the test results

After the test run, you can view the run results in the following ways:

A visual report

In the Extensions tab of the run results, you can see a report including the following parts: 

  • The UFT Report section shows the test run status, as well as links to the UFT One report and archive, if they were uploaded to Azure Storage.

  • The Run Summary shows the number of tests that ran and the percentage of each status.

  • If you selected the Generate ‘Failed Tests’ report option, the Failed Tests section shows a detailed breakdown of any failed steps.

Note:  

  • If you are working with a release pipeline, these results are available on the Stage level.

  • Sometimes, the Extensions tab is not displayed if you abort the job in the middle of the test run. Even if the Extensions tab is available and you selected the Generate 'Failed Tests' report option, the Failed Tests section is not shown.

Retrieve run result files

  • The detailed failure results are stored in JUnit format in the junitreport.xml file located in the UFTWorking/res/Report_<build run number> folder.

  • If the UFT One html report and the archive file were uploaded to Azure Storage, you can access these in the storage container.

  • If you abort a job in the middle of a test run, the results of all tests executed before the job stopped are saved into a Results###.xml file in the %UFT_LAUNCHER%\res\Report_###\ folder.

If you run your UFT One tests from Team Foundation Server (TFS)

You can also view UFT One run results in the Summary tab of the TFS build run results.

For example:

Back to top

Release the UFT One license when a build is aborted

When you abort a build before it completes the test run, the UFT One license still remains in use.

To release the license, prepare a clean.vbs script and run the script directly from the command line after the build is aborted.

Alternatively, you can add a script task to your pipeline to let the task automatically release the license for you.

Note: If UFT One is visible and busy when the script runs, the cleanup.vbs might not be able to interrupt it. In that case, you need to stop UFT One manually and close it to release the license.

To release the license by running the script from the command line:

  1. On the client machine where UFT One is running, create a clean.vbs file with the following contents:

    Copy code
    On Error Resume Next
    Set qtApp = CreateObject("QuickTest.Application")
    If qtApp.Launched Then
    qtApp.Test.Stop
    qtApp.Test.Close
    End If
    qtApp.Quit
    Set qtApp = Nothing
  2. Run the script from the command line after you abort the build.

  3. Check whether the license was successfully released. See Check whether the license was released.

To add a script task that automatically releases the license to your pipeline:

  1. Prepare a clean.vbs file. See Step 1.

  2. In your pipeline , add a Command line task.

  3. In the new task, enter the following: 

    • In the Script box, enter:

      cscript <path to your clean.vbs file>

    • In the Run this task option, select Even if a previous task has failed, even if the build was canceled.

    This Script task always runs automatically and closes UFT One to release the license at the end of the build, whether it was aborted prematurely or not.

    Note: We recommend that you run the remote Azure agent as the admin.

After the build ends, check whether the license was successfully released. See Check whether the license was released.

To check whether the license was released:

  1. On AutoPass License Server, click the Usage Reports menu.

  2. Under UFT Enterprise Concurrent User, click the History tab.

  3. In the table on the bottom of the page, check the CLIENT INFO column and find the record about your license.

    • If you can find the license record there, your license was successfully released.

    • If your license is on the In Use tab, your license remains in use.

Back to top

See also: