Use Azure DevOps Server or Services to run a local test

This topic describes how to run a UFT One test stored on your local machine from Azure DevOps Server (formerly known as TFS).

In this topic:

Run a test saved on your local machine

In Azure DevOps Server:

1. Create a pipeline. (Note: If you are using TFS, skip this step)

   1. Create a build pipeline or a release pipeline, with the Empty job template.

      Note: A build pipeline is the pipeline type created when you do not explicitly create a release pipeline.

   2. Select the agent pool that includes the agent on which you want to run your tests.

   3. In the pipeline variables, add a UFT_LAUNCHER variable. The variable's value should be the full path to the UFTWorking folder.

   For more details, see the Microsoft Azure documentation.

2. To run tests using the open UFT One visible instance, if it exists, instead of opening a new UFT One instance for each run:

   Define a new pipeline variable: LEAVE_UFT_OPEN_IF_VISIBLE, and set its value to true (or yes or 1).

   UFT One remains open and the end of the run as well.

3. Add a UFT One task and place the step in the correct place in the build order.

   Note: If you are working with a release pipeline, add the task in the relevant stage, then place the step in the build.

   1. From the Task catalog, select the Test tab. A list of all available test tasks is displayed.

   2. From the Test tab, select the UFT One File System Run task and click Add. A new empty task is added as part of your build pipeline.

      Note: If you are working with a release pipeline, the task is added as part of your deployment process.

4. Provide the following information for your build step.

   Option	Description
   Display name	By default, the Azure DevOps Server CI system uses a preset descriptor for the task. Provide a more meaningful name for your step.
   Tests	The test, test batch file, or folder containing the tests to run. For multiple tests, the test path, name, or test result path should be separated by a comma.
   Timestamp pattern	Define the value format of the timestamp field used for reporting in the Extensions tab. The default value is yyyy-MM-dd HH:mm:ss.
   Timeout	The amount of time (in seconds) to wait if there is a problem opening or running a test. If the field is left blank, there is no timeout.
   Cancel run on first failure	Select whether to cancel the entire job if one test in your test run failed.
   Generate 'Failed Tests' report	Select whether to generate a report which provides information on your tests' failed steps. After the run, the report is available in the Extensions tab.

   Alternative: Use a batch file

   Alternatively, indicate a batch file that contains a list of tests and their parameters.

   • Specify multiple tests, or the same test several times with different parameters each time.
   • Use the reportPath parameter to define a specific path for your test result.

   The batch file should have an .mtbx extension and use the following syntax:

   <Mtbx>
    <Test name="test1" path="c:\tests\APITest1">
    <Parameter name="A" value="abc" type="string"/>
    ....
    </Test>
    <Test name="test2" path="c:\tests\test2">
     <Parameter name="p1" value="123" type="int"/>
     <Parameter name="p4" value="123" type="float"/>
      ....
    </Test>
    <Test name="test3" path="c:\tests\APITest3" reportPath="c:\reports\APITest3">
   </Mtbx>

5. (Optional) Configure the run step to upload UFT One's test result information to an Azure Storage location.

   Uploading the UFT One results to Azure Storage lets you to access the results from the Azure DevOps portal after the test run.

   Make sure you performed the steps described in Set up Azure Storage for your UFT One test results.

   Then enter the following options in your UFT One File System Run build step:

   Option	Description
   Do you want to upload the UFT report to the storage account?	Select Yes.
   Artifacts to upload	Select whether to upload only the html run results report, an archive of all the UFT One result files, or both. Note: When running an API test we recommend uploading the archive, as the standalone html run results report does not contain any information on the captured data. The content for RequestBody and HttpRawRequest steps is available in the html file contained in the archive.
   Report file name	Accept the default file name, which is based on the pipeline name and build number, or enter a name of your choice.

6. To run a local test on a mobile device or cloud browser, configure the Digital Lab options.

   Settings you define in the Azure task override the ones defined in the UFT One Record and Run settings. For anything you don't specify in Azure, the Record and Run settings are used.

   When upgrading a task from a version earlier than 24.2.0: The Digital Lab settings in the Azure tasks are now grouped under the Use Digital Lab option. When you upgrade your Azure tasks to the new extension, you must select this option so the Digital Lab settings continue to take effect.

   Configure the connection to Digital Lab:

   Option	Description
   Server	Provide the address of your Digital Lab server, in the format of http[s]://<server name>:<port>.
   Authentication type	Select the authentication mode to use for connecting to Digital Lab: Basic authentication. Authenticate using user name and password. Access key authentication. Authenticate using an access key you receive from Digital Lab. User name and Password. If you selected Basic authentication, enter your login credentials for the Digital Lab server. Access key. If you selected Access key authentication, enter the access key you received from Digital Lab.
   Use proxy settings	Provide the following information if you selected to connect using a proxy: Proxy Server: The address of your proxy server, in the format of <proxy server name>:<port>. Use proxy credentials. Enables you to connect to the Digital Lab server using a specific account. Proxy user name and password. The credentials used to connect to the Digital Lab server, if Use proxy credentials is selected.

   Select Use Device Lab to specify details about the device and application you want to test:

   Option	Description
   Device	Specify the device information of your mobile test, including DeviceID, Manufacturer, Model, OSType, and OSVersion. You can retrieve the values of these fields by running the Get Digital Lab Resources task (formerly named UFT Mobile Get Resources). If the value of DeviceID is provided, the other fields will be ignored. Examples: DeviceID: "123456789" Manufacturer: "Samsung", Model: "SM-G920F" OSType: "ANDROID", OSVersion: "7.0"
   Launch on start	Select which app you want to launch automatically at the start of your test. Home Screen: The device home screen. System App: Pre-installed system application on your device, such as camera. Digital Lab App: An application hosted on Digital Lab. System Application. Specify the system application that you want to test on your device. Main Digital Lab Application. Specify the main application that you want to test if you select Digital Lab App. The value format must be Identifier: "<app identifier>", Packaged: "<Yes/No>", where Identifier is required and Packaged is optional. You can get the app identifier by running the Get Digital Lab Resources task (formerly named UFT Mobile Get Resources).
   Additional Digital Lab Applications	Specify additional applications that you want to test. The value format must be Identifier: "<app identifier>", Packaged: "<Yes/No>", where Identifier is required and Packaged is optional. You can get the app identifier by running the Get Digital Lab Resources task (formerly named UFT Mobile Get Resources).
   Install	Select to install your application at the start of your test.
   Restart	Select to restart your application at the start of your test.
   Uninstall	Select to uninstall your application at the end of your test.

   Select Use Cloud Browser Lab to describe the browser to open for this test run:

   • Specify the URL of the application you want to test. The browser opens to this web page.

   • Specify the browser information, including it's geographic Location, the Operating System to run it on, the Browser type, and its Version.

     You can retrieve the values to use for these fields by running the Get Digital Lab Resources task.

     You can specify a specific version or one of the following:

     • latest: the most recent version supported by your Digital Lab server.

     • latest-1: the second most recent version supported by your Digital Lab server.

     • latest-2: the third most recent version supported by your Digital Lab server.

     For the list of supported versions, see the Digital Lab Support Matrix.

7. Configure the CI system control options for the step, including:

   Enabled	Specify whether the step should be run as part of this build.
   Continue on error	Instruct the CI system to stop or continue the build if there is an error on this step.

8. Set your pipeline to run: 

   Build pipeline: Save and queue the pipeline.

   Release pipeline: Create a release and deploy the pipeline.

   When the pipeline runs, the UFT One tests run as part of the task you added.

Back to top

View the test results

After the test run, you can view the run results in the following ways:

A visual report

In the Extensions tab of the run results, you can see a report including the following parts: 

• The UFT Report section shows the test run status, as well as links to the UFT One report and archive, if they were uploaded to Azure Storage.

• The Run Summary shows the number of tests that ran and the percentage of each status.

• If you selected the Generate ‘Failed Tests’ report option, the Failed Tests section shows a detailed breakdown of any failed steps.

Retrieve run result files

• The detailed failure results are stored in JUnit format in the junitreport.xml file located in the UFTWorking/res/Report_<build run number> folder.

• If the UFT One html report and the archive file were uploaded to Azure Storage, you can access these in the storage container.

• If you abort a job in the middle of a test run, the results of all tests executed before the job stopped are saved into a Results###.xml file in the %UFT_LAUNCHER%\res\Report_###\ folder.

If you run your UFT One tests from Team Foundation Server (TFS)

You can also view UFT One run results in the Summary tab of the TFS build run results.

For example:

Back to top

Release the UFT One license when a build is aborted

When you abort a build before it completes the test run, the UFT One license still remains in use.

To release the license, prepare a clean.vbs script and run the script directly from the command line after the build is aborted.

Alternatively, you can add a script task to your pipeline to let the task automatically release the license for you.

To release the license by running the script from the command line:

1. On the client machine where UFT One is running, create a clean.vbs file with the following contents:

   Copy code

   On Error Resume Next
   Set qtApp = CreateObject("QuickTest.Application")
   If qtApp.Launched Then
   qtApp.Test.Stop
   qtApp.Test.Close
   End If
   qtApp.Quit
   Set qtApp = Nothing

2. Run the script from the command line after you abort the build.

3. Check whether the license was successfully released. See Check whether the license was released.

To add a script task that automatically releases the license to your pipeline:

1. Prepare a clean.vbs file. See Step 1.

2. In your pipeline , add a Command line task.

3. In the new task, enter the following: 

   • In the Script box, enter:

     cscript <path to your clean.vbs file>

   • In the Run this task option, select Even if a previous task has failed, even if the build was canceled.

   This Script task always runs automatically and closes UFT One to release the license at the end of the build, whether it was aborted prematurely or not.

   Note: We recommend that you run the remote Azure agent as the admin.

After the build ends, check whether the license was successfully released. See Check whether the license was released.

To check whether the license was released:

1. On AutoPass License Server, click the Usage Reports menu.

2. Under UFT Enterprise Concurrent User, click the History tab.

3. In the table on the bottom of the page, check the CLIENT INFO column and find the record about your license.

   • If you can find the license record there, your license was successfully released.

   • If your license is on the In Use tab, your license remains in use.

Back to top

See also:

• UFT One and Azure DevOps Server or Services
• Known issues - Running tests from Azure DevOps