AI-based testing in UFT One

GUI UFT Mobile and Web tests only

This topic describes how to install and use UFT One's Artificial Intelligence (AI) Features to identify objects the way a person would. This enables you to run the same test on different platforms and versions, regardless of the objects' implementation.

UFT One AI-based testing support

The following table describes the level of AI-based testing support provided in each UFT One version.

UFT One version Mobile apps Desktop web
15.0 Supported Not supported
15.0.1 Supported Technical preview
15.0.2 Supported Supported

Back to top

AI-based testing overview

UFT One's AI Features enable your tests to interact with the application you are testing in the same way a person would. UFT One uses AI to identify objects visually, based on a wide variety of images, context, and sometimes text.

For example, UFT One AI can identify many forms of search fields, user profile areas, input fields, button, shopping carts and more.

Some advantages of AI-based object identification are: 

1. More intuitive test scripts.

2. Tests are technology agnostic, identifying objects visually, regardless of the UI technology details used behind the scenes.

3. Tests are easier to maintain, as an object changing location, UI framework, or even shape, won’t break the test script as long as the object remains visually similar or its purpose remains clear.

Back to top

Install UFT One's AI Features

To use UFT One's AI features, you must install and enable them.


Before installing the AI features, make sure you have the following installed: 

  • A 64-bit operating system

  • Microsoft Visual C++ 2013 Redistributable

  • The required Windows feature, depending on your Windows operating system:

    For the most up-to-date instructions on enabling Windows features, see the Microsoft documentation.

Install UFT One's AI features

Expand one of the sections below, depending on the UFT One version you are using: 

Note: AI Codeless Testing is not part of UFT One. It is a standalone tool that lets you experiment with AI Codeless testing. Installing AI Codeless Testing requires additional prerequisites. See the AI Codeless Testing Help Center.

Back to top

Inspect your application for objects

Use AI inspection to identify objects in your application that you can use in your test steps.


If you are testing an application on a mobile device If you are testing a desktop web application
  1. Make sure the Mobile Add-in is installed and loaded.

    To test web applications on your mobile device, make sure the Web Add-in is installed and loaded.

  2. Connect to UFT Mobile and open your mobile device and the application you want to test. For details, see Open a remote access window and application manually.

  1. Make sure the Web Add-in is installed and loaded.

  2. Open the application.

    For best identification results:

    • Maximize the browser window and set zoom level to 100%.
    • Set your Windows display settings to display all text and apps at 100%.
    • Make sure no banners are displayed in the browser in addition to your web page.

Supported browsers: Chrome, Chromium-based Edge, Firefox, Internet Explorer.

Headless browsers are not supported.

Inspect your application

Help design the future of AI-based testing in UFT One

Click How is the detection? Help us improve to open the Feedback Tool and send Micro Focus feedback about the object detection. For details, see AI-based testing Feedback Tool.

Back to top

Inspect application mockups for objects

Supported on UFT One versions 15.0.2 and later

Use AI Mockup Identification to inspect application mockups and identify objects you could use in your test. In this way, you can prepare your test much earlier and design your test even before your application is fully developed.


AI Mockup Identification requires no web or mobile application. Instead, you need to load images in .jpg, .jpeg, or .png format from a local folder.

Inspect application mockups

  1. In the UFT One toolbar, click the down arrow near the AI Identification button and select the AI Mockup Identification .
  2. In the dialog box that opens, select a folder containing images, select Web or Mobile as the AI identification context, and click Open.

    The AI Mockup Identification View loads all of the images in the folder by file name and creates an image gallery.

    The AI Mockup Identification View inspects the image that appears first in the file name order and highlights all identified visual elements by default.

  3. To manage images in the folder, edit settings in the image gallery:

    • Select File Name or Modified Time to sort the images by file name or modified time.
    • Click Refresh to refresh the images if the images are changed.
    • Click Choose different to select a different folder of images to inspect.
  4. To achieve optimal identification, edit settings on the inspected page:

    • Select Visual Element or Texts to show either the objects that UFT One identified visually or areas of text.
    • Select Web or Mobile to change your AI identification context again.

Help design the future of AI-based testing in UFT One

Click How is the detection? Help us improve to open the Feedback Tool and send Micro Focus feedback about the object identification. For details, see AI-based testing Feedback Tool.

Back to top

(Optional) Verify support for selecting from a combo box

Supported on UFT One versions 15.0.1 and later

Selecting items from a combo box is a complex operation. UFT One can perform a Select operation on a combo box identified by AI only if it can also identify this combo box as a WebList or a MobileDropDown.

When inspecting your application for objects using AI, if UFT One identifies a combo box, it displays a Verify icon on the object. Click this icon to check whether UFT One can run Select operations on this object before adding such steps to your test.

UFT One relies on the relative location of a combo box to verify the object. If you re-size the browser after inspecting a web page and change the object location, the verification fails.

Tip: Try verifying more than once. Depending on various environment conditions, even if verification failed, UFT One may find the necessary object on subsequent attempts.

Back to top

Add AI-based steps to your tests

Set the AI context

Before you can run any AI-based steps in your test, you must set the context in which to use AI. Add a statement to your test similar to this one: 

AIUtil.SetContext Device("device")

UFT One 15.0: The context must be a mobile Device test object that exists in your object repository.

UFT One 15.0.1 and later: The AI context can be one of the following: 

  • A mobile Device test object stored in your object repository.

  • A web Browser test object stored in your object repository or described using descriptive programming.

    The web Browser can run on a Desktop or Mobile device, and be opened manually or from the Record and Run settings.

Note: UFT One must be able to identify the context object throughout the test run. If necessary, use additional SetContext steps to select a different object for the context.

Add test steps

You can add AI-based steps to your test in one of the following ways:

  • Drag and Drop the identified Object or Text from the Object Spy, Object Identification Center, AI Identification View or AI Mockup Identification View window into your test.

    The step added for each object includes any information used to identify it uniquely, such as associated text or the objects relative location on the screen. See Associating text with objects and Identifying objects by relative location.

    Note: Avoid adding steps with objects that were incorrectly identified. For example, if a button is identified as a text box, or a check mark is identified as a button, such objects may be identified inconsistently and fail in subsequent test runs.

  • Use AI Record to automatically create AI-based test steps in your test editor. For details, see Record an AI-based test.

  • Edit your test using the AIUtil object.

    Next, use the AIUtil object to identify the control by type, text, or both, and to perform operations on the objects, such as Click, Type, or Search.

    For details, see AI-based Testing in the UFT One Object Model Reference.


The following example clicks a settings control on a mobile device and then types a user name in the appropriate field. Both controls are identified visually using AI.

Next, the Log In text is located on the screen and clicked.

After logging in, this example clicks an "ON" button. Since the application contains a few similar buttons, the example specifies that button to click is the third button from the top.

AIUtil.SetContext Device("device") 'Set the context for AI
AIUtil("input", "USER NAME").Type "admin"
AIUtil.FindTextBlock("Log In").Click
AIUtil("button", "ON", "FromTop", 3).Click

Tip: UFT One generates steps that use FindTextBlock, as the spy identifies full blocks of text. When you create your own steps, consider using the FindText method, which will match the text you provide, even if it is only part of a larger text block.

Add checkpoints to your test (optional)

Supported on UFT One versions 15.0.2 and later

You can add different types of checkpoints, checking the existence of an object, or its state.

Enter checkpoint steps in your test manually, or add them in an AI Record session. For details, see Add a checkpoint during recording.

Checkpoint method Description Condition

AIObject.CheckExists or AITextObject.CheckExists

Check the existence or absence of an object in your application.

Specify whether you expect the object to exist or not when this step runs.

Check the state of objects that can be set to Off or On, such as check boxes, toggles, or radio buttons.

Specify the expected state of the object.

The checkpoint passes if the application meets the expectation. Otherwise, a step failure is reported in the run results.

Back to top

Inspect the next application page/screen

When you finish creating test steps for one page or screen in your application and you want to continue on another one, navigate to the desired location in the application itself. Then, in the AI Identification View window, click REFRESH to load the new application screen and re-inspect it.

  • When working with a mobile application, navigate to the correct location in your application directly on the device or using the remote access window.

  • If multiple remote access windows or browser windows are open, the inspection session interacts with only one.

    UFT One 15.0.1 or later: You can switch to inspect other browser windows or mobile devices that are open on your computer. Click CHOOSE in the AI Identification View window and select a different browser or device.

Next, add steps from this page or screen to your test.

Back to top

Run AI-based tests

After inspecting your application and creating test steps, run your AI-based test as you would run any other UFT One test. See Run / Debug tests.

You can run the same test on different operating systems and versions, as it is not based on implementation details.

Back to top

Convert technology-based tests to AI-based tests

The AI Transformation Assistant can help you convert property-based test steps in your existing tests to AI-based testing steps:

  1. Enable the assistant.

    In Tools > Options > GUI Testing > AI, select Replace run results with AI object suggestions.

  2. Run your tests.

    The test steps are performed on the application and the run report contains AI object suggestions instead of the results of your test run. Each step contains a suggestion for transforming the technology-based test step into an AI-based testing step.

  3. Modify your test steps (AI conversion).

    Replace existing steps with the suggested AI object steps to create a more resilient and platform-agnostic test.

Back to top


AI text identification requires the Windows mediaserver.exe service to be running. Otherwise the following may occur:

  • AI inspection cannot find objects By Text
  • All FindText and FindTextBlock steps fail to return an AITextObject when running a test
  • An error message indicates that an error occurred when calling the Media Server OCR service.


Open the Windows Services Manager and make sure the mediaserver.exe is running. Otherwise, start the service manually.

Back to top

See also: