AI-based testing in UFT One

GUI UFT Mobile and Web tests only

This topic explains how to use UFT One's Artificial Intelligence (AI) Features in your tests to identify objects the way a person would. This enables you to run the same test on different platforms and versions, regardless of the objects' implementation.

Note: AI Codeless Testing is not part of UFT One. It is a standalone tool that lets you experiment with AI Codeless testing with its own installation and prerequisites. For details, see the AI Codeless Testing Help Center.

AI-based testing overview

UFT One's AI Features enable your tests to interact with the application you are testing in the same way a person would. UFT One uses AI to identify objects visually, based on a wide variety of images, context, and sometimes text.

For example, UFT One AI can identify many types of search fields, user profile areas, input fields, buttons, shopping carts and more.

Some advantages of AI-based object identification are: 

  • More intuitive test scripts.

  • Tests are technology agnostic, identifying objects visually, regardless of the technology details used behind the scenes.

  • Tests are easier to maintain, as an object changing location, framework, or even shape, won’t break the test script as long as the object remains visually similar or its purpose remains clear.

AI Service

AI Service in UFT One is computationally intensive. We recommend that you use a powerful computer to benefit from the AI Features and achieve optimal performance. For recommended system requirements, see Support Matrix.

If you have a faster and powerful computer, you can use it as an AI server and use the remote AI service to provide more efficient AI capabilities for your less strong UFT One computers. For details, see Use the Remote AI Service.

AI Model

UFT One's AI Features use incorporated AI models to assist UFT One in learning your application and identifying test objects in a more effective way.

Updated AI models can improve performance, accuracy, and identification capabilities. When new models are available, you can download them and use them for your AI-based testing. For details, see Update AI models.

Back to top

Prerequisites

To use UFT One's AI features, they must be installed and enabled. By default, UFT One's installation is set up to install and enable UFT One's AI Features.

If the AI Features are not installed (the Options dialog box does not display the GUI Testing > AI pane), do one of the following: 

To enable or disable the AI Features after installation, use the Tools > Options > GUI Testing > AI > AI active option.

Back to top

Inspect your application for objects

Use AI inspection to identify objects in your application that you can use in your test steps.

Prerequisites

If you are testing an application on a mobile device
  1. Make sure the Mobile Add-in is installed and loaded.

    To test web applications on your mobile device, make sure the Web Add-in is installed and loaded.

  2. Connect to UFT Mobile and open your mobile device and the application you want to test. For details, see Open a remote access window and application manually.

If you are testing a desktop web application
  1. Make sure the Web Add-in is installed and loaded.

  2. Make sure that your browser is set up with the UFT extension. For details, see Test Web apps on specific browsers.

  3. Open the application.

    For best identification results:

    • Maximize the browser window and set the zoom level to 100%.
    • Set your Windows display settings to display all text and apps at 100%.
    • Make sure no banners are displayed in the browser aside from your web page.

Supported browsers: Chrome, Chromium-based Edge, Firefox, Internet Explorer.

Headless browsers are not supported.

To identify single objects

In the Object Spy or Object Identification Center, as you spy on objects in mobile or web applications, UFT One also presents the AI object you could use to represent the same object.

When spying on a single object Once you click on an object in the application, an AI object is suggested only for the object you clicked. AI objects are not suggested for any other object in the hierarchy displayed in the Spy.

When spying on multiple objects using the OIC

AI object suggestions are available in the Edit view of each spied object. To open the edit view, hover over an object in the list of spied objects and click the Edit button.

To identify all objects

  1. Open the AI Inspection window to inspect your application and detect all AI objects in it.

    To open the window, do one of the following: 

    • Select AI > AI Identification.

    • Click the AI Identification toolbar button.

      Tip: Depending on the version you are using, the button might look different.

    • In the Object Spy or Object Identification Center, click INSPECT.

      Tip: This button is available when the AI Auto Inspection option is enabled.

  2. Click a web application or the remote access window displaying your mobile application. If you click an unsupported context or press Esc, Select Application button appears to prompt you to click again.

    Note: You can skip this step if you began the inspection with a Web browser or Mobile device as the top-level object in your spied hierarchy. UFT One automatically selects that browser or device for inspection.

    The Live Application tab displays the current screen of the application, highlighting all of the detected objects.

  3. Decide whether to show Visual Element, Text, or both, to see either the objects that UFT One detected visually, or areas of text in the application, or both.

Help design the future of AI-based testing in UFT One

Click How is the detection? Help us to improve to open the Feedback Tool and send Micro Focus feedback about the object detection. For details, see AI-based testing Feedback Tool.

Back to top

Inspect application mockups for objects

Use AI Mockup Identification to inspect application mockups and identify objects to use in your test. This enables you to design and prepare your test even before your application is fully developed.

Prerequisites

AI Mockup Identification does not require a web or mobile application. Instead, you need to load images in .jpg, .jpeg, or .png format from a local folder.

Inspect application mockups

  1. In the UFT One toolbar, click the down arrow near AI Identification button , and select AI Mockup Identification.
  2. In the MOCKUP IMAGES tab of the AI Inspection window, select Web or Mobile as the inspection context and click Browse Folder to select a folder containing images.

    The AI Inspection window inspects the image that appears first in the file name order and highlights all identified visual elements by default.

  3. Decide whether to show Visual Element, Text, or both, to see either the objects that UFT One detected visually, or areas of text in the application, or both.
  4. (Tech preview) If your application contains container objects such as tables and you want to inspect the lower-level objects inside the tables, toggle on Drill-down to initiate a re-inspection.
  5. Decide whether to change the AI identification context type by selecting Web or Mobile.
  6. Click any highlighted object.

    On the tooltip that opens, modify the step action, edit the value field, and click Add to test. Alternatively, click Edit to modify additional step details including Text, Position, Relation, and Action in the right pane.

    Note:  

    • For cells inside a table, the tooltip also displays the object hierarchy, row, and column information.

    • You cannot add a position and a relation for an object at the same time.

  7. Click the down arrow near the folder icon to select a different folder, synchronize with the current folder, or go to the current folder.
  8. To view all images in your folder, click the down arrow on the top to show the image gallery. In the gallery, you can:

    • Navigate between images in the folder with arrows.

    • Click the Search button to search for a specific image.

    • Select whether to view the images in Grid View or Line View.

    • Select Name or Date to sort images by the file name or modified time.

Help design the future of AI-based testing in UFT One

Click How is the detection? Help us to improve to open the Feedback Tool and send Micro Focus feedback about the object identification. For details, see AI-based testing Feedback Tool.

Back to top

Add AI-based steps to your tests

Set the AI context

Before you can run any AI-based steps in your test, you must set the context in which to use AI.

You can set the AI context for your test in one of the following ways:

  • In the Live Application tab of AI Inspection window, add a context step from the right pane, or click the down arrow near Context and select Add as "SetContext" step to test.

  • Manually add a SetContext step to your test before you add any other steps.

    For details, see SetContext method in UFT One Object Model Reference for GUI Testing.

  • The SetContext step is automatically added if you use AI Record.

UFT One must be able to identify the context object throughout the test run. If necessary, use additional SetContext steps to select a different object for the context.

Tip: Within a web browser context, you can also add AIUtil.Context.SetBrowserScope steps, to specify whether to inspect the whole browser for objects or only the web page itself. Inspecting the whole browser is useful when your test needs to respond to a pop-up dialog box that opens in the browser. Note that UFT One cannot identify any pop-up dialog boxes hidden behind the browser window.

Add test steps

You can add AI-based steps to your test in one of the following ways:

Add checkpoints to your test (optional)

You can add different types of checkpoints to check the existence of an object or its state.

Add checkpoint steps in one of the following ways:

  • Enter checkpoint steps in your test manually.

  • Add checkpoint steps in an AI Record session. For details, see Add a checkpoint during recording.

  • Select Verify as the action and select Exists or Does Not Exist to verify the existence of the object. Use On or Off to verify the object's state when you edit a step in AI Inspection window.

Checkpoint method Description Condition

AIObject.CheckExists or AITextObject.CheckExists

Check the existence or absence of an object in your application.

Specify whether you expect the object to exist or not when this step runs.
AIObject.CheckState

Check the state of objects that can be set to Off or On, such as check boxes, toggles, or radio buttons.

Specify the expected state of the object.

The checkpoint passes if the application is in the expected conditions. Otherwise, a step failure is reported in the run results.

Back to top

Inspect the next application page/screen

When you finish creating test steps for one page or screen in your application and you want to continue on another one, follow the steps below:

To inspect the next application page/screen

  1. Click the down arrow near Context , select Go to inspected application , and navigate to the desired location in the application.

  2. In the AI Inspection window, click Re-inspect to load the new application page or screen and reinspect it.

    • If you don't reload and try to run a step from the AI Inspection window, UFT One runs the step based on the previous page’s inspection, resulting in an error or performing the operation on the new page.

    • If multiple remote access windows or browser windows are open, the inspection session interacts with only one.

    • You can switch to inspect other browser windows or mobile devices that are open on your computer. Click the down arrow near Context and then select Select a different application in the AI Inspection window.

    • If you need to perform steps on the application to prepare it for inspection, you can use the Delayed re-inspect.

      Click the down arrow near Delayed re-inspect and set the required delay.

      Click Go to begin the countdown, open the application, and perform steps such as hover or menu clicks to bring the application to the state you want to inspect.

      When the delay timer expires, the application is re-inspected.

  3. Add steps from the new page or screen to your test.

    Tip:
  1. You may need to set the delay to a larger value if you have a slow network.

  2.  

Back to top

Run AI-based tests

After inspecting your application and creating test steps, run your AI-based test as you would run any other UFT One test. See Run / Debug tests.

You can run the same test on different operating systems and versions since it is not based on implementation details.

Back to top

Convert technology-based tests to AI-based tests

The AI Transformation Assistant can help you convert property-based test steps in your existing tests to AI-based testing steps:

  1. Enable the assistant.

    In Tools > Options > GUI Testing > AI > General, select Replace run results with AI object suggestions.

  2. Run your tests.

    The test steps are performed on the application and the run report contains AI object suggestions instead of the results of your test run. Each step contains a suggestion for transforming the technology-based test step into an AI-based testing step.

    Note: The AI Transformation Assistant does not check the object hierarchy inside table or calendar objects, and does not suggest AITable or AICalendar containers.

  3. Modify your test steps (AI conversion).

    Replace existing steps with the suggested AI object steps to create a more resilient and platform-agnostic test.

Back to top

Troubleshooting

AI text identification requires the Windows mediaserver.exe service to be running. Otherwise the following may occur:

  • AI inspection cannot find objects By Text.
  • All FindText and FindTextBlock steps fail to return an AITextObject when running a test.
  • An error message indicates that an error occurred when calling the Media Server OCR service.

Solution: 

Open the Windows Services Manager and make sure the mediaserver.exe is running. Otherwise, start the service manually.

Back to top

See also: