AI-based testing in UFT One

Mobile, web, and windows-based SAP GUI tests

This topic explains how to use UFT One's Artificial Intelligence (AI) Features in your tests to identify objects the way a person would. This enables you to run the same test on different platforms and versions, regardless of the objects' implementation.

AI-based testing overview

UFT One's AI Features enable your tests to interact with the application you are testing in the same way a person would. UFT One uses AI to identify objects visually, based on a wide variety of images, context, and sometimes text.

For example, UFT One AI can identify many types of search fields, user profile areas, input fields, buttons, and shopping carts.

Some advantages of AI-based object identification are: 

Easier to edit

Test scripts are more intuitive.
Test multiple environments

Tests are technology agnostic, identifying objects visually, regardless of the technology details used behind the scenes.
Test resilience

Tests are easier to maintain, as an object changing location, framework, or even shape, won’t break the test script as long as the object remains visually similar or its purpose remains clear.

AI Object-Detection service

The AI Object-Detection (AIOD) service that provides UFT One's AI Features is computationally intensive. We recommend that you use a powerful computer to benefit from the AI Features and achieve optimal performance. For recommended system requirements, see Support Matrix.

UFT One's AI Object-Detection service can run on, and be consumed from, different sources. The service can run locally on your UFT One computer, remotely, on a designated machine in your network, or on the cloud. The local AI Object-Detection service is used by default.

You can choose to use the remote AIOD service or the AIOD Cloud Service, depending on your needs:

  • Use the remote AIOD service on one powerful computer, letting it provide AI capabilities for multiple weaker UFT One computers. For details, see Use the remote AI Object-Detection service.

  • Use the AIOD Cloud Service to reduce local use of resources and get the most up-to-date AI support. For details, see Use the AI Object-Detection Cloud Service.

    Note: The AIOD Cloud Service is available in UFT One versions 2022 and later as a Technical Preview upon request. It is not available in UFT One out of the box. To learn more, contact your account team and send an email to AIODCloudService@opentext.com.

Back to top

Prerequisites

To use UFT One's AI Features, they must be installed and enabled. By default, UFT One's installation is set up to install and enable UFT One's AI Features.

If the AI Features are not installed (the Options dialog box does not display the GUI Testing > AI pane), do one of the following: 

To enable or disable the AI Features after installation, use the Tools > Options > GUI Testing > AI > AI active option.

Prepare your test solution and create a test for a web, mobile, or SAP GUI for Windows application.

To start identifying AI objects to use in your test, continue with one of the following tasks:

Back to top

Inspect your application for objects

Use AI inspection to identify objects in your application that you can use in your test steps.

The instructions below describe how to prepare your application for testing, how to use AI to identify objects in your application, and how to send Micro Focus feedback on the identification, if you would like to.

Prerequisites

If you are testing an application on a mobile device

  1. Make sure the Mobile Add-in is installed and loaded.

    To test web applications on your mobile device, make sure the Web Add-in is also installed and loaded.

  2. Connect to Digital Lab (UFT Mobile) and open your mobile device and the application you want to test. For details, see Open a remote access window and application manually.

If you are testing a desktop web application

  1. Make sure the Web Add-in is installed and loaded.

  2. Make sure that your browser is set up with the UFT extension. For details, see Test Web apps on specific browsers.

  3. Open the application.

    For best identification results:

    • Maximize the browser window and set the zoom level to 100%.

    • Set your Windows display settings to display all text and apps at 100%.

    • Make sure no banners are displayed in the browser aside from your web page.

Supported browsers: Chrome, Chromium-based Edge, Firefox, Internet Explorer.

Headless browsers are not supported.

If you are testing an SAP GUI for Windows application

(UFT One version 2022 and later)

  1. Make sure that you perform the following tasks as described in Enable support for SAP GUI for Windows.

    1. Install SAP GUI Scripting.

    2. Enable SAP GUI Scripting on the server side.

    3. Enable SAP GUI Scripting on the client side.

  2. Make sure the SAP Solutions Add-in is installed and loaded.

  3. Open the application.

Note: AI-based testing for SAP GUI for Windows applications is officially supported on the Classic and Signature themes.

To identify single objects

In the Object Spy or Object Identification Center, as you spy on objects in mobile, web, or SAP GUI for Windows applications, UFT One also presents the AI object you could use to represent the same object.

When spying on a single object Once you click on an object in the application, an AI object is suggested only for the object you clicked. AI objects are not suggested for any other object in the hierarchy displayed in the Spy.

When spying on multiple objects using the OIC

AI object suggestions are available in the Edit view of each spied object. To open the edit view, hover over an object in the list of spied objects and click the Edit button.

To identify all objects

  1. Open the AI Inspection window to inspect your application and detect all AI objects in it.

    To open the window, do one of the following: 

    • Select AI > AI Identification.

    • Click the AI Identification toolbar button.

      Tip: Depending on the version you are using, the button might look different.

    • In the Object Spy or Object Identification Center, click INSPECT.

      Tip: This button is available when the AI Auto Inspection option is enabled.

  2. Click a web application, an SAP GUI for Windows application, or the remote access window displaying your mobile application. If you click an unsupported context or press Esc, the Select Application button appears to prompt you to click again.

    Note: You can skip this step if you began the inspection with a web Browser, mobile Device, or SAPGUISession test object as the top-level object in your spied hierarchy. UFT One automatically selects that test object for inspection.

    The Live Application tab displays the current screen of the application, highlighting all of the detected objects.

  3. Decide whether to show Visual Elements, Texts, or both, to see either the objects that UFT One detected visually, or areas of text in the application, or both.

  4. (Tech preview) If your application contains container objects such as tables and you want to inspect the lower-level objects inside the tables, toggle on Containers to initiate a reinspection. (The switch name may vary between versions.)

    Note: Table cell objects are identified using ABBYY OCR.

    UFT One versions 2021 R1 and later: To identify TableCell objects, you must have the ABBYY OCR Engine feature installed in UFT One.

  5. After identifying the AI objects, you can Add AI-based steps to your tests.

Help design the future of AI-based testing in UFT One

Click How is the detection? Help us to improve to open the Feedback Tool and send Micro Focus feedback about the object detection. For details, see AI-based testing Feedback Tool.

Back to top

Inspect application mockups for objects

Use AI Mockup Identification to inspect application mockups and identify objects to use in your test. This enables you to design and prepare your test even before your application is fully developed.

Prerequisites

AI Mockup Identification does not require a ready application. Instead, create a local folder containing images in .jpg, .jpeg, or .png format.

Inspect application mockups

  1. In the UFT One toolbar, click the down arrow near AI Identification button , and select AI Mockup Identification.

  2. In the MOCKUP IMAGES tab of the AI Inspection window, select the inspection context and click Browse Folder to select a folder containing images.

    Inspection contexts: Non-Mobile/Mobile or Web/Mobile depending on your UFT One version.

    The AI Inspection window inspects the image that appears first in the file name order and highlights all identified visual elements by default.

  3. Decide whether to show Visual Elements, Texts, or both, to see either the objects that UFT One detected visually, or areas of text in the application, or both.

  4. (Tech preview) If your application contains container objects such as tables and you want to inspect the lower-level objects inside the tables, toggle on Containers to initiate a reinspection. (The switch name may vary between versions.)

    Note: Table cell objects are identified using ABBYY OCR.

    UFT One versions 2021 R1 and later: To identify TableCell objects, you must have the ABBYY OCR Engine feature installed in UFT One.

  5. Decide whether to change the AI identification context type by selecting Non-Mobile or Mobile.

  6. Click the down arrow near the folder icon to select a different folder, synchronize with the current folder, or open the current folder.

  7. To view all images in your folder, click the down arrow on the top to show the image gallery. In the gallery, you can:

    • Navigate between images in the folder with arrows.

    • Click the Search button to search for a specific image.

    • Select whether to view the images in Grid View or Line View.

    • Select Name or Date to sort images by the file name or modified time.

  8. After identifying the AI objects, you can Add AI-based steps to your tests.

Help design the future of AI-based testing in UFT One

Click How is the detection? Help us to improve to open the Feedback Tool and send Micro Focus feedback about the object identification. For details, see AI-based testing Feedback Tool.

Back to top

Add AI-based steps to your tests

To add AI-based steps in your tests:

  • Set the AI context

  • Add test steps

  • Add checkpoints to your test (optional)

For details, see Add AI-based steps to your tests.

Back to top

Inspect the next application page/screen

When you finish creating test steps for one page or screen in your application and you want to continue on another one, follow the steps below:

To inspect the next application page/screen

  1. Click the down arrow near Context , select Go to inspected application , and navigate to the desired location in the application.

  2. In the AI Inspection window, click Re-inspect to load the new application page or screen and reinspect it.

    • If you don't reload and try to run a step from the AI Inspection window, UFT One runs the step based on the previous page’s inspection, resulting in an error or performing the operation on the new page.

    • If multiple remote access windows or browser windows are open, the inspection session interacts with only one.

    • You can switch to inspect other browser windows or mobile devices that are open on your computer. Click the down arrow near Context and then select Select a different application in the AI Inspection window.

    • If you need to perform steps on the application to prepare it for inspection, you can use the Delayed re-inspect.

      Click the down arrow near Delayed re-inspect and set the required delay.

      Click Go to begin the countdown, open the application, and perform steps such as hover or menu clicks to bring the application to the state you want to inspect.

      When the delay timer expires, the application is re-inspected.

  3. Add steps from the new page or screen to your test.

Tip: You may need to set the delay to a larger value if you have a slow network.

Back to top

Run AI-based tests

After inspecting your application and creating test steps, run your AI-based test as you would run any other UFT One test. See Run / Debug tests.

You can run the same test on different operating systems and versions since it is not based on implementation details.

Back to top

Convert technology-based tests to AI-based tests

The AI Transformation Assistant can help you convert property-based test steps in your existing tests to AI-based testing steps:

  1. Enable the assistant.

    In Tools > Options > GUI Testing > AI > General, select Replace run results with AI object suggestions.

  2. Run your tests.

    The test steps are performed on the application and the run report contains AI object suggestions instead of the results of your test run. Each step contains a suggestion for transforming the technology-based test step into an AI-based testing step.

    Note: The AI Transformation Assistant does not check the object hierarchy inside table or calendar objects, and does not suggest AITable or AICalendar containers.

  3. Modify your test steps (AI conversion).

    Replace existing steps with the suggested AI object steps to create a more resilient and platform-agnostic test.

Back to top

Troubleshooting

AI text identification requires the Windows mediaserver.exe service to be running. Otherwise the following may occur:

  • AI inspection cannot find objects By Text.
  • All FindText and FindTextBlock steps fail to return an AITextObject when running a test.
  • An error message indicates that an error occurred when calling the Media Server OCR service.

Solution: 

Open the Windows Services Manager and make sure the mediaserver.exe is running. Otherwise, start the service manually.

Back to top

See also: