AI-based testing in UFT One
GUI UFT Mobile and Web tests only
This topic describes how to install and use UFT One's Artificial Intelligence (AI) Features to identify objects the way a person would. This enables you to run the same test on different platforms and versions, regardless of the objects' implementation.
UFT One AI-based testing support
The following table describes the level of AI-based testing support provided in each UFT One version.
UFT One version | Mobile apps | Desktop web |
---|---|---|
15.0 | Supported | Not supported |
15.0.1 | Supported | Technical preview |
15.0.2 | Supported | Supported |
AI-based testing overview
UFT One's AI Features enable your tests to interact with the application you are testing in the same way a person would. UFT One uses AI to identify objects visually, based on a wide variety of images, context, and sometimes text.
For example, UFT One AI can identify many forms of search fields, user profile areas, input fields, button, shopping carts and more.
Some advantages of AI-based object identification are:
1. More intuitive test scripts.
2. Tests are technology agnostic, identifying objects visually, regardless of the UI technology details used behind the scenes.
3. Tests are easier to maintain, as an object changing location, UI framework, or even shape, won’t break the test script as long as the object remains visually similar or its purpose remains clear.
Install UFT One's AI Features
To use UFT One's AI features, you must install and enable them.
Prerequisites
Before installing the AI features, make sure you have the following installed:
-
A 64-bit operating system
-
Microsoft Visual C++ 2013 Redistributable
-
The required Windows feature, depending on your Windows operating system:
Windows OS - Microsoft Media Feature Pack-
From the Control Panel, select Programs and Features > Turn Windows features on or off.
-
In the Windows Feature window, turn on Media Features.
-
Restart the computer to take effect.
Windows Server 2008 - Desktop Experience1. From the Control Panel, select Programs and Features > Turn Windows features on or off.
2. In the Server Manager window, in the right pane, click Add Features.
3. In the Add Features Wizard window, turn on Desktop Experience.
4. Click Install.
5. Restart the computer for your changes to take effect.
Windows Server 2012 and later - Microsoft Media Foundation1. From the Control Panel, select Programs and Features > Turn Windows features on or off.
2. In the Add Roles and Features Wizard window, click Next until you reach the Features tab.
3. Turn on Media Foundation.
4. Click Install.
5. Restart the computer for your changes to take effect.
For the most up-to-date instructions on enabling Windows features, see the Microsoft documentation.
-
Install UFT One's AI features
Expand one of the sections below, depending on the UFT One version you are using:
When you run the UFT One installation program, AI Features is selected by default in the list of features to include in the installation. If you deselected it when you first installed UFT One, run the installation program again to Modify the installation and select AI Features to install.
Alternatively, run the UFT One silent installation and specify AI Features for installation using the ADDLOCAL parameter. For details, see Silent commands for installing individual UFT One features.
Note: If UFT AI Features 15.0.1 or earlier was previously installed, this installation removes the older one.
Once installed, AI Features are enabled by default. If you ever need to disable them, open Tools > Options > GUI Testing > AI, and clear the AI active option.
After installing UFT One, run the separate UFT AI Features installation program.
Use the version of AI Features that is delivered with your UFT One version.
-
For UFT One 15.0, install UFT One AI Features version 2.0
-
For UFT One 15.0.1, install UFT One AI Features version 15.0.1
You can run the installation manually, using the setup wizard or you can run it silently from a command line.
Manual installation | Silent installation |
---|---|
From the UFT One DVD, run AI\setup.exe installation file and follow the on-screen instructions. When selecting the features to install, you only need the AI Engine. |
msiexec /qn /i "<UFT DVD>\AI\AI_Installer.msi" ADDLOCAL="AI_Services" |
Enable UFT One's AI features
- Open the Options dialog box to the AI pane of the GUI Testing tab (Tools > Options > GUI Testing > AI).
- Select the AI active option.
- Restart UFT One.
If you use the Generate Script option in the Options dialog box, the AI Active option is included in the script. For details, see General Pane (Options Dialog Box > GUI Testing Tab).
Add a statement to the generated script to restart UFT One if this option's value is modified. For details, see the Options Object > AI Property section of the UFT One Automation Object Model Reference.
Note: AI Codeless Testing is not part of UFT One. It is a standalone tool that lets you experiment with AI Codeless testing. Installing AI Codeless Testing requires additional prerequisites. See the AI Codeless Testing Help Center.
Inspect your application for objects
Use AI inspection to identify objects in your application that you can use in your test steps.
Prerequisites
If you are testing an application on a mobile device | If you are testing a desktop web application |
---|---|
|
Supported browsers: Chrome, Chromium-based Edge, Firefox, Internet Explorer. Headless browsers are not supported. |
Access the AI Spy from within the Object Spy:
-
In UFT One, click the Object Spy toolbar button to open the Object Spy.
-
In the Object Spy, click the AI spy .
-
Click on your mobile application in the remote access window.
The spy's AI Identification View displays an image of the application, highlighting all of the detected objects.
-
You can switch between By Object and By Text modes, to see either the objects that UFT One detected visually, or areas of text in the application.
In the Object Spy or Object Identification Center, as you spy on objects in mobile or web applications, UFT One also presents the AI object you could use to represent the same object.
When spying on a single object | Once you click on an object in the application, an AI object is suggested only for the object you clicked. AI objects are not suggested for any other objects in the hierarchy displayed in the Spy. |
When spying on multiple objects using the OIC (15.0.2 or later) |
AI object suggestions are available in the edit view of each spied object. To open the edit view, hover over an object in the list of spied objects and click the Edit button. |
Drag the AI object into your test to create a step that performs an operation on that object.
To toggle off the AI suggestions, toggle off AI Auto Inspection in the Object Spy window or in the Object Identification Center settings.
Tip: This setting remains in effect in subsequent Spy sessions and subsequent UFT One sessions, until you toggle it on again.
-
Open the AI Identification View window to inspect your application and detect all AI objects in it.
To open the window, do one of the following:
-
Click the AI Identification toolbar button.
Tip: Depending on the version you are using, the button might look different.
-
In the Object Spy or Object Identification Center, click INSPECT.
Tip: This button is available when the AI Auto Inspection option is enabled.
-
-
Click on a web application or on the remote access window displaying your mobile application.
Note: You can skip this step if you began the inspection with a Web browser or Mobile device as the top-level object in your spied hierarchy. UFT One automatically selects that browser or device for inspection.
The AI Identification View displays an image of the application, highlighting all of the detected objects.
- You can decide whether to show Visual Element or Text, to see either the objects that UFT One detected visually, or areas of text in the application.
Help design the future of AI-based testing in UFT One
Click How is the detection? Help us improve to open the Feedback Tool and send Micro Focus feedback about the object detection. For details, see AI-based testing Feedback Tool.
Inspect application mockups for objects
Supported on UFT One versions 15.0.2 and later
Use AI Mockup Identification to inspect application mockups and identify objects you could use in your test. In this way, you can prepare your test much earlier and design your test even before your application is fully developed.
Prerequisites
AI Mockup Identification requires no web or mobile application. Instead, you need to load images in .jpg, .jpeg, or .png format from a local folder.
Inspect application mockups
- In the UFT One toolbar, click the down arrow near the AI Identification button and select the AI Mockup Identification .
-
In the dialog box that opens, select a folder containing images, select Web or Mobile as the AI identification context, and click Open.
The AI Mockup Identification View loads all of the images in the folder by file name and creates an image gallery.
The AI Mockup Identification View inspects the image that appears first in the file name order and highlights all identified visual elements by default.
-
To manage images in the folder, edit settings in the image gallery:
- Select File Name or Modified Time to sort the images by file name or modified time.
- Click Refresh to refresh the images if the images are changed.
- Click Choose different to select a different folder of images to inspect.
-
To achieve optimal identification, edit settings on the inspected page:
- Select Visual Element or Texts to show either the objects that UFT One identified visually or areas of text.
- Select Web or Mobile to change your AI identification context again.
Help design the future of AI-based testing in UFT One
Click How is the detection? Help us improve to open the Feedback Tool and send Micro Focus feedback about the object identification. For details, see AI-based testing Feedback Tool.
(Optional) Verify support for selecting from a combo box
Supported on UFT One versions 15.0.1 and later
Selecting items from a combo box is a complex operation. UFT One can perform a Select operation on a combo box identified by AI only if it can also identify this combo box as a WebList or a MobileDropDown.
When inspecting your application for objects using AI, if UFT One identifies a combo box, it displays a Verify icon on the object. Click this icon to check whether UFT One can run Select operations on this object before adding such steps to your test.
UFT One relies on the relative location of a combo box to verify the object. If you re-size the browser after inspecting a web page and change the object location, the verification fails.
Tip: Try verifying more than once. Depending on various environment conditions, even if verification failed, UFT One may find the necessary object on subsequent attempts.
Add AI-based steps to your tests
Set the AI context
Before you can run any AI-based steps in your test, you must set the context in which to use AI. Add a statement to your test similar to this one:
AIUtil.SetContext Device("device")
UFT One 15.0: The context must be a mobile Device test object that exists in your object repository.
UFT One 15.0.1 and later: The AI context can be one of the following:
-
A mobile Device test object stored in your object repository.
-
A web Browser test object stored in your object repository or described using descriptive programming.
The web Browser can run on a Desktop or Mobile device, and be opened manually or from the Record and Run settings.
Note: UFT One must be able to identify the context object throughout the test run. If necessary, use additional SetContext steps to select a different object for the context.
Add test steps
You can add AI-based steps to your test in one of the following ways:
-
Drag and Drop the identified Object or Text from the Object Spy, Object Identification Center, AI Identification View or AI Mockup Identification View window into your test.
The step added for each object includes any information used to identify it uniquely, such as associated text or the objects relative location on the screen. See Associating text with objects and Identifying objects by relative location.
Note: Avoid adding steps with objects that were incorrectly identified. For example, if a button is identified as a text box, or a check mark is identified as a button, such objects may be identified inconsistently and fail in subsequent test runs.
-
Use AI Record to automatically create AI-based test steps in your test editor. For details, see Record an AI-based test.
-
Edit your test using the AIUtil object.
Next, use the AIUtil object to identify the control by type, text, or both, and to perform operations on the objects, such as Click, Type, or Search.
For details, see AI-based Testing in the UFT One Object Model Reference.
Example:
The following example clicks a settings control on a mobile device and then types a user name in the appropriate field. Both controls are identified visually using AI.
Next, the Log In text is located on the screen and clicked.
After logging in, this example clicks an "ON" button. Since the application contains a few similar buttons, the example specifies that button to click is the third button from the top.
AIUtil.SetContext Device("device") 'Set the context for AI AIUtil("settings").Click AIUtil("input", "USER NAME").Type "admin" AIUtil.FindTextBlock("Log In").Click AIUtil("button", "ON", "FromTop", 3).Click
Tip: UFT One generates steps that use FindTextBlock, as the spy identifies full blocks of text. When you create your own steps, consider using the FindText method, which will match the text you provide, even if it is only part of a larger text block.
Add checkpoints to your test (optional)
Supported on UFT One versions 15.0.2 and later
You can add different types of checkpoints, checking the existence of an object, or its state.
Enter checkpoint steps in your test manually, or add them in an AI Record session. For details, see Add a checkpoint during recording.
Checkpoint method | Description | Condition |
---|---|---|
AIObject.CheckExists or AITextObject.CheckExists |
Check the existence or absence of an object in your application. |
Specify whether you expect the object to exist or not when this step runs. |
AIObject.CheckState |
Check the state of objects that can be set to Off or On, such as check boxes, toggles, or radio buttons. |
Specify the expected state of the object. |
The checkpoint passes if the application meets the expectation. Otherwise, a step failure is reported in the run results.
Inspect the next application page/screen
When you finish creating test steps for one page or screen in your application and you want to continue on another one, navigate to the desired location in the application itself. Then, in the AI Identification View window, click REFRESH to load the new application screen and re-inspect it.
-
When working with a mobile application, navigate to the correct location in your application directly on the device or using the remote access window.
-
If multiple remote access windows or browser windows are open, the inspection session interacts with only one.
UFT One 15.0.1 or later: You can switch to inspect other browser windows or mobile devices that are open on your computer. Click CHOOSE in the AI Identification View window and select a different browser or device.
Next, add steps from this page or screen to your test.
Run AI-based tests
After inspecting your application and creating test steps, run your AI-based test as you would run any other UFT One test. See Run / Debug tests.
You can run the same test on different operating systems and versions, as it is not based on implementation details.
Convert technology-based tests to AI-based tests
The AI Transformation Assistant can help you convert property-based test steps in your existing tests to AI-based testing steps:
-
Enable the assistant.
In Tools > Options > GUI Testing > AI, select Replace run results with AI object suggestions.
-
Run your tests.
The test steps are performed on the application and the run report contains AI object suggestions instead of the results of your test run. Each step contains a suggestion for transforming the technology-based test step into an AI-based testing step.
-
Modify your test steps (AI conversion).
Replace existing steps with the suggested AI object steps to create a more resilient and platform-agnostic test.
Troubleshooting
AI text identification requires the Windows mediaserver.exe service to be running. Otherwise the following may occur:
- AI inspection cannot find objects By Text
- All FindText and FindTextBlock steps fail to return an AITextObject when running a test
- An error message indicates that an error occurred when calling the Media Server OCR service.
Solution:
Open the Windows Services Manager and make sure the mediaserver.exe is running. Otherwise, start the service manually.
See also: