Improve AI-based test object identification

Supported for web and mobile testing

This topic discusses some of the elements that UFT Developer's Artificial Intelligence (AI) Features use to support unique identification of objects in your application.

Associating text with objects

The text associated with an object can help identify an object uniquely. For example:

browser.describe(AiObject.class, new AiObjectDescription("button", "ADD TO CART")).Click 

Using the ADD TO CART text in this step to describe the object, ensures that we click the correct button.

When detecting objects in an application, if there are multiple labels around the field, UFT Developer uses the one that seems most logical for the object's identification.

However, if you decide to use a different label in your object description, UFT Developer still identifies the object.

Example: When detecting objects in an application, a button is associated with the text on the button, but a field is associated with its label, as opposed to its content.

If a field has multiple labels and UFT Developer chooses one for detection, UFT Developer will still identify this field correctly when running a test step that uses a different label to describing the field.

In some cases, UFT Developer AI combines multiple text strings that are close to each other into one text string to identify one object.

You can edit the combined string and leave just one to use for object identification. Make sure to remove a whole string and not part of it.

Example:  

For the text box below, AI Inspection combines Password and Forgot? into one string to identify the object and uses the string "Password Forgot?" to describe the object in the code it generates.

You can remove a whole string from your script and change the code to use just "Password" for the description without causing test failure.

Back to top

Text recognition in multiple languages

To enable UFT Developer's AI features to recognize text in languages other than English, select the relevant OCR languages in the AI tab of the UFT Developer runtime engine settings. For details, see AI Settings.

You can also customize the languages temporarily within a test run by adding AI run settings steps to your test. For details, see the AIRunSettings Class in the Java SDK reference.

For example: 

Copy code
// Configure OCR settings to enable detection for English, German, French, Hebrew and Traditional Chinese
AiRunSettings.updateOCRSettings((new AiOCRSettings()).setLanguages(new String[]{"en", "de", "fr", "he", "zht"}))

Back to top

Identifying objects by ordinal position

UFT Developer must be able to identify an object uniquely to run a step on the object. When multiple objects match your object description, you can add the object's ordinal position location to provide a unique identification.

To describe an object's ordinal location

Provide the following information:

  • The object's occurrence, is it first, second, third, and so on.

  • The orientation, the direction in which to count occurrences: FromLeft, FromRight, FromTop, FromBottom.

For example, the following code snippet clicks the first twitter image from the right in your application.

Copy code
browser.describe(AiObject.class, new AiObjectDescription.Builder()
.aiClass("twitter")
.locator(new com.hp.lft.sdk.ai.Position(com.hp.lft.sdk.ai.Direction.FROM_RIGHT, 0)).build()).click();

Back to top

Automatic scrolling

When running a test, if the object is not displayed in the application but the web page or mobile app is scrollable, UFT Developer automatically scrolls further in search of the object. Once an object matching the description is identified, no further scrolling is performed. Identical objects displayed in subsequent application pages or screens will not be found.

UFT Developer scrolls similarly when running a checkpoint that requires that an object not be displayed in the application. The checkpoint passes only if the object is not found even when scrolling to the next pages or screens.

By default, UFT Developer scrolls down twice. You can customize the direction of the scroll and the maximum number of scrolls to perform, or disable scrolling if necessary.

  • Globally customize the scrolling in the AI tab of the UFT Developer runtime engine settings. For details, see AI Settings.

  • Customize the scrolling temporarily within a test run by adding AI run settings steps to your test. For details, see the AIRunSettings Class in the Java SDK reference.

    For example:

    Copy code
    // Configure autoscroll settings to enable scrolling up
    AiRunSettings.updateAutoScrollSettings((new AiAutoScrollSettings()).enable(ScrollDirection.UP, 10));

Back to top

See also: