Text recognition in run-time

Relevant for: GUI tests and components

When working with tests and scripted components, you can use the text and text area checkpoint or output value commands to verify or retrieve text in your objects.

In addition, when working with tests, keyword or scripted components, and function libraries, you can insert steps to capture the text from objects in your application using the .GetVisibleText, the .GetTextLocation test object methods, the TextUtil.GetText or TextUtil.GetTextLocation reserved object methods, or the .GetText (for Terminal Emulator objects).

UFT 15.0.1 or later (tech preview): Use descriptive programming to create TextObject test objects for specific texts in your application. You can then perform operations on these test objects, such as Click, Drag, Drop, Hover, and Type. For example, you can use TextObject test objects to support HTML5 Canvas. For details, see the Insight > TextObject Object topic in the UFT Object Model Reference for GUI Testing.

Note: Text recognition is not supported for objects in the Active Screen.

When you use one of these options, UFT identifies text in your application uses an OCR (optical character recognition) mechanism. When using this OCR engine, you can use one of the following text recognition engines:

  • The Abbyy OCR (the default option)
  • The Tesseract OCR engine
  • The Google Cloud OCR engine
  • The Baidu Cloud OCR engine

Note: Cloud OCR engines are supported only in UFT versions 15.0.1 or later.

Using a cloud OCR engine requires setting up an account with the relevant vendor and obtaining an access token or key used to connect to the cloud service.

When UFT uses the OCR mechanism, a number of factors can affect the text it retrieves. Depending on the characteristics of the text you want to retrieve, you can adjust several OCR configuration options to optimize the way the text is captured. You use the Text Recognition Pane (Options Dialog Box > GUI Testing Tab) to specify the preferred text recognition mechanism and OCR-specific settings.

Note: If UFT cannot connect to the cloud OCR service using the configured details, it uses Abbyy instead.

OCR's accuracy depends on font and image quality and uniformity. You should also note the following considerations for performing more effective text recognition:

Fonts in your text

(For the Abbyy and Tesseract OCR engines only)

  • Single text block mode and multiple text block mode sometimes result in different captured text. If you are not sure which text block mode to use, use the default multiple block mode. If the results are not what you expect, then try using the single text block mode.

    If you want to use the text recognition mechanism for a large area containing different fonts and backgrounds, we recommend creating several steps to capture the text for each single text block instead of creating one step to capture a multiple text block.

  • If the text recognition mechanism retrieves unwanted text information (such as hidden text and shadowed text that appears as multiple copies of the same string) when using the multiple text block mode, use the single text block mode option. To do this, in the Text Recognition pane of the Options dialog box (Tools > Options > GUI Testing tab > Text Recognition node), select Single text block mode. For details, see Text Recognition Pane (Options Dialog Box > GUI Testing Tab).
  • If your application uses small fonts (less than 10 pt.) you should use the Tesseract OCR engine with the Preprocess the image before using text recognition option selected.
Colors and color contrast
  • The color scheme of the background should be permanent and without gradient.
  • High contrast between the background and text is best for text recognition (for example, black text on a white background).
Text within images
  • If your text is found within an image, we recommend using the Preprocess the image before using text recognition option in the Text Recognition pane of the Options dialog box (Tools > Options > GUI Testing tab > Text Recognition mode).
Dimension for text recognition
  • Try to keep the dimensions of the selected text area as small as possible to prevent additional unwanted characters in recognized text.

    At the same time, if you are using a cloud vendor, avoid very small windows or elements and do not specify such a small area for a text area checkpoint that the recognition fails:

    When using Baidu: Make sure that your rectangle's length and width are no shorter than 15 pixels and no longer than 4096 pixels.

    When using Google: Make sure that your rectangle's length and width are no longer than 20 Megapixels. Recommended size: 1024 x 768.

  • Consider the potential movement (change of coordinates) of the object within the window. For example, the screen resolution is often different on different computers, and this can affect the coordinates of the object in the application. Also, during the design and development stages of an application, an object may be moved to make room for other objects or for aesthetic purposes.

  • Consider that the operating system, installed service packs, installed toolkits, and so on, can all affect the size and location of an object in an application. Make sure that the dimensions of the selected text area are large enough for different system configurations.

  • The size of the application window also affects the text recognition. Experiment with application window size to achieve the best recognition. Then, make sure to record and run your tests on that size.

Tests created in UFT 15.0 or earlier

In UFT 15.0.1, the Abbyy OCR engine was upgraded to a much newer version. As a result, you may see changes in the text recognition of tests created in earlier versions of UFT.

OCR Engine consistency Once you determine which OCR engine works best with your tests, we recommend using that engine consistently. Using different engines for different runs may produce different results.

See also: