Interpret run results
Relevant for: GUI tests and components, API testing, and business process tests and flows
This task describes how to navigate and interpret the run results to help you isolate and solve problems in your application.
Set run result reporting options
Before you begin a test run, set options and properties to change the information included with the test results:
Option | UI location | Description |
---|---|---|
Instruct UFT One to automatically open the run results | Run Sessions pane (General tab > Run Sessions node) | Select the View results when run session ends option. |
Select the format of the run results | Run sessions pane (General tab > Run Sessions node) |
Select either the HTML Report or Run Results Viewer Report option. If you are running a business component, the HTML report option is supported only from ALM versions 12.53 and higher. |
Instruct UFT One to automatically export the run results | Run sessions pane (General tab > Run Sessions node) |
Select the Automatically export run results when run session ends option. You can export test results in following formats:
For details, see Automatically export run results when run session ends. |
Take screen captures of steps | Screen Capture pane (GUI Testing tab > Screen Capture pane) |
Select the Save still image captures to results option. You can specify to save screen captures for:
For mobile tests, save as GIF. Instructs UFT One to capture animated GIF files for the specified steps, rather than still images. (Requires using Digital Lab (UFT Mobile) 3.4 or later) |
Capture movies of the run session | Screen Capture pane (GUI Testing tab > Screen capture pane) |
Select the Save movie to results option. You can specify to capture the movie:
|
Save HTTP request/response body with the run results | Properties pane for an HTTP request activity in an API test (Properties pane > General tab) |
Set values to one or both of the following properties: Extension type for saved request body Extension type for saved response body Run results with the request / response body are saved in the <Results>\Report\Data folder |
View step details
In the run results, view details for each test step:
-
In the lower part of the run results, display the Test Flow for a full list of all steps in the test run.
-
From the step tree, select the step to display its details. These include screen captures (if enabled), and call stack information for any errors.
Tip: If you have a test with a large number of steps, instead of expanding each node individually, click the Expand All or Collapse All buttons to quickly view and hide all nodes in the results tree.
Analyze errors
The run results also display a special section detailing the errors and warnings that occurred during the test run. View these errors without the test flow to determine the root cause of the error:
-
In the lower part of the run results, display the Errors List.
-
Select an error from the list. A summary of the error details are displayed.
Tip: If you are in the Test Flow view, use the Previous Error and Next Error buttons to quickly jump to the next error in the list.
-
Use the available details to isolate the cause of the error including:
-
A description of the error
-
The test object being used in the step
-
The properties of the test object
-
The call stack of the current step
-
Analyze checkpoint results
For each checkpoint step, view information about the checkpoint:
For a checkpoint that succeeded |
|
For a checkpoint that failed |
|
View the included data sources
If your test uses a data source, this data source is attached to the test as an external file. This enables you to see exactly which data was used for this test run.
The location of the data differs depending on the test type and the type of data source:
Test type | Data source type | Location |
---|---|---|
GUI test or component | Data table |
A link named Test Data displayed in the Show More section above the Test Flow and Details area of the run results. The data table is opened as an Excel file. You can also find the Excel file in the run results folder: <run results folder>\Report\Default.xls |
API test |
|
A link named Test Data displayed in the Show More section above the Test Flow and Details area of the run results. This link opens a new browser page in which the specific data sources for the test are displayed. You can click on the name of a data source to view it as an external file. The actual run-time values used in each step (taken from the data source) are displayed in the Details section for each test step. |
Note: The run results do not sync the selected step in the Test Flow with the data source.
View the call stack to isolate errors in the test flow
When you have an error in your test, use the Call Stack to determine exactly where this error occurs. This helps you isolate the specific line in the test that contains the order.
-
In the lower section of the run results, display the Errors.
-
From the list of errors and warnings, select an error. The summary of the error is displayed
-
In the error details, find the section containing the StackTrace. This section displays the following:
-
The section of the test containing the error (an action, function library)
-
The specific function containing the error (if the error occurred in the context of a function call)
-
The full script line in which the error occurred
-
The line number of the error in the relevant document
-
View the step properties capture for an API test step
When you run an API test, each test step requires specific property values to run the step. In the run results, view the properties and the values used in this specific test run from the run results:
-
In the lower section of the run results, display the Test Flow.
-
In the Test Flow, select the step you want to view. A summary of the test step is displayed.
-
In the summary, view the Captured Data section of the summary.
If the Captured Data contains links to external resources (for example a Web service Request/Response), you can click the link in the Captured Data and a floating window displays the detailed data.
View custom messages sent to the run results
For GUI tests |
Use the Reporter object to send custom messages to the run results. These messages appear in the Test Flow in the step in which you inserted the statement. |
For API tests |
When you send a custom message to the run results from an step's event handler, it is displayed as part of the step's captured data.
|
Send the run results by email
In the tab displaying the run results, right-click the tab name and select Send by Email. A mail message opens in your default mail application with the run results attached.
In Windows 7, close the mail window to return to UFT One.