Summary reports

The Analysis Summary Report provides general information about the execution of the load test scenario. It lists statistics about the scenario run and provides links to the following graphs: Running Vusers, Throughput, Hits Per Second, HTTP Responses per Second, Transaction Summary, and Average Transaction Response Time.

To access

Session Explorer > Reports > Summary Report

Important information

The Summary report for Diagnostics provides a usage chart that links to and displays each individual transaction's web, application, and database layers, and provides the total usage time for each transaction.

Relevant tasks

You can save the Summary Report to an Excel file by selecting View > Export Summary to Excel or by clicking on the toolbar.

Summary report

User interface elements are described below:

UI Element

Description

Scenario details

This section shows the basic details of the load test scenario being analyzed.

Scenario Schedule

This graph shows the schedule of the Controller scenario.

You can use this information to identify problems. For example, if the maximum number of users in your scenario exceeds your licenses, you will see a gap between the number of users in this schedule graph and the actual number of users that ran the scenario.

Statistics

This section shows a breakdown of the transaction statistics, and status for SLAs defined for each measurement. Click on a measurement to open the relevant graph for the measurement.

Top Transactions by Duration

This table shows the five transactions with the highest average response time.

Scenario Behavior Over Time

This section shows how each transaction performed in terms of the SLA over time intervals. The green squares show time intervals where the transaction performed within the SLA boundary. Red squares show time intervals where the transaction failed and gray squares show where no relevant SLA was defined.

Note: The time intervals displayed in the Scenario Behavior Over Time section may vary for each interval. The time interval set in the tracking period of the SLA is only the minimum time interval that will be displayed.

It is only the display that varies. The SLA is still determined over the time interval you choose in the Advanced Settings section.

Analysis allows you to analyze a specific transaction in more detail. You open the Analyze Transaction tool from the Scenario Behavior Over Time section in one of the following ways:

  • Select the transaction to analyze from the list and enter the time interval in the From and To boxes. Then click Analyze Transaction.
  • Drag the mouse over the desired transaction and time range to analyze. Then click Analyze Transaction.

For more information on Transaction Analysis Reports, see Analyze Transactions dialog box.

Transactions

This section displays a table containing the load test scenario's diagnostics data.

Using the table header bar you can filter by SLA status, search for transactions or values in the table, and filter columns by clicking on the column title. At the end of the table you can scroll between pages.

This table includes the following:

  • Network Virtualization: If your script runs with network virtualization, you will see each latency option as a group under the overall transaction.
  • APDEX column: Displays the overall APDEX rating for each transaction, if an SLA was defined for APDEX scores.

  • x Percent column: Indicates the maximum response time for that percentage of transactions performed during the run. All transaction response times are calculated based on "passed" transactions data.

    You can change the value in the percentile column in the Summary Report section of the General tab of the Options dialog box. Open the dialog box (Tools > Options) and enter the desired percentage. Alternatively, you can also change the value in the Summary Filter (View > Summary Filter).

HTTP Responses

This section shows the number of HTTP status codes returned from the web server during the load test scenario, grouped by status code.

Note: There are additional Diagnostics sections that may appear at the end of the Summary report, depending on the configuration of your system.

Back to top

Summary report with SLA

This table describes the elements of the report indicating SLA compliance.

Note: This feature is available for version 2020 SP3 and later.

Name

Description

Scenario details

Basic details of the load test scenario being analyzed.

Statistics Summary

A breakdown of the transaction statistics.

X Worst Transactions/ SLA Violations

The worst transactions in terms of how often the transactions exceeded the SLA boundary during the run, and by how much.

Transaction Response Time (Average)

This section shows:

  • Failure Ratio. The percentage of time intervals where the transaction exceeded the SLA. You can see this graphically in the Scenario Behavior Over Time section below.
  • Failure Value. The average percentage by which the transaction exceeded the SLA over the whole run.

The following example shows the 5 transactions with the worst average response time.

Transaction Response Time (Percentile)

This section shows an SLA violation report with the following information:

  • Goal. The goal value for the SLA rule.
  • Actual. The percentile (90th by default) of the transaction.
  • Violation. The average percent by which the values exceed the threshold.

In the following example, in the first transaction, the Goal value of 5 was exceeded on average by 190.5%.

The table will only display transactions that have exceeded the threshold.

If we would change the goal to 14, only two transactions would be marked as 'Failed' and only those two would be shown.

Back to top

Summary reports for cross result graphs

User interface elements are described below:

UI Element

Description

<graphs>

Displays summary information for the scenarios that you are comparing. The information is displayed in a way that enables you to compare data from the different scenarios. Includes the same type of information as the regular Summary report except for the following:

  • SLA information

  • Diagnostics information

  • Scenario behavior over time

Back to top

See also: