Summary Report

The Analysis Summary Report provides general information about the execution of the load test scenario.

Summary Report overview

The Summary Report lists statistics about the scenario run and provides links to the following graphs: Running Vusers, Throughput, Hits Per Second, HTTP Responses per Second, Transaction Summary, and Average Transaction Response Time.

To access the Summary Report, navigate to Session Explorer > Reports > Summary Report.

To save the Summary Report to an Excel file, select View > Export Summary to Excel or by click the Export to Excel button on the toolbar.

Tip: The Summary Report for Diagnostics provides a usage chart that links to and displays each individual transaction's web, application, and database layers, and provides the total usage time for each transaction.

User interface

The following table describes the columns in the Summary Report:

UI Element

Description
Scenario details

This section shows the basic details of the load test scenario being analyzed.
Scenario Schedule

This graph shows the schedule of the Controller scenario.

You can use this information to identify problems. For example, if the maximum number of users in your scenario exceeds your licenses, you will see a gap between the number of users in this schedule graph and the actual number of users that ran the scenario.

If the scenario includes Gremlin disruption events, the start and end times for the events are shown on the graph.
Statistics

This section shows a breakdown of the transaction statistics, and status for SLAs defined for each measurement. Click on a measurement to open the relevant graph for the measurement.
Scenario Behavior Over Time

This section shows how each transaction performed in terms of the SLA over time intervals. The green squares show time intervals where the transaction performed within the SLA boundary. Red squares show time intervals where the transaction failed and gray squares show where no relevant SLA was defined.

Note: The time intervals displayed in the Scenario Behavior Over Time section may vary for each interval. The time interval set in the tracking period of the SLA is only the minimum time interval that is displayed.

It is only the display that varies. The SLA is still determined over the time interval you choose in the Advanced Settings section.

Analysis allows you to analyze a specific transaction in more detail. You open the Analyze Transaction tool from the Scenario Behavior Over Time section in one of the following ways:

  • Select the transaction to analyze from the list and enter the time interval in the From and To boxes. Then click Analyze Transaction.
  • Drag the mouse over the relevant transaction and time range to analyze. Then click Analyze Transaction.

For more information on Transaction Analysis Reports, see Analyze Transactions dialog box.
Transactions

This section displays a table containing the load test scenario's diagnostics data. By default, transactions are listed according to their execution order in the script.

Tip: After changing the list order by sorting or filtering, you can revert to the default display order by using the Tools > Templates option to reapply the default template (for details, see Apply/Edit Template dialog box).

Using the table header bar you can filter by SLA status, search for transactions or values in the table, and filter columns by clicking on the column title. At the end of the table you can scroll between pages.

This table includes the following:

  • Network Virtualization: If your script runs with NV, you see each latency option as a group under the overall transaction.

  • APDEX column: Displays the overall APDEX rating for each transaction, if an SLA was defined for APDEX scores.

  • x Percent column: Indicates the maximum response time for that percentage of transactions performed during the run. All transaction response times are calculated based on "passed" transactions data.

    You can change the value in the percentile column in the Summary Report section of the General tab of the Options dialog box. Open the dialog box (Tools > Options) and enter the required percentage. Alternatively, you can also change the value in the Summary Filter (View > Summary Filter).
HTTP Responses

This section shows the number of HTTP status codes returned from the web server during the load test scenario, grouped by status code.

Note: There are additional Diagnostics sections that may appear at the end of the Summary Report, depending on the configuration of your system.

Back to top

Summary Report with SLA

In the Summary Report with SLA, the Statistics summary section provides a breakdown of the transaction statistics.

This Transactions section shows different columns, depending on the type of SLA rule.

For the Percentile SLA rule, the Transactions columns show the transaction response time information per transaction name and its compliance with the SLA:

  • SLA Status. The general status of the transaction: In compliance ü or in violation û or no data /.
  • Goal. The goal value of the SLA rule.
  • Violation (%). The average percent by which the values exceeded the threshold.

  • Minimum: The minimum time it took for the transaction to complete.

  • Average: The average time it took for the transaction to complete.

  • Maximum: The maximum time it took for the transaction to complete.

  • Std. Deviation: The standard deviation of the transaction response time.

  • 90 Percent. The percentile (90th by default) of the transaction.
  • Pass: The number of transactions that ended successfully without errors.
  • Fail: The number of transactions that failed to run until the end.
  • Stop: The number of transactions that were stopped before their completion.

In the following example, in the first transaction, the Goal value of 1 was exceeded on average by 0.2%.

For the Average SLA rule, the Transactions columns show the transaction response time information per transaction name and its violation of the threshold defined in the SLA:

  • SLA Status. The general status of the transaction: In compliance ü or in violation û or no data /.
  • Threshold. The threshold defined in the SLA rule.
  • Violation (%). The average percent by which the values exceeded the threshold.

  • Minimum: The minimum time it took for the transaction to complete.

  • Average: The average time it took for the transaction to complete.

  • Maximum: The maximum time it took for the transaction to complete.

  • Std. Deviation: The standard deviation of the transaction response time.

  • 90 Percent. The percentile (90th by default) of the transaction.
  • Pass: The number of transactions that ended successfully without errors.
  • Fail: The number of transactions that failed to run until the end.
  • Stop: The number of transactions that were stopped before their completion.

The following example shows the 3 transactions that exceeded the threshold of 1.

For the APDEX SLA rule, the Transactions columns show the APDEX score and the number of transactions that satisfied the rule and the number tolerated by the rule.

  • APDEX. The APDEX (Application Performance Index) score of this transaction, with 1 indicating full compliance in which all transactions satisfied the SLA rule. The APDEX score is a ratio value of the number of satisfied and tolerating requests to the total requests made. Each satisfied request counts as one request, while each tolerating request counts as half a satisfied request.
  • Satisfied. The threshold needed to comply with the SLA rule.
  • Tolerating. The threshold tolerated by the SLA rule.

  • Minimum: The minimum time it took for the transaction to complete.

  • Average: The average time it took for the transaction to complete.

  • Maximum: The maximum time it took for the transaction to complete.

  • Std. Deviation: The standard deviation of the transaction response time.

  • 90 Percent. The percentile (90th by default) of the transaction.
  • Pass: The number of transactions that ended successfully without errors.
  • Fail: The number of transactions that failed to run until the end.
  • Stop: The number of transactions that were stopped before their completion.

The following example shows that only one transaction received a score of 1.

Back to top

Summary Report for cross result graphs

User interface elements are described below:

UI Element

Description
<graphs>

Displays summary information for the scenarios that you are comparing. The information is displayed in a way that enables you to compare data from the different scenarios. Includes the same type of information as the regular Summary Report except for the following:

  • SLA information

  • Diagnostics information

  • Scenario behavior over time

Back to top