Trend reports

This topic describes how to use trend reports to provide visibility of your application's performance.

In this topic:

About trend reports

Defining trend reports enables you to compare performance test run data (transactions and web resources) over several instances of a test run. This provides better visibility into your application's performance and helps you identify improvements or regressions.

For each trend in the list, summary information is displayed and provides a high-level overview of the trend.

Create a new trend

You can create a new trend to analyze changes in performance test run data over time.

To create a new trend:

1. From the menu, select Trends. The Trends page opens.

2. Click the Create button . The Create a new trend wizard opens.

3. On the Select a test page, enter a name for the trend and select a test from the list. The number of available runs for each test is displayed in the Runs column. Click Next.

4. On the Select runs page:

   1. Select the runs to include in the trend.

   2. Select a comparison method.

      Method	Description
      Compare to Baseline	Select one test run as the baseline. All measurements are compared to the measurements contained in the baseline.
      Compare to Previous	Each measurement in a test run is compared to the measurement in the run that immediately precedes it.

      For differences between the two comparison methods, see Analyze trends.

   3. If you use Compare to Baseline, select the checkbox next to a run and click Set Baseline. The Baseline icon is displayed next to the test run ID.

   4. (Optional) To make runs comparable by duration, select the runs and click Edit duration. By default, data from the entire test run is used for trending. To include only part of a run, define the start and end times, and click Apply.

   5. Click Next.

5. On the Set thresholds page, define thresholds to track performance differentials between the compared measurements. Thresholds help identify significant performance improvements or regressions. You can define thresholds for Major regression, Minor regression, and Improvement.

   1. Enter percentage values to define improvements and regressions for each of the following measurements.

      Measurement	Details
      Transaction response time (TRT)	Measures whether the average transaction response time of the transactions over a specified time interval exceeds the defined threshold.
      Total errors	Measures the total number of errors within the trended time range.
      Hits per second	Measures the total number of transaction hits per second to the web server within the trended time range.
      Throughput	Measures the total data received from web servers within the trended time range.
      Passed transactions	Measures the actual number of transactions that passed within the trended time range.

   2. For TRT, you can also specify a threshold value in seconds. For example, you can define a major regression as a 25% increase in TRT with a minimum TRT of 0.2 seconds (200 milliseconds).

6. Save the trend.

   A trend report is generated and the Runs overview tab of the trend is displayed. For details, see Trends list.

Trends list

The Trends list displays summary information for all the defined trends. This includes passed transactions ratio, transaction performance distribution (percentage major regressions, minor regressions, improvements, and not in threshold), and transactions with the highest major regressions. The trends list provides you with basic overview trending information, without having to open a trend view. In the trends list, basic trend information is shown by upward or downward arrows that indicate performance improvements or regressions.

You can perform the following actions on this page.

Action	Details
Rename a trend.	Select the trend and click the Rename button .
Delete a trend.	Select the trend and click the Delete button .
Add a new trend.	For details, see Create a new trend.
Open a more detailed view of the Trend report.	Click the trend number link or click View. The Runs overview tab is displayed.

Runs overview tab

The Trend > Runs overview tab provides an overview of the runs included in the trend. This gives you a basic overview of trending information without opening the TRT performance trend view.

You can use this tab to make sure all runs are comparable and adjust them if needed, for example by deleting runs or including other measurements in the trend. For example, if one run has significantly more regressions, you may want to delete it from the trend.

In Runs overview, you can perform the following actions.

Action	Details
Edit the trend	Click the Edit Trend button and make changes to your trend: add runs, remove runs, and change the comparison method or thresholds.
Set a baseline	Select the test run to use as the baseline run for comparing test runs, and click the Set Run as Baseline button . The selected baseline test run is indicated by the Baseline icon . For details on comparison methods, see Create a new trend.
Edit a test run name	Select a test run and click the Rename button to change the test run name.
Remove a run	Select a run and click the Remove button . Note: A test run cannot be deleted from the system unless it is first removed from all trend reports.
Change the duration of a run	Select a run and click the Change Duration button . Enter the start and end times for the run.
View more details of a run	Click the Run ID link to open the Summary page for the run.
Select columns	To change the columns displayed in the list of runs, select the Available columns button . Select or clear the checkboxes next to the column names to show or hide the columns.
Export	Click the Export button to download the trend report as a CSV file.

TRT performance tab

On the TRT performance tab of a trend report, you can see the baseline (if relevant) and the runs with the thresholds that you set for major regressions, minor regressions, and improvements.

You can select how the data is displayed:

• TRT data can be aggregated using either percentiles or an average.
  

  Measurement	Details
  Percentile	Measures the percentage of transactions whose transaction response time falls below a specific threshold. For example, the 95th percentile is the value where 95% of all measurements are below it, and 5% of measurements are above it. Available values: 85th, 90th, 95th, and 99th percentile.
  Average	Measures whether the average transaction response time of the transactions over a specified time interval exceeds the defined threshold.

• You can filter the data. Click the Filter button and select one or more values: major regression, minor regression, improvement, or not in threshold.

• Use the View menu to switch between:

  • Showing and hiding the differences. For example, if you have many runs and want to view details on one screen, you can hide the differences. When you hide the differences, TRT values are color coded by severity.

  • Showing the difference as a percentage or a value in milliseconds.

Click the Export button to download the TRT performance report as a CSV file for further analysis.