Trend reports

Defining trend reports enables you to compare performance test run data (transactions and web resources) over several instances of a test run. This provides better visibility of your application's performance and enables you to identify improvements or regressions in performance. For each trend in the list, summary information is displayed, providing a high-level overview of the trend.

In this topic:

Create a new trend

You can create a new trend to view changes in performance test run data over time.

To create a new trend:

1. Select Trends from the OpentText Core Performance Engineering Analysis menu. The Trends page open.

2. Click the Create button . The Trends Definition wizard opens.

3. In the Select a test page of the wizard, give your trend a name and select a test from the list. For each test, the number of runs is listed in the Runs column.

4. Select runs to add to the trend.

5. Select a comparison method.

Method	Description
Compare to Baseline	You select one performance test run and define it as the baseline. All measurements in the report are then compared to the measurements contained in the baseline.
Compare to Previous	All measurements in a performance test are compared to the measurements in the performance test run that immediately precedes it.

For more details on the differences between the two comparison methods, see Analyze trends.

To set a test run as the baseline, select the checkbox next to the run, and click Set baseline. The Baseline icon is displayed next to the Test Run ID.

6. (Optional) Edit the duration of selected runs to make the runs comparable. By default, data collected from the whole test run is available for trending. To make only part of the test runs available for trending, select the runs and click Edit duration. Define the start and end time, and click Apply.

7. Set thresholds.

   To identify significant improvements or regressions in performance, you can define unique thresholds to track differentials between measurements being compared. You can set thresholds for a major regression, a minor regression, and for an improvement in the following measurements.

   Measurement	Details
   Transaction Response Time (TRT)	Measures whether the average transaction response time of the transactions over a specified time interval exceeds the defined threshold.
   Total Errors	Measures the total number of errors within the trended time range.
   Hits per second	Measures the total number of transaction hits per second to the web server within the trended time range.
   Throughput	Measures the total data received from web servers within the trended time range.
   Passed transactions	Measures the actual number of transactions that passed within the trended time range.

   Enter the percentage differential between instances of the measurement that are considered an improvement, a minor regression, and a major regression in performance. For TRT, in addition to a percentage value, you can also define the threshold value in seconds. For example, you can define a major regression as a 25% increase in TRT, and with a TRT of at least 0.2 (200 milliseconds).
   

8. After you save a new trend, a trend report is generated and the Runs overview tab of the trend is displayed. For more details, see Trends list.

Back to top

Trends list

The Trends list displays summary information for all the defined trends. This includes passed transactions ratio, transaction performance distribution (percentage major regressions, minor regressions, improvements, and not in threshold), and transactions with the highest major regressions. The trends list provides you with basic overview trending information, without having to open a trend view. In the trends list, basic trend information is shown by upward or downward arrows that indicate performance improvements or regressions.

You can perform the following actions on this page.

Action	Details
Rename a trend.	Select the trend and click the Rename button .
Delete a trend.	Select the trend and click the Delete button .
Add a new trend.	For details, see Create a new trend.
Open a more detailed view of the Trend report.	Click the trend number link or click View. The Runs overview tab is displayed.

Back to top

Runs overview tab

The Trend > Runs overview tab provides an overview of the runs included in the trend. This provides you with a basic overview of trending information without having to open the TRT performance trend view.

You can use this tab to make sure all your runs are comparable, and adjust if necessary, such as deleting runs or including other measurements in the trend. For example, if one run has significantly more regressions you may want to delete it from the trend.

In Runs overview, you can perform the following actions.

Action	Details
Edit the trend	Click the Edit Trend button and make changes to your trend: Add runs, remove runs, change the comparison method or thresholds.
Set a baseline	Select the test run to use as the baseline run for comparing test runs, and click the Set Run as Baseline button .The selected baseline test run is indicated by the Baseline icon . For details on comparison methods, see Create a new trend.
Edit a test run name	Select a test run and click the Rename button to change the test run name.
Remove a run	Select a run and click the Remove button . Note: A test run cannot be deleted from the system unless it is first removed from all trend reports.
Change the duration of a run	Select a run and click the Change Duration button . Enter the start and end times for the run.
View more details of a run	Click the Run ID link to open the Summary page for the run.
Select columns	To change the columns displayed in the list of runs, select the Available Columns button . Select or clear the checkboxes next to the column names to show or hide the columns respectively.
Export	Click the Export button to download the trend report as a .CSV file.

Back to top

TRT performance tab

On the TRT performance tab of a trend report, you can see the baseline (if relevant), and the runs with the thresholds that you set for major regressions, minor regressions, and improvements.

You can select how the data is displayed:

• TRT data can be aggregated using either percentiles or an average.
  

  Measurement	Details
  Percentile	Measures the percentage of transactions whose transaction response time falls below a specific threshold. For example, the 95th percentile is the value where 95% of all measurements are below it, and 5% of measurements are above it. Available values: 85th, 90th, 95th, and 99th percentile.
  Average	Measures whether the average transaction response time of the transactions over a specified time interval exceeds the defined threshold.

• You can filter the data. Click the Filter button and select one or more values: major regression; minor regression; improvement; not in threshold.

• Use the View menu to switch between:

  • Showing and hiding the differences. For example, if you have a lot of runs and want to view the details on one screen, you can hide the differences . When you hide the differences, TRT values are color coded by severity.

  • Showing the difference as a percentage or a value in milliseconds.

Click the Export button to download the TRT performance report as a .CSV file for further manipulation of the data.

Back to top