Viewing performance improvements and regressions

This lesson explains how to view changes in performance from one performance test to another using a trend report.

Trend reports

A trend report is a feature that enables you to view changes in performance from one performance test to another, or across several performance tests. By analyzing these changes, you can easily identify improvements or regressions in the measurement's performance.

For example, if you would like to analyze the performance trend of the average transaction response time measurement of Transaction X, then the trend report would display the changes in the response time from one test to the next, clearly indicating whether this measurement improved (where the response time decreased) or regressed (where it increased).

What steps are involved in working with trend reports?

The process of creating and viewing a trend report involves the following steps:

Step 1 - Creating the trend report. Creating the basic trend report by defining its name, description and selecting a template.
Step 2 - Adding test runs to the trend report. Extracting test run data from Analysis to the trend report.
Step 3 - Viewing trend report data. Opening the trend view and analyzing the trending information.

Creating the Trend Report

You create the trend report from the main Trend Reports page.

To create the trend report:

Open the main Performance Trending page.

In the banner, click the module name or arrow and selectReports > Performance Trending.
Define a new trend report.

On the Performance Trending page, click the Create Trend Report button . Enter a name and description for the report and click Create.

Add performance test run data to the report.

Click Add Runs and select a project and test set. All analyzed instances of the performance test appear in the grid.
Select the analyzed instances of the test that you want to add to the trend report.

Note: To add test runs to the trend report, there must be a data processor in your project's host pool. For details, contact the administrator.
By default, data collected from the whole test run is available for trending. To make only part of the test run available for trending, click Time Range, and select Trend Part of Run.

Select the part of the run to trend by entering the start and end times in the Start Time and End Time boxes, and click OK.

To revert back to the default setting, select Trend complete run.
Click Add or Add & Close. The test runs are uploaded from Analysis and added to the trend report. The trend report opens displaying the Trend Overview tab.

Viewing Trend Report data

In the following section, we will look at trending information for the average transaction response time measurement only.

To view trending information:

On the trend report, click the Performance tab.

In the trend view that opens, you see a table that displays the transactions that were present in the test run, as well as the average transaction response times for each selected instance of the test run.

By comparing these average transaction response time figures, you can identify if the performance of the transaction improved or regressed from one test run to the next.

The following example illustrates this point.

Transaction Response Time (Compare to baseline)

Name	Type	Average
Name	Type	run 3 [Base]	run 4	run 5	run 6
All	TRT	4.567	1.22 (-73.29%)	2.32 (-49.2%)	12.455 (+172.72%)
TRX_01	TRT	2.045	4.073 (+99.17%)	2.035 (-0.49%)	1.05 (-48.66%)
TRX_02	TRT	1.045	2.07 (+98.09%)	1.015 (-2.87%)	1.051 (+0.57%)
TRX_03	TRT	3.053	3.067 (+0.46%)	2.009 (-34.2%)	2.654 (-13.07%)
TRX_04	TRT	6.055	6.868 (+13.43%)	5.011 (-17.24%)	7.05 (+16.43%)

In the trend view shown above, four transactions (TRX_01, TRX_02, TRX_03, and TRX_04) are displayed and their average Transaction Response Time measurement is being trended from four performance test runs: 3, 4, 5, and 6.

Test run 3 has been automatically defined as the baseline run, (as indicated by the word Base in brackets). This means that the average transaction response times contained in the other tests are compared to test run 3 only.

In test run 3, the average transaction response time for TRX_01 was 2.045. The average transaction response time for the same transaction in test run 4 was 4.073, which represents a slower response time and therefore a regression in the performance of this measurement. The percentage difference between the two figures is displayed in parenthesis, in this case +99.17%.

In test run 6, the average transaction response time for TRX_01 was 1.05, which represents a faster response time that test run 3, and therefore a performance improvement. The percentage difference between the two figures is displayed in parenthesis, in this case -48.66%.