Show anomalies

Available in versions: OneAnalysis version 2020 R1 and later

This section describes how to view anomalies in graphs, which enable you to quickly, intuitively, and effectively identify abnormal application behavior in performance tests.

View anomalies in graphs

You can view anomalies to identify abnormal application behavior in performance tests. Anomalies are where data points (measurements) significantly deviate from their normal behavior. These insights help speed your investigation into system performance, and determine the root cause of detected deviations.

You can view anomalies in real-time while a performance test is running, or after a test run has finished.

To view anomalies:

  1. Select a run to analyze as described in Select a run to analyze.

  2. Select a graph and click Graph Display. The Graph Display dialog box opens.

  3. Toggle the Anomalies button to Show to identify plot bands which indicate where anomalies occurred.

  4. To zoom in on a metric anomaly, hold down the left mouse button and drag the slider over the section of the graph when the anomaly occurred to increase the magnification.

    Click Reset zoom to restore the default graph size.

Back to top

How OneAnalysis determines when an anomaly has occurred

OneAnalysis uses an algorithm to determine anomalies. The algorithm is split into two main parts:

1. Detecting if a point in a series is abnormal, relative to other points.

OneAnalysis uses a statistical assumption that the series approximately follows a normal distribution, and deviates in a standard manner over time.

For every measurement, it continuously calculates the mean and standard deviation. Then, for each point, weighted versions of the mean and standard deviation (which give higher priority to more recent points) are calculated. These weighted results are used to create the sleeve. The sleeve is then 6 “weighted standard deviations” around the “weighted mean” of the measurement.

Every point in the series that is above, or below the sleeve, is considered abnormal.

2. Notifying the user when a measurement behaves abnormally within a certain time range.

Not all deviations from the sleeve are considered anomalies; only when a measurement deviates from the sleeve for a significant period of time.

OneAnalysis determines this by keeping a baseline of the last 30 points for each measurement. Each point that deviates from the sleeve is given a value of 1, and 0 if it is inside the sleeve. For example, a measurement with a baseline sum of 0 means that all points are in the sleeve, whereas a baseline sum close to 30 means that most points are deviating.

OneAnalysis notifies the user of an anomaly when the baseline sum for a measurement is above 18 (this means that 60% of the points in the baseline are above or below the sleeve).

Note: OneAnalysis gives more weight to measurements with a baseline that has consecutive deviations (0000011111) than to an unstable baseline (0101010101). This is because measurements that spike are not deviating in a standard way, and are more likely to result in false-positive notifications.

Back to top

See also: