Share this page

# SLA rules

This topic describes how to define SLA performance goals for your tests.

In this topic:

## About SLA rules

You can define rules for your DevWeb script tests, to determine if the AUT performance levels meet your service level agreement (SLA) goals. Determine the criteria for success or failure by defining rule thresholds that you associate with the script.

Whenever the test is executed, the rules are applied, and a report is created to show the results of the rule calculations. This enables you to determine at which point the application failed to meet minimum performance levels.

Note: SLA rules are supported only for DevWeb tests run locally in OpenText Performance Engineering for Developers.

Back to top

## Supported rules

The following SLA rules are supported.

- Transaction Response Time (type: averageTRT). This rule measures the average response time for the transaction during the test, and compares it with a defined threshold (averageTime, in milliseconds). If the average time is above this value, the SLA fails.

  The rule can be applied to multiple transactions.

- Failing Transactions to Total Transaction Ratio (type: failureRate). This rule measures the overall failure rate for transactions during the test. The SLA fails if the percentage of failed transactions is greater than a defined percentage (a whole integer between 1-100).

- Total Passing Transactions (type: passingTransactions). This rule compares the total amount of transactions that passed against a defined threshold amount. The SLA fails if the number is below the threshold.

- Transaction Rank Percentile (type: TRP). This rule compares defined percentile ranks against the ranges for the transaction duration, for both passed and failed transactions. The SLA fails if the target percentage is not reached.

- Passed Transaction Rank Percentile (type: PTRP). This rule compares defined percentile ranks against the ranges for the transaction duration, for passed transactions only. The SLA fails if the target percentage is not reached.

  For a specified transaction for the TRP and PTRP rules, you define the following arguments:

  Argument | Definition | Example
  --- | --- | ---
  ranges | The ranges (starting from zero) are defined using range markers, in milliseconds (ms). | If the range markers are defined as [2000, 5000, 10000], this delimits the following duration ranges: 0-2000 ms 0-5000 ms 0-10000 ms
  ranks | This defines the percentile rank required for each specified range. You must define the same number of ranks as ranges. | If the ranks are defined as [88, 98, 100], this means the following results are required for the SLA to succeed: 88% of transactions must have a duration of 2000 ms or less. 98% of transactions must have a duration of 5000 ms or less. 100% of transactions must have a duration of 10000 ms or less.

Back to top

## Defining SLA rules

Define the criteria for your SLA rules in the sla_rules.yml file.

Add the sla_rule.yml to your script folder, before running the test. If the rules in the file are enabled, then at the end of test execution, the replay summary displays a pass or fail status, and generates a report file.

Two report formats are supported:

- JUnit: Format for the JUnit unit testing framework, in an XML file.
- TAP: Format for the TAP unit testing framework, in a TAP file.

To run a test with SLA rules:

1. Copy the sla_rules.yml file from <DevWeb root>\examples\SLAJUnit to your script folder.

2. Open the sla_rules.yml file and edit the contents, and add extra rules as required, as relevant for your script. Use one of the examples given below in SLA rules file example as a guide.

   For each transactionName value, enter the name of the transaction as it appears in the script.

   You can enable or deactivate the whole file, or individual rules.

3. Define the report settings:

   - The type of report file (reporterType) to generate, JUnit or Tap.

   - The name for the report, in the format <name>.xml for JUnit reports, or <name>.tap for TAP reports.

   The report is generated inside the script folder.

4. Execute the test. See Run scripts.

5. At the end of the test, check the SLA rule results (after the replay summary) :

6. Open the script folder and locate the JUnit or TAP report. Review the report to see more detailed information on the rule calculations.

## SLA rules file example

Example of an sla_rules.yml file:

Copy code

    name: devWeb
    version: 2020.2.0

    enabled: true                                                 # Allows the user to enable or disable all the rules in a category.
    reporterType: JUnit                                           # Allows to change the reporter type, supported reporters: [JUnit, Tap].
    reportName: SLA-report-JUnit.xml                              # The report name, which is generated inside the script directory.
    rules:                                                        # List of the rules which the SLA enforces.
      - rule: Transaction Rank Percentile                         # The SLA rule validates if transaction duration measurements meet percentile rank specified in the arguments.
        type: TRP                                                 # The transaction type.
        enabled: true                                             # Allows the user to enable or disable a specific rule.
        arguments:                                                # The arguments which a given type is expecting (at least one entry).
          - transactionName: Login                                # The name of the transaction for which we want to calculate the transaction percentile rank.
            ranges: [3000, 3500, 4000, 4500]                      # Defines the duration ranges in milliseconds. For example, 3500 defines the range 0-3500.
            threshold:                                            # The number of threshold ranks must match the number of ranges.
              ranks: [80, 85, 90, 95]                             # Defines the percentile ranks for the ranges. For example, 85% of transactions are required to have a duration of 0-3500 milliseconds.
          - transactionName: Search
            ranges: [2700, 3500, 3800]
            threshold:
              ranks: [80, 85, 90]
          - transactionName: Order
            ranges: [5000, 7000]
            threshold:
              ranks: [85, 90]
          - transactionName: Logout
            ranges: [3200, 4200, 5200, 6999]
            threshold:
              ranks: [70, 80, 85, 90]
      - rule: Passed Transaction Rank Percentile                  # The SLA rule validates if transaction duration measurements meet percentile rank specified in the arguments.
        type: passedTRP                                           # The transaction type.
        enabled: true                                             # Allows the user to enable or disable a specific rule.
        arguments:                                                # The arguments which a given type is expecting (at least one entry).
          - transactionName: Login                                # The name of the transaction for which we want to calculate the transaction percentile rank.
            ranges: [3000, 3500, 4000, 4500]                      # Defines the duration ranges in milliseconds. For example, 3500 defines the range 0-3500.
            threshold:                                            # The number of threshold ranks must match the number of ranges.
              ranks: [80, 85, 90, 95]                             # Defines the percentile ranks for the ranges. For example, 85% of transactions are required to have a duration of 0-3500 milliseconds.
          - transactionName: Search
            ranges: [2700, 3500, 3800]
            threshold:
              ranks: [80, 85, 90]
          - transactionName: Order
            ranges: [5000, 7000]
            threshold:
              ranks: [85, 90]
          - transactionName: Logout
            ranges: [3200, 4200, 5200, 6999]
            threshold:
              ranks: [70, 80, 85, 90]

Back to top

See also:

- Script structure
- Integrate with CI/CD systems

https://admhelp.microfocus.com Performance Engineering for Developers 8942
Tips to improve your search results... OpenText™ Performance Engineering for Developers Help Center OpenText™ Performance Engineering for Developers Help 26.1 All View Local Help Center View Online Help Center Version: